CN115905715A

CN115905715A - Internet data analysis method and platform based on big data and artificial intelligence

Info

Publication number: CN115905715A
Application number: CN202211735448.9A
Authority: CN
Inventors: 刘曼曼
Original assignee: Xuzhou Hai Qing Mdt Infotech Ltd
Current assignee: Xuzhou Hai Qing Mdt Infotech Ltd
Priority date: 2022-12-30
Filing date: 2022-12-30
Publication date: 2023-04-04

Abstract

The application provides an internet data analysis method and a platform based on big data and artificial intelligence, and relates to the technical field of artificial intelligence. In the method, one target internet user in a target internet user group is marked as a first target internet user, and each target internet user outside the first target internet user group is marked as a second target internet user; performing image anomaly identification processing on an internet image to be analyzed corresponding to a first target internet user to obtain an initial image anomaly degree, and screening out a related historical internet image matched with the internet image to be analyzed from a historical internet image corresponding to a second target internet user; and updating the initial image abnormal degree based on the historical image abnormal degree corresponding to the related historical internet image to form the target image abnormal degree. Based on the content, the reliability of the internet data anomaly analysis can be improved to a certain extent.

Description

Internet data analysis method and platform based on big data and artificial intelligence

Technical Field

The application relates to the technical field of artificial intelligence, in particular to an internet data analysis method and platform based on big data and artificial intelligence.

Background

The development of internet technology has led to the increasing of application scenes and application objects, and has become a technology that is commonly used. Based on the application of internet technology, content internet data, including text data, image data, etc., is generated. In addition, artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a digital computer-controlled computation, senses the environment, acquires knowledge and uses the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

In the prior art, for massive internet data, sometimes it is necessary to perform processing such as data anomaly monitoring, for example, anomaly analysis may be performed on the internet data based on an artificial intelligence technology, so that a corresponding anomaly analysis result may be obtained. However, in the prior art, the reliability of the anomaly analysis is poor in the process of performing the anomaly analysis on the internet data.

Disclosure of Invention

In view of this, an object of the present application is to provide an internet data analysis method and platform based on big data and artificial intelligence, so as to improve reliability of internet data anomaly analysis.

In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

a big data and artificial intelligence based internet data analysis method comprises the following steps:

performing user group identification processing on an internet user distribution relationship network to be analyzed by utilizing an optimized internet user group identification neural network to obtain a target internet user group in the internet user distribution relationship network to be analyzed, wherein the internet user distribution relationship network to be analyzed comprises a plurality of internet users, attribute data of each internet user in the internet user distribution relationship network to be analyzed is network behavior data of the internet user, and a distribution position relationship between every two internet users in the internet user distribution relationship network to be analyzed is determined based on a user correlation relationship between every two internet users;

any one target internet user in the target internet user group is marked as a first target internet user, and each target internet user except the first target internet user in the target internet user group is marked as a second target internet user;

performing image anomaly identification processing on the to-be-analyzed internet images corresponding to the first target internet users to obtain initial image anomaly degrees corresponding to the to-be-analyzed internet images, and screening out at least one frame of related historical internet images matched with the to-be-analyzed internet images from the historical internet images corresponding to each second target internet user;

and updating the initial image abnormal degree based on the historical image abnormal degree corresponding to the at least one frame of related historical internet image to form a target image abnormal degree corresponding to the internet image to be analyzed.

In a possible embodiment, in the internet data analysis method based on big data and artificial intelligence, the step of performing image anomaly identification processing on the to-be-analyzed internet image corresponding to the first target internet user to obtain an initial image anomaly degree corresponding to the to-be-analyzed internet image, and screening out at least one frame of related historical internet image matched with the to-be-analyzed internet image from the historical internet images corresponding to each of the second target internet users includes:

performing image key information mining processing on the to-be-analyzed internet image corresponding to the first target internet user by using an image key information mining unit included in a target image abnormality recognition neural network so as to output an to-be-analyzed image description vector corresponding to the to-be-analyzed internet image;

respectively carrying out image key information mining processing on the historical internet image corresponding to each second target internet user by using an image key information mining unit included in the target image abnormality recognition neural network so as to output a historical image description vector corresponding to each frame of historical internet image;

performing image anomaly identification processing on the image description vector to be analyzed by using an image anomaly identification unit included in the target image anomaly identification neural network so as to output an initial image anomaly degree corresponding to the internet image to be analyzed;

and screening out at least one frame of related historical internet image matched with the internet image to be analyzed from the historical internet image corresponding to each second target internet user according to the vector correlation degree between the historical image description vector corresponding to each frame of the historical internet image and the image description vector corresponding to the internet image to be analyzed.

In a possible embodiment, in the internet data analysis method based on big data and artificial intelligence, before the step of performing, by using the optimized internet user group recognition neural network, user group recognition processing on the internet user distribution relationship network to be analyzed to obtain a target internet user group in the internet user distribution relationship network to be analyzed, the internet data analysis method based on big data and artificial intelligence further includes:

acquiring a first exemplary internet user distribution relationship network and a first user group identification data cluster corresponding to the first exemplary internet user distribution relationship network, wherein the first user group identification data cluster comprises first user group identification data of an internet user group corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network;

utilizing each of a plurality of intermediate internet user group recognition neural networks to perform user group recognition processing operation on the first exemplary internet user distribution relationship network respectively so as to output a user group recognition possibility coefficient cluster corresponding to each of the intermediate internet user group recognition neural networks, wherein each of the user group recognition possibility coefficient clusters comprises a user group recognition possibility for each of the exemplary internet users in the first exemplary internet user distribution relationship network to recognize as each of a plurality of internet user groups, and each of the plurality of intermediate internet user group recognition neural networks is formed based on different network optimization of the first internet user group recognition neural network;

for each exemplary internet user in the first exemplary internet user distribution relationship network, analyzing and obtaining a confidence parameter of first user group identification data corresponding to the exemplary internet user according to differences among user group identification possibilities corresponding to the exemplary internet user in a plurality of user group identification possibility coefficient clusters;

carrying out identification data updating processing on first user group identification data of which the corresponding confidence parameters are smaller than preset reference confidence parameters in the first user group identification data cluster so as to form corresponding updated first user group identification data;

combining to form a second user group identification data cluster corresponding to the first exemplary internet user distribution relationship network according to each updated first user group identification data and other first user group identification data in the first user group identification data cluster; and performing network optimization processing on the first internet user group recognition neural network according to the second user group identification data cluster to form a corresponding optimized internet user group recognition neural network.

In a possible embodiment, in the above internet data analysis method based on big data and artificial intelligence, the number of the plurality of internet user groups is equal to the first number;

the step of analyzing, for each exemplary internet user in the first exemplary internet user distribution relationship network, a confidence parameter of first user group identification data corresponding to the exemplary internet user according to a difference between user group identification possibilities corresponding to the exemplary internet user in a plurality of user group identification possibility coefficient clusters, includes:

polling a first number of internet user groups;

according to the difference between the user group identification possibilities corresponding to the A-th internet user group identified by the exemplary internet users in the user group identification possibility coefficient clusters, analyzing and obtaining a confidence parameter of the user group identification possibility corresponding to the exemplary internet users in the A-th internet user group to obtain a candidate confidence parameter of the exemplary internet users, wherein A is smaller than or equal to the first number;

and determining a candidate confidence parameter from the first number of candidate confidence parameters corresponding to the exemplary internet user, wherein the candidate confidence parameter is marked as the confidence parameter of the first user group identification data corresponding to the exemplary internet user.

In a possible embodiment, in the method for analyzing internet data based on big data and artificial intelligence, the step of analyzing a confidence parameter of the user group recognition possibility corresponding to the exemplary internet user in the a-th internet user group according to the difference between the user group recognition possibilities corresponding to the exemplary internet user recognition in the plurality of user group recognition possibility coefficient clusters includes:

analyzing the discrete degree of the user group identification possibility corresponding to the A-th internet user group identified by the exemplary internet user in the plurality of user group identification possibility coefficient clusters to output a corresponding first discrete degree coefficient;

and performing labeling processing on the negative correlation coefficient of the first discrete degree coefficient so as to label a confidence parameter forming the recognition possibility of the user group corresponding to the exemplary internet user under the A-th internet user group.

analyzing a user group attribution characterization parameter for reflecting whether the Internet user group corresponding to the exemplary Internet user belongs to the A-th Internet user group or not according to the first user group identification data corresponding to the exemplary Internet user;

identifying the exemplary internet user as the user group identification possibility corresponding to the A-th internet user group in the plurality of user group identification possibility coefficient clusters, and performing discrete degree analysis on the user group attribution characterization parameters corresponding to the exemplary internet user to output a corresponding second discrete degree coefficient; and labeling the negative correlation coefficient of the second discrete degree coefficient to form a confidence parameter of the user group identification possibility corresponding to the exemplary internet user under the A-th internet user group.

In a possible embodiment, in the method for analyzing internet data based on big data and artificial intelligence, the step of determining a candidate confidence parameter from among the first number of candidate confidence parameters corresponding to the exemplary internet user to mark as the confidence parameter of the first user group identification data corresponding to the exemplary internet user includes:

and marking the candidate confidence parameter with the minimum value in the first number of candidate confidence parameters corresponding to the exemplary internet user to be the confidence parameter of the corresponding first user group identification data of the exemplary internet user.

In a possible embodiment, in the above internet data analysis method based on big data and artificial intelligence, the first number of internet user groups includes a second number of large-scale internet user groups and small-scale internet user groups, and a difference between the first number and the second number is equal to 1;

the step of determining a candidate confidence parameter from the first number of candidate confidence parameters corresponding to the exemplary internet user to label as the confidence parameter of the first user group identification data corresponding to the exemplary internet user comprises:

analyzing a second number of candidate confidence parameters corresponding to the second number of large-scale internet user groups from the first number of candidate confidence parameters corresponding to the exemplary internet users;

and in the second quantity of candidate confidence parameters, marking the candidate confidence parameter with the minimum value as the confidence parameter of the first user group identification data corresponding to the exemplary Internet user.

In a possible embodiment, in the above internet data analysis method based on big data and artificial intelligence, the internet data analysis method further includes:

determining a user group identification information cluster, wherein the user group identification information cluster comprises user group identification information of each exemplary internet user in the first exemplary internet user distribution relationship network, the user group identification information of the exemplary internet user is used for reflecting an internet user group corresponding to the exemplary internet user, and the user group identification information of the exemplary internet user is obtained based on the user group identification possibility that the exemplary internet user is identified as each internet user group in the user group identification possibility coefficient cluster;

determining a third number of to-be-processed confidence parameters from the first number of candidate confidence parameters of the exemplary internet users according to the user group identification information of each exemplary internet user in the user group identification information cluster and the difference between the first user group identification data of the corresponding exemplary internet user in the first user group identification data cluster, wherein the third number is less than or equal to the first number;

and determining a candidate confidence parameter from the third number of to-be-processed confidence parameters corresponding to the exemplary internet users to mark the candidate confidence parameter as the confidence parameter of the identification data of the first user group corresponding to the exemplary internet users.

The application also provides an internet data analysis platform based on big data and artificial intelligence, which comprises a processor and a memory, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program so as to realize the internet data analysis method based on big data and artificial intelligence.

According to the internet data analysis method and platform based on big data and artificial intelligence, an optimized internet user group recognition neural network can be used for carrying out user group recognition processing on an internet user distribution relationship network to be analyzed to obtain a target internet user group in the internet user distribution relationship network to be analyzed; marking a target internet user in a target internet user group as a first target internet user, and marking each target internet user except the first target internet user as a second target internet user; performing image anomaly identification processing on an internet image to be analyzed corresponding to a first target internet user to obtain an initial image anomaly degree, and screening out a related historical internet image matched with the internet image to be analyzed from a historical internet image corresponding to a second target internet user; and updating the initial image abnormal degree based on the historical image abnormal degree corresponding to the related historical internet image to form a target image abnormal degree. Based on the steps, a target internet user group is determined, so that after the image abnormity identification processing is carried out on the to-be-analyzed internet image corresponding to the first target internet user to obtain the initial image abnormity degree, the initial image abnormity degree can be reliably updated based on the history image abnormity degree corresponding to the related history internet image corresponding to the second target internet user belonging to the same user group, the formed target image abnormity degree is higher in reliability, the reliability of the internet data abnormity analysis is improved to a certain extent, and the defects in the prior art are overcome; for example, compared with a scheme of analyzing only internet data of an internet user, the scheme not only combines the internet data of other internet users, but also has higher correlation when the other internet users belong to the same user group, and the internet data is also matched related internet data and also has correlation, so that the combination reliability is higher, and the reliability of the result is guaranteed.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

Fig. 1 is a block diagram of a structure of an internet data analysis platform based on big data and artificial intelligence provided in an embodiment of the present application.

Fig. 2 is a schematic flowchart illustrating steps included in a big data and artificial intelligence-based internet data analysis method according to an embodiment of the present application.

Fig. 3 is a flowchart illustrating steps included in an internet user processing method based on big data and artificial intelligence according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

FIG. 1 illustrates a big data and artificial intelligence based Internet data analytics platform, which may be, for example, a server with data processing capabilities, including a bus or other communication component for communicating information, and a processor coupled to the bus for processing information. The big data and artificial intelligence based internet data analytics platform also includes a main memory, such as a Random Access Memory (RAM) or other dynamic storage device, coupled to the bus for storing information and instructions to be executed by the processor. The main memory may also be used for storing location information, temporary variables, or other intermediate information during execution of instructions by the processor. The big data and artificial intelligence based internet data analytics platform may further include a Read Only Memory (ROM) or other static storage device coupled to the bus for storing static information and instructions for the processor. Coupled to the bus for persistently storing information and instructions, are storage devices, such as solid state devices, magnetic disks, or optical disks.

It should be understood that in some embodiments, the big-data and artificial intelligence based internet data analytics platform may be coupled via a bus to a display (such as a liquid crystal display, or active matrix display) for displaying information to a user.

It should be appreciated that in some embodiments, the big-data and artificial intelligence based internet data analytics platform may include a communications adapter (such as a network adapter). A communications adapter may be coupled to the bus and may be configured to enable communications with a computing or communications network and/or other computing systems. In various illustrative embodiments, any type of networking configuration may be implemented using a communications adapter.

According to various embodiments, the processes of the embodiments described herein may be carried out by the big data and artificial intelligence based internet data analytics platform in response to a processor executing an arrangement of instructions contained in main memory. Such instructions may be read into main memory by another computer-readable medium, such as a storage device. Execution of the arrangement of instructions contained in main memory enables the big-data and artificial intelligence based internet data analytics platform to perform the illustrative processes described herein.

With reference to fig. 2, an embodiment of the present application further provides an internet data analysis method based on big data and artificial intelligence, which is applicable to the internet data analysis platform based on big data and artificial intelligence. The method steps defined by the flow related to the internet data analysis method based on big data and artificial intelligence can be realized by the internet data analysis platform based on big data and artificial intelligence. The specific process shown in FIG. 2 will be described in detail below.

And step S110, carrying out user group identification processing on the Internet user distribution relationship network to be analyzed by utilizing the optimized Internet user group identification neural network so as to obtain a target Internet user group in the Internet user distribution relationship network to be analyzed.

In the embodiment of the present invention, the internet data analysis platform (i.e., the above-mentioned internet data analysis platform based on big data and artificial intelligence) may perform user group recognition processing on the internet user distribution relationship network to be analyzed by using an optimized internet user group recognition neural network (for example, the optimized internet user group recognition neural network may be formed by performing network optimization according to corresponding exemplary data), so as to obtain a target internet user group in the internet user distribution relationship network to be analyzed. The to-be-analyzed internet user distribution relationship network comprises a plurality of internet users, and attribute data of each internet user in the to-be-analyzed internet user distribution relationship network is network behavior data of the internet user (i.e. behaviors performed based on the internet, such as shopping, browsing, gaming, social contact, query and other network behaviors), and in the to-be-analyzed internet user distribution relationship network, a distribution position relationship between every two internet users is determined based on a user correlation relationship between every two internet users (for any two internet users, the closer the user correlation relationship between the two internet users is, the smaller the corresponding distribution position distance is, and conversely, the larger the corresponding distribution position distance is, and further, the user correlation relationship between the two internet users can be determined based on the corresponding network behavior data, or can be determined on the basis of the corresponding network behavior data by combining information such as physical world positions of the users, and the like).

Step S120, any one target Internet user in the target Internet user group is marked as a first target Internet user, and each target Internet user except the first target Internet user in the target Internet user group is marked as a second target Internet user.

In an embodiment of the present invention, the internet data analysis platform (i.e., the above-mentioned internet data analysis platform based on big data and artificial intelligence) may mark any one target internet user in the target internet user group as a first target internet user, and mark each target internet user other than the first target internet user in the target internet user group as a second target internet user (at least one second target internet user may be).

Step S130, performing image anomaly identification processing on the to-be-analyzed internet image corresponding to the first target internet user to obtain an initial image anomaly degree corresponding to the to-be-analyzed internet image, and screening out at least one frame of related historical internet image matched with the to-be-analyzed internet image from the historical internet image corresponding to each second target internet user.

In an embodiment of the present invention, the internet data analysis platform (that is, the above-mentioned internet data analysis platform based on big data and artificial intelligence) may perform image anomaly recognition processing (which may be implemented by using an image anomaly recognition neural network) on an internet image to be analyzed corresponding to the first target internet user to obtain an initial image anomaly degree corresponding to the internet image to be analyzed, and screen out at least one frame of related historical internet image matched with the internet image to be analyzed from historical internet images corresponding to each second target internet user.

Step S140, updating the initial image abnormal degree based on the historical image abnormal degree corresponding to each of the at least one frame of related historical internet image, so as to form a target image abnormal degree corresponding to the internet image to be analyzed.

In this embodiment of the present invention, the internet data analysis platform (i.e. the above-mentioned internet data analysis platform based on big data and artificial intelligence) may update the initial image anomaly degree based on the historical image anomaly degree corresponding to each of the at least one frame of related historical internet image (the historical image anomaly degree corresponding to each of the related historical internet images may be obtained by analyzing the related historical internet image as an internet image to be analyzed historically or may be configured historically), so as to form a target image anomaly degree corresponding to the internet image to be analyzed (for example, the historical image anomaly degree corresponding to each of the at least one frame of related historical internet image may be calculated by weighted average based on the correlation between each frame of related historical internet image and the internet image to be analyzed, to obtain a weighted history image abnormality degree, then, the initial image abnormality degree may be updated based on the weighted history image abnormality degree, if the weighted history image abnormality degree is greater than the initial image abnormality degree, the initial image abnormality degree may be expanded, the expanded magnitude may be positively correlated with a ratio between the weighted history image abnormality degree and the initial image abnormality degree, if the weighted history image abnormality degree is less than the initial image abnormality degree, the initial image abnormality degree may be reduced, the reduced magnitude may be negatively correlated with a ratio between the weighted history image abnormality degree and the initial image abnormality degree, if the weighted history image abnormality degree is equal to the initial image abnormality degree, the initial image abnormal degree may be used as a corresponding target image abnormal degree, and the target image abnormal degree may be an image violation degree of the internet image to be analyzed, or the like).

Based on the foregoing steps (e.g., step S110-step S140), since the target internet user group is determined first, after the image abnormality recognition processing is performed on the to-be-analyzed internet image corresponding to the first target internet user to obtain the initial image abnormality degree, the initial image abnormality degree can be reliably updated based on the history image abnormality degree corresponding to the related history internet image corresponding to the second target internet user belonging to the same user group, so that the reliability of the formed target image abnormality degree is higher, thereby improving the reliability of the internet data abnormality analysis to a certain extent and overcoming the defects in the prior art; for example, compared with a scheme of analyzing only internet data of an internet user, the scheme not only combines the internet data of other internet users, but also has higher correlation when the other internet users belong to the same user group, and the internet data is also matched related internet data and also has correlation, so that the reliability of the combination is higher, and the reliability of the result is guaranteed, namely higher reliability.

It should be understood that, in some embodiments, for step S130 in the above description, it may further include the following details:

performing image key information mining on the to-be-analyzed internet image corresponding to the first target internet user by using an image key information mining unit included in a target image abnormality recognition neural network to output a to-be-analyzed image description vector corresponding to the to-be-analyzed internet image (the image key information mining may include performing feature space mapping on the to-be-analyzed internet image, and then performing convolution operation on a result of the feature space mapping to realize filtering, so as to output the to-be-analyzed image description vector corresponding to the to-be-analyzed internet image, and the target image abnormality recognition neural network may be obtained by performing network optimization processing based on corresponding exemplary data);

performing image anomaly identification processing on the image description vector to be analyzed by using an image anomaly identification unit included in the target image anomaly identification neural network so as to output an initial image anomaly degree corresponding to the internet image to be analyzed (the image anomaly identification unit may include a softmax function so as to realize excitation mapping output and obtain a corresponding initial image anomaly degree);

according to the vector correlation degree between the historical image description vector corresponding to each frame of historical internet image and the image description vector corresponding to the internet image to be analyzed, at least one frame of relevant historical internet image matched with the internet image to be analyzed is screened out from the historical internet image corresponding to each second target internet user (based on this, because the historical image description vector and the image description vector to be analyzed are obtained based on the target image abnormality recognition neural network, the historical image description vector and the image description vector to be analyzed both pay more attention to the abnormal information in the image, so that the vector correlation degree of the calculation output is more emphasized in the dimension of the abnormal information, and therefore, the reliability of the update of the abnormal degree of the determined relevant historical internet image is higher, namely, the abnormal information in the image is more concerned).

It should be appreciated that, in some embodiments, for step S110 in the above description, before that, the big-data and artificial-intelligence-based internet data analysis method may further include the following steps (to optimize and form the optimized internet user population recognition neural network):

collecting a first exemplary internet user distribution relationship network and a first user group identification data cluster corresponding to the first exemplary internet user distribution relationship network, where the first user group identification data cluster includes first user group identification data of an internet user group corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network (that is, the first user group identification data is used to reflect a group where the exemplary internet user is located, and the first user group identification data cluster may be user group identification data with low reliability, that is, formed by coarse-grained labeling, that is, user group identification data that may be incorrect exists, and in a case where an internet user group reflected by the first user group identification data of an exemplary internet user is not identical to an actual internet user group of the exemplary internet user, the first user group identification data of the exemplary internet user may be regarded as user group identification data of the user that is incorrect, and in a case where an internet user group reflected by the first user group identification data of an exemplary internet user is identical to an actual internet user group of the exemplary internet user, the first user group identification data of the exemplary user group may be regarded as user group identification data of the first user group of the exemplary user group of the internet user;

utilizing each of a plurality of intermediate Internet user group recognition neural networks to perform user group recognition processing operation on the first exemplary Internet user distribution relational network respectively so as to output a user group recognition possibility coefficient cluster corresponding to each of the intermediate Internet user group recognition neural networks, wherein each of the user group recognition possibility coefficient clusters comprises the user group recognition possibility that each of the first exemplary Internet user distribution relational network recognizes as each of a plurality of Internet user groups, and each of the plurality of intermediate Internet user group recognition neural networks is formed based on performing different network optimization on the first Internet user group recognition neural network (for example, the times of performing network optimization are different or exemplary data are different, and the like);

for each exemplary internet user in the first exemplary internet user distribution relationship network, analyzing to obtain a confidence parameter of first user group identification data corresponding to the exemplary internet user according to differences between user group identification possibilities corresponding to the exemplary internet user in a plurality of user group identification possibility coefficient clusters;

updating identification data of the first user group identification data in the first user group identification data cluster, wherein the corresponding confidence parameter of the first user group identification data cluster is smaller than the preconfigured reference confidence parameter, so as to form corresponding updated first user group identification data (that is, the analyzed unpaired first user group identification data can be updated and adjusted to obtain updated first user group identification data, that is, the paired first user group identification data and the correct first user group identification data);

according to each updated first user group identification data and other first user group identification data in the first user group identification data cluster, combining to form a second user group identification data cluster corresponding to the first exemplary internet user distribution relationship network; and performing network optimization processing on the first internet user group identification neural network according to the second user group identification data cluster to form a corresponding optimized internet user group identification neural network (based on this, the user group identification data in the second user group identification data cluster can be regarded as correct user group identification data, or the accuracy of the second user group identification data cluster is higher compared with that of the first user group identification data cluster, so that the recognition reliability of the optimized internet user group identification neural network formed by performing network optimization processing on the basis of the second user group identification data cluster is higher).

It should be understood that, in some embodiments, the number of the plurality of internet user groups may be equal to the first number, and based on this, for each of the exemplary internet users in the first exemplary internet user distribution relationship network in the above description, the step of analyzing the confidence parameter of the first user group identification data corresponding to the exemplary internet user according to the difference between the user group identification possibilities corresponding to the exemplary internet user in the plurality of user group identification possibility coefficient clusters may further include the following specific contents:

polling a first number of internet user groups;

analyzing a confidence parameter of the user group identification possibility corresponding to the exemplary internet user in the A-th internet user group (namely, the internet user group currently polled in the first number of internet user groups) according to the difference between the user group identification possibilities corresponding to the exemplary internet user (each of which is processed separately) identified as the A-th internet user group (in the plurality of user group identification possibility coefficient clusters) so as to obtain a candidate confidence parameter of the exemplary internet user, wherein A is less than or equal to the first number;

It should be understood that, in some embodiments, for the difference between the user group recognition possibilities of the exemplary internet user recognition as the a-th internet user group in the user group recognition possibility coefficient clusters according to the plurality of user group recognition possibility coefficients in the above description, the step of analyzing the confidence parameter of the user group recognition possibility corresponding to the exemplary internet user in the a-th internet user group may further include the following specific contents:

analyzing the discrete degree of the user group identification possibility corresponding to the A-th internet user group identified by the exemplary internet user in the plurality of user group identification possibility coefficient clusters to output a corresponding first discrete degree coefficient; and labeling the negative correlation coefficient of the first discrete degree coefficient to form a confidence parameter of the recognition possibility of the user group corresponding to the exemplary internet user in the a-th internet user group (that is, the negative correlation coefficient of the discrete degree corresponding to the recognition possibility of the user group can be used as the confidence parameter, that is, the greater the difference between the recognition possibilities of the user groups, the lower the confidence, the smaller the difference between the recognition possibilities of the user groups, and the higher the confidence).

analyzing a user group attribution characterization parameter for reflecting whether the internet user group corresponding to the exemplary internet user belongs to the a-th internet user group or not according to the first user group identification data corresponding to the exemplary internet user (for example, if the first user group identification data reflects that the internet user group corresponding to the exemplary internet user belongs to the a-th internet user group, the user group attribution characterization parameter may be 100%, and if the first user group identification data reflects that the internet user group corresponding to the exemplary internet user does not belong to the a-th internet user group, the user group attribution characterization parameter may be 0%);

identifying the exemplary internet user in the multiple user group identification possibility coefficient clusters as the user group identification possibility corresponding to the A-th internet user group, and analyzing the discrete degree of the user group attribution characterization parameter corresponding to the exemplary internet user to output a corresponding second discrete degree coefficient; and labeling the negative correlation coefficient of the second discrete degree coefficient to form a confidence parameter of the user group identification possibility corresponding to the exemplary internet user under the A-th internet user group.

In some embodiments, it should be understood that, in the above description, for the difference between the user group recognition possibilities of the exemplary internet user recognition as the a-th internet user group in the plurality of user group recognition possibility coefficient clusters, the step of analyzing the confidence parameter of the user group recognition possibility of the exemplary internet user corresponding to the a-th internet user group under the a-th internet user group may further include the following specific contents:

analyzing the user group recognition possibility corresponding to the A-th Internet user group identified by the exemplary Internet users in the plurality of user group recognition possibility coefficient clusters based on a configured discriminant probability model to form an optimized user group recognition possibility (the discriminant probability model may be used to improve the accuracy of each user group recognition possibility in the plurality of user group recognition possibility coefficient clusters, and the discriminant probability model may be a conditional probability distribution model P (Y | X) representing a Markov random field of another set of output random variables Y given a set of input random variables X);

according to the difference between the optimal user group identification possibilities corresponding to the A-th internet user group identified by the exemplary internet user identification, a confidence parameter of the user group identification possibility corresponding to the exemplary internet user in the A-th internet user group is obtained through analysis (for example, based on the dispersion degree analysis, the corresponding confidence parameter is determined).

It should be appreciated that, in some embodiments, for the step of determining a candidate confidence parameter from the first number of candidate confidence parameters corresponding to the exemplary internet user in the above description to label the candidate confidence parameter as the confidence parameter of the first user group identification data corresponding to the exemplary internet user, the following may be further included:

It should be understood that, in some embodiments, the first number of internet user groups may include a second number of large-scale internet user groups and a small-scale internet user group, and a difference between the first number and the second number is equal to 1 (i.e. the total number of the second number of large-scale internet user groups and the small-scale internet user group is the first number, and the large-scale internet user group and the small-scale internet user group may refer to how many internet users are included relative to each other, such as the number of included internet users is relatively large, and may be used as the large-scale internet user group, such as the number of included internet users is relatively small, and may be used as the small-scale internet user group), based on which, for the step of determining one candidate confidence parameter from the first number of candidate confidence parameters corresponding to the exemplary internet users in the above description to mark as the confidence parameter of the first user group identification data corresponding to the exemplary internet users, may further include the following specific contents:

and in the second quantity of candidate confidence parameters, performing labeling processing on the candidate confidence parameters with the minimum value to label the confidence parameters as the identification data of the first user group corresponding to the exemplary internet user (since the small-scale internet user group has a smaller scale, corresponding exclusion can be performed, so that the confidence parameters can be screened from the second quantity of candidate confidence parameters corresponding to the second quantity of large-scale internet user groups to serve as a representative).

It should be appreciated that in some embodiments, the big data and artificial intelligence based internet data analysis method may further include the following steps:

determining a user group identification information cluster, where the user group identification information cluster includes user group identification information of each exemplary internet user in the first exemplary internet user distribution relationship network, the user group identification information of the exemplary internet user is used to reflect an internet user group corresponding to the exemplary internet user, and the user group identification information of the exemplary internet user is obtained based on a user group identification possibility that the exemplary internet user is identified as each internet user group in the user group identification possibility coefficient cluster (for example, an internet user group corresponding to a user group identification possibility having a maximum value may be used as an internet user group corresponding to the exemplary internet user reflected by the user group identification information);

the step of determining a third number of confidence parameters to be processed among the first number of confidence parameters of the exemplary internet users, the third number being less than or equal to the first number, according to the difference between the user group identification information of each exemplary internet user in the user group identification information cluster and the first user group identification data of the corresponding exemplary internet user in the first user group identification data cluster (in this way, determining a confidence parameter candidate from the first number of confidence parameters of the exemplary internet users to be marked as the confidence parameter of the first user group identification data of the corresponding exemplary internet user may further include the specific content of determining a confidence parameter candidate (such as the minimum) from the third number of confidence parameters to be processed of the corresponding exemplary internet user to be marked as the confidence parameter of the first user group identification data of the corresponding exemplary internet user.

It should be understood, that in some embodiments, for the difference between the user group identification information of each exemplary internet user in the user group identification information cluster and the first user group identification data of the corresponding exemplary internet user in the first user group identification data cluster in the above description, the step of determining a third number of confidence parameters to be processed from among the first number of candidate confidence parameters of the exemplary internet users may further include the following specific contents:

polling a first number of candidate confidence parameters corresponding to the exemplary internet user;

for the a candidate confidence parameter for the exemplary internet user (i.e. the currently polled candidate confidence parameter in the first number of candidate confidence parameters), analyzing a user group attribution representative parameter for reflecting whether the internet user group corresponding to the exemplary internet user belongs to the a internet user group or not according to the user group identification information corresponding to the exemplary internet user (e.g. the internet user group corresponding to the exemplary internet user belongs to the a internet user group, the user group attribution representative parameter may be configured to be one value, such as 100%, the internet user group corresponding to the exemplary internet user does not belong to the a internet user group, the user group attribution representative parameter may be configured to be another value, such as 0%), and analyzing a user group attribution representative parameter for reflecting whether the internet user group corresponding to the exemplary internet user belongs to the a internet user group according to the first user group identification data corresponding to the exemplary internet user (as described above);

analyzing whether to mark the A-th candidate confidence parameter of the exemplary internet user according to the difference between the representative parameter of the user group attribution corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network and the characterization parameter of the user group attribution corresponding to the exemplary internet user, so as to mark the result as a confidence parameter to be processed, and forming a third number of confidence parameters to be processed.

It should be understood that, in some embodiments, for the difference between the user group affiliation representative parameter corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network and the user group affiliation characterizing parameter corresponding to the exemplary internet user in the above description, the step of analyzing whether to mark the a-th candidate confidence parameter of the exemplary internet user as a result of one to-be-processed confidence parameter may further include the following specific contents:

determining an overlap degree characterization parameter between the user group attribution representative parameter corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network and the user group attribution characterization parameter corresponding to each exemplary internet user in the first exemplary internet user distribution relationship network (for example, first determining an intersection and a union of the user group attribution representative parameter and the user group attribution characterization parameter, and then performing quotient value calculation on the parameter quantity in the intersection and the parameter quantity in the union to obtain an overlap degree characterization parameter);

and under the condition that the contact ratio characterization parameter exceeds a pre-configured reference contact ratio characterization parameter, marking the A-th candidate confidence parameter of the exemplary Internet user as a confidence parameter to be processed (the specific value of the reference contact ratio characterization parameter can be configured according to actual application requirements, and is not specifically limited and described here).

It should be understood that, in some embodiments, the number of the plurality of internet user groups may be equal to the first number, and based on this, the step of performing the identification data updating process on the first user group identification data in the first user group identification data cluster whose corresponding confidence parameter is smaller than the pre-configured reference confidence parameter to form the corresponding updated first user group identification data in the above description may further include the following specific contents:

for first user group identification data (which may be each processed correspondingly) whose corresponding confidence parameter exceeds the reference confidence parameter, obtaining an a-th candidate confidence parameter of a first exemplary internet user (the first exemplary internet user is an exemplary internet user corresponding to the first user group identification data) corresponding to the first user group identification data, and obtaining an a-th representative user group recognition possibility of the first exemplary internet user, where the a-th candidate confidence parameter of the first exemplary internet user is used to reflect the user group recognition possibility of the first exemplary internet user in the a-th internet user group, and the a-th representative user group recognition possibility of the first exemplary internet user is used to reflect a mean value of the user group recognition possibilities of the a-th internet user group corresponding to the first exemplary internet user in the plurality of user group recognition possibility coefficient clusters;

analyzing and outputting an updated representative coefficient of the first exemplary internet user under the A-th internet user group according to the A-th candidate confidence parameter of the first exemplary internet user and by combining the A-th representative user group identification possibility of the first exemplary internet user;

screening out a first updated representative coefficient from the updated representative coefficients of the first exemplary internet user in each internet user group, and marking the internet user group corresponding to the first updated representative coefficient to be updated first user group identification data corresponding to the first user group identification data.

Wherein, it should be understood that, in some embodiments, for the above description, the step of analyzing and outputting the updated representative coefficient of the first exemplary internet user under the a-th internet user group in combination with the a-th representative user group identification possibility of the first exemplary internet user according to the a-th candidate confidence parameter of the first exemplary internet user may further include the following specific contents:

performing quotient calculation (for example, dividing the latter by the former) on the a candidate confidence parameter of the first exemplary internet user and the a representative user group identification possibility of the first exemplary internet user to output an updated representative coefficient of the first exemplary internet user in the a representative user group (based on which, in the updated representative coefficient of the first exemplary internet user in each of the internet user groups, a first updated representative coefficient is filtered out, and the internet user group corresponding to the first updated representative coefficient is subjected to labeling processing to label the updated first user group identification data corresponding to the first user group identification data, specific content may be further included in that the updated representative coefficient of the first exemplary internet user having the minimum value in the updated representative coefficients of each of the internet user groups to label the updated representative coefficient to form a corresponding first updated representative coefficient, and the internet user group corresponding to the first updated representative coefficient is subjected to labeling processing to label the updated representative coefficient corresponding to the first user group identification data.

With reference to fig. 3, an embodiment of the present application further provides an internet user processing method based on big data and artificial intelligence, which is applicable to the internet data analysis platform based on big data and artificial intelligence. The method steps defined by the flow related to the internet user processing method based on big data and artificial intelligence can be realized by the internet data analysis platform based on big data and artificial intelligence. The internet user processing method based on big data and artificial intelligence can comprise the following steps:

performing user group identification processing on the internet user distribution relationship network to be analyzed by using the optimized internet user group identification neural network to obtain a target internet user group in the internet user distribution relationship network to be analyzed (including the detailed description of the foregoing step S110);

tagging any one target internet user in the target internet user group as a first target internet user, and tagging each target internet user other than the first target internet user in the target internet user group as a second target internet user (including the foregoing detailed description of step S120);

performing image anomaly identification processing on the to-be-analyzed internet images corresponding to the first target internet users to obtain initial image anomaly degrees corresponding to the to-be-analyzed internet images, and screening out at least one frame of related historical internet images matched with the to-be-analyzed internet images from the historical internet images corresponding to each second target internet user (including the detailed description of the step S130);

updating the initial image abnormal degree based on the historical image abnormal degree corresponding to each of the at least one frame of related historical internet image to form a target image abnormal degree corresponding to the internet image to be analyzed (including the detailed description of the foregoing step S140);

performing mean value calculation based on the target image abnormal degree corresponding to the to-be-analyzed internet image and the historical image abnormal degree corresponding to the latest frame of historical internet image corresponding to each second target internet user, and performing group abnormal analysis processing on the target internet user group based on the result of the mean value calculation to obtain a group abnormal result corresponding to the target internet user group (the group abnormal result can be used for reflecting the abnormal degree of the target internet user group, and the abnormal degree is positively correlated with the result of the mean value calculation).

In summary, according to the internet data analysis method and the internet data analysis platform based on big data and artificial intelligence, the optimized internet user group recognition neural network can be used for carrying out user group recognition processing on the internet user distribution relationship network to be analyzed to obtain a target internet user group in the internet user distribution relationship network to be analyzed; marking a target internet user in a target internet user group as a first target internet user, and marking each target internet user except the first target internet user as a second target internet user; performing image anomaly identification processing on an internet image to be analyzed corresponding to a first target internet user to obtain an initial image anomaly degree, and screening out a related historical internet image matched with the internet image to be analyzed from a historical internet image corresponding to a second target internet user; and updating the initial image abnormal degree based on the historical image abnormal degree corresponding to the related historical internet image to form the target image abnormal degree. Based on the steps, a target internet user group is determined, so that after the image abnormity identification processing is carried out on the to-be-analyzed internet image corresponding to the first target internet user to obtain the initial image abnormity degree, the initial image abnormity degree can be reliably updated based on the history image abnormity degree corresponding to the related history internet image corresponding to the second target internet user belonging to the same user group, the formed target image abnormity degree is higher in reliability, the reliability of the internet data abnormity analysis is improved to a certain extent, and the defects in the prior art are overcome; for example, compared with a conventional scheme of analyzing only internet data of an internet user, the scheme not only combines the internet data of other internet users, but also the other internet users and the analyzed internet user belong to the same user group, the correlation is high, the internet data is also matched related internet data, and the correlation is also high, so that the reliability of the combination is high, and the reliability of the result is guaranteed, namely, the reliability is high.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An internet data analysis method based on big data and artificial intelligence is characterized by comprising the following steps:

2. The internet data analysis method based on big data and artificial intelligence according to claim 1, wherein the step of performing image anomaly recognition processing on the to-be-analyzed internet image corresponding to the first target internet user to obtain an initial image anomaly degree corresponding to the to-be-analyzed internet image, and screening out at least one frame of related historical internet image matching with the to-be-analyzed internet image from the historical internet images corresponding to each of the second target internet users comprises:

respectively carrying out image key information mining processing on historical internet images corresponding to each second target internet user by using an image key information mining unit included in the target image abnormity identification neural network so as to output historical image description vectors corresponding to each frame of historical internet image;

3. The big data and artificial intelligence based internet data analysis method according to claim 1 or 2, wherein before the step of performing the user group recognition processing on the internet user distribution relationship network to be analyzed by using the optimized internet user group recognition neural network to obtain the target internet user group in the internet user distribution relationship network to be analyzed, the big data and artificial intelligence based internet data analysis method further comprises:

4. The big data and artificial intelligence based internet data analysis method of claim 3, wherein the number of the plurality of internet user groups is equal to a first number;

the step of analyzing, for each exemplary internet user in the first exemplary internet user distribution relationship network, a confidence parameter of the first user group identification data corresponding to the exemplary internet user according to a difference between the user group identification possibilities corresponding to the exemplary internet user in the plurality of user group identification possibility coefficient clusters, includes:

polling a first number of internet user groups;

according to the difference between the user group recognition possibilities corresponding to the A-th internet user group identified by the exemplary internet user in the plurality of user group recognition possibility coefficient clusters, analyzing and obtaining a confidence parameter of the user group recognition possibility corresponding to the exemplary internet user in the A-th internet user group to obtain a candidate confidence parameter of the exemplary internet user, wherein A is smaller than or equal to the first number;

5. The internet data analysis method based on big data and artificial intelligence according to claim 4, wherein the step of analyzing the confidence parameter of the recognition possibility of the user group corresponding to the exemplary internet user in the a-th internet user group according to the difference between the recognition possibilities of the user group corresponding to the exemplary internet user recognition in the plurality of user group recognition possibility coefficient clusters comprises:

performing discrete degree analysis on the user group identification possibility corresponding to the A-th internet user group identified by the exemplary internet users in the plurality of user group identification possibility coefficient clusters to output a corresponding first discrete degree coefficient;

and performing marking processing on the negative correlation coefficient of the first discrete degree coefficient so as to mark and form a confidence parameter of the user group identification possibility corresponding to the exemplary internet user under the A-th internet user group.

6. The internet data analysis method based on big data and artificial intelligence according to claim 4, wherein the step of analyzing the confidence parameter of the recognition possibility of the user group corresponding to the exemplary internet user in the a-th internet user group according to the difference between the recognition possibilities of the user group corresponding to the exemplary internet user recognition in the plurality of user group recognition possibility coefficient clusters comprises:

7. The big data and artificial intelligence based internet data analysis method of claim 4, wherein the step of determining a confidence candidate parameter among the confidence candidate parameters of the first number of internet users corresponding to the example internet user to label as the confidence parameter of the identification data of the first user group corresponding to the example internet user comprises:

8. The big-data and artificial-intelligence based internet data analysis method of claim 4, wherein the first number of internet user groups comprises a second number of large-scale internet user groups and a small-scale internet user group, and a difference between the first number and the second number is equal to 1;

the step of determining a candidate confidence parameter from among the first number of candidate confidence parameters corresponding to the exemplary internet user to mark as the confidence parameter of the first user group identification data corresponding to the exemplary internet user comprises:

9. The big data and artificial intelligence based internet data analysis method of claim 4, wherein the internet data analysis method further comprises:

10. An internet data analysis platform based on big data and artificial intelligence, comprising a processor and a memory, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program to realize the method of any one of claims 1 to 9.