CN111552857B - Feature event identification method and device, electronic equipment and storage medium - Google Patents
Feature event identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111552857B CN111552857B CN202010373434.1A CN202010373434A CN111552857B CN 111552857 B CN111552857 B CN 111552857B CN 202010373434 A CN202010373434 A CN 202010373434A CN 111552857 B CN111552857 B CN 111552857B
- Authority
- CN
- China
- Prior art keywords
- target data
- internet channel
- event
- internet
- identifying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000002596 correlated effect Effects 0.000 claims abstract description 20
- 230000000875 corresponding effect Effects 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 229920001690 polydopamine Polymers 0.000 description 2
- 230000008707 rearrangement Effects 0.000 description 2
- 239000004984 smart glass Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computer Security & Cryptography (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
One or more embodiments of the present disclosure provide a method and apparatus for identifying a feature event, an electronic device, and a storage medium. The method comprises the following steps: respectively determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event, wherein the grade is positively correlated with the association degree; distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel; and analyzing the collected target data to identify characteristic events contained in the target data.
Description
Technical Field
One or more embodiments of the present disclosure relate to the field of network information processing technologies, and in particular, to a method and apparatus for identifying a feature event, an electronic device, and a storage medium.
Background
With the development of the internet, more and more users tend to spread information through the internet, and information security of the internet needs to be managed with enhanced. For information management of the internet, at present, information is generally captured by manually browsing pages, collecting pages and marking, and a large amount of labor cost is required.
Disclosure of Invention
In view of this, one or more embodiments of the present disclosure provide a method and apparatus for identifying a feature event, an electronic device, and a storage medium
In order to achieve the above object, one or more embodiments of the present disclosure provide the following technical solutions:
according to a first aspect of one or more embodiments of the present disclosure, a method for identifying a feature event is provided, including:
respectively determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event, wherein the grade is positively correlated with the association degree;
distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel;
and analyzing the collected target data to identify characteristic events contained in the target data.
According to a second aspect of one or more embodiments of the present specification, there is provided an identification device for a characteristic event, including:
the determining module is used for respectively determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event, and the grade is positively correlated with the association degree;
the distribution module is used for distributing acquisition resources to each Internet channel and respectively acquiring target data from each Internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel;
and the identification module is used for identifying the characteristic event contained in the target data by analyzing the collected target data.
According to a third aspect of one or more embodiments of the present specification, there is provided an electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of identifying a characteristic event as described in any preceding claim by executing the executable instructions.
According to a fourth aspect of one or more embodiments of the present description, a computer-readable storage medium is provided, on which computer instructions are stored, which instructions, when executed by a processor, implement the steps of the method for identifying a characteristic event according to any of the above.
Drawings
FIG. 1 is a schematic diagram of a system architecture for implementing feature event recognition as shown in an exemplary embodiment of the present description.
Fig. 2 is a flowchart illustrating a method for identifying a feature event according to an exemplary embodiment of the present disclosure.
Fig. 3 is a flowchart illustrating another method for identifying a characteristic event according to an exemplary embodiment of the present disclosure.
Fig. 4 is a flowchart illustrating another method for identifying a characteristic event according to an exemplary embodiment of the present disclosure.
Fig. 5 is a schematic structural view of an apparatus shown in an exemplary embodiment of the present specification.
Fig. 6 is a schematic block diagram of an apparatus for identifying a characteristic event according to an exemplary embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
In an embodiment, the recognition scheme of the feature event of the present disclosure may be applied to an electronic device, for example, the electronic device may include any type of mobile phone, tablet device, notebook computer, palm computer (PDAs, personal Digital Assistants), wearable device (such as smart glasses, smart watch, etc.), and the present disclosure is not limited thereto.
Fig. 1 is a schematic diagram of a system architecture for implementing feature event recognition according to an exemplary embodiment of the present disclosure. As shown in fig. 1, the system may include a server 11, a network 12, a number of electronic devices, such as a cell phone 13, a cell phone 14, a cell phone 15, and the like.
The server 11 may be a physical server comprising a separate host, or the server 11 may be a virtual server carried by a cluster of hosts. During the running process, the server 11 may run a program on the server side of a certain application to implement the relevant service functions of the application, for example, when the server 11 runs a program of the identification application of the feature event, it may be implemented as a server of the identification application of the feature event.
It should be noted that the mobile phones 13-15 are only one type of electronic device that can be used by the user, and in fact, it is obvious that the user can also use electronic devices such as the following types: tablet devices, notebook computers, palm top computers (PDAs, personal Digital Assistants), wearable devices (e.g., smart glasses, smart watches, etc.), etc., as one or more embodiments of the present description are not limited in this regard. And the network 12 for interaction between the handsets 13-15 and the server 11 may comprise various types of wired or wireless networks. In one embodiment, the network 12 may include a public switched telephone network (Public Switched Telephone Network, PSTN) and the internet.
Fig. 2 is a flowchart illustrating a method for identifying a feature event according to an exemplary embodiment of the present disclosure. As shown in fig. 2, the method may include the steps of:
step 202, determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event.
Among other things, the Internet channel may include, but is not limited to: forum, official network, APP (application), microblog, weChat applet, public number etc. In this embodiment, the status of the internet channel is not limited, and the internet channel may be an internet channel which completes domain name registration and has handled internet information service and has not handled internet information service and has expired.
The feature event is an internet event to be monitored, for example, information that the internet channel issues once and contains preset target content is called a feature event. The target content can be set according to the actual requirement and can be sensitive content, negative content and the like.
In step 202, the level of the internet channel is positively correlated with the degree of correlation, and the higher the level of the internet channel is, the lower the level of correlation of the internet channel with the feature event is, and the higher the level of the internet channel is.
In one embodiment, the degree of association between the internet channels and the feature event may be determined according to historical empirical data, so as to rank each internet channel. Taking the example that the characteristic event is the release sensitive information, if the probability of releasing the sensitive information on the microblog (the internet channel) is larger than the probability of releasing the sensitive information on the forum (the internet channel) according to the historical experience data, the association degree of the microblog and the characteristic event is higher than the association degree of the forum and the characteristic event, and the microblog grade can be determined to be higher than the forum grade.
In another embodiment, the initial collection of the target data may be performed on each internet channel, and the grade of the internet channel is determined by analyzing the target data collected initially, for example, whether the target data includes a preset keyword, so as to determine the association degree of the internet channel and the feature event. For initial collection of target data, the method can be realized by adopting the modes of open data downloading, API interface, web crawler and the like, but is not limited to.
It can be understood that the content of the page presented on the internet channel is updated from time to time, so that whether the internet channel generates the characteristic event cannot be accurately judged through the target data acquired at one time, and therefore the following steps are needed to acquire the target data of the internet channel in a circulating way and analyze the target data to determine whether the internet channel generates the characteristic event.
And 204, distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources.
The resource quantity of the acquisition resources allocated to each Internet channel is positively correlated with the grade corresponding to each Internet channel. Acquisition resources may be, but are not limited to, storage resources, computing resources, IP resources, etc. used in data acquisition.
It can be understood that the characteristic events on the internet have the characteristics of high burst frequency, fast transmission, wide audience, various forms and the like, and the wider the target data acquisition range of the internet channel is, the higher the frequency is, the more accurate and timely the characteristic events can be identified, which is required to be based on the premise of distributing enough acquisition resources.
However, in the case where the acquisition resources are limited, sufficient acquisition resources cannot be allocated for each internet channel. In step 204, corresponding resource amounts of the acquisition resources are allocated to each internet channel according to the grade of the internet channel, and the internet channel with higher grade can allocate more acquisition resources, so that the target data of the internet channel can be acquired with higher acquisition frequency or shorter acquisition period, and key monitoring of the internet channel with higher grade is realized. The acquisition resources may, but are not limited to, acquiring text type target data, picture type target data, video type target data, voice type target data, etc. of the internet channel.
Step 206, analyzing the collected target data to identify the characteristic event contained in the target data.
In step 206, the target data is analyzed, which may be to determine whether the target data contains a preset keyword, and determine that the internet channel may generate a feature event when the target data contains the preset keyword. The preset keywords can be keywords related to the feature event and can be set according to actual requirements. For example, taking the example of a characteristic event being the release of sensitive information containing illegal financial activity content, keywords may be, but are not limited to, "high interest credits," "high interest," and the like.
Because the internet channels of different grades are allocated with the acquisition resources of different resource amounts, the target data can be acquired in different acquisition periods (or acquisition frequencies) and acquisition ranges for the internet of different grades, the acquisition periods of the internet channels of different grades are overlapped differently, which means that the total acquisition of the target data can not be carried out on all the internet channels at the same time, and the data amount of the target data acquired in each time period or each moment can not be too large, so that the acquired target data can be analyzed in real time in step 206, and the characteristic events of the internet channels can be identified in time. By full-size collection is meant the collection of all data provided by the internet channel, including the page data of the top page, the page data of the sub-pages, etc.
In the embodiment, the resource quantity of the acquisition resources allocated to the internet channel is matched with the grade of the internet channel, the association degree of the internet channel with the characteristic event with higher grade is higher, the acquisition resources with more resource quantity are allocated, and the acquisition resources are subjected to key acquisition and monitoring, so that the acquisition frequency and range are enlarged, and the characteristic event generated in the moment of the internet channel can be captured; the correlation degree between the internet channels with lower grades and the characteristic events is lower, and acquisition resources with smaller resource quantity are allocated; therefore, the acquisition of a large amount of effective data and the maximization of the utility of the acquisition result are realized under the condition of the existing acquisition resource consumption.
In another embodiment, if the target data is identified to include the feature event, the pre-warning information may also be sent to the preset alert object, so that the preset alert object can timely cope with and process the feature event. The early warning information may include the reason for generating the feature event, the domain name of the website generating the feature event, the publisher and/or registrar of the website, the web content generating the feature event, and the like. The early warning information can be sent to a preset warning object in a short message, mail and other modes, so that early discovery and early warning of the characteristic event are realized; the user interface for viewing the characteristic event can also be provided for the preset alarm object, and the characteristic event can be prompted through a page popup window. The preset alert object may also use the user interface to set keywords for the feature event.
In another embodiment, the ranking of the plurality of Internet channels may also be updated based on the target data collected in step 204. In step 202, the grades of the internet channels are preliminary grades of the internet channels, and the degree of correlation between the internet channels and the feature event is changed along with the data update of the internet channels, so that in order to improve the accuracy of feature event identification, the grade grades of the internet channels can be adjusted according to the target data collected again in step 204.
If the target data of the internet channel is periodically collected in step 204, the target data collected in each collection period can be analyzed and the grading of the internet channel can be adjusted under the condition that the target data of each collection period is completely collected. Therefore, the accuracy of grading can be improved by circularly collecting the target data and repeatedly correcting the grading of the internet channel, the reasonable distribution of the collection resources is facilitated, and the accuracy of feature event identification is improved.
Fig. 3 is a flowchart of another method for identifying a characteristic event according to an exemplary embodiment of the present disclosure, in this embodiment, by performing an initial acquisition of target data on an internet channel, and determining a ranking of the internet channel according to the initially acquired target data. As shown in fig. 3, the method may include the steps of:
step 302, initial collection of target data is carried out on each internet channel.
In step 302, the initial collection of the target data may be implemented by, but not limited to, an open data download, an API interface, a web crawler, etc.
And 304, determining the association degree of each Internet channel and the characteristic event according to the target data initially collected from each Internet channel.
The association procedure of the internet channel and the characteristic event can be determined according to keywords contained in the target data, and the user can preset the keywords associated with the characteristic event. If the target data does not contain the content matched with the preset keywords, primarily determining that the Internet channel is irrelevant to the characteristic event; if the target data includes content matched with the preset keywords, the internet channel is initially determined to be associated with the feature event, and the degree of association between the internet channel and the feature event needs to be further quantified, which can be, but is not limited to, implemented by the following ways:
and matching keywords in the target data, scoring the association degree of the Internet channels according to the number of the keywords and the weight of the keywords in the target data, wherein the score is positively correlated with the association degree, the higher the score is, the higher the association degree of the Internet channels and the characteristic events is, and the lower the score is, the lower the association degree of the Internet channels and the characteristic events is.
For ease of understanding, one possible implementation of quantifying the degree of association is described below taking as an example that the feature event is the release of sensitive information containing illegal financial activity content:
the "high interest" and "P2P second mark" can be preset as keywords related to the feature event, and corresponding weights are set for each keyword, wherein the weights of the keywords can be set according to actual requirements. For example, "high interest credits" and "P2P seconds" are associated with a higher degree of unlawful financial activity, and may be set with a higher weight, set to 0.8; the degree of correlation of "high interest" with illegal financial activity is low, and a lower weight can be set to 0.2. Therefore, the relevance degree scoring can be performed on each internet channel according to the number of keywords contained in the target data and the corresponding weight, and a calculation formula of the relevance degree scoring can be represented as follows:
Z＝a×A+b×B+c×C；
wherein Z represents the score of the relevancy score; A. b, C the number of keywords; a. c and b represent weights of keywords.
It will be appreciated that the weights corresponding to the same keyword in different types of feature events may be the same or different. For example, taking the three keywords as examples, in the characteristic event of sensitive information monitoring, the value of a is 0.8, the value of b is 0.2, and the value of c is 0.8; in the characteristic event of negative information monitoring, the value of a is 0.5, the value of b is 0.4, and the value of c is 0.5.
It should be noted that the number of keywords can be set according to the actual requirement, and is not limited to the above-provided 3 keywords, but can be 5 keywords, 6 keywords or even more keywords. Under the condition that the number of the keywords is changed, the corresponding calculation formula needing to modify the relevance scoring is correspondingly modified.
Step 306, determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event.
Wherein, the grade of the internet channel is positively correlated with the degree of association.
In one embodiment, the relevance scoring scores may be divided into different value ranges, each range corresponds to a level, the range with high scoring score corresponds to a higher level, the range with low scoring score corresponds to a lower level, for example, the internet channels with relevance scoring scores in the range of 1-0.7 correspond to important monitoring levels, the internet channels with relevance scoring scores in the range of 0.7-0.5 correspond to medium monitoring levels, the internet channels with relevance scoring scores in the range of 0.5-0.2 correspond to low monitoring levels, and the internet channels with relevance scoring scores in the range of 0.2-0 correspond to very low monitoring levels.
It should be noted that the number of the classification levels of the internet channel is not limited to the above-provided 4 levels, and the number of the levels may be set according to the actual requirements, for example, to 3 levels or 5 levels.
And 308, distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources.
The resource quantity of the acquisition resources allocated to each Internet channel is positively correlated with the grade corresponding to each Internet channel. The higher-grade internet channel description generates more frequent characteristic events, more resource quantity of acquisition resources can be allocated, so that target data of the internet channel can be acquired with higher acquisition frequency (shorter acquisition period) and larger acquisition range, and key monitoring of the internet channel with higher grade is realized. The lower the level of the internet channel description, the less probability of generating a characteristic event, and less acquisition resources can be allocated to acquire target data of the internet channel with a lower acquisition frequency and a smaller acquisition range (for example, only acquiring page data of part of sub-pages).
For example, assuming that the total resource amount of the collection resources is divided into 4 levels, 60% of the total resource amount of the collection resources can be distributed to the internet channels of the key monitoring level for collecting the target data of the internet channels of the key monitoring level, so that the target data is collected once in 1 minute; 30% of the total resource amount of the acquisition resources is distributed to the Internet channel with the medium monitoring level and is used for acquiring target data of the Internet channel with the medium monitoring level, so that the target data is acquired once in 1 day; distributing 8% of the total resource amount of the acquired resources to the Internet channels with medium monitoring level, and acquiring target data of the Internet channels with low monitoring level, so as to acquire the target data once in 1 week; and distributing 2% of the total resource amount of the acquisition resources to the Internet channel with the extremely low monitoring level, and acquiring the target data of the Internet channel with the extremely low monitoring level, so as to acquire the target data once in 1 month. Therefore, the grade matching of the acquisition resources and the Internet channels is realized, and the utility maximization of the acquisition result is realized under the existing acquisition resource consumption.
In another embodiment, the ranking of the internet channels may also be updated using the target data collected for each collection period. Specific:
and re-determining the association degree of each internet channel and the characteristic event according to the target data acquired in each acquisition period, and updating the grade of the internet channel according to the re-determined association degree. The quantization process of the association degree is similar to that of step 304, and will not be described herein. Therefore, the grade of the internet channel is repeatedly adjusted according to the circularly acquired target data, and the accuracy of grade division of the internet is improved.
Step 310, analyzing the collected target data to identify the characteristic event contained in the target data.
In one embodiment, for the target data of the type herein, the target data may be analyzed by, but not limited to, semantic analysis, web phrase translation, variant font recognition, etc., to identify characteristic events contained in the target data; for the video-voice type target data, the voice data can be analyzed by utilizing a voice recognition technology so as to recognize characteristic events contained in the target data; for the target data of the video type, image recognition and classification technologies of multiple context structures and depth feature mining can be utilized, features of different semantic hierarchies are fused, and the image and the video are analyzed by combining with emotion analysis technologies. If the target data contains at least two types of data, the weighted summation can be carried out on the analysis results of the at least two types of data, and finally, the characteristic event contained in the target data is identified.
In another embodiment, a pre-trained event recognition model may also be used for feature event recognition, specifically: inputting the text type target data into an event recognition model, recognizing the characteristic event contained in the Internet channel according to the confidence coefficient output by the event recognition model, and representing the association degree of the Internet channel and the characteristic event by the confidence coefficient.
The event recognition model is obtained by training a Hough Forest (Random Forest) or a yolo neural network or other convolution neural networks by adopting text pairs marked with characteristic event labels. Different models may be trained for different types of feature events, such as feature events containing sensitive information, feature events containing negative information. For an event recognition model for recognizing a characteristic event containing sensitive information, training by using a text marked with a sensitive information label as a training sample; for an event recognition model for recognizing a characteristic event containing negative information, training is carried out by adopting a text marked with the negative information label as a training sample. The specific training process of the model is not described in detail here.
The event recognition model can accurately recognize the characteristic events according to the target data and exclude the invalid characteristic events. By invalid feature event is meant that although the target data contains some keywords related to the feature event, the content belongs to active promotional content and content in public service advertisements, and should not be identified as the feature event.
In another embodiment, if the collected target data includes target data of a picture type, the target data of the picture type is first converted into target data of a text type, and then the target data is input into the event recognition model. Text conversion may be, but is not limited to, employing OCR (optical character recognition) technology.
In another embodiment, before the target data is input into the event detection model, the target data may be further subjected to processing such as cleaning, normalization, rearrangement, and the like, and the target data subjected to data cleaning, data normalization, and data rearrangement is input into the event detection model.
In this embodiment, according to different levels of internet channels, different amounts of acquisition resources are adopted to acquire target data from the internet, the internet channel with higher level has higher association degree with the characteristic event, more resource amounts can be allocated to perform key acquisition monitoring, and the acquisition frequency and range are enlarged, so that the characteristic event of the instant behavior of the website can be captured, for example, a certain page is released in the internet channel for 10 minutes, namely, the web page is put down. The internet channels with lower grades have lower association degree with characteristic events and can be distributed with less resource quantity, so that more useful information can be acquired under the existing resource consumption, and the utility of acquisition results is maximized.
In this embodiment, the collection resources with different resource amounts are allocated to the internet channels with different levels, so that the target data of the internet channels are collected with different collection periods (or collection frequencies) and/or different collection ranges, and the data amount of the target data collected in each time period or each time is not too large, so that the characteristic event can be analyzed and identified for the target data process while the target data is collected, the characteristic event can be found in time, and early warning information can be generated.
Fig. 4 is a flowchart of another method for identifying a characteristic event according to an exemplary embodiment of the present disclosure, which is substantially the same as the method for identifying a characteristic event according to fig. 3, except that the present embodiment further uses the data update frequency of the internet channel and/or the resource consumption amount when collecting the target data as a reference basis when quantifying the association degree. As shown in fig. 3, the method may include the steps of:
step 402, initial collection of target data is carried out on each internet channel.
In step 402, the target data of each internet channel may be collected synchronously or asynchronously. It can be understood that the efficiency of data acquisition in a synchronous acquisition mode is higher, more target data can be obtained in an asynchronous acquisition mode under the condition of limited acquisition resources, and a user can adopt the data acquisition mode according to actual demands.
If the internet channels are acquired by adopting a synchronous acquisition mode, acquisition resources with the same resource amount can be allocated to each internet channel, and initial acquisition of the target data is respectively carried out on each internet channel by using initial acquisition resources with the same resource amount.
Step 404, determining the data update frequency of each internet channel and/or the resource consumption when collecting the target data.
Step 406, determining the association degree of each internet channel and the characteristic event according to the target data and the data updating frequency which are initially acquired from each internet channel and/or the resource consumption when the target data are acquired.
In the process of determining the association degree, association degree scoring can be performed according to the keyword quantity contained in the target data, then weighting and summing are performed on scoring values, data updating frequency and resource consumption, and the association degree of the internet channel and the characteristic event is represented by using the weighted and summed result. Specific implementation of the association score may refer to step 304, which is not described herein.
Step 408, determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event.
And step 410, distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources.
Step 412, analyzing the collected target data to identify the characteristic event contained in the target data.
The specific implementation process of steps 408 to 412 is similar to that of steps 306 to 310, and will not be repeated here.
In another embodiment, the ranking of the Internet channels may also be updated based on the target data collected in step 410. If the target data from each internet channel is periodically collected in step 410, in the process of level update, the relevance degree scoring can be performed according to the keyword amount contained in the target data and the resource consumption amount of each collection period when the collection of the target data is completed in each collection period, the data update frequency of the internet channel in the collection period is determined, the three results are weighted and summed, and the relevance degree of the internet channel and the characteristic event is represented by using the weighted and summed results.
Fig. 5 is a schematic structural view of an apparatus according to an exemplary embodiment of the present specification. Referring to fig. 5, at the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a nonvolatile memory 510, although other hardware may be included as needed for other services. The processor 502 reads the corresponding computer program from the non-volatile memory 510 into the memory 508 and then runs, forming the recognition means of the characteristic event at the logic level. Of course, in addition to software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.
Referring to fig. 6, in a software implementation, the identifying device for a feature event may include:
a determining module 61, configured to determine a level of each internet channel based on a degree of association between the internet channel and the feature event, where the level is positively related to the degree of association;
the distribution module 62 is configured to distribute the acquisition resources to each internet channel, and collect the target data from each internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel;
the identifying module 63 is configured to identify a feature event contained in the collected target data by analyzing the target data.
Optionally, the determining module includes:
the acquisition unit is used for carrying out initial acquisition of target data on each Internet channel by initial acquisition resources with the same resource quantity;
and the determining unit is used for determining the association degree of each internet channel and the characteristic event according to the target data initially collected from each internet channel.
Optionally, the determining unit is configured to:
keyword matching is carried out on the target data;
and scoring the association degree of the Internet channels according to the number of keywords and the weight of the keywords contained in the target data, wherein the score is positively correlated with the association degree.
Optionally, the determining module is further configured to:
determining the data updating frequency of each internet channel and/or the resource consumption when collecting target data;
the degree of association is positively correlated to the page update frequency and/or the resource consumption of the data acquisition.
Optionally, the identification module includes:
the input unit is used for inputting the text type target data into an event recognition model, wherein the event recognition model is obtained by training a neural network through a text marked with a characteristic event label;
and the identification unit is used for identifying the characteristic event contained in the Internet channel according to the confidence coefficient output by the event identification model.
Optionally, the identifying device of the characteristic event further includes:
the conversion module is used for converting the target data of the picture type into the target data of the text type under the condition that the collected target data contains the target data of the picture type, so as to be used for inputting the event recognition model.
Optionally, the distribution module periodically collects target data from each internet channel;
wherein the acquisition period is inversely related to the amount of resources of the acquisition resources allocated to the corresponding internet channel.
Optionally, the identifying device further includes:
and the updating module is used for updating the grade of each internet channel according to the target data acquired by the acquisition resources from the internet channels.
Optionally, the updating module includes:
the determining unit is used for redefining the association degree of the internet channel and the characteristic event according to the target data acquired by the acquisition resources in each acquisition period;
and the adjusting unit is used for adjusting the grade of the Internet channel according to the redetermined association degree.
Optionally, the identifying device further includes:
and the early warning module is used for sending early warning information to a preset warning object under the condition that the target data is identified to contain the characteristic event.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.
In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both permanent and non-permanent, removable and non-removable media, may implement information storage by any method or technology, having stored thereon a computer program (information) that when executed by a processor performs the method steps provided by any of the embodiments described above. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present description to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
The foregoing description of the preferred embodiment(s) is (are) merely intended to illustrate the embodiment(s) of the present invention, and it is not intended to limit the embodiment(s) of the present invention to the particular embodiment(s) described.
Claims (13)
1. A method of identifying a characteristic event, comprising:
respectively determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event, wherein the grade is positively correlated with the association degree;
distributing acquisition resources to each Internet channel, and respectively acquiring target data from each Internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel;
and analyzing the collected target data to identify characteristic events contained in the target data.
2. The method for identifying a feature event according to claim 1, determining a degree of association of an internet channel with the feature event, comprising:
respectively carrying out initial acquisition of target data on each Internet channel by using initial acquisition resources with the same resource quantity;
and determining the association degree of each Internet channel and the characteristic event according to the target data initially collected from each Internet channel.
3. The method for identifying a feature event according to claim 2, wherein determining the degree of association between the internet channel and the feature event according to the initially collected target data comprises:
keyword matching is carried out on the target data;
and scoring the association degree of the Internet channels according to the number of keywords and the weight of the keywords contained in the target data, wherein the score is positively correlated with the association degree.
4. The method for identifying a feature event according to claim 2, wherein the determining the association degree of each internet channel with the feature event comprises:
determining the data updating frequency of each internet channel and/or the resource consumption when collecting target data;
the degree of correlation is positively correlated with the frequency of data update and/or the consumption of resources when the target data is collected.
5. The method for identifying a feature event according to claim 1, wherein the feature event contained in the target data is identified by analyzing the collected target data, comprising:
inputting target data of a text type into an event recognition model, wherein the event recognition model is obtained by training a neural network by adopting a text marked with a characteristic event label;
and identifying the characteristic event contained in the Internet channel according to the confidence coefficient output by the event identification model.
6. The method for identifying a characteristic event according to claim 5, further comprising:
and under the condition that the collected target data comprises target data of a picture type, converting the target data of the picture type into target data of a text type for inputting the event recognition model.
7. The method for identifying characteristic events according to claim 1, wherein the collecting target data from each internet channel through the distributed collection resources comprises:
periodically collecting target data from each internet channel;
wherein the acquisition period is inversely related to the amount of resources of the acquisition resources allocated to the corresponding internet channel.
8. The method of identifying a feature event according to claim 1, the method further comprising:
and updating the grade of each internet channel according to the target data acquired by the acquisition resources from the internet channels.
9. The method for identifying a characteristic event according to claim 8, updating the level of each internet channel, comprising:
the association degree of the internet channel and the characteristic event is redetermined according to the target data acquired by the acquisition resources in each acquisition period;
and adjusting the grade of the Internet channel according to the redetermined association degree.
10. The method of identifying a feature event according to claim 1, the method further comprising:
and under the condition that the target data is identified to contain the characteristic event, sending early warning information to a preset warning object.
11. An identification device for a characteristic event, comprising:
the determining module is used for respectively determining the grade of each internet channel based on the association degree of the internet channel and the characteristic event, and the grade is positively correlated with the association degree;
the distribution module is used for distributing acquisition resources to each Internet channel and respectively acquiring target data from each Internet channel through the distributed acquisition resources; wherein, the resource quantity of the distributed acquisition resources is positively correlated with the grade corresponding to each Internet channel;
and the identification module is used for identifying the characteristic event contained in the target data by analyzing the collected target data.
12. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of identifying a characteristic event according to any of claims 1-10 by executing the executable instructions.
13. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of identifying a characteristic event according to any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010373434.1A CN111552857B (en) | 2020-05-06 | 2020-05-06 | Feature event identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010373434.1A CN111552857B (en) | 2020-05-06 | 2020-05-06 | Feature event identification method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111552857A CN111552857A (en) | 2020-08-18 |
| CN111552857B true CN111552857B (en) | 2023-09-19 |
Family
ID=72004566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010373434.1A Active CN111552857B (en) | 2020-05-06 | 2020-05-06 | Feature event identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111552857B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107463686A (en) * | 2017-08-10 | 2017-12-12 | 深圳市腾讯计算机系统有限公司 | A kind of method and device of calculating network public sentiment temperature |
| CN107766481A (en) * | 2017-10-13 | 2018-03-06 | 国家计算机网络与信息安全管理中心 | A kind of method and system for finding internet financial platform |
| CN109840183A (en) * | 2018-12-05 | 2019-06-04 | 平安科技（深圳）有限公司 | Data center's grading forewarning system method, apparatus and storage medium |
| CN110557608A (en) * | 2019-07-29 | 2019-12-10 | 视联动力信息技术股份有限公司 | resource monitoring method, device and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10135712B2 (en) * | 2016-04-07 | 2018-11-20 | At&T Intellectual Property I, L.P. | Auto-scaling software-defined monitoring platform for software-defined networking service assurance |
| US10153983B2 (en) * | 2016-11-04 | 2018-12-11 | Bank Of America Corporation | Optimum resource routing using contextual data analysis |
-
2020
- 2020-05-06 CN CN202010373434.1A patent/CN111552857B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107463686A (en) * | 2017-08-10 | 2017-12-12 | 深圳市腾讯计算机系统有限公司 | A kind of method and device of calculating network public sentiment temperature |
| CN107766481A (en) * | 2017-10-13 | 2018-03-06 | 国家计算机网络与信息安全管理中心 | A kind of method and system for finding internet financial platform |
| CN109840183A (en) * | 2018-12-05 | 2019-06-04 | 平安科技（深圳）有限公司 | Data center's grading forewarning system method, apparatus and storage medium |
| CN110557608A (en) * | 2019-07-29 | 2019-12-10 | 视联动力信息技术股份有限公司 | resource monitoring method, device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111552857A (en) | 2020-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ma et al. | A new aspect on P2P online lending default prediction using meta-level phone usage data in China | |
| Bansal et al. | On predicting elections with hybrid topic based sentiment analysis of tweets | |
| CN110163647B (en) | Data processing method and device | |
| US12052321B2 (en) | Determining session intent | |
| JP5454357B2 (en) | Information processing apparatus and method, and program | |
| US20130263181A1 (en) | Systems and methods for defining video advertising channels | |
| TW201935292A (en) | Compliant report class sorting method and apparatus | |
| US20100100443A1 (en) | User classification apparatus, advertisement distribution apparatus, user classification method, advertisement distribution method, and program used thereby | |
| CN109977312B (en) | Knowledge base recommendation system based on content tags | |
| WO2011031973A1 (en) | Determining client system attributes | |
| CN111538931A (en) | Big data-based public opinion monitoring method and device, computer equipment and medium | |
| US11886964B2 (en) | Provisioning interactive content based on predicted user-engagement levels | |
| CN106033455B (en) | Method and equipment for processing user operation information | |
| US20220100809A1 (en) | Recommending information to present to users without server-side collection of user data for those users | |
| CN107688533A (en) | Applied program testing method, device, computer equipment and storage medium | |
| CN110858326B (en) | Method, device, equipment and medium for training model and acquiring additional characteristic data | |
| CN110020196B (en) | User analysis method and device based on different data sources and computing equipment | |
| CN110717788A (en) | Target user screening method and device | |
| WO2020204812A1 (en) | Privacy separated credit scoring mechanism | |
| CN110197375A (en) | A kind of similar users recognition methods, device, similar users identification equipment and medium | |
| CN115204881A (en) | Data processing method, device, equipment and storage medium | |
| CN112330426A (en) | Product recommendation method, device and storage medium | |
| CN111552857B (en) | Feature event identification method and device, electronic equipment and storage medium | |
| CN112084406A (en) | Short message processing method and device, electronic equipment and storage medium | |
| CN111078972B (en) | Questioning behavior data acquisition method, questioning behavior data acquisition device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40035837 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240927 Address after: Room 302, 3rd Floor, Building 1, Yard 1, Danling Street, Haidian District, Beijing, 100080 Patentee after: Sasi Digital Technology (Beijing) Co.,Ltd. Country or region after: China Address before: 310000 801-11 section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province Patentee before: Alipay (Hangzhou) Information Technology Co.,Ltd. Country or region before: China |