CN116186149A - Target depth mining method and system based on self-defined association relation - Google Patents
Target depth mining method and system based on self-defined association relation Download PDFInfo
- Publication number
- CN116186149A CN116186149A CN202310227963.4A CN202310227963A CN116186149A CN 116186149 A CN116186149 A CN 116186149A CN 202310227963 A CN202310227963 A CN 202310227963A CN 116186149 A CN116186149 A CN 116186149A
- Authority
- CN
- China
- Prior art keywords
- target
- index
- mining
- relation
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Game Theory and Decision Science (AREA)
- Economics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of data mining, and provides a target depth mining method and system based on a self-defined association relation, wherein the method comprises the following steps: obtaining mining target information; performing general target detection on mass data existing in a target network through big data to obtain a general target detection result; according to the general target detection result and the mining target information, carrying out associated index parameter analysis to determine a self-defined association relation; performing key network detection on the general target detection result to obtain the actual flow of the key network; performing target flow detection analysis on the actual flow of the key network by using the self-defined association relationship to obtain a high-value flow clue; and (5) according to the high-value flow clues, mining target information, and re-identifying the target relevance to obtain a target clue identification result. The method can solve the technical problem of low target data acquisition rate and accuracy caused by overlarge data volume in the target network.
Description
Technical Field
The application relates to the technical field of data mining, in particular to a target depth mining method and system based on a self-defined association relation.
Background
Data mining, which is a hotspot problem in current artificial intelligence and database research, refers to a non-trivial process of revealing implicit, previously unknown and potentially valuable information from a vast amount of data in a database. Data mining is a decision support process, and is mainly based on artificial intelligence, machine learning, databases and the like, and is highly automated to analyze data, make inductive reasoning and mine valuable data from the data.
With the advent of the big data age, data is expanding and becoming larger rapidly, and how to use massive data in a network for data mining and application is a key problem. Because the amount of data existing in the target network is too large, the traditional target data acquisition mode cannot meet the requirements of people.
In summary, the technical problem in the prior art is that the rate and accuracy of target data acquisition are low due to the excessive data volume in the target network.
Disclosure of Invention
Based on this, it is necessary to provide a target depth mining method and system based on a custom association relationship in order to solve the above technical problems.
A target depth mining method based on a self-defined association relation is applied to a target depth mining system and comprises the steps of obtaining mining target information; according to the mining target information, performing universal target detection on mass data existing in a target network through big data to obtain a universal target detection result; carrying out associated index parameter analysis according to the general target detection result and the mining target information to obtain a related index set, and determining a self-defined association relation according to the related index set; performing key network detection on the general target detection result to obtain the actual flow of a key network; performing target flow detection analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue; and according to the high-value flow clues and the mining target information, performing target relevance re-identification to obtain target clue identification results.
In one embodiment, further comprising: performing index confidence analysis on the relevant index set and the mining target information, and determining each index confidence; according to the confidence coefficient of each index, index screening is carried out to determine a target index; according to the target index, carrying out index relation analysis to obtain a target index relation; and setting a self-defined relation according to the target index relation to obtain the self-defined association relation.
In one embodiment, further comprising: setting a custom rule list, wherein the custom rule list comprises an index occupation ratio and an index association degree; determining the number of indexes and the cross-correlation degree of each index according to the target index relation; determining an index combination relation according to the index number, the target index relation and the index occupation ratio; according to the cross association degree of each index, carrying out combination association degree calculation on the index combination relation to determine the index combination association degree; and matching the index combination association degree based on the index association degree in the custom rule list to obtain the custom association relation.
In one embodiment, further comprising: constructing an adaptability function for mining target information and a related index set based on the target index relation; determining the index requirements of each level of relation according to the index quantity and the index occupation ratio; adding the relation index requirements of each level as constraint conditions into the fitness function; and carrying out iterative optimization by using a global optimization algorithm based on the fitness function to obtain index combination relations corresponding to each level of relation.
In one embodiment, further comprising: obtaining a multi-layer association relationship according to the target association relationship; respectively carrying out target flow detection analysis on the actual flow of the key network by utilizing the multi-layer association relationship to obtain a flow clue set; constructing a multi-level clue map according to the multi-level association relation and the flow clue set; determining each level of mark value based on the multi-level cue map, analyzing the overlapping rate of each cue, determining the overlapping rate, and determining the overlapping mark value based on the overlapping rate; and screening high-value flow clues according to the multi-level mark value, the overlapped mark value and the preset weight value to obtain the high-value flow clues.
In one embodiment, further comprising: inputting the high-value flow clues into a feature recognition model, and carrying out feature recognition extraction to obtain clue feature sets; obtaining core features according to the semantic feature influence of the cue feature set and the high-value flow cues; performing influence analysis by using the core features and the mining target information to obtain clue relevance; screening all core features according to a preset cue number threshold based on the cue relevance to obtain target mining features; and carrying out fusion analysis according to the target mining characteristics to obtain the target clue identification result.
A target depth mining system based on a custom association, comprising:
the mining target information acquisition module is used for acquiring mining target information;
the universal target detection module is used for carrying out universal target detection on mass data existing in a target network through big data according to the mining target information to obtain a universal target detection result;
the self-defined incidence relation determining module is used for carrying out incidence index parameter analysis according to the generic object detection result and the mining object information to obtain a relevant index set, and determining the self-defined incidence relation according to the relevant index set;
the key network detection module is used for carrying out key network detection on the general target detection result to obtain the actual flow of the key network;
the target flow detection and analysis module is used for carrying out target flow detection and analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue;
and the target cue identification result obtaining module is used for carrying out target relevance re-identification according to the high-value flow cue and the mining target information to obtain a target cue identification result.
The target depth mining method and system based on the self-defined association relation can solve the technical problem that the target data acquisition rate and accuracy are low due to the fact that the data volume in the target network is too large. And obtaining a generic object detection result through big data detection based on the mining object information, and further determining a self-defined association relation. And performing target flow detection analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue. And finally, target relevance re-identification is carried out according to the high-value flow clues and the mining target information, and a target clue identification result is obtained. The method can automatically realize the relevance retrieval of mass data in the target network, and improve the speed and accuracy of obtaining the target flow clues.
The foregoing description is only an overview of the technical solutions of the present application, and may be implemented according to the content of the specification in order to make the technical means of the present application more clearly understood, and in order to make the above-mentioned and other objects, features and advantages of the present application more clearly understood, the following detailed description of the present application will be given.
Drawings
Fig. 1 is a schematic flow chart of a target depth mining method based on a user-defined association relation;
FIG. 2 is a schematic flow chart of determining a custom association in a target depth mining method based on the custom association;
FIG. 3 is a schematic flow chart of obtaining high-value flow clues in a target depth mining method based on a self-defined association relation;
fig. 4 is a schematic structural diagram of a target depth mining system based on a custom association relationship.
Reference numerals illustrate: the system comprises an excavation target information obtaining module 1, a general target detection module 2, a self-defined association relation determining module 3, a key network detecting module 4, a target flow detecting and analyzing module 5 and a target clue recognition result obtaining module 6.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
As shown in fig. 1, the present application provides a target depth mining method based on a custom association relationship, where the method is applied to a target depth mining system, and the method includes:
step S100: obtaining mining target information;
step S200: according to the mining target information, performing universal target detection on mass data existing in a target network through big data to obtain a universal target detection result;
specifically, mining target information including information of a target data range, a target data type, a target data feature, and the like is obtained. And determining a target network range according to the mining target information, and detecting and acquiring data related to the mining target information by mass data existing in a target network through a big data technology to obtain a general target detection result, wherein the general target detection result refers to the data related to the mining target information. And by obtaining the generic object detection result, data support is provided for next step data correlation analysis.
Step S300: carrying out associated index parameter analysis according to the general target detection result and the mining target information to obtain a related index set, and determining a self-defined association relation according to the related index set;
as shown in fig. 2, in one embodiment, step S300 of the present application further includes:
step S310: performing index confidence analysis on the relevant index set and the mining target information, and determining each index confidence;
step S320: according to the confidence coefficient of each index, index screening is carried out to determine a target index;
step S330: according to the target index, carrying out index relation analysis to obtain a target index relation;
step S340: and setting a self-defined relation according to the target index relation to obtain the self-defined association relation.
Specifically, performing associated index parameter analysis on the generic object detection result and the mining object information, wherein the associated index parameter analysis refers to mining index parameters with association between the generic object detection result and the mining object information to obtain a related index set, and the related index set refers to a set of the associated index parameters. And then, carrying out index confidence analysis according to the related index set and the mining target information, wherein the confidence refers to the probability of the confidence between the related index set and the mining target information, for example, AB is used for representing two commodities, the number of times of AB simultaneous occurrence is divided by the number of times of A simultaneous occurrence, namely, the probability that a customer buys B again on the premise of buying A, the probability is also called as the confidence, the meaning of the formula indicates how much confidence exists for the behavior that the customer buys A simultaneously and buys B, how much confidence holds, and the commodity combination is considered to be relevant, so that each index confidence is obtained. The method comprises the steps of presetting an index confidence coefficient threshold, wherein the index confidence coefficient threshold can be set in a self-defined mode according to the accuracy requirement of target data searching, conducting index screening on the related index set according to the index confidence coefficient threshold, and determining a target index, wherein the target index is an index with confidence coefficient higher than the confidence coefficient threshold. And then carrying out association analysis based on the related index set and the mining target information according to the target index to obtain a target index relation. And then, according to the target index relationship, carrying out custom relationship setting to obtain the custom association relationship. For example: when the customer satisfaction is mined by data, indexes such as a customer shopping list, a customer shopping price, a customer refund type and number, a customer complaint type and number and the like are included in the target index relation, and when the custom relation is set as the customer satisfaction, the customer refund type and number and the customer complaint type and number are the custom association relation. And carrying out confidence analysis on the related index set and the mining target information, and screening the confidence of each index, so as to obtain the self-defined association relationship, thereby improving the accuracy of obtaining the target flow clues.
In one embodiment, step S340 of the present application further includes:
step S341: setting a custom rule list, wherein the custom rule list comprises an index occupation ratio and an index association degree;
step S342: determining the number of indexes and the cross-correlation degree of each index according to the target index relation;
step S343: determining an index combination relation according to the index number, the target index relation and the index occupation ratio;
in one embodiment, step S343 of the present application further comprises:
step S3431: constructing an adaptability function for mining target information and a related index set based on the target index relation;
step S3432: determining the index requirements of each level of relation according to the index quantity and the index occupation ratio;
step S3433: adding the relation index requirements of each level as constraint conditions into the fitness function;
step S3434: and carrying out iterative optimization by using a global optimization algorithm based on the fitness function to obtain index combination relations corresponding to each level of relation.
Step S344: according to the cross association degree of each index, carrying out combination association degree calculation on the index combination relation to determine the index combination association degree;
step S345: and matching the index combination association degree based on the index association degree in the custom rule list to obtain the custom association relation.
Specifically, a custom rule list is set, wherein the custom rule list comprises an index occupation ratio and an index association degree. The index occupation ratio refers to the index occupation ratio condition of the index in the custom relation, for example, 100 indexes are in total in the custom rule list, wherein 78 indexes are associated with the custom relation, and the index occupation ratio is 78%. The index association degree refers to the degree of closeness of the index and the custom relationship. And then determining the number of indexes and the cross-correlation degree of each index according to the target index relation, wherein the number of indexes refers to the number of indexes contained in the target index relation. The cross-correlation degree refers to the degree of mutual correlation among the indexes. And constructing an adaptability function for mining target information and related index sets based on the genetic algorithm based on the target index relation. The fitness function is used for measuring the fitness of the index individuals in the related index set. And then determining index relationships of a plurality of levels according to the index number and the index occupation ratio. The index relationship level is represented by the product of the index number multiplied by the index ratio, wherein the larger the product is, the higher the level of the index relationship is. For example: the index number is 10, the primary index relationship is 2, the secondary index relationship is 5, the tertiary index relationship is 7, the ratio is 20% -50% of the primary index relationship, 50% -70% of the secondary index relationship, and 70% -100% of the tertiary index relationship. And then adding the index relation requirements of each level as constraint conditions of individuals in a genetic algorithm into the fitness function, performing iterative optimization by using a global optimization algorithm, firstly selecting a group, wherein the group refers to the related index set, then randomly selecting a plurality of individuals from the group to form a set, wherein the individuals are a plurality of indexes in the related index set, selecting next generation individuals by taking fitness as a selection principle, randomly selecting the same positions of two individuals for the next generation individuals, exchanging according to the crossover probability, and then mutating a certain position of the individuals according to the principle of gene mutation by using mutation probability. When the fitness of the optimal individual reaches a given threshold or the fitness of the optimal individual and the fitness of the group are not increased any more, the algorithm is ended, and the index combination relation corresponding to each level of relation is obtained. And then, carrying out combination relevance calculation on the index combination relation according to the index cross relevance, namely adding the sum of the index cross relevance to obtain the index combination relevance. And finally, inputting the combined association degree into the index association degree in the custom rule list for matching, namely matching the corresponding grades in the custom rule list according to the combined association degree to obtain the custom association relation, wherein the custom association relation comprises a plurality of custom grades and corresponding index combinations. The self-defined association relation is obtained through a genetic algorithm, so that the optimal index combination in the self-defined range can be obtained, and the accuracy of obtaining the target flow clues is further improved.
Step S400: performing key network detection on the general target detection result to obtain the actual flow of a key network;
step S500: performing target flow detection analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue;
as shown in fig. 3, in one embodiment, step S500 of the present application further includes:
step S510: obtaining a multi-layer association relationship according to the target association relationship;
step S520: respectively carrying out target flow detection analysis on the actual flow of the key network by utilizing the multi-layer association relationship to obtain a flow clue set;
step S530: constructing a multi-level clue map according to the multi-level association relation and the flow clue set;
step S540: determining each level of mark value based on the multi-level cue map, analyzing the overlapping rate of each cue, determining the overlapping rate, and determining the overlapping mark value based on the overlapping rate;
step S550: and screening high-value flow clues according to the multi-level mark value, the overlapped mark value and the preset weight value to obtain the high-value flow clues.
Specifically, source analysis is performed on the generic object detection result to obtain a plurality of network source types, namely the key network, and then data acquisition is performed on the key network to obtain the actual flow of the key network. And obtaining a multi-layer association relationship according to the custom association relationship, wherein the multi-layer association relationship corresponds to a plurality of custom levels. And then screening the actual flow of the key network according to the multi-layer association relation to obtain a flow clue set. And then constructing a multi-level cue map according to the multi-level association relations and the flow cue set corresponding to each level of association relations. And determining all levels of marking values according to the multi-level cue patterns, wherein the marking values of all levels refer to index parameters corresponding to the non-level cue patterns. And carrying out overlap ratio analysis on each cue according to the marking value, namely judging the overlap number of index parameters in each cue, determining the overlap ratio according to the overlap number, and carrying out overlap marking according to the overlap ratio. And finally, carrying out high-value flow clue screening according to the multi-level mark value, the overlapped mark value and a preset weight, wherein the preset weight refers to a range obtained by setting the flow clue, and the range can be represented by number or time range to obtain the high-value flow clue. By obtaining the high value traffic cue, data support is provided for obtaining the target cue in the next step.
Step S600: and according to the high-value flow clues and the mining target information, performing target relevance re-identification to obtain target clue identification results.
In one embodiment, step S600 of the present application further includes:
step S610: inputting the high-value flow clues into a feature recognition model, and carrying out feature recognition extraction to obtain clue feature sets;
step S620: obtaining core features according to the semantic feature influence of the cue feature set and the high-value flow cues;
step S630: performing influence analysis by using the core features and the mining target information to obtain clue relevance;
step S640: screening the core features according to a preset cue number threshold based on the cue relevance to obtain target mining features; and carrying out fusion analysis according to the target mining characteristics to obtain the target clue identification result.
Specifically, a feature recognition model is constructed according to the mining target information, wherein the feature recognition model is a neural network model which can be subjected to continuous iterative optimization in machine learning, and is obtained through monitoring training through a training data set. Inputting the high-value flow clues into a feature recognition model to perform feature recognition extraction to obtain clue feature sets; and analyzing the semantic feature influence of the high-value flow clues according to the clue feature set to obtain core features with higher importance, and performing influence analysis on the core features and the mining target information, wherein the influence analysis refers to judging the degree of association between the core features and the mining target information to obtain clue association. Presetting a clue quantity threshold, wherein the clue quantity threshold can be set in a self-defined mode according to the data quantity, screening the core features according to the clue quantity threshold to obtain screened core features, namely target mining features, and finally carrying out data mining on the high-value flow clues according to the target mining features to obtain the target clue identification result. The method can solve the technical problem of low target data acquisition speed and accuracy caused by overlarge data volume in the target network, automatically realize relevance retrieval of mass data in the target network, and improve the target flow clue acquisition speed and accuracy.
In one embodiment, as shown in fig. 4, a target depth mining system based on a custom association relationship is provided, including: the system comprises an excavation target information obtaining module 1, a general target detection module 2, a self-defined association relation determining module 3, a key network detecting module 4, a target flow detecting and analyzing module 5, a target clue identification result obtaining module 6, wherein:
the mining target information acquisition module 1 is used for acquiring mining target information;
the universal target detection module 2 is used for detecting the universal target of mass data existing in a target network through big data according to the mining target information to obtain a universal target detection result;
the self-defined incidence relation determining module 3 is used for carrying out incidence index parameter analysis according to the generic object detection result and the mining object information to obtain a relevant index set, and determining a self-defined incidence relation according to the relevant index set;
the key network detection module 4 is used for performing key network detection on the general target detection result to obtain the actual flow of the key network;
the target flow detection and analysis module 5 is used for carrying out target flow detection and analysis on the actual flow of the key network by utilizing the self-defined association relationship, so as to obtain a high-value flow clue;
and the target cue identification result obtaining module 6 is used for carrying out target relevance re-identification according to the high-value flow cue and the mining target information to obtain a target cue identification result.
In one embodiment, the system further comprises:
the index confidence determining module is used for carrying out index confidence analysis on the relevant index set and the mining target information and determining each index confidence;
the target index determining module is used for screening indexes according to the confidence coefficient of each index to determine target indexes;
the target index relation obtaining module is used for carrying out index relation analysis according to the target index to obtain a target index relation;
the self-defined association relation obtaining module is used for carrying out self-defined relation setting according to the target index relation to obtain the self-defined association relation.
In one embodiment, the system further comprises:
the rule list setting module is used for setting a custom rule list, wherein the custom rule list comprises an index occupation ratio and an index association degree;
the index information determining module is used for determining the number of indexes and the cross-correlation degree of each index according to the target index relation;
the index combination relation determining module is used for determining an index combination relation according to the index number, the target index relation and the index occupation ratio;
the index combination association degree determining module is used for calculating the combination association degree of the index combination relation according to the cross association degree of each index and determining the index combination association degree;
the custom incidence relation obtaining module is used for matching the index combination incidence degrees based on the index incidence degrees in the custom rule list to obtain the custom incidence relation.
In one embodiment, the system further comprises:
the fitness function construction module is used for constructing a fitness function for mining target information and related index sets based on the target index relation;
the relation index requirement determining module is used for determining relation index requirements of all levels according to the index quantity and the index occupation ratio;
the constraint condition adding module is used for adding the relation index requirements of each level as constraint conditions into the fitness function;
the index combination relation obtaining module is used for carrying out iterative optimization by utilizing a global optimization algorithm based on the fitness function to obtain the index combination relation corresponding to each level of relation.
In one embodiment, the system further comprises:
the multi-layer association relation obtaining module is used for obtaining a multi-layer association relation according to the target association relation;
the flow clue set obtaining module is used for respectively carrying out target flow detection analysis on the actual flow of the key network by utilizing the multi-layer association relation to obtain a flow clue set;
the multi-level cue pattern construction module is used for constructing a multi-level cue pattern according to the multi-level association relation and the flow cue set;
the overlapping mark value determining module is used for determining each level of mark value based on the multi-level cue map, analyzing the overlapping rate of each cue, determining the overlapping rate and determining the overlapping mark value based on the overlapping rate;
and the high-value flow clue obtaining module is used for screening the high-value flow clues according to the multi-level mark value, the overlapped mark value and the preset weight value to obtain the high-value flow clues.
In one embodiment, the system further comprises:
the clue feature set obtaining module is used for inputting the clue of the high-value flow into a feature recognition model, carrying out feature recognition extraction and obtaining a clue feature set;
the core feature acquisition module is used for acquiring core features according to the semantic feature influence of the cue feature set and the high-value flow cues;
and a thread relevance obtaining module. The clue relevance obtaining module is used for carrying out influence analysis on the core characteristics and the mining target information to obtain clue relevance;
the target clue identification result obtaining module is used for screening all core features according to a preset clue quantity threshold value based on clue relevance to obtain target mining features; and carrying out fusion analysis according to the target mining characteristics to obtain the target clue identification result.
In summary, the application provides a target depth mining method and system based on a self-defined association relation, which have the following technical effects:
1. the method solves the technical problem of low target data acquisition speed and accuracy caused by overlarge data volume in the target network, can automatically realize relevance retrieval of mass data in the target network, and improves the target flow clue acquisition speed and accuracy.
2. And carrying out confidence analysis on the related index set and the mining target information, screening the confidence coefficient of each index, and obtaining the self-defined association relationship by utilizing a genetic algorithm, so that the optimal index combination in the self-defined range can be obtained, and the accuracy of obtaining the target flow clues is further improved.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (7)
1. The target depth mining method based on the self-defined association relation is characterized by being applied to a target depth mining system, and comprises the following steps:
obtaining mining target information;
according to the mining target information, performing universal target detection on mass data existing in a target network through big data to obtain a universal target detection result;
carrying out associated index parameter analysis according to the general target detection result and the mining target information to obtain a related index set, and determining a self-defined association relation according to the related index set;
performing key network detection on the general target detection result to obtain the actual flow of a key network;
performing target flow detection analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue;
and according to the high-value flow clues and the mining target information, performing target relevance re-identification to obtain target clue identification results.
2. The method of claim 1, wherein determining a custom association from the set of correlation metrics comprises:
performing index confidence analysis on the relevant index set and the mining target information, and determining each index confidence;
according to the confidence coefficient of each index, index screening is carried out to determine a target index;
according to the target index, carrying out index relation analysis to obtain a target index relation;
and setting a self-defined relation according to the target index relation to obtain the self-defined association relation.
3. The method of claim 2, wherein performing a custom relationship setting according to the target index relationship to obtain the custom association relationship comprises:
setting a custom rule list, wherein the custom rule list comprises an index occupation ratio and an index association degree;
determining the number of indexes and the cross-correlation degree of each index according to the target index relation;
determining an index combination relation according to the index number, the target index relation and the index occupation ratio;
according to the cross association degree of each index, carrying out combination association degree calculation on the index combination relation to determine the index combination association degree;
and matching the index combination association degree based on the index association degree in the custom rule list to obtain the custom association relation.
4. The method of claim 3, wherein determining an index combination relationship based on the index number, a target index relationship, and the index occupancy ratio comprises:
constructing an adaptability function for mining target information and a related index set based on the target index relation;
determining the index requirements of each level of relation according to the index quantity and the index occupation ratio;
adding the relation index requirements of each level as constraint conditions into the fitness function;
and carrying out iterative optimization by using a global optimization algorithm based on the fitness function to obtain index combination relations corresponding to each level of relation.
5. The method of claim 1, wherein performing target flow detection analysis on the actual flow of the key network by using the target association relationship to obtain a high-value flow clue, comprises:
obtaining a multi-layer association relationship according to the target association relationship;
respectively carrying out target flow detection analysis on the actual flow of the key network by utilizing the multi-layer association relationship to obtain a flow clue set;
constructing a multi-level clue map according to the multi-level association relation and the flow clue set;
determining each level of mark value based on the multi-level cue map, analyzing the overlapping rate of each cue, determining the overlapping rate, and determining the overlapping mark value based on the overlapping rate;
and screening high-value flow clues according to the multi-level mark value, the overlapped mark value and the preset weight value to obtain the high-value flow clues.
6. The method of claim 1, wherein performing target relevance re-recognition based on the high-value traffic clues and the mined target information to obtain target clue recognition results comprises:
inputting the high-value flow clues into a feature recognition model, and carrying out feature recognition extraction to obtain clue feature sets;
obtaining core features according to the semantic feature influence of the cue feature set and the high-value flow cues;
performing influence analysis by using the core features and the mining target information to obtain clue relevance;
screening all core features according to a preset cue number threshold based on the cue relevance to obtain target mining features; and carrying out fusion analysis according to the target mining characteristics to obtain the target clue identification result.
7. A target depth mining system based on a custom association relationship, the system comprising:
the mining target information acquisition module is used for acquiring mining target information;
the universal target detection module is used for carrying out universal target detection on mass data existing in a target network through big data according to the mining target information to obtain a universal target detection result;
the self-defined incidence relation determining module is used for carrying out incidence index parameter analysis according to the generic object detection result and the mining object information to obtain a relevant index set, and determining the self-defined incidence relation according to the relevant index set;
the key network detection module is used for carrying out key network detection on the general target detection result to obtain the actual flow of the key network;
the target flow detection and analysis module is used for carrying out target flow detection and analysis on the actual flow of the key network by utilizing the self-defined association relationship to obtain a high-value flow clue;
and the target cue identification result obtaining module is used for carrying out target relevance re-identification according to the high-value flow cue and the mining target information to obtain a target cue identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310227963.4A CN116186149A (en) | 2023-03-03 | 2023-03-03 | Target depth mining method and system based on self-defined association relation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310227963.4A CN116186149A (en) | 2023-03-03 | 2023-03-03 | Target depth mining method and system based on self-defined association relation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116186149A true CN116186149A (en) | 2023-05-30 |
Family
ID=86436510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310227963.4A Pending CN116186149A (en) | 2023-03-03 | 2023-03-03 | Target depth mining method and system based on self-defined association relation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116186149A (en) |
-
2023
- 2023-03-03 CN CN202310227963.4A patent/CN116186149A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111882446B (en) | Abnormal account detection method based on graph convolution network | |
| CN110245981B (en) | Crowd type identification method based on mobile phone signaling data | |
| Wu et al. | A data mining approach for spatial modeling in small area load forecast | |
| CN110674840B (en) | Multi-party evidence association model construction method and evidence chain extraction method and device | |
| CN108985380B (en) | Point switch fault identification method based on cluster integration | |
| CN111178611B (en) | Method for predicting daily electric quantity | |
| CN110929939B (en) | Landslide hazard susceptibility spatial prediction method based on clustering-information coupling model | |
| CN110287292B (en) | Judgment criminal measuring deviation degree prediction method and device | |
| CN104573130A (en) | Entity resolution method based on group calculation and entity resolution device based on group calculation | |
| WO2022237213A1 (en) | Resident travel chain model construction method and resident travel chain acquisition method | |
| CN112149922A (en) | Method for predicting severity of accident in exit and entrance area of down-link of highway tunnel | |
| CN107679550A (en) | A kind of appraisal procedure of data set classification availability | |
| CN115794803B (en) | Engineering audit problem monitoring method and system based on big data AI technology | |
| CN107016416A (en) | The data classification Forecasting Methodology merged based on neighborhood rough set and PCA | |
| CN113569946A (en) | Open source map and professional data source road network adaptive matching method | |
| CN112330441A (en) | Method for evaluating business value credit loan of medium and small enterprises | |
| CN112560915A (en) | Urban expressway traffic state identification method based on machine learning | |
| CN113688870B (en) | Group renting room identification method based on user electricity behavior by adopting hybrid algorithm | |
| Afandizadeh Zargari et al. | A Structural Comparison between the Origin‐Destination Matrices Based on Local Windows with Socioeconomic, Land‐Use, and Population Characteristics | |
| CN116484266B (en) | Fine urban land type recognition model training method | |
| CN117540303A (en) | Landslide susceptibility assessment method and system based on cross semi-supervised machine learning algorithm | |
| CN115481841A (en) | Material demand prediction method based on feature extraction and improved random forest | |
| CN117291085A (en) | Expressway asphalt pavement maintenance scheme decision method based on Bayes multi-modal neural network | |
| CN116186149A (en) | Target depth mining method and system based on self-defined association relation | |
| CN113268936B (en) | Key quality characteristic identification method based on multi-objective evolution random forest characteristic selection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |