CN110852085A - Hotspot topic mining method and system - Google Patents
Hotspot topic mining method and system Download PDFInfo
- Publication number
- CN110852085A CN110852085A CN201910762850.8A CN201910762850A CN110852085A CN 110852085 A CN110852085 A CN 110852085A CN 201910762850 A CN201910762850 A CN 201910762850A CN 110852085 A CN110852085 A CN 110852085A
- Authority
- CN
- China
- Prior art keywords
- module
- chinese
- data object
- database
- scoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000005065 mining Methods 0.000 title claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims abstract description 25
- 238000004590 computer program Methods 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 5
- 230000000366 juvenile effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 3
- 241000208125 Nicotiana Species 0.000 description 1
- 235000002637 Nicotiana tabacum Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Abstract
The invention discloses a method and a system for mining a hot topic, and belongs to the technical field of information. The system comprises a theme database module, a keyword database module, a Chinese word segmentation module, a Chinese name word segmentation module, a similarity analysis module and a comprehensive scoring module. The method is that a user of a data object examines the data object, and the system selects a Chinese word segmentation module to perform Chinese word segmentation, a subject database and a keyword database on the content of the data object to obtain a subject and a keyword to which the data object belongs; meanwhile, the similarity analysis module calculates the content similarity between the data objects through the data object database, and returns a combinable data object list after comprehensive scoring. According to the method, the data object with high content relevance is quickly extracted to be referred to the user through judging the content, the theme and the keywords of the data object, and the efficiency and the accuracy are greatly improved.
Description
Technical Field
The invention relates to the technical field of information, in particular to a hot topic mining method and system.
Background
From the statistical report of the development conditions of the Chinese Internet in the information center of the Chinese Internet, 6 months in 2014, the number of Chinese netizens reaches 6.32 hundred million, the popularity of the Internet reaches 46.9 percent, and the popularity of the Internet is improved by 1.1 percent compared with that of the Internet at the end of 2013. The net citizen proportion of the mobile phone on the internet accounts for 83.4%, and is increased by 2.4% compared with the net citizen proportion at the end of 2013, and the net citizen proportion exceeds the proportion of the traditional PC on the internet for the first time (80.9%). The popularity of the internet shows a rapid rising trend, especially the development of the mobile internet drives the development of the whole internet, and the netizens have comprehensively networked lives.
The continuous expansion of the internet scale is accompanied by the explosive growth of network information resources, but in the face of information resources of the internet in a huge amount like the tobacco, how to effectively and quickly obtain contents required by the user or interested by the user, so that the massive network information resources on the internet are really fully utilized and information consumption is promoted, which is still a difficult problem to be solved urgently. By analyzing the internet in depth to search for related technologies, a problem was found: for a server for pushing information to a user, the obtained data is messy, the data information is redundant, the storage space of a database is occupied, the content depth of a data object is insufficient, and the viscosity between the data object and the user cannot be deeply established. Due to the repeated occurrence of a large amount of similar information, the time for a user to acquire required accurate information is too long, and the experience psychology of the user is further influenced.
Disclosure of Invention
1. Technical problem to be solved by the invention
In order to overcome the technical problem, the invention provides a hotspot topic mining method and system. By judging the content, the theme and the keywords of the data object, the data object with high content relevance is quickly extracted to be referred to a user, and the efficiency and the accuracy are greatly improved.
2. Technical scheme
In order to solve the problems, the technical scheme provided by the invention is as follows:
a hot topic mining system comprises a topic database module, a keyword database module, a Chinese word segmentation module, a Chinese name word segmentation module, a similarity analysis module and a comprehensive scoring module, wherein the topic database module is used for dividing a data object into a certain topic; the keyword database module is used for extracting keywords related to the data objects; the Chinese word segmentation module is used for extracting Chinese names in the data objects; the Chinese noun scoring module is used for calculating the weight of each Chinese name in the data object; the similarity analysis module is used for calculating the similarity between the data objects; and the comprehensive scoring module is used for carrying out overall scoring on the data objects.
Preferably, the theme database module divides the data objects into a certain theme by the accumulated themes.
Preferably, the subject matter is juvenile child protection; science and technology, economy and agriculture.
Preferably, the keyword database module stores the latest national issued regional database in storage, including all provinces, cities and counties, villages, towns and streets in storage, collects national garden information and building names, and uses the information and the names as the keyword database to extract keywords related to the data objects.
Preferably, the Chinese word segmentation module adopts a Chinese word segmentation IKAnalyzer, and extracts the Chinese name in the data object by matching with the collected Chinese word stock.
Preferably, the Chinese noun scoring module calculates the weight of each word in the data object through a TF-IDF weight scoring algorithm, and screens out the keywords in the Chinese noun scoring module.
Preferably, the Chinese lexicon is an information lexicon.
Preferably, the similarity analysis module calculates the similarity between the data objects using a cosine algorithm.
Preferably, the comprehensive scoring module synthesizes content similarity between the data objects, and takes keywords and topics as parameters to perform overall scoring.
A hotspot topic mining method comprises the following steps: selecting a data object to be transacted by a user, submitting the data object to the hotspot topic mining system for analysis, starting to inspect the data object, selecting a Chinese word segmentation module to perform Chinese word segmentation on the content of the data object, ranking all Chinese words after scoring by using the Chinese word segmentation module, selecting words with high weight as a keyword list, and obtaining a topic and a keyword which the data object belongs to according to the keyword list, a topic database and a keyword database; meanwhile, the similarity analysis module calculates the content similarity among the data objects through the data object database, and returns a combinable data object list for the user to refer after comprehensively scoring the keyword, the theme and the content similarity.
An apparatus, the apparatus comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the methods described above.
A storage medium storing a computer program which, when executed by a processor, implements the method described above.
3. Advantageous effects
By adopting the technical scheme provided by the invention, the data object with higher content relevance is quickly extracted for the reference of the user through judging the content, the theme and the keywords of the data object, and the efficiency and the accuracy are greatly improved.
Drawings
Fig. 1 is a block diagram of a hotspot topic mining system provided by the present invention.
Fig. 2 is a schematic flow chart of a hot topic mining method provided by the present invention.
Fig. 3 is a schematic flow diagram of a preferred embodiment of fig. 2.
Fig. 4 is a schematic diagram of a mechanism of an apparatus provided by the present invention.
Detailed Description
For a further understanding of the present invention, reference is made to the following detailed description taken in conjunction with the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Example 1
A hot topic mining system comprises a topic database module, a keyword database module, a Chinese word segmentation module, a Chinese name word segmentation module, a similarity analysis module and a comprehensive scoring module, wherein the topic database module is used for dividing a data object into a certain topic; the keyword database module is used for extracting keywords related to the data objects; the Chinese word segmentation module is used for extracting Chinese names in the data objects; the Chinese noun scoring module is used for calculating the weight of each Chinese name in the data object; the similarity analysis module is used for calculating the similarity between the data objects; and the comprehensive scoring module is used for carrying out overall scoring on the data objects.
And the theme database module divides the data objects into a certain theme through the accumulated theme. The theme is juvenile child protection. The keyword database module stores the latest national issued regional database in storage, including all provinces, cities and counties in the country, villages, towns and streets in storage, collects national garden information and building names at the same time, and uniformly serves as a keyword database for extracting keywords related to data objects. The Chinese word segmentation module adopts Chinese word segmentation IKAnalyzer and matches with a collected Chinese word stock to extract Chinese names in the data objects, and the Chinese word stock is an information word stock.
The Chinese noun scoring module calculates the weight of each word in the data object through a TF-IDF weight scoring algorithm, and screens out the keywords in the Chinese noun scoring module. The similarity analysis module calculates the similarity between the data objects by using a cosine algorithm. The comprehensive scoring module synthesizes the content similarity between the data objects, and takes keywords and topics as parameters to carry out overall scoring.
Example 2
The traditional data object merging mode is manually processed, similar data objects are manually selected, problems (subjects) and areas related to the contents of the similar data objects are analyzed, whether the similar data objects belong to the data objects of the same subject is judged, and whether merging is considered. Therefore, the method is time-consuming, labor-consuming, low in efficiency and high in misjudgment rate.
As shown in fig. 2 and 3, a method for mining a hot topic includes the steps of: selecting a data object to be transacted by a user of a data object (referring to data information to be mined), submitting the data object to be transacted to a hotspot topic mining system for analysis in embodiment 1, starting to examine the data object, selecting a Chinese word segmentation module to perform Chinese word segmentation on the content of the data object according to an information word bank, ranking after scoring all Chinese words by using a Chinese word scoring module (namely a weight scoring module shown in figure 1), selecting a word with high weight as a keyword list, and correspondingly and respectively obtaining a topic and a keyword to which the data object belongs according to the keyword list, the topic database and the keyword database; meanwhile, the similarity analysis module (i.e. the similar document analysis module shown in fig. 1) calculates the content similarity between the data objects by using a cosine algorithm through the data object database to obtain similar data objects, and returns a combinable data object list for the user to refer after comprehensively scoring the keyword, the topic and the content similarity.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, for example, each of the described units may be a software program provided in a computer or a mobile intelligent device, or may be a separately configured hardware device. Wherein the designation of a unit or module does not in some way constitute a limitation of the unit or module itself.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the present application. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Example 3
An apparatus, the apparatus comprising: one or more processors; memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to perform a method as described above.
A storage medium storing a computer program which, when executed by a processor, implements the method as described in the above embodiments.
Fig. 4 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
As shown in fig. 4, as another aspect, the present application also provides an apparatus 500 including one or more Central Processing Units (CPUs) 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the apparatus 500 are also stored. The CPU501, ROM502, and RAM503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to embodiments disclosed herein, the method described in any of the above embodiments may be implemented as a computer software program. For example, embodiments disclosed herein include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method described in any of the embodiments above. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511.
As yet another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus of the above-described embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described herein.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, for example, each of the described units may be a software program provided in a computer or a mobile intelligent device, or may be a separately configured hardware device. Wherein the designation of a unit or module does not in some way constitute a limitation of the unit or module itself.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the present application. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A hot topic mining system is characterized by comprising a topic database module, a keyword database module, a Chinese word segmentation module, a Chinese name word segmentation module, a similarity analysis module and a comprehensive grading module, wherein the topic database module is used for dividing a data object into a certain topic; the keyword database module is used for extracting keywords related to the data objects; the Chinese word segmentation module is used for extracting Chinese names in the data objects; the Chinese noun scoring module is used for calculating the weight of each Chinese name in the data object; the similarity analysis module is used for calculating the similarity between the data objects; and the comprehensive scoring module is used for carrying out overall scoring on the data objects.
2. The system of claim 1, wherein the topic database module divides the data objects into topics by accumulated topics.
3. The system of claim 2, wherein the topic is juvenile child protection; science and technology, economy and agriculture.
4. The system of claim 1, wherein the keyword database module is configured to store a national latest region database, which includes all provinces, cities and counties in the country, and information about villages, towns and streets, and collect national region information and building names, and unify the information as a keyword database for extracting keywords related to the data objects.
5. The system of claim 1, wherein the chinese participle module extracts chinese names in the data object by using a chinese participle IKAnalyzer in combination with a collected chinese lexicon.
6. The system of claim 1, wherein the Chinese noun scoring module is configured to select the keywords in the Chinese noun scoring module by calculating the weight of each term in the data object through a TF-IDF weight scoring algorithm; the comprehensive scoring module synthesizes the content similarity between the data objects, and takes keywords and topics as parameters to carry out overall scoring.
7. The system of claim 5, wherein the chinese lexicon is an information lexicon; the similarity analysis module calculates the similarity between the data objects by using a cosine algorithm.
8. A hotspot topic mining method is characterized by comprising the following steps: selecting a data object to be transacted by a user, submitting the data object to a hotspot topic mining system for analysis as claimed in claim 1, starting to inspect the data object, selecting a Chinese word segmentation module to perform Chinese word segmentation on the content of the data object, ranking all Chinese words after scoring by using the Chinese word segmentation module, selecting a word with high weight as a keyword list, and obtaining a topic and a keyword to which the data object belongs according to the keyword list, a topic database and a keyword database; meanwhile, the similarity analysis module calculates the content similarity among the data objects through the data object database, and returns a combinable data object list for the user to refer after comprehensively scoring the keyword, the theme and the content similarity.
9. An apparatus, characterized in that the apparatus comprises:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method recited in claim 8.
10. A storage medium storing a computer program, characterized in that the program, when executed by a processor, implements the method as claimed in claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910762850.8A CN110852085A (en) | 2019-08-19 | 2019-08-19 | Hotspot topic mining method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910762850.8A CN110852085A (en) | 2019-08-19 | 2019-08-19 | Hotspot topic mining method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110852085A true CN110852085A (en) | 2020-02-28 |
Family
ID=69594846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910762850.8A Pending CN110852085A (en) | 2019-08-19 | 2019-08-19 | Hotspot topic mining method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852085A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326355A (en) * | 2021-07-29 | 2021-08-31 | 湖南正宇软件技术开发有限公司 | Proposal scoring method, device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731946A (en) * | 2015-04-01 | 2015-06-24 | 成都金税电子技术有限公司 | Internet information data mining method |
| CN107766449A (en) * | 2017-09-26 | 2018-03-06 | 杭州云赢网络科技有限公司 | Focus method for digging, device, electronic equipment and storage medium |
| CN108334591A (en) * | 2018-01-30 | 2018-07-27 | 天津中科智能识别产业技术研究院有限公司 | Industry analysis method and system based on focused crawler technology |
| CN108763213A (en) * | 2018-05-25 | 2018-11-06 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Theme feature text key word extracting method |
-
2019
- 2019-08-19 CN CN201910762850.8A patent/CN110852085A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104731946A (en) * | 2015-04-01 | 2015-06-24 | 成都金税电子技术有限公司 | Internet information data mining method |
| CN107766449A (en) * | 2017-09-26 | 2018-03-06 | 杭州云赢网络科技有限公司 | Focus method for digging, device, electronic equipment and storage medium |
| CN108334591A (en) * | 2018-01-30 | 2018-07-27 | 天津中科智能识别产业技术研究院有限公司 | Industry analysis method and system based on focused crawler technology |
| CN108763213A (en) * | 2018-05-25 | 2018-11-06 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Theme feature text key word extracting method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326355A (en) * | 2021-07-29 | 2021-08-31 | 湖南正宇软件技术开发有限公司 | Proposal scoring method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106649818B (en) | Application search intention identification method and device, application search method and server | |
| CN106383887B (en) | Method and system for collecting, recommending and displaying environment-friendly news data | |
| WO2017020451A1 (en) | Information push method and device | |
| CN108572990B (en) | Information pushing method and device | |
| CN106844640B (en) | Webpage data analysis processing method | |
| CN104598532A (en) | Information processing method and device | |
| CN107544988B (en) | Method and device for acquiring public opinion data | |
| CN110232126B (en) | Hot spot mining method, server and computer readable storage medium | |
| CN103577478A (en) | Web page pushing method and system | |
| Wu et al. | Extracting topics based on Word2Vec and improved Jaccard similarity coefficient | |
| US20150294005A1 (en) | Method and device for acquiring information | |
| Alves et al. | A spatial and temporal sentiment analysis approach applied to Twitter microtexts | |
| CN110287314A (en) | Long text credibility evaluation method and system based on Unsupervised clustering | |
| CN114330329A (en) | Service content searching method and device, electronic equipment and storage medium | |
| CN110851562A (en) | Information acquisition method, system, equipment and storage medium | |
| CN112149422A (en) | Enterprise news dynamic monitoring method based on natural language | |
| US20170235835A1 (en) | Information identification and extraction | |
| CN111930949B (en) | Search string processing method and device, computer readable medium and electronic equipment | |
| US10853429B2 (en) | Identifying domain-specific accounts | |
| CN110852085A (en) | Hotspot topic mining method and system | |
| US20150193444A1 (en) | System and method to determine social relevance of Internet content | |
| CN116226494B (en) | Crawler system and method for information search | |
| CN110580279A (en) | Information classification method, system, equipment and storage medium | |
| CN111538903A (en) | Method and device for determining search recommended word, electronic equipment and computer readable medium | |
| CN112183093A (en) | Enterprise public opinion analysis method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200228 |