KR20190040046A - Information collection system, information collection method and recording medium - Google Patents

Abstract

An object of the present invention is to efficiently collect interest information from a large amount of various types of contents. The information collection system 100 includes a learning unit 110 and an extraction unit 130. The learning unit 110 uses the learning data to generate parser rules in which strings to be analyzed, i.e., strings containing specific information are extracted from the data. The extracting unit 130 extracts the strings to be analyzed from the data using parser rules.

Description

Information collection system, information collection method and recording medium

The present invention relates to an information collection system, an information collection method, and a recording medium.

The necessary pieces of information are collected from web contents provided in various services on the Internet, including social networking services (SNS), blogs, homepages, etc., and analysis and control of the collected information is performed in various ways. Because these Web contents are vast, it is required to collect information by crawling based on the program and to automate analysis and control of collected information.

Web content is typically described by HyperText Markup Language (HTML), and the locations where the necessary information to be collected on the content (e.g., the user's message part in the SNS) is described, They are different. Also, in many cases, these locations are not disclosed as definite specifications by service providers. Therefore, in order to collect information from web contents, a developer needs to develop a parser program for analyzing contents for each service and extracting necessary information. In this case, the parser program needs to be changed every time the specification of the contents is changed, and the programming cost is also increased. As described above, there is a problem in that, in information collection from web contents, it is not possible to collect desired information efficiently from such a large amount of contents in various formats.

As a related art related to information collection from Web contents, for example, Patent Document 1 discloses a technique for classifying micro blog messages according to classification rules. Patent Document 2 discloses a technique in which a user's interests and preferences are learned from a document created by a user, and the learned interests and preferences are used for recommendation of other users and blogs and for transmission of advertisements. Patent Document 3 discloses a technique for extracting an article in accordance with an extraction condition from articles such as news and updating the extraction condition according to the amount of comments to the article.

Japanese translation of PCT International Patent Application Publication No. 2012-529717 Japanese Patent Application Laid-Open Publication No. 2009-098964 Japanese Patent Application Laid-Open Publication No. 2014-049094

However, in any of the above prior art documents, a technique for efficiently collecting information of interest from various types of data of all the vast amounts is not disclosed.

An exemplary object of the present invention is to provide an information collecting system, an information collecting method, and a recording medium that can solve the above-mentioned problems and efficiently collect desired information from a large amount of various types of data .

An information collection system according to an exemplary aspect of the present invention includes learning means for generating a parser rule for extracting a target character string, which is a character string including specific information, from data using learning data; And extraction means for extracting the target character string from the data using the parser rule.

An information collecting method according to an exemplary aspect of the present invention includes: generating a parser rule for extracting a target character string, which is a character string including specific information, from data using learning data; And extracting the target character string from the data using the parser rule.

According to an exemplary aspect of the present invention, there is provided a computer-readable medium having embodied thereon a program for causing a computer to execute: generating a parser rule for extracting a target character string, which is a character string containing specific information, from data using learning data; And extracting the target character string from the data using the parser rule.

The effect of the present invention is to efficiently collect desired information from a large amount of various types of contents.

1 is a block diagram showing a characteristic configuration of the first exemplary embodiment.
2 is a block diagram showing a configuration according to the first exemplary embodiment.
3 is a block diagram illustrating the configuration of an information gathering system 100 implemented by a computer in a first exemplary embodiment.
Fig. 4 is a flowchart showing a learning process in the first exemplary embodiment. Fig.
5 is a diagram showing a concrete example of the learning process in the first exemplary embodiment.
6 is a diagram showing examples of parser rules stored in the parser rule storage unit 120 in the first exemplary embodiment.
7 is a flowchart showing an extraction process in the first exemplary embodiment.
8 is a diagram showing a specific example of extraction processing in the first exemplary embodiment.
9 is a diagram showing examples of tables stored in the character string storage unit 140 in the first exemplary embodiment.
10 is a diagram showing an example of a processing rule input screen in the first exemplary embodiment.
11 is a flow chart showing the analysis and control processing in the first exemplary embodiment.
12 is a diagram showing an example of analysis and control processing in the first exemplary embodiment.
13 is a diagram showing an example of an analysis result screen in the first exemplary embodiment.
14 is a diagram showing an example of an alarm mail in the first exemplary embodiment.
15 is a block diagram showing another configuration according to the first exemplary embodiment.
16 is a block diagram showing another configuration of the first exemplary embodiment.
17 is a diagram showing an example of processing rules in the second exemplary embodiment.
18 is a diagram showing an example of a table stored in the character string storage unit 140 in the second exemplary embodiment.
19 is a diagram showing an example of the analysis and control processing in the second exemplary embodiment.
20 is a diagram showing a specific example of learning processing in the second exemplary embodiment.

[EXEMPLARY EMBODIMENT]

Exemplary embodiments of the present invention are described in detail with reference to the drawings. In the exemplary embodiments described in the drawings and specification, it is noted that like elements are denoted by like reference numerals, and the description thereof is appropriately omitted.

(First Exemplary Embodiment)

First, a configuration according to the first exemplary embodiment will be described. 2 is a block diagram showing a configuration according to the first exemplary embodiment. The information collection system 100 is connected to the server apparatus 200 and the terminal apparatus 300 via a network 400 such as an internetwork. The server device 200 is a server that provides content data (hereinafter also simply referred to as data) in a service such as SNS. The data is described using a predetermined data description language, e.g., HTML.

The information collection system 100 extracts a " target string ", which is a string subjected to various analysis and control processing, from data acquired from the server device 200. [ The target string is a character string including " specific information " such as, for example, a message issued by the user in the SNS and a user name of the user, Here, the data format defining the storage method of the message, the user name, the transmission date and the like in the data differs depending on the operator providing the services. The information collection system 100 extracts a target character string using a parser rule corresponding to each of the services. The information collection system 100 performs analysis and control processing based on the extracted target character string.

2, the information collecting system 100 according to the first exemplary embodiment includes: a learning unit 110; a parser rule storage unit 120; an extraction unit 130; a character string storage unit 140; (150); A processing rule input unit 160; And a processing rule storage unit 170.

The learning unit 110 acquires data to be learned (hereinafter also referred to as learning data) from the server apparatus 200, and generates a parser rule using the learning data. The parser rule is a rule for extracting the above-described target character string from the data. The learning unit 110 generates a parser rule using one or more pieces of learning data for each service, for example. The service is identified by a domain name included in, for example, a uniform resource locator (URL).

The parser rule is specified by, for example, a description pattern of a data description language. The learning unit 110 generates, as a parser rule, the description pattern of the data description language-the description pattern is a description pattern that can specify a character string including a keyword specified with the learning data in the learning data. For example, when data is described by HTML, the learning unit 110 generates, as a parser rule, a pattern of tags (surrounding a target character string) capable of specifying a target character string.

The parser rule storage unit 120 stores a parser rule generated by the learning unit 110 in association with a service.

The extracting unit 130 acquires data (hereinafter, referred to as processing data or simply data) to be extracted from the server 200 from the server 200 and uses the parser rules associated with the data- Extract the target string from the data.

The character string storage unit 140 stores the target character string extracted by the extraction unit 130. [ The target string is stored in a table format such as a KVS (Key-Value Store).

The processing unit 150 performs analysis and control processing based on the extracted target character strings in accordance with a processing rule specified by an administrator or the like. The processing rules define the methods of analysis and control.

The processing rule input unit 160 accepts input of a processing rule from an administrator or the like.

The processing rule storage unit 170 stores one or more processing rules input from an administrator or the like.

The information collecting system 100 may be a computer including a central processing unit (CPU) and a storage medium storing a program, and operated by control based on a program.

3 is a block diagram illustrating the configuration of an information gathering system 100 implemented by a computer in a first exemplary embodiment.

In this case, the information collection system 100 includes a CPU 101; A storage device 102 (storage medium) such as a hard disk and a memory; An input / output device 103 including a keyboard, a display, and the like; And other devices and the like. The CPU 101 executes a program for implementing the learning unit 110, the extraction unit 130, the processing unit 150 and the processing rule input unit 160. [ The storage device 102 stores information in the parser rule storage unit 120, the character string storage unit 140, and the processing rule storage unit 170. [ The input / output device 103 accepts designation of learning data, processing data, and processing rules from an administrator or the like. In addition, the communication device 104 can accept designation of learning data, processing data, and processing rules from other devices.

In addition, some or all of the components of the information collection system 100 may be implemented by a general purpose or dedicated circuit, a processor, and a combination thereof. These circuits and processors can be constituted by a single chip and can be constituted by a plurality of chips connected to each other via a bus. In addition, some or all of the components of the information collecting system 100 may be implemented by a combination of the above-described circuit and the like and the program.

When some or all of the components of the information collection system 100 are implemented by a plurality of information processing apparatuses, circuits, etc., the plurality of information processing apparatuses, circuits, and the like may be concentratedly arranged or distributed . For example, information processing devices, circuits, and the like may be implemented as a system in which each component is connected to each other via a communication network, such as a client and server system, a cloud computing system, and the like.

Next, the operation of the first exemplary embodiment will be described.

Here, in the learning process, a parser rule for extracting a target character string from data of the SNS is generated. In the analysis and control processing, a predetermined keyword included in the target character string is analyzed and a notice about the analysis result is issued As an example.

<Learning process>

Fig. 4 is a flowchart showing a learning process in the first exemplary embodiment. Fig. 5 is a diagram showing a concrete example of the learning process in the first exemplary embodiment.

First, the learning unit 110 accepts designation of learning data from an administrator or the like (step S101).

For example, as shown in FIG. 5, the learning unit 110 assigns three pieces of learning data pieces to the URLs " URL 1 ", " URL 2 ", and " URL 3 ". Key words "Keyword 1", "Keyword 2", and "Keyword 3" are specified along with each learning data piece. For each of the keywords, a character string of the user's message (hereinafter also referred to as a user message) in the corresponding learning data piece is designated. In addition, the table name " SNS 1 " and the data label " Post " used in the extraction process to be described later are specified along with these learning data pieces. The table name is the name of the table that stores the target strings extracted from the process data. A data label is the name of a column that stores user messages in a table.

The learning unit 110 acquires the designated learning data from the server apparatus 200 (step S102).

For example, as shown in FIG. 5, the learning unit 110 acquires a learning data piece designated by URLs "URL 1", "URL 2", and "URL 3" from the server apparatus 200 do.

The learning unit 110 generates a parser rule using the acquired learning data (step S103).

For example, the learning unit 110 generates tag patterns &quot; <a> <b Class = "post" id = xx> <c name = "userx"> Keywordx </ c> </ b> </a> ". Then, as shown in Fig. 5, the learning unit 110 compares the attributes of the tags and the tags between the tag patterns identified for each piece of learning data, <C> Value </ c> </ b> </a> "by extracting a property having a common value. Here, Value indicates a target character string to be extracted.

As shown in Fig. 5, the learning unit 110 also stores the table name " SNS 1 ", the data label " Post ", the URL " http://hoge.com " b Class = "post"> <c> Value </ c> </ b> </a>. Here, the URL &quot; http://hoge.com &quot; is a URL corresponding to the service &quot; SNS 1 ", and is commonly included in the URLs " URL 1 ", " URL 2 & Domain name.

The learning unit 110 also performs learning processing on user names (data label " User "), and transmission dates and times of messages (data label " Date ") in the same manner in addition to user messages. Note that, in the learning data of Fig. 5, user names are designated by the attribute " name " of the tag " c ". In the learning data of Fig. 5, the transmission dates and times are omitted.

The learning unit 110 stores the generated parser rule in the parser rule storage unit 120 (step S104).

6 is a diagram showing examples of parser rules stored in the parser rule storage unit 120 in the first exemplary embodiment.

6, for example, the learning unit 110 generates a user message (data label "Post") of the service "SNS 1", a user name (data label "User" In the label " Date "), the generated parser rules are stored in the parser rule storage unit 120. [

The learning unit 110 also performs the learning process for the other services " SNS 2 " and " SNS 3 ". As a result, the parser rule storage unit 120 stores parser rules for the user message, the user name, and the transmission date and time of each service, as shown in Fig.

The learning unit 110 may further generate a script, a program module, or a source code thereof to extract a target character string from the data in accordance with a parser rule, and may generate the generated script, program module, or source code into a parser rule storage unit 120 ). &Lt; / RTI &gt;

<Extraction Treatment>

7 is a flowchart showing an extraction process in the first exemplary embodiment. 8 is a diagram showing a specific example of extraction processing in the first exemplary embodiment. The extraction process is executed after the parser rule is generated by the learning process.

First, the extracting unit 130 accepts designation of data from an administrator or the like (step S201).

For example, as shown in FIG. 8, the extracting unit 130 accepts the designation of data for the service "SNS 1" by URL "http://hoge.com/index.html".

The extracting unit 130 acquires the designated data from the server apparatus 200 (step S202).

For example, as shown in Fig. 8, the extracting unit 130 acquires the data designated by the URL " http://hoge.com/index.html " from the server apparatus 200. [

The extracting unit 130 acquires the parser rule associated with the data service from the parser rule storage unit 120 (step S203).

For example, as shown in Fig. 8, the extracting unit 130 extracts from the parser rules of Fig. 6 the URL " http://hoge.com " obtained from the domain name of the URL specifying the data, and <C> Value </ c> </ b> </a> associated with the data label "Post".

The extracting unit 130 extracts the target character string from the data using the acquired parser rule (step S204).

For example, as shown in FIG. 8, the extracting unit 130 extracts, from the data, the tag pattern "<a> <b Class =" post "> <c> Value </ c> </ b &gt; &quot; &gt; ", as target strings. Note that the extracting unit 130 can also extract, as object IDs, attribute values of a predetermined tag for identifying a set of related target strings together with each of the target strings. In the example of Fig. 8, it is assumed that the set of the user message, user name, and transmission date and time is identifiable by the attribute " id " of the tag " b ". In this case, the extracting unit 130 extracts the values of the attribute " id " of the tag " b &quot; together with the user messages as the object ID, as shown in Fig.

In addition to user messages, the extracting unit 130 also extracts target strings in the same manner for user names (data label "User") and transmission dates and times of messages (data label "Date").

The extracting unit 130 stores the extracted target character strings in the character string storage unit 140 (step S205).

9 is a diagram showing an example of tables stored in the character string storage unit 140 in the first exemplary embodiment.

9, the extracting unit 130 extracts user messages (data label " Post "), user names (data label " User " (Data label " Date "), and stores this table in the character string storage unit 140. [ Here, the extracting unit 130 sets a user message, a user name, and a transmission date / time with the same Object ID in each column of the table.

The extracting unit 130 also performs extraction processing for other services "SNS 2" and "SNS 3" in the same manner. As a result, the strings "SNS 1", "SNS 2", and "SNS 3" are stored in the character string storage unit 140 as shown in FIG.

<Analysis and control processing>

10 is a diagram showing an example of a processing rule input screen in the first exemplary embodiment. In the input screen of Fig. 10, the parts for constituting the processing rule and the editing area of the processing rule are displayed. The manager or the like edits processing rules by, for example, selecting parts from the displayed parts, placing the selected parts in the editing area, and setting the parameters. In the example of FIG. 10, the processing rule "Rule 1" is input. When the number of target strings including keywords "earthquake" and "shake" in the tables "SNS 1", "SNS 2" and "SNS 3" as the processing rule "Rule 1" Quot; disaster " to the target strings, and an alarm mail is transmitted.

Here, the processing rule input unit 160 receives input of the processing rule "Rule 1" as shown in FIG. 10 from the manager or the like in advance through the input / output device 103 and receives the accepted processing rule "Rule 1 Quot; is stored in the processing rule storage unit 170. [

The processing rule input unit 160 may further generate a script, a program module, or a source code thereof for performing the analysis and control processing in accordance with the processing rule, and transmit the generated script, program module, or source code to a processing rule storage unit 170). &Lt; / RTI &gt;

11 is a flow chart showing the analysis and control processing in the first exemplary embodiment. 12 is a diagram showing an example of analysis and control processing in the first exemplary embodiment. The analysis and control processing is executed after the object strings are extracted by the extraction processing.

Initially, the processing unit 150 accepts, from an administrator or the like, the conditions for the target character strings subjected to the analysis and control processing and the specification of the processing rule to be used (step S301).

13 is a diagram showing an example of an analysis result screen in the first exemplary embodiment.

The processing unit 150 outputs an analysis result screen as shown in FIG. 13, for example, to an administrator through the input / output device 103, and sets a condition (transmission date " date & 15 ") and the processing rule " Rule 1 ".

The processing unit 150 acquires, from the character string storage unit 140, target strings that satisfy the specified condition (step S302).

For example, the processing unit 150 acquires from the tables " SNS 1 ", " SNS 2 ", and " SNS 3 " in FIG. 9 the condition of acquisition date " date> = 2015/12/15 " Get target strings.

The processing unit 150 acquires the specified processing rule from the processing rule storage unit 170 (step S303).

For example, the processing unit 150 acquires the processing rule "Rule 1" as shown in FIG.

The processing unit 150 performs analysis and control processing based on the obtained target character strings in accordance with the acquired processing rule (step S304).

For example, in accordance with the processing rule " Rule 1 " in FIG. 10, the processing unit 150 sets the number of target strings including the keyword " earthquake & Count as in 12. As a result of the counting, if the number is greater than or equal to 20, the processing unit 150 sets the type (data label " Type ") to target strings including the keyword " As a "disaster".

As shown in FIG. 13, the processing unit 150 sets target strings to which the type "disaster" is assigned to the analysis result of the analysis result screen, and outputs the analysis result screen to the manager or the like. The processing unit 150 can also update the tables stored in the character string storage unit 140 for the target character strings to which the type "disaster" is assigned.

14 is a diagram showing an example of an alarm mail in the first exemplary embodiment.

The processing unit 150 also transmits an alert mail such as that shown in Fig. 14 indicating occurrence of a disaster (earthquake) to the terminal device 300 registered as a notification receiver in advance via the network 400. Fig. Here, the processing unit 150 sets the contents of the mail according to, for example, a processing rule. In this case, the processing unit 150 can acquire and set contents of mail from other devices specified by the processing rule.

As described above, even when user messages and the like in data of SNSs are provided by various types of data for each service, messages can be efficiently collected and analyzed.

Thus, the operations of the first exemplary embodiment are completed.

In the above-described embodiment, the case where data is described by HTML has been described as an example. However, as long as specific information can be extracted as target strings by analyzing a technique by data description languages, &Lt; / RTI &gt; For example, the data may be described by other markup languages such as Extensible Markup Language (XML), and may be written in languages other than markup languages including JAVA (registered trademark) Script Object Notation &Lt; / RTI &gt;

Further, in the above-described exemplary embodiment, the extracting unit 130 acquires, as processing data, data represented by URLs specified by an administrator or the like. However, the present invention is not limited to this, and the extracting unit 130 can acquire data by crawling through the network 400. [ In this case, the extracting unit 130 can acquire the data by passing a predetermined number of links from the data indicated by the URL of a starting point, and use the acquired data. Also, the extracting unit 130 can acquire the data provided by the predetermined number of URLs acquired by the search result of the search engine, and use the acquired data.

In addition, the extracting unit 130 can acquire, as processing data, data transmitted and received via the network 400 using a network device such as a proxy server.

In the above-described exemplary embodiment, analysis and control processing are executed by the processing unit 150 based on a plurality of target characters extracted by the extraction unit 130. [ However, the present invention is not limited to this, and the processing unit 150 can execute analysis and control processing based on the extracted target character each time the extraction unit 130 acquires the processing data and extracts the target character string.

Further, in the above-described exemplary embodiment, in the learning process, parser rules are generated for extracting user messages, user names, transmission date and time, etc. from the data of the SNSs as target strings. Further, in the analysis and control processing, the number of predetermined keywords included in the target strings is calculated, and a notification is issued about the calculation result.

However, the present invention is not limited to this, and the information extracted as the target character string by the parser rule may be arbitrary information as long as the information is acquired by analyzing the description in the data description language. In addition, the analysis and control processing may be any processing as long as the analysis and control processing is performed based on the extracted target character string.

For example, as a target string, a blacklist may be extracted from data to provide a list of IP (internet protocol) addresses or domain names (i.e., blacklists) with high operability and vulnerability. Further, as the target string, a malware list can be extracted from data for providing file names of malware and URLs (i.e., a malware list). In this case, in the analysis and control processing, a policy for blocking communication related to the black list or the malware list can be set.

15 is a block diagram showing another configuration according to the first exemplary embodiment. The information collecting system 100 in FIG. 15 includes the same components as those of the information collecting system 100 in FIG.

15, the server device 200 is a server that provides the above-described black list or malware list. The information collection system 100 is connected to an intrusion prevention system (IPS) The IPS device 500 controls the communication between the network 400 and the terminal device 601 connected to the local network 600 according to a policy.

For example, the learning unit 110 in FIG. 15 generates a parser rule using the learning data acquired from the server apparatus 200 and the black list or the malware list included in the learning data designated as the keyword.

The extraction unit 130 extracts a target character string (black list or malware list) from the data acquired from the server apparatus 200 by using the parser rule.

The processing unit 150 extracts, from the target character string, IP addresses or domain names included in the black list, or file names or URLs included in the malware list, according to the processing rule. Thereafter, the processing unit 150 sets a policy for interrupting the communication with the extracted IP addresses or domain names, or the communication of the messages including the extracted file names and URLs, to the IPS device 500.

In this way, even if the black list or the malware list is provided by various types of data for each service, the black list or the malware list can be efficiently collected according to the services, .

Also, as a target string, for example, a user comment on a blog page can be extracted. In this case, in the analysis and control processing, an inappropriate expression included in the user comment can be detected, the writing of the user comment can be blocked, or a log of the user comment can be obtained.

16 is a block diagram showing another configuration of the first exemplary embodiment. The information collecting system 100 in FIG. 16 includes the same components as those of the information collecting system 100 in FIG.

In Fig. 16, the server device 200 is a server that provides blogs. The information collection system 100 is connected to the proxy server 700. The proxy server 700 controls communication between the network 400 and the terminal device 801 connected to the local network 800. [

For example, the learning unit 110 shown in Fig. 16 is configured to include learning data including user comments to blogs, which are transmitted from the terminal device 801 to the server apparatus 200, Using the included user comments, a parser rule is generated.

The extraction unit 130 acquires data transmitted from the terminal device 801 to the server device 200 via the proxy server 700. [ The extracting unit 130 extracts target strings (user comments) from the acquired data using the parser rule.

The processing unit 150 extracts an inappropriate expression, such as a representation corresponding to a malignant middle ear, from the target strings in accordance with the processing rule. Thereafter, the processing unit 150 controls the proxy server 700 so as to block transmission of the extracted data to the server apparatus 200. In addition, the processing unit 150 can additionally store in the log storage unit (not shown) a log indicating the IP address or the like of the terminal device 801, which is the transmission source of the data in which the inappropriate representation is detected.

In this way, even if the writing to the SNSs and blogs is performed by various types of contents for each of the services, it is possible to efficiently collect the entries written according to the services, and to detect the improperly written entries .

Also, news strings, weather forecasts, program information, and the like can be extracted from the content data of news sites as target strings, for example. Also, as the target strings, analysis results and the like can be extracted from content data of sites that analyze and provide information including networks of people on the Internet, interests of people, and the like.

Further, for example, in the analysis and control processing, the file name of the malware can be detected from the messages of the SNS acquired as the target strings, and the malware can be analyzed in other analyzers and the like. Further, in the analysis and control processing, it is possible to extract networks and interests of a specific person, or extract persons with a particular interest, from analysis results of networks of people and interest of people acquired as target strings.

Next, the characteristic configuration of the first exemplary embodiment will be described. 1 is a block diagram showing a characteristic configuration of the first exemplary embodiment.

Referring to FIG. 1, the information collection system 100 includes a learning unit 110 and an extraction unit 130. The learning unit 110 generates a parser rule for extracting a target character string, which is a character string including specific information, from the data using the learning data. The extracting unit 130 extracts the target character string from the data using the parser rule.

The beneficial effects of the first exemplary embodiment will now be described.

According to the first exemplary embodiment, target information can be efficiently collected from a large amount of various types of contents. The reason for the above is that the learning unit 110 generates a parser rule for extracting a target character string which is a character string including specific information by using the learning data and the extracting unit 130 extracts the parser rule To extract the target string from the data.

In this way, a parser program for extracting necessary information from various types of contents can be created by the developer in a short time without analyzing the contents. Also, even if the specifications of the contents are changed, the reprogramming of the parser program can be executed at low cost. In addition, the notation of the parser program is integrated for various types of content, thus improving its readability and reusability.

Further, according to the first exemplary embodiment, it is possible to efficiently perform various analyzes and controls based on information extracted from a large amount of various types of contents. The reason for the above is that the processing section 150 performs the analysis and control processing based on the target strings extracted by the extraction section 130 according to the processing rule.

Further, according to the first exemplary embodiment, various methods of analysis and controls can be efficiently executed. The reason for the above is that the processing rule storage unit 170 stores one or more processing rules and the processing unit 150 performs analysis and control processing in accordance with processing rules specified by an administrator or the like.

(Second Exemplary Embodiment)

Next, a second exemplary embodiment will be described.

The second exemplary embodiment differs from the first exemplary embodiment in that the processing unit 150 determines new learning data based on the extracted target strings.

The processing unit 150 determines new learning data based on the extracted target characters in accordance with the processing rule. Here, when the target character strings include a specific keyword and URL, the processing unit 150 determines data indicated by the URL as new learning data.

The learning unit 110 generates a parser rule using the new learning data.

17 is a diagram showing an example of processing rules in the second exemplary embodiment. In the example of Fig. 17, the processing rule " Rule 2 " is defined. In the processing rule " Rule 2 ", when a plurality of target strings including a specific keyword " earthquake " and a common domain name URL are detected in the table " SNS 1 & The processing to specify is described. The new learning data is specified by the detected URL. In addition, the keyword " earthquake " is designated as a string to be extracted.

Here, the processing rule input unit 160 receives the input of the processing rule " Rule 2 " as shown in Fig. 17 from the manager or the like in advance and sends the received processing rule " Rule 2 " to the processing rule storage unit 170 And preserve it.

18 is a diagram showing an example of a table stored in the character string storage unit 140 in the second exemplary embodiment.

6 are stored in the parser rule storage unit 120 in the same manner as in the first exemplary embodiment and the character string storage unit 140 stores the parser rules as shown in FIG. It is assumed that the table " SNS 1 " is stored.

19 is a diagram showing an example of the analysis and control processing in the second exemplary embodiment.

The processing unit 150 searches the target character string in the table of Fig. 18 for the target character string including the URL of the keyword " earthquake " and the common domain name " news.com " Are detected as shown in Fig. The processing unit 150 stores the URLs "http://news.com/news1.html" and "http://news.com/news2.html" of the detected target strings as URLs representing new learning data, And inputs it to the learning unit 110 together with " earthquake ".

20 is a diagram showing a specific example of learning processing in the second exemplary embodiment.

The learning unit 110 acquires the learning data designated by each of the URLs, and generates a parser rule. Hereinafter, when the learning unit 110 includes the keyword " earthquake " specified by the learning data, the learning unit 110 specifies a tag pattern for extracting a character string including the keyword, and, as shown in FIG. 20, "<x> <y> Value </ y> </ x>". The learning unit 110 associates the URL "http://news.com" and the new parser rule "<x> <y> Value </ y> </ x>" with each other as shown in FIG. 20, In the parser rule storage unit 120. [

Thereafter, extraction of the target character strings by the extraction unit 130 using the new parser rule, determination of new learning data by the processing unit 150, and generation of a new parser rule by the learning unit 110 are repeated .

In this manner, parser rules for extracting target characters from various services, in addition to the service initially designated as learning data, are efficiently generated with respect to information related to a specific keyword, without specifying the learning data by the administrator.

Next, the beneficial effects of the second exemplary embodiment of the present invention will be described.

In accordance with the second exemplary embodiment of the present invention, in addition to the beneficial effects of the first exemplary embodiment, a parser rule for extracting target strings from various services can also be efficiently generated. The reason for the above is that the processing unit 150 determines new learning data based on the extracted target strings in accordance with the processing rule, and the learning unit 110 uses the determined new learning data, Because it generates rules.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, the present invention is not limited to these embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

This application is based on and claims the benefit of priority from Japanese Patent Application No. 2016-187064, filed on September 26, 2016, the disclosure of which is incorporated herein by reference in its entirety.

[Reference List]
100 information collection system
101 CPU
102 storage device
103 Input / output devices
104 communication device
110 Department of Education
120 parser rule storage
130 extraction unit
140 String storage unit
150 processor
160 Processing rule input unit
170 processing rule storage unit
200 server device
300 terminal device
400 network
500 IPS devices
600 local network
601 terminal device
700 Proxy Server
800 local network
801 terminal device

Claims (9)

As an information collection system,
Learning means for generating a parser rule for extracting a target character string from data using training data, the target character string being a character string including specific information; And
And extracting means for extracting the target character string from the data using the parser rule. The method according to claim 1,
And processing means for performing at least one of predetermined analysis and control based on the extracted target character string in accordance with a processing rule. 3. The method of claim 2,
The processing means determines new learning data based on the extracted target character string in accordance with the processing rule,
Wherein the learning means further generates the parser rule using the determined new learning data. The method of claim 3,
And said processing means determines other data designated in said extracted target character string as said new learning data. 5. The method according to any one of claims 1 to 4,
The learning means stores the generated parser rule in the parser rule storage means in association with the service related to the learning data,
Wherein the extracting means extracts the target character string from the data using the parser rule associated with the service related to the data. 6. The method according to any one of claims 1 to 5,
Said data being described by a predetermined data description language,
Wherein said learning means generates, as said parser rule, a description pattern in said predetermined data description language that can specify a character string including a specified keyword in said learning data. 5. The method according to any one of claims 2 to 4,
Further comprising processing rule storage means for storing one or more processing rules,
Wherein the processing means performs at least one of the predetermined analysis and control according to a specified processing rule among the one or more processing rules. As an information collection method,
Generating a parser rule for extracting a target character string from the data using the training data, wherein the target character string is a character string containing specific information; And
And using the parser rule to extract the target character string from the data. A computer-readable recording medium recording a program that causes a computer to perform a method,
The method comprising:
Generating a parser rule for extracting a target character string from the data using the training data, wherein the target character string is a character string containing specific information; And
And using the parser rule to extract the target character string from the data.

Images