KR101843519B1 - Method and apparatus for automatic collecting and analyzing data based on changeable three dimensional sturcure meshup keyward - Google Patents

Abstract

A method and an apparatus for automatically collecting and analyzing data by using a variable three-dimensional rectangular network keyword are disclosed. The method for automatically collecting and analyzing data by using a variable three-dimensional rectangular network keyword comprises the steps of: generating a variable three-dimensional rectangular network around a central keyword; and filling a plurality of network empty bins on the variable three-dimensional network with a keyword related to the central keyword.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for automatic data collection and analysis using a variable three-dimensional rectangular mesh keyword,

The present invention relates to a data collection and analysis method, and more particularly, to a method and apparatus for automatic data collection and analysis using a variable three-dimensional rectangular network keyword.

While the demand for analyzing and insighting the numerous data pouring into the advent of the Fourth Industrial Revolution continues to increase, there has been an increase in the difficulty of collecting and analyzing certain data due to increased data and increased sources.

As a result, data collection and analysis became important. In collecting and analyzing data, it is necessary to check and collect data contents one by one. It is very tricky to do.

In order to collect data, the center of the data to be searched, abstract keyword, must exist. If we try to collect data based on abstract keywords, we collect data that is too broad. In order to extract desired data, we need to retrieve the data in the collected data and retrieve the relevant data again. Collection and analysis.

Therefore, there is a problem that a considerable amount of time and manpower is required for collecting and analyzing desired data, and a solution method is needed.

10-2014-0114029

One aspect of the present invention provides an automatic data collection and analysis method using a variable three-dimensional square net keyword.

Another aspect of the present invention provides an apparatus for performing an automatic data collection and analysis method using a variable three-dimensional rectangular network keyword.

According to an aspect of the present invention, there is provided an automatic data collection and analysis method using a variable three-dimensional rectangular network keyword, comprising the steps of: generating a variable-size three-dimensional rectangular network around a central keyword; And filling the associated keywords.

Meanwhile, the variable-shaped three-dimensional rectangular network may have a plurality of mesh bins based on the center keyword, and the variable-size three-dimensional rectangular network may be in the form of a three-dimensional rectangular grid allocated with n,

Also, the keyword is determined based on an optimal score, and the optimal score can be determined based on a direct association index, an association index, and an appearance frequency index of the data based on the center keyword.

The optimal score may be determined by assigning different weights to the direct association index, the association index, and the frequency index of appearance of the data.

Also, an automatic data collection and analysis method using a variable three-dimensional rectangular network keyword may include generating a network completion signal when a plurality of network empty bins on the variable rectangular solid network are filled in and generating a new variable network based on the network complete signal, And a second step of performing a second step.

According to another aspect of the present invention, there is provided a data analysis system for performing automatic data collection and analysis based on a variable three-dimensional rectangular network keyword, the apparatus comprising: a processor for generating a variable rectangular solid network around a center keyword, And fill the plurality of mesh empty spaces on the variable rectangular solid network with keywords related to the center keyword.

Meanwhile, the variable-shaped three-dimensional rectangular network may have a plurality of mesh bins based on the center keyword, and the variable-size three-dimensional rectangular network may be in the form of a three-dimensional rectangular grid allocated with n,

Also, the keyword is determined based on an optimal score, and the optimal score can be determined based on a direct association index, an association index, and an appearance frequency index of the data based on the center keyword.

The optimal score may be determined by assigning different weights to the direct association index, the association index, and the frequency index of appearance of the data.

In addition, the processor may be configured to generate a mesh completion signal when a plurality of mesh blanks on the variable rectangular mesh are filled, and to generate a new mesh based on the mesh completion signal.

The method and apparatus for automatic data collection and analysis using a variable three-dimensional rectangular network keyword according to an embodiment of the present invention are very fast and effective in automatically collecting and analyzing a keyword, can do.

1 is a flowchart illustrating an automatic data collection and analysis method using a variable three-dimensional rectangular mesh keyword according to an embodiment of the present invention.
2 is a conceptual diagram illustrating an automatic data collection and analysis method using a variable three-dimensional rectangular mesh keyword according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a method for determining an optimal score and a best / best score, an evaluation score, according to an embodiment of the present invention.
4 is a conceptual diagram illustrating a data analysis system according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

In order to effectively collect and analyze large amounts of data (Bigdata), it is necessary to generate a dictionary, create learning data, grasp patterns and re-process the data.

To do this, we create a virtual variable network and select the center keyword to select the data that has a direct relevance index (or relation level score), association index (or support score) and high frequency (or frequently score) And a pre-cube that is constantly changed and refreshed can be generated.

1 is a flowchart illustrating an automatic data collection and analysis method using a variable three-dimensional rectangular mesh keyword according to an embodiment of the present invention.

In FIG. 1, a procedure for performing automatic data collection and analysis using a variable three-dimensional rectangular network keyword is disclosed.

In an automatic data collection and analysis method and apparatus using a variable three-dimensional rectangular mesh-up keyword according to an embodiment of the present invention, an abstract core keyword (or a core keyword) Can be selected and input.

The goal of creating a variable three-dimensional rectangular mesh is to collect and analyze meaningful data more easily and quickly by creating a mesh centered on interests and issues for massive data and analysis.

The following steps may be performed to generate a mesh keyword based on the core keyword according to the present invention.

1) Step of making a lattice shape of 6 points (+, - 3point) in horizontal, vertical, and height based on the center point of the mesh empty block (step S110).

2) Data that are related to the central keyword and have a direct association index (or relation level score), data with a high association score (or support score) and data with a high frequency of occurrence (or frequent score) Calculating an optimal score obtained by summing the scores, and repeatedly generating a keyword that is a mesh point (step S120).

3) setting a direct relationship (Core Keyword Relation Level) between the keyword and the central keyword (step S130).

4) Based on the central keyword, the support score and the relation level score (1 ~ 3) are ranked in order from the closest lattice centered on the central axis. (Step S140).

5) After all the mesh points in the mesh have been input, the data for the keyword consisting of each point in each solid rectangle composed of 8 points (one grid) And automatically collecting and loading (step S150).

6) Automating / analyzing data collected at each mesh point (step S160).

7) If the Meshup Empty Block in the mesh is filled with the collected data (the block size can be specified by the user), it is judged that the life cycle has been completed. A step S 170 of generating a new three-dimensional rectangular network suitable for a new environment and collecting variable data according to an environment change.

Hereinafter, a specific variable three-dimensional rectangular network is disclosed.

2 is a conceptual diagram illustrating an automatic data collection and analysis method using a variable three-dimensional rectangular mesh keyword according to an embodiment of the present invention.

In FIG. 2, an automatic data collection and analysis method using a variable three-dimensional rectangular mesh keyword is disclosed.

Referring to FIG. 2, a variable shape solid square mesh may be generated around a core keyword for data collection and analysis.

A Meshup Empty Block is created at the center of the mesh of the mesh, and a three dimensional rectangular grid shape can be generated by n points (for example, 6 points (+, - 3point)) in the horizontal, vertical and height directions.

After the center keyword is selected, initial data for the center keyword is generated for a predetermined period (for example, 24 hours) from the start of collection, and the initial data can be stored in the initial data repository.

A relation level score, a support score, and an appearance frequency index of data on a content content of an initial data collected based on a center keyword in order to find data to be input to a blank web site, score) can be determined. The scores are weighted in the order of direct association with the central keyword, association index, and data appearance frequency index, and the scores can be combined (optimal score). Based on the summation result, a keyword centered on the central keyword can be determined.

The collected data and collected keywords can be stored in the initial data store to fill all of the data in the mesh blank.

The process of collecting data and determining the keywords to fill all of the data in the mesh blank can be repeated until the space on the mesh blank is filled.

Information on the keyword, lattice location information, and date information may be included in the mesh empty house. After the fillets are all filled, a complete signal is generated and the collection of data and the determination of keywords may no longer be performed.

When a network complete signal is received, a new network is created, and the center keyword of the new network can be determined by summing up the direct association with the central keyword, the association index, and the frequency of occurrence of the data. . ≪ / RTI > If the duplication of data collected in the existing network and the new network in the repeated process occurs more than a threshold percentage (for example, 20%), it can be judged that the lifetime of the old network is exhausted.

Hereinafter, a method for determining an optimal score and a best / best score, an evaluation score, according to an embodiment of the present invention is disclosed.

In order to find the best / optimal score according to an embodiment of the present invention, it is necessary to create a data model that creates and evaluates a dictionary set of learning data. As described above, an optimal score can be calculated based on a direct relevance index (or relation level score) for a central keyword, data having a high association score (or support score), and frequency of occurrence (or a frequent score) have.

In addition, an evaluation may be performed for each keyword collected using an evaluation item and an evaluation score as a column. This method of evaluation can be scored by summing up the data gathering qunatity, the amount of data after data refinement, and the life cycle of the network.

Specifically, an optimal score, a best / optimal score and an evaluation score can be calculated by the following equation.

&Quot; (1) "

Optimal score = weight 1 * relevance level + weight 2 * association score + weight 3 * appearance frequency

Best optimal score = score data obtained by taking the highest score among the evaluation items between the keyword and the keyword through the learning data

Evaluation Score = Amount of raw data collected by keywords and related keywords / Daily + Amount of data after purification / Daily + Life cycle

The optimal score and the best / optimal score, evaluation score can be utilized in various ways.

3 is a conceptual diagram illustrating a data analysis system according to an embodiment of the present invention.

3, the data analysis system includes a central keyword input unit 300, a variable three-dimensional network generating unit 310, a data storing unit 320, an optimal score generating unit 330, a network completion determining unit 340, A network life determination unit 350 and a processor 360. [

The central keyword input unit 300 may be implemented to receive a core keyword for data collection and analysis.

The variable shape solid mesh generation unit 310 may be implemented to generate a variable shape solid mesh for data acquisition and analysis.

The variable-shape three-dimensional mesh generation unit 310 includes a meshup empty block at the center of the mesh grid, and generates a three-dimensional mesh of the mesh shape by n points (for example, 6 points (+, - 3point) Can be generated in a grid form. The collected data may be located in the mesh blank on the variable shape mesh generating unit 310.

When the mesh completion signal is received, the variable shape mesh generator 310 may be determined to generate a new mesh.

The data storage unit 320 may be configured to store initial data for a central keyword for a predetermined period (for example, 24 hours) from the start of collection after the central keyword is selected.

The optimal score generation unit 330 generates a relation score, a relation score, a data relevance index, and the like, on the content content of the initial data collected on the basis of the center keyword, A frequency score, and to generate an optimal score based on the weights for each.

The mesh completion determiner 340 may be implemented to generate a mesh complete signal (Complete Signal) after all of the mesh empty spaces have been filled.

The variable mesh network lifetime determiner 350 may be implemented to determine whether the life of the variable mesh is reached.

The processor 360 includes a central keyword input unit 300, a variable three-dimensional network generating unit 310, a data storing unit 320, an optimum score generating unit 330, a network completion determining unit 340, (Not shown).

The above-described methods may be implemented in an application or may be implemented in the form of program instructions that may be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention and may be those known and used by those skilled in the computer software arts.

Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It will be possible.

Claims (10)

An automatic data collection and analysis method using a variable three-
Generating a variable rectangular solid network around a center keyword; And
Filling a plurality of mesh blanks on the variable three-dimensional rectangular mesh with keywords related to the center keyword,
Wherein the variable-shaped three-dimensional rectangular network has the plurality of mesh bins based on the center keyword,
Wherein the variable shape rectangular mesh network is in the form of a three-dimensional square lattice having n, n, and h points assigned thereto.
delete delete delete delete A data analysis system for performing automatic data collection and analysis based on a variable three-dimensional rectangular network keyword,
Wherein the data analysis system comprises a processor,
The processor generates a variable three-dimensional rectangular network around a center keyword,
Wherein a plurality of mesh blanks on the variable three-dimensional rectangular mesh are filled with a keyword related to the center keyword,
Wherein the variable-shaped three-dimensional rectangular network has the plurality of mesh bins based on the center keyword,
Wherein the variable-shape three-dimensional rectangular network is in the form of a three-dimensional rectangular lattice with n, n, and h points assigned thereto.
delete delete delete The method according to claim 6,
The processor generates a mesh completion signal when a plurality of mesh bins on the variable rectangular mesh are filled,
And a new variable network is generated based on the network completion signal.

Images