KR20220104098A - Big data analysis system and method of search service for marketing company thereof - Google Patents

Abstract

The present invention relates to a big data analysis system and a marketing company search service method using the system. More specifically, the big data analysis system searches for information from sources (websites) on selected networks based on internal customer information and keywords, collects the search results in an information collection unit, stores the collected information in a distributed manner, taking into account characteristics of multiple information collection units, and analyzes the distributed stored information using a method including value analysis through item association analysis to provide analysis results. The provided analysis results are stored again as data, and through the repeated process of analyzing the association between the data and newly accumulated and analyzed data, a diffusion state of customer information that changes in a time-series manner can be effectively analyzed. Therefore, even if an individual company's own analysis information is limited, the present invention provides effective marketing support information or management support information. This configuration can be established by a specialized analysis company instead of individual companies, and the collected information can be used universally to provide marketing support information that individual client companies desire through the analysis method that the individual client companies require. In this case, individual client companies can secure the marketing support information or management support information through big data analysis at low cost.

Description

Analysis system using big data and marketing company search service method using it { BIG DATA ANALYSIS SYSTEM AND METHOD OF SEARCH SERVICE FOR MARKETING COMPANY THEREOF }

The present invention relates to an analysis system using big data and a marketing company search service method using the same, and more particularly, to collect user information through various media, analyze the collected information as big data, and use it to suit the user. A marketing method suitable for users by a service providing information of a marketing company and an analysis system using big data that provides information on a company acting on behalf of the user, and a marketing company search service method using the same.

Recently, as the SNS advertising platform of the one-person company type has grown, various types of advertising and marketing service methods have been proposed.

Conventional general promotional marketing is a temporary marketing method that emphasizes event characteristics, and has more utility as a marketing method for individuals rather than companies.

As a result, advertisers have to constantly find new targets to continuously market new products, and the consumption of time, money, and manpower increases accordingly.

In order to overcome these problems, most advertisers spend a considerable amount of money to entrust marketing to the marketing agency.

On the other hand, as interest in collecting and analyzing big data increases, companies that have many members and can collect vast amount of member usage log information, such as portals, large distributors, or manufacturers, are certain to analyze member usage logs. The use of big data, such as calculating trend information or classifying target groups, is being considered.

Basically, the processing of big data includes technology for distributed storage and distributed processing to effectively process vast amounts of information, technology to collect and related analysis of various structured and unstructured data, technology to calculate meaningful information from information obtained without purpose, etc. This is complexly applied, and since the effect of such big data analysis has not yet been visibly proven, big data analysis is mainly focused on large companies that can secure a large number of data to analyze and the cost of building a system for massive computation and analysis of numerous data. Research and attempts to utilize it in marketing are being conducted.

Among them, considerable technological progress and utilization have been made in the distributed storage and distributed processing of large-capacity data to cope with the rapid increase in web content or member usage logs, but there are still many studies on calculating meaningful information from big data. is needed.

In particular, various analysis methods for practical use of such big data analysis are being studied. Therefore, when big data analysis is currently used in business activities, it is mainly analyzed internal data provided by the company's customers for statistics or trend analysis, etc. Some of them are being used, and the results of such big data analysis are not yet fully trusted. However, due to the extensive use of various mobile devices, the use of various social networks, and the expansion of various network infrastructures, various user usage log information that can be tracked and collected is exponentially accumulating. No one can deny that it will be essential marketing support or management support information across the country.

As a result, small and medium-sized enterprises (SMEs) that do not have enough standardized customer information that are easy to collect and utilize or cannot afford system costs for big data analysis access the market through big data analysis, such as large portals, financial institutions, and large enterprises. There is a high possibility that it will develop into a situation of being eliminated or subordinated in competition with companies that do business.

Large portals and large corporations can or will soon be able to utilize such management information through big data analysis using capital or accumulated customer information. It's bound to be a difficult day.

On the other hand, in the case of large portals, there are cases where they want to dominate the platform market by providing free big data analysis tools that can utilize the vast amount of information they have. However, the distributed storage of large data and the degree of static analysis through known analysis methods are still common. A methodology for small and medium-sized enterprises or venture companies to customize these systems to their own or to effectively analyze market conditions that change over time. And it is still difficult to directly develop and apply the analysis process. Therefore, they give up securing business information through big data analysis and follow traditional marketing methodologies or use only unreliable simple auxiliary data even if big data analysis is performed.

Therefore, in the case of small and medium-sized enterprises or venture companies, it is possible to effectively obtain optimized management or marketing support information suitable for their own circumstances, and appropriate data even in cases where it is difficult to collect and time-series correlation analysis is required due to scattered information such as viral marketing. There is a need for a new type of big data analysis system and method that can collect and provide desired analysis results.

An object of the embodiments of the present invention for improving the above-described problems is to search for information from sources (websites) on a network selected based on internal customer information and keywords, collect the results in an information collection unit, and collect a plurality of information In consideration of the characteristics of the collection unit, the collected information is distributed and stored, and the distributed and stored information is analyzed in an analysis method including value analysis through item correlation analysis and the result is provided, but the analysis result and the provision result are stored as data again. Through an iterative process of analyzing the correlation with the data that is then accumulated and analyzed, it is possible to effectively provide marketing support information or management support information even if the self-contained analysis information is limited by effectively analyzing the time-series change state of customer information. It is to provide a big data analysis system and method for marketing.

Another object of the embodiments of the present invention is to actively collect required information, not limited information of its customers, on the web, but collect various atypical items according to the collection target in order to effectively analyze various atypical information according to the collection target The process of reanalyzing the analyzed information based on the new analysis information and items is repeated by configuring the information collection unit that formalizes the information to a manageable level, enabling value analysis of variously collected information on the web based on the items. It is to provide a big data analysis system and method for marketing that enables correlation analysis of 1:N relationships based on items to enable analysis of time series analysis and viral marketing.

Another object of the embodiments of the present invention is to store the analyzed information in the utilization database and then process the content of the utilization database by the user's request or the result providing application and provide it to the user as a result, and in the analysis process for providing such a result By analyzing the calculated events and storing some of them in the database to be distributed or re-used so that they can be reused, the information generated according to the experience of the actual work processor for providing the results is additionally reflected in the analysis and can be recycled by other users. It is to provide a big data analysis system and method for marketing.

In order to achieve the above object, a big data analysis system for marketing according to an embodiment of the present invention searches for information according to a search criterion set from a preset type of website and then parses the searched information to provide an information collecting unit provided for each type of website that converts the hierarchical common code and the format based on the item; a distributed storage unit for distributing and storing the information converted by the information collection unit with reference to a common code and size; The distributed storage unit accesses the distributed and stored information in a distributed processing method, analyzes it according to analysis processes including item-based value analysis, and stores the results in the utilization database, and stores the results of previous value analysis and new data previously stored in the utilization database. an analysis unit for re-analyzing the analyzed value analysis result based on the item and further storing the result in the utilization database; It reanalyzes the analysis result of the utilization database as a desired process or searches according to the requested query and outputs the result, and analyzes the output content and provides it as data to the distributed storage unit according to the data format or stores it in the utilization database for reuse Includes results provider.

In addition, it may include a user support unit that provides the search criteria of the information collection unit, updates the analysis process of the analysis unit, provides a reanalysis process or query to the result providing unit, and provides the result to the user.

The information collection unit includes member information preset as a search criterion, and may search for information related to the member.

On the other hand, the information collection unit includes a keyword as a search criterion, and divides the information into a hierarchical common keyword prepared to search information according to the keyword from a plurality of websites as basic information for general-purpose users and classify the search result, It can be parsed and converted into a format including items for which relevance information is set.

The information collection unit can anonymize collected personal information and encrypt or delete identification information.

In addition, the information collection unit converts the format based on the item, but may include unstructured data.

The information collection unit may access each website for each open website including a search portal site, a social network site, and a cloud site, and collect information according to the search criteria through a search or an open API. Here, it is preferable that the information collection unit is configured separately for each type of website, and each information collection unit has a different criterion for classifying the search result into a common code or a criterion for parsing the search result based on the item according to the type of the website. .

The analysis unit may provide the analysis result back as data to the distributed storage unit.

It is preferable that the analysis unit repeatedly collects data for 1:N correlation analysis by storing the result of the value analysis performed on the basis of the item in the utilization database.

The result providing unit may be an interface for linking with an external analysis solution or an external analysis solution, or may include an analysis configuration that can be customized on an external user site.

A big data analysis method for marketing according to another embodiment of the present invention searches for information according to a search criterion set by an information collection unit that is classified according to a preset type of website, and sets the search content in advance with a hierarchical common code and item an information collection step of converting to a format based on ; a distributed storage step of distributing and storing the received information based on a common code and size by a distributed storage unit receiving the converted information collected in the information collection step; In the distributed storage step, an analysis unit that performs a required analysis using the distributed and stored information accesses the distributed stored information in a distributed processing method, performs analysis according to analysis processes including item-based value analysis, and then utilizes the results an analysis step of storing the data in the database, reanalyzing the previous value analysis result and the newly analyzed value analysis result previously stored in the utilization database based on the items, and storing the result back in the utilization database; The result providing unit that provides the results to the user reanalyzes the analysis result of the utilization database as a desired process or searches according to the requested query and outputs the result, and analyzes the output and provides it as data to the distributed storage unit according to the data format; It includes a step of providing the result of storing it in the utilization database for reuse.

receiving and processing a user's request through a user support unit that provides the search criteria to the information collection unit, updates the analysis process in the analysis unit, provides a reanalysis process or query to the result provision unit, and provides the result to the user may include

In the information collection step, the information collection unit may include member information preset as a search criterion and search for information related to the member.

In the information collection step, the information collection unit includes a keyword as a search criterion, and is divided into hierarchical common keywords prepared to search information according to the keyword from a plurality of websites as basic information for general-purpose users and classify the search results, , it is possible to parse the information and convert it into a format including items for which relevance information is set.

In the information collection step, the information collection unit accesses each open website for each open website including a search portal site, a social network site, and a cloud site, and collects information according to the search criteria through a search or an open API. . In addition, these information collection units are configured separately for each type of website, and each information collection unit may have different criteria for classifying search results into a common code or criteria for parsing search results based on items depending on the type of website. .

In the analysis step, the analysis unit may provide the analysis result back to the distributed storage unit as data.

In the analysis step, the analysis unit may repeatedly collect data for 1:N correlation analysis by storing the result of the value analysis performed on the basis of the item in the utilization database.

A big data analysis system and method for marketing according to an embodiment of the present invention searches for information from sources (websites) on a network selected based on internal customer information and keywords, and collects the results in an information collection unit, The collected information is distributed and stored in consideration of the characteristics of the information collection unit of Through an iterative process of analyzing the correlation with the newly accumulated and analyzed data after storage, it effectively analyzes the customer information diffusion status that changes in time series, so that even if the analysis information possessed by the company is limited, the marketing support information or management support information can be effectively analyzed. Not only is there an effect that can be provided, but also the marketing support information desired by individual customers can be provided through the analysis method required by individual customers by using the collected information universally by configuring this configuration by a specialized analysis company rather than an individual company. It is also possible to provide a method, and in this case, individual customers have the effect of securing marketing support information or management support information through big data analysis at a low cost.

The big data analysis system and method for marketing according to the embodiment of the present invention actively collects necessary information on the web rather than limited information of its customers, but in order to effectively analyze various atypical information according to the collection target, depending on the collection target It constitutes an information collection unit that standardizes the collected information collected in various atypical items to a processable level and stores it in a distributed manner. By repeating the process of re-analysis based on and items, it enables the correlation analysis of 1:N relationships based on items, thereby economically providing fast and reliable analysis results for time series analysis and viral marketing. there is

The big data analysis system and method for marketing according to an embodiment of the present invention stores the analyzed information in the utilization database, processes the contents of the utilization database by the user's request or the result providing application, and provides the result to the user as a result. By analyzing the events generated in the process of analysis to provide results and storing some of them in the database to be distributed or reused again, the information generated according to the experience of the actual work processor for providing the results is additionally reflected in the analysis. By allowing other users to recycle it, the quality of the provided marketing information is improved, and reliability and satisfaction increase according to its use.

1 is a block diagram of a conventional big data analysis system.
2 is a conceptual diagram of a big data analysis system according to an embodiment of the present invention.
3 is a block diagram of a big data analysis system according to an embodiment of the present invention.
4 is a configuration diagram of an information collection unit according to an embodiment of the present invention;
5 is a configuration diagram of a distributed storage unit according to an embodiment of the present invention.
6 is a configuration diagram of an analysis unit according to an embodiment of the present invention;
7 is a configuration diagram of a result providing unit according to an embodiment of the present invention.

The present invention as described above will be described in detail with reference to the accompanying drawings and embodiments.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as meanings generally understood by those of ordinary skill in the art to which the present invention belongs, unless otherwise defined in the present invention, and excessively comprehensive It should not be construed as a human meaning or in an excessively reduced meaning. In addition, when the technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be understood by being replaced with a technical term that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted as defined in advance or according to the context before and after, and should not be interpreted in an excessively reduced meaning.

Also, the singular expression used in the present invention includes the plural expression unless the context clearly dictates otherwise.

In the present invention, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various elements or several steps described in the invention, and some of the elements or some steps are included. It should be construed that it may not, or may further include additional components or steps.

Also, terms including ordinal numbers such as first, second, etc. used in the present invention may be used to describe the components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted.

In addition, in the description of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

2 is a conceptual diagram of a big data analysis system according to an embodiment of the present invention. As shown, an information search and collection system 10 that actively collects numerous user information generated on the web, rather than using only internal information possessed. And, a distributed storage system 20 that distributes and stores the vast amount of information collected in this way, and an analysis system 30 that accesses distributed and stored information in a distributed processing method to perform analysis and distribute and store recyclable information among the analysis results ), and receive and provide the analyzed results through the analysis system 30, allow the analysis system 30 to re-perform the necessary analysis based on the analyzed results, or retrieve the analysis results according to a query and display the results Search result information that can be provided and recyclable is provided as data to the distributed storage system 20 according to the type, and if it can be utilized as the analyzed itself, the result providing system 40 that provides it for recycling in the analysis system 30 And, the information search and collection system 10 provides search criteria for the information to be collected, provides the analysis system 30 with the contents to be analyzed (scripts, analysis algorithms, processes such as settings), or updates the contents to be analyzed, , the result providing system 40 includes a user support system 50 that provides content (processes such as script, analysis algorithm, setting, etc.) or query for re-analysis and provides the result to the user. Here, the user support system 50 may be a user interface that allows access to each of the systems, but may be a system having some functions of each system, and thus the specific configuration may vary.

In the end, the configuration shown is not a conventional method of collecting and analyzing standardized internal information, but a desired It is a method of collecting information suitable for analysis target and analyzing information for desired marketing or management support from them.

If some information about the member is provided, various related information may be collected in a manner that further collects information related to the member from various websites as well as internal information about the member.

Websites that collect information may be, for example, various portals such as Google, Naver, Daum, Bing, Nate, Yahoo, and Baidu, and information using specific search results or member identifiers may be collected from these portals. Web pages, news, blogs, documents, etc. provided as a result can be the target of information collection. Such portals also provide an open API (Application Program Interface) that enables search and information collection for various services provided by the portal as well as search. The website that collects such information may be, for example, various social networks such as Facebook, Twitter, YouTube, Pinterest, Cyworld, Line, Instagram, Me2day, Tumblr, Listen Me, and Last FM. Most of these social networks often provide open APIs that allow external users to retrieve or collect internal information of the social network.

In addition, information can be collected from various websites such as various cloud service sites and mobile service sites, and these sites also often provide open APIs.

Therefore, since it is possible to collect information using robots, it is possible to collect vast amounts of information according to desired criteria (keywords, time, popularity, filtering conditions, etc.).

However, simply collecting various information on the web does not mean that it can be directly utilized for big data analysis. not.

Accordingly, the embodiment of the present invention basically introduces two new methods. First, the method of composing the information collection units corresponding to the type of website so as to standardize the collected information to some extent, and for the viral marketing analysis, which increases in importance in time-series analysis or online marketing, value analysis is performed based on the items and then stored In addition, a method is introduced that enables potential value analysis for marketing support by enabling correlation analysis of 1:N relationships based on items through an iterative process of analyzing correlations with newly collected and analyzed data.

In addition, result information is generated and provided to users according to analysis requests (scripts, algorithms, settings, etc. for desired analysis) or search queries from actual marketing experts. By applying a method that allows users to do this, analysis information for marketing support or management support that is practically meaningful can be provided rather than simply analyzing large amounts of data. In addition, when the system according to the embodiment of the present invention is specialized as a professional service for general-purpose users rather than a subordinate system for a specific company, information collected in general is analyzed according to the analysis needs of each service-using company. You can then provide the results. In this case, the system utilization increases, so the burden on service users can be reduced, and the results analyzed by the marketers of each service user can be recyclably collected so that these various service users can quickly and diversely obtain the desired results. do. In addition, it is possible to increase the efficiency of time-series analysis or correlation analysis based on items by integrating the analysis results with newly collected information and re-analyzing them.

A more specific example to which such a configuration is applied will be described with reference to FIG. 3 .

3 is a block diagram of a big data analysis system according to an embodiment of the present invention. As shown, information is searched for from a preset type of website according to a set search criterion, and then the searched information is parsed and a layer prepared in advance. The information collection unit 100 prepared for each type of website, which converts it into a format based on common codes and items, and the information converted by the information collection unit 100, with reference to the common code and size, distributed storage 300 The distributed storage unit 200 that stores the distributed data in the database unit, and the distributed storage unit 200 accesses the distributed and stored information in a distributed processing method and analyzes the results according to analysis processes including item-based value analysis, and then displays the results in the database unit The analysis unit 500 that stores in 400, re-analyzes the previous value analysis result and the newly analyzed value analysis result previously stored in the database unit 400 based on the item, and further stores the result in the database unit 400 And, the analysis result of the database unit 400 is reanalyzed as a desired process or searched according to a requested query and the result is output, and the output is analyzed and provided as data to the distributed storage unit according to the data format or a database for reuse The result providing unit 600 to be stored in the unit 400 and the search criteria of the information collecting unit 100 are provided, the analysis process of the analysis unit 500 is updated, and the result providing unit 600 is re-analyzed or and a user support unit 700 that provides a query and provides the results to a user.

Here, the user support unit 700 means an integrated user interface, and not only a simple interface, but also an extended function for each related functional unit (information collecting unit 100 , analysis unit 500 , and result providing unit 600 ). may be provided, and in some cases, may be included in the corresponding functional unit as a partial function of each functional unit. Accordingly, the user support unit 700 is not limited to the configuration according to the embodiment of the present invention because it may be configured in various ways or may have various functions.

On the other hand, the illustrated distributed storage unit 200 and the distributed storage 300 can use various distributed storage methods that classify data according to the size and store the data in several storage places, which are widely known in this embodiment. The Hard File System (HDFS) is the basis for distributed storage. In this case, the analysis unit 500 may utilize Map Reduce using the Hadoop file system as a basis for distributed processing.

The illustrated database unit 400 includes a metadata database 410 that stores information on files that are distributed and stored or a search criterion for search of the information collection unit 100, and information on a hierarchical integration code for classification. The code database 420, the analysis unit 500 and the result providing unit 600 store the analysis results for analysis, accumulate the item-based value analysis results, and a utilization database 420 that stores the results for recycling among the search results. ) is included.

That is, the database unit 400 substantially described in the description of FIG. 3 described above means the utilization database 420 . Other databases 410 and 430 will be further described later.

Meanwhile, in the illustrated configuration, the information collection unit 100 includes a member information acquisition unit 110 and an information search collection unit 120 , among which the member information acquisition unit 110 may be selectively applied. The illustrated member information acquisition unit 110 collects internal member information or internal member usage information when collecting information, and information related to member information when the information search and collection unit 120 collects information from various websites as needed We can provide information that enables us to collect

Of course, the information collection unit 100 may configure only the information search and collection unit 120 that collects information from websites set based on a keyword set by a user through the user support unit 700 .

As described above, in the embodiment of the present invention, an information collection unit that searches for and collects desired information from various websites, parses them, and converts them into a format having an appropriate classification code and item is configured for each type of website.

This will be described in more detail with reference to FIG. 4 .

4 is a meta data showing the configuration of the information search and collection unit 120 according to an embodiment of the present invention, a metadata database 410 in which search criteria for information search of the information search and collection unit 120 are set, and the corresponding search criteria. The configuration of the user support unit 700 provided to the data database 410 is shown.

In an embodiment of the present invention, the illustrated information search and collection unit 120 collects various information to enable big data analysis, which will be described in more detail.

First of all, it is not standardized internal information, but various result provision formats or provided results (web pages, documents, bulletin boards, comments, etc.) Let it be classified as a hierarchical integration code. In the above, an information search unit that analyzes search results provided differently according to each type of website in a parsing method and converts it into a format based on an integrated code and item is configured according to the type of website. Through this, even if the regulations of the website itself change or the status of the search result changes, only the relevant information search unit needs to be updated, and when a new website is added, the corresponding information search unit needs to be added, so that management for information search is improved. In addition to making it easy, it is possible to convert the collected information into data that can be analyzed in an integrated manner even if the information collected varies according to various websites. Of course, even with this format, it is possible to classify the search results to the extent that an item is determined by selecting one of the items with similarity, but it is very difficult to match the data type of the item as well. Therefore, even if the data is collected as a related item, the data may be structured or unstructured data, and the analysis unit selects and processes one of several known methods for interworking structured data and unstructured data.

To this end, the illustrated information search and collection unit 120 searches for information according to the search criteria (keywords, search target information creation time, target website, search cycle, search degree, member information, etc.) set in the metadata 410 . Search by the collection management module 122, the information search module 121 that searches for information by accessing the website 800 set according to the request of the collection management module 122, and the information search module 121 The data generation module 123 for converting the format based on the items in which the relationship is defined after parsing the obtained information in consideration of the characteristics of the website, and the distributed storage unit ( 200), and a data providing module 124 provided.

Here, the data generation module 123 may add a classification to which the corresponding information belongs as hierarchical common code information while parsing the searched information and converting it into a standardized format. For example, if the corresponding information is information about a recently used lipstick, it may be common code information corresponding to women's products-cosmetics-lipstick.

In addition, if necessary, the data generation module 123 may perform security-related processing of anonymizing collected personal information and encrypting or deleting identification information.

On the other hand, since the searched information is information obtained from an arbitrary website, items provided on a specific website may not be provided on other websites, and items identified on a specific webpage may not be confirmed on other webpages. For big data analysis, items with relationships set are prepared, and items suitable for the website are selected from them and organized in a prepared format. For example, for items such as a user's gender, age, and location, information corresponding to the related item may or may not be collected. Accordingly, in this case, as the basic information to be collected, gender, age, location, etc. may be provided as related items, and information obtained from the website may be set as a corresponding item among them. As another example, the user's location information may be collected as user IP information on a certain website, as information that can confirm the approximate user location, on another website as address information, and as GPS information on another website. However, they can all have a relationship as a location item. However, they each have a different data structure. Accordingly, these may be converted into specific information (address or GPS location) to fit the format, or the format may be configured by classifying them into appropriate detailed items among location items having a relationship.

Since such item information can be utilized in value analysis according to the relationship between items or correlation analysis according to 1:N information generation based on items during subsequent analysis, the format setting can be said to be an important part, and the implementation of the present invention In the example, conversion optimized for the characteristics of the collection target is possible by parsing the format setting by reflecting the characteristics of the website.

5 is a configuration example of the distributed storage unit 200 according to an embodiment of the present invention. As shown in FIG. 5, the distributed storage unit 200 includes a plurality of units corresponding to a plurality of information collection units separately provided for each type of website. A distributed storage unit 210 is provided, and this unit distributed storage unit 210 collects search information in a format provided by the information collection unit according to its own conversion rules, and checks the integrated code for it in the code database 430. A classification identifier is added and the size is divided and stored in the distributed storage 300 .

Each unit distributed storage unit 210 distributes and stores the collected information in the distributed storage 300 and a collection agent module that provides information about the location, file name, group code, input date, etc. of the distributed storage to the metadata database 410 . 211 and a data flow control module 212 configured to divide and store the collected data according to the size of the data.

6 shows the configuration of the analysis unit 500 according to an embodiment of the present invention. As shown, data distributed and stored in the distributed storage 300 by the distributed storage unit 200 is queried or distributed by the MapReduce method, etc. A distributed processing module 510 that processes and, if necessary, stores the same format as data among the analysis results as data in the distributed storage 300 for recycling, and various analysis processes (analysis scripts, An analysis process module 530 that manages algorithms, settings, etc.), a data analysis module 520 that automatically performs an analysis according to each analysis process of the analysis process module 530 and calculates the result, and a data process module The individual analysis processes of 530 are registered, updated, and removed, and if necessary, integrated code information is checked from the code database 430 and provided to the data analysis module 520, and the data analysis module 520 analyzes the result. and an analysis control module 540 for collecting and recording in the utilization database 420 . The user support unit 700 may provide the analysis unit 500 with an analysis process for a desired analysis, update an existing process, or provide information on deleting an existing process.

The analysis control module 540 performs analysis processes including value analysis by item-based correlation analysis for effective viral marketing analysis (inquiry, structured and unstructured linkage analysis, statistical analysis, text analysis, machine learning, batch analysis, data After analyzing the collected data using correlation identification, pattern extraction, ranking, issue analysis, timing analysis, related word analysis, correlation analysis, regression analysis, etc.), the result is stored in the utilization database 420 and the utilization database The previous value analysis result and the newly analyzed value analysis result previously stored in the are re-analyzed based on the item, and the previous result is further stored in the utilization database 420 . When this process is repeated, 1:N analysis information can be accumulated for items, and through this, correlation analysis is possible, so that various information about changes in analysis results can be checked. For example, it is possible to check how the usage information for a specific lipstick spreads over time or changes in what direction (age, region, specific affiliation, website standard direction), and the effect of viral marketing is analyzed in relation to the execution of advertisements. can be checked.

Meanwhile, the data analysis module 520 or the analysis control module 540 may store the analyzed result as data again in the distributed storage 300 .

Here, the utilization database 420 is a database that supports high-speed database analysis to effectively perform Hadoop distributed processing. HBase, a NoSQL-based database for distributed processing of big data with a length of several trillion rows, can be used. have. This is configured on an in-memory basis to increase the speed of big data analysis, and stores the analyzed results through the analysis unit 500 so that the stored contents can be quickly utilized for various services. Of course, the utilization database 420 may have the form of a memory store and an interface in order to provide compatibility with various other types of databases in addition to the HBase. Therefore, it can operate as various types of memory-based databases.

7 shows the configuration of the result providing unit 600 according to an embodiment of the present invention. As shown, the analysis result of the utilization database 420 is reanalyzed as a desired process or searched according to a requested query and the result is displayed. The information distribution module 610 for output, the service application module 620 for providing a process or query to utilize the utilization database 420, and the distributed storage of events that can be logged and analyzed according to the data format by analyzing the result output contents It may include an event analysis module 630 that provides data to the distributed storage through the department and stores the result event that can be used and serviced immediately in the utilization database 420 for reuse.

Here, the service application module 620 may be an interface that is linked to the user support unit 700 as a system that interworks with the user, and an actual analysis process or query may be provided through the user support unit 700 .

On the other hand, some functions of the user support unit 700 may be integrated with the result providing unit 600 . In this case, the information distribution module 610 performs analysis if necessary in addition to the function of utilizing the analysis results stored in the utilization database 420 . It is also possible to provide a kind of dispatcher function to provide a new analysis process to the unit 500 to collect and output the analysis result according to the analysis result to the user. In this case, the result providing unit 600 is an interface for linking with an external analysis solution or an external analysis solution, or may include an analysis configuration that can be customized to an external user site, and the service application module is an analysis engine ( For example, it may be an analysis engine configured through R-studio, a tool for analysis of distributed processing databases.

On the other hand, if a user, such as a marketer, manager, or advertiser, collects information by providing search criteria such as keywords for analysis that they need, the big data analysis system according to the embodiment of the present invention periodically analyzes information according to the search criteria. collect with The collected information is distributed and stored, and the information is accumulated by repeatedly analyzing the analysis information to enable correlation analysis by approaching and analyzing it in a distributed processing method. If necessary, the accumulated analysis information can be queried according to the desired criteria to produce results, or the analysis results can be utilized by suggesting a new analysis process. By storing the results according to such a practical analysis process or query in the utilization database, the analysis results for the related analysis data can be immediately reused when requested, and the continuous user usage records according to such analysis are stored in the analysis unit in a machine learning method. can be learned to increase the reliability of the related analysis.

In the above, preferred embodiments according to the present invention have been shown and described. However, the present invention is not limited to the above-described embodiments, and without departing from the gist of the present invention appended in the claims, anyone with ordinary skill in the art to which the invention pertains will be able to implement various modifications. .

10: information search and collection system 20: distributed storage system
30: analysis system 40: result providing system
50: user support system 100: information collection unit
200: distributed storage 300: distributed storage
400: database unit 500: analysis unit
600: result providing unit 700: user support unit

Claims (1)

an information collection unit provided for each type of website, which searches for information from a preset type of website according to a set search criterion, parses the searched information, and converts the searched information into a format based on a hierarchical common code and item;
a distributed storage unit for distributing and storing the information converted by the information collection unit with reference to a common code and size;
After the distributed storage unit accesses the distributed and stored information in a distributed processing method and analyzes it according to analysis processes including item-based value analysis, the result is stored in the utilization database, and the previous value analysis result stored in the utilization database and an analysis unit for re-analyzing the newly analyzed value analysis result based on the item and further storing the result in the utilization database;
The analysis result of the utilization database is reanalyzed as a desired process or searched according to a requested query and the result is output, and the output is analyzed and provided as data to the distributed storage according to the data format or stored in the utilization database for reuse Big data analysis system for marketing, characterized in that it comprises a result providing unit to store.

Images