KR20230099820A - System of analyzing market based on ai and method thereof - Google Patents

Abstract

The present invention relates to an AI-based market research and analysis system and a market research and analysis method using the same. More specifically, the AI-based market research and analysis system collects and analyzes big data for market analysis depending on market demand, performs quantified data analysis such as market research, industry trend research, statistical surveys, and public opinion surveys. Also, AI produces professional-level reports to suggest trends and policy directions and simultaneously presents the basis for the generated reports. Accordingly, it is possible to verify the reliability of the generated reports and the calculation results, and through AI iterative learning, the reliability and verification of report generation can be improved. According to the present invention, the difficulties that small and medium-sized enterprises are experiencing in new product and service development, sales, advertising, public relations, and marketing due to a lack of professional planning personnel can be eased.

Description

AI-based market research and analysis system and market research and analysis method using the same {SYSTEM OF ANALYZING MARKET BASED ON AI AND METHOD THEREOF}

The present invention relates to an AI-based market research and analysis system and a market research and analysis method using the same, and more specifically, market research, industry trend research, and statistical research by collecting and analyzing big data for market analysis according to market demand. , quantified data analysis such as public opinion polls, AI produces professional-level reports to present trends and policy directions, and at the same time presents the basis for calculation of generated reports, thereby verifying the reliability of generated reports and the results of their calculations. It is about an AI-based market research and analysis system and a market research and analysis method using the AI-based market research and analysis system that enables this and improves the reliability and verification ability of report generation through repeated AI learning.

All domestic and foreign companies use the web, news, policy materials, other reports, social media, and SNS to make important decisions such as product and service development, marketing, advertising, public relations, and business plan establishment. , Industry trends, competitor trends, market needs, consumer needs, trends, etc. are collected and used to prepare market research reports.

Companies use these market research data in the stages of market research, analysis, design, and development, which are important factors for success in developing, selling, and marketing new products and services.

According to the 2019 CMO Survey, leading companies around the world are allocating and utilizing about 32% of their marketing budgets for market research and insight analysis, but companies that do not have financial conditions are not properly conducting market research and analysis. am.

More than a few small and medium-sized enterprises (SMEs) where conditions allow, conduct market research through their own planning personnel or purchase and use reports to secure market data for the industry to which the company belongs by using a service that provides market analysis reports for each specialized industry. However, due to the lack of planning manpower or cost and report quality problems of industry-specific market analysis reports, only small and medium-sized enterprises (SMEs) or higher with financial resources can do so. market research is required.

In particular, in the case of trend research on overseas markets, the difficulty is increasing in the market research stage besides the KOTRA market report.

In addition, most domestic and foreign companies have problems with the reliability of information collected on the web or SNS, and due to information overload, incorrect market research and market analysis reports are written, and reports containing incorrect information are reflected in important decision-making of the company. situation is going on.

In particular, SMEs that do not have or lack professional planning personnel do not properly conduct such market research or prepare market research and analysis reports based on incorrect information on the web and SNS to develop new products and services, establish advertising promotion marketing strategies, and market And there is a problem that can be expanded to a problem that is directly connected to a serious misjudgment in business planning in predicting consumer requirements, surveying consumer responses, and the like.

As of the end of 2017, the number of SMEs by industry in Korea was about 6.3 million, accounting for more than 90% of all companies, 82.9% of all employees, 97% of domestic job creation, and 401 world-class products. SMEs have an important position and value in the industry.

However, as SMEs with less than 30 employees who do not have professional planning personnel account for 95.6% of the total business operators, SMEs face problems in securing professional manpower for market research and analysis, cost problems, and report quality problems. The use of market analysis report services and market research research institutes is also limited.

As such, it is difficult to secure market data for market research and analysis, which is important for SMEs, and problems arise in making experience-based decisions in important decisions in business management.

In particular, venture companies and start-up companies, which are sensitive to the latest market trends and trends, and whose business establishments are increasing recently have limitations in market research and production of analysis reports due to lack of professional planning personnel as well as their own personnel and experience. In a situation where periodic information collection on trends is absolutely necessary, each report takes several weeks and there is a limit to handling outsourcing reports for market research worth at least millions of won.

KR 10-2018-0075852 A KR 10-2018-0059167 A

The present invention for solving the above problems automatically generates professional market research and analysis reports through the development of key word extraction, key sentence extraction, and sentence writing functions from data collected based on AI, and professional market research and analysis techniques. The purpose is to provide an AI-based market research and analysis system that can provide verification technology to secure the reliability of the generated report by developing a deep learning learning model for this applied AI, and a market research and analysis method using the same do.

In addition, the present invention is AI-based to develop market data collection / classification / analysis technology from the web, SNS, policy data, statistical data, and open data, develop a multi-processing data crawling module interface, and develop a data pre-processing function for collected data. Its purpose is to provide a market research and analysis system and a market research and analysis method using the same.

In addition, the present invention presents the grounds used for generating a report in the form of an appendix when generating a final report on market research, develops a technology that enables data to be attached, and develops an HTML5-based intelligent content publishing technology when generating a final report. Its purpose is to provide an AI-based market research and analysis system and a market research and analysis method using the same.

The present invention, conceived to solve the above problems, is an analysis system that collects market data through the web 400 and analyzes the collected data to create a report on market conditions. Based on the AI agent technology, a data collection unit 102 that collects super-capacity market data existing on the web 400; a data DB (104) for storing the data collected by the data collection unit (102); A data pre-processing module 106 for extracting important words from the data and extracting important words through a frequency keyword algorithm that calculates the frequency of words after extracting them word by word from the document base collected on the web 200; A deep learning learning module 108 that applies an automatic summary technique to the data using an AI-based deep learning learning method, extracts sentences from the collected data, and summarizes documents including the sentences; A report creation module 110 for creating reports necessary for market research and analysis reports based on AI on the data collected based on the intelligent content publishing function;

The data collection unit 102 includes a data crawling module 102a for collecting a plurality of data through the web 400; A data collection module (102b) for automatically extracting key words and sentences based on AI from the data collected through the web (400); A data classification module 102c that executes a key word extraction algorithm so that a user can selectively check through keyword search and efficiently generate a desired market research and analysis report through this; and a data storage module (102d) for storing data on the extracted key words in the data DB (104).

The analysis system allows the AI to automatically generate a report by applying specialized market research and analysis techniques, builds a deep learning learning model to improve the AI performance, and performs verification to secure reliability of the generated report, , presents the basis for market research and analysis reports, and is characterized by implementing large-capacity market data collection technology.

The report writing module 110 extracts key words and sentences by comparing them with structured/atypical market data collected according to user requirements, and generates a market research and analysis report through a combination of the extracted key words and sentences, The AI provides a template-based report of content generated based on professional market research and analysis techniques, and is characterized by generating and applying dynamic content such as HTML5-based charts and graphs when generating a report.

The present invention according to another embodiment is an analysis method using the analysis system of the preceding paragraph, comprising the steps of inputting user requirements including structured/atypical market data related to products and markets; analyzing the requirements input by the user to understand the main goal; Collecting, by the data collection unit 102, market data on a plurality of webs 400 connected through the Internet 200; pre-processing the data collected through the Internet 200 by the data pre-processing module 106 and classifying data directly related to the content of the report; The report writing module 110 extracts words using a natural language processing (NLP) engine or converts text based on understanding of sentences, lists words at a level that can be understood by humans, extracts sentences, and generates content to generate market reports. Writing a; includes.

According to the present invention, there is an effect of solving a situation in which small and medium-sized enterprises are experiencing difficulties in new product and service development, sales, advertising, promotion, marketing, etc. due to a lack of professional planning personnel.

In addition, in the artificial intelligence-related market that is growing explosively at home and abroad, it is effective to secure the ability to respond to the domestic market by providing solutions and services based on its own technology in the domestic market where foreign solutions have entered.

In addition, there is an effect of developing and providing services that can be practically used by small and medium-sized enterprises for artificial intelligence technology and related technologies, which are concentrated in large companies or IT companies.

1 is a conceptual diagram showing the workflow of a platform that provides market research and analysis reports using the analysis system of the present invention.
Figure 2 is a block diagram showing the configuration of the analysis system of the present invention.
Figure 3 is a flow chart showing a process of creating a market research and analysis report using the analysis system.
Figure 4 is a conceptual diagram showing the automatic generation function of market research and analysis reports.
5 is a conceptual diagram illustrating a sentence extraction method using topic modeling for data collected through crawling;

Hereinafter, "AI-based market research and analysis system and market research and analysis method using the same" according to an embodiment of the present invention will be described with reference to the drawings.

Figure 1 is a conceptual diagram showing the workflow of a platform that provides market research and analysis reports using the analysis system of the present invention, Figure 2 is a block diagram showing the configuration of the analysis system of the present invention, Figure 3 is using the analysis system Figure 4 is a conceptual diagram showing the automatic generation function of market research and analysis reports, Figure 5 is a sentence extraction method using topic modeling for data collected through crawling it is a concept

The "AI-based market research and analysis system" (hereinafter referred to as 'analysis system') of the present invention collects and analyzes market data according to user requirements by AI (artificial intelligence), and market trends necessary for users and companies. , Industry trends, product trends, competitor trends, consumer trends, etc. AI learned in accordance with the market research and analysis stage functions to provide a service platform that automatically generates and provides market research and analysis reports required by companies.

A number of analysis systems (100) through the Internet (200) are used for each step in the NPD Process used by small and medium-sized businesses to develop new products and services (1) AI technology to automatically generate reports by applying specialized market research and analysis techniques , (2) deep learning learning model to improve AI performance, (3) verification technology to secure the reliability of generated reports, (4) market research and analysis report presentation technology, (5) mass market data collection technology, etc. run

In addition, the analysis system 100 uses professional market research and analysis techniques (STEEP, PEST (PESTLE), U&A analysis, customer VOC analysis, customer preference analysis, purchase route confirmation, etc. ) is applied, AI is trained using major structured/unstructured data, and reports are created and provided customized to user requirements from data collected through AI with improved performance through repetitive learning and reinforcement learning. In addition, it can provide a service that can be used for new product and service development, sales and marketing activities by providing quality market research and analysis reports that can be immediately used by small and medium-sized enterprises that do not have professional planning and marketing personnel.

The analysis system 100 compares structured/unstructured market data collected according to user requirements, extracts key words, extracts sentences, and develops technology to generate market research and analysis reports through combination of extracted key words and sentences. The learned AI creates and provides reports based on professional market research and analysis techniques, and provides reports that can be used based on templates. And when creating a report, implement the technology to create and apply dynamic content such as HTML5-based charts and graphs.

The analysis system 100 develops and provides a learning model to which AI professional market research and analysis techniques are applied in generating market research and analysis reports, and deep learning for AI for performance improvement and decision making. Design and apply the model. In addition, it provides an iterative learning model through an AI learning data set, and a reinforcement learning model that infers the optimal result by AI decision-making.

The analysis system 100 develops technology for ensuring reliability of market data through verification of market research and analysis reports generated by AI, and applies a cross-verification technique to the generated reports. In addition, it provides a function to improve the level of report production by re-registering the final report generated by applying the over-bias classification technique to the generated report and continuously applying the latest data.

The data collection unit 102 included in the analysis system 100 collects super-capacity market data existing on the web and SNS based on multi-processing crawling technology and distributed AI agent technology and stores them in the data DB 104. Market data can include web and social media crawling, policy data, market news and statistics, open data, document files, and internal company data. The data collection technology of the analysis system 100 may be implemented to provide a document file and system possessed by the user and a RESTFul method interlocking specification in order to provide a market analysis report.

In addition, supporting data on the source of the report generated through AI is tracked, and data such as graphs and images are provided in the form of an appendix. It can be used as basic data to help users make decisions about reports by providing evidence and sources.

There is no limit to the market range in which the analysis system 100 of the present invention can research and analyze data, and consumer manufacturing such as electrical/electronic devices, fashion/clothing/miscellaneous goods, mobile accessories, and household appliances will be the main application fields. will be. It can also be applied to technology-based ventures/startups such as intelligent robot manufacturing, O2O service, blockchain, and biohealth.

In addition, it can be applied to educational service industries such as kindergartens, daycare centers, and specialized academies, and it will also be applicable to financial service industries such as insurance companies and financial institutions.

The report writing module 110 extracts key words and sentences based on AI from the data collected based on the intelligent content publishing function, classifies the importance according to the number of frequencies, and extracts the relationship between words with high relevance, and based on this, market research and Write the report by composing the sentences necessary for the analysis report. In order to prepare a market research and analysis report, understand the requirements according to the user's requirements, extract similar or highly related key words and sentences from the collected data, compose the sentences to meet the user's requirements, and provide each main supporting data Summarizes documents on to generate professional market research and analysis reports.

The data crawling module 102a and the data collection module 102b automatically extract key words and sentences based on AI from the data collected through the web 400.

The data classification module 102c develops and applies a key word extraction algorithm so that a user can selectively check through a keyword search and efficiently generate a desired market research and analysis report.

The data pre-processing module 106 extracts important words from the data collected by the Unsupervised Learning technique without three correct answers, and extracts word by word from the document base collected on the web 200, and then calculates the frequency of words. Frequency keyword algorithm After extracting important words through and calculating weights for important words, the differences in text characteristics between groups or the relationship between tokens are analyzed to select keywords with the top K number of appropriate weights.

In addition, it extracts the relationship between key words and sentences of the collected data by document unit. The purpose of relation extraction is to reduce the ambiguity that may exist in the input information by extracting structured information from unstructured information, and to simplify the process in the process of processing the information to analyze the process more quickly and accurately.

Relationship extraction is largely divided into two types: Global Level Relation Extraction and Mention Level Relation Extraction. In the present invention, global-level relationship extraction is performed, but mention-level relationship extraction In parallel, omission of information is prevented as much as possible to maintain performance and completeness.

In the case of languages in which subjects and objects are frequently omitted, such as Korean, in extracting relationships from data, there is a weakness in that the extracted result does not properly express the relationship between entities corresponding to the omitted subject or object, but the relationship between each entity is stored in an external memory. This weakness was overcome by developing a relation extraction model that extracts the interrelationships between entities expressed over several sentences by storing and analyzing them in the .

The analysis system 100 is composed of a fragmentary relationship extraction model and an external memory neural network, and through repeated training for AI, training of each fragmentary relationship extraction model, memory enhancement neural network training for global relationships, and finally, the result of memory enhancement neural network training. Retraining of the relationship extraction model reflecting

In addition, automatic summary of collected data and similar data search technology were implemented. That is, sentences were extracted from the collected data using K-Means Clustering, an unsupervised learning algorithm, and documents containing the sentences were summarized.

The automatic summary technology for the collected data solves the problem of word mismatch by using the AI-based deep learning method of the deep learning learning module (108) rather than the vector generation using the conventional TF-IDF method when generating sentence vectors. , a clustering-based extraction summary method was used as a summary technique after generating a sentence vector.

Then, by projecting the collected data into a virtual vector space, reducing its dimensionality, and then visualizing it, a technology that can intelligently search for similar documents is applied so that it can be used as a similar data search method for the collected data.

If the collected data is projected onto a virtual vector space, the degree of similarity between data can be quantified using a vector space model, and similar documents can be searched for from this. In order to convert it into a fixed-length vector, it can be used for similar data search using text and sentence embedding technology and document automatic summary technology.

In addition, the projected vector space can be transformed into less than three dimensions, a space that humans can visually perceive, through dimension reduction techniques such as using t-distributed Stochastic Neighbor Embedding (t-SNE). To use it, use TensorBoard.

Although the preferred embodiments of the present invention have been described with reference to the accompanying drawings, the technical configuration of the present invention described above is another specific form without changing the technical spirit or essential features of the present invention by those skilled in the art to which the present invention belongs. It will be understood that it can be implemented as. Therefore, the embodiments described above should be understood as illustrative and not restrictive in all respects, and the scope of the present invention is indicated by the claims to be described later rather than the detailed description, and the meaning and scope of the claims and All changes or modified forms derived from the equivalent concept should be construed as being included in the scope of the present invention.

100: analysis system 200: Internet
300: user terminal 400: web

Claims (5)

As an analysis system that collects data on the market through the web 400 and analyzes the collected data to create a report on the market situation,
Based on multi-processing crawling technology and distributed AI agent technology, a data collection unit 102 that collects super-large market data existing on the web 400;
a data DB (104) for storing the data collected by the data collection unit (102);
A data pre-processing module 106 for extracting important words from the data and extracting important words through a frequency keyword algorithm that calculates the frequency of words after extracting them word by word from the document base collected on the web 200;
A deep learning learning module 108 that applies an automatic summary technique to the data using an AI-based deep learning learning method, extracts sentences from the collected data, and summarizes documents including the sentences;
An AI-based market research and analysis system including a report creation module (110) that creates reports required for AI-based market research and analysis reports on the data collected based on the intelligent content publishing function. According to claim 1,
The data collection unit 102
a data crawling module (102a) for collecting a plurality of data through the web (400);
A data collection module (102b) for automatically extracting key words and sentences based on AI from the data collected through the web (400);
A data classification module 102c that executes a key word extraction algorithm so that a user can selectively check through keyword search and efficiently generate a desired market research and analysis report through this;
A data storage module (102d) for storing the data for the extracted key words in the data DB (104); including, AI-based market research and analysis system. According to claim 1,
The analysis system allows the AI to automatically generate a report by applying specialized market research and analysis techniques, builds a deep learning learning model to improve the AI performance, and performs verification to secure reliability of the generated report, , AI-based market research and analysis system, which presents the basis for market research and analysis reports and implements large-capacity market data collection technology. According to claim 1,
The report writing module 110
Key words and sentences are extracted by comparing with structured/atypical market data collected according to user requirements, and market research and analysis reports are generated through the combination of the extracted key words and sentences. An AI-based market research and analysis system that provides reports based on templates for content generated based on analysis techniques, and generates and applies dynamic content such as HTML5-based charts and graphs when generating reports. An analysis method using the analysis system of any one of claims 1 to 4,
inputting user requirements including structured/unstructured market data related to products and markets;
analyzing the requirements input by the user to understand the main goal;
Collecting, by the data collection unit 102, market data on a plurality of webs 400 connected through the Internet 200;
pre-processing the data collected through the Internet 200 by the data pre-processing module 106 and classifying data directly related to the content of the report;
The report writing module 110 extracts words using a natural language processing (NLP) engine or converts text based on understanding of sentences, lists words at a level that can be understood by humans, extracts sentences, and generates content to generate market reports. Comprising; AI-based market research and analysis method.

Images