KR20220111219A - Analytical methods of systems for setting data processing cycles based on growth rate of data in real time - Google Patents

Abstract

Disclosed is an analysis method of an in-memory computing-based system. The analysis method includes the steps of: dividing data acquired in real time into a plurality of content groups according to time and storing the data in a memory; acquiring a plurality of types of analysis content by pre-processing each of the plurality of content groups according to at least one index keyword; if at least one search word and condition are input, performing a search for the plurality of content groups and identifying at least one type of content by a plurality of cores; and performing analysis using at least one type of analysis content matching the identified content.

Description

An analysis method of a system that sets the data processing cycle according to the data growth rate in real time

The present disclosure relates to an analysis method of a system for setting a data processing cycle according to an increase rate of data in real time, and more particularly, to a system for improving analysis speed by constructing data in real time according to in-memory computing. .

In-memory computing methods have been devised to process real-time and/or large-capacity big data.

In the in-memory computing method, most of the data is stored on the hard disk, breaking away from the existing method in which a lot of time was consumed for input/output of used result data, and data is directly stored in the memory composed of RAM so that the core can quickly retrieve the data. The way it is processed/processed.

In the case of the in-memory computing method, a plurality of cores efficiently divide the load to process data in the memory in parallel, thereby increasing the processing speed.

Patent Publication No. 10-2019-0005578 (2019.01.16)

The present disclosure provides a system and an analysis method for improving the speed of search and analysis operations by pre-classifying and pre-processing data in memory based on in-memory computing.

Objects of the present disclosure are not limited to the above-mentioned purposes, and other objects and advantages of the present disclosure that are not mentioned may be understood by the following description, and will be more clearly understood by examples of the present disclosure. Moreover, it will be readily apparent that the objects and advantages of the present disclosure may be realized by the means and combinations thereof indicated in the claims.

An analysis method of an in-memory computing-based system according to an embodiment of the present disclosure includes dividing data obtained in real time into a plurality of content groups according to time and storing the data in a memory; Pre-processing according to the index keyword to obtain a plurality of analyzed contents, and when at least one search word and condition are input, a plurality of cores perform a search in a parallel processing manner on the plurality of content groups to identify at least one content and performing analysis using at least one analysis content matching the identified content.

The acquiring of the plurality of analyzed contents may include, based on at least one of an appearance frequency of the index keyword, an appearance order of the index keyword, classification information according to the index keyword, and pattern information based on the index keyword, You can preprocess each of the content groups of .

And, in the step of identifying the at least one content, when a data size of at least one content group among the plurality of content groups is larger than a data size preset according to the search performance of each of the plurality of cores, the content group is selected The search can be performed in a parallel processing manner by partitioning.

On the other hand, the analysis method of the system, when a plurality of contents are identified according to a search performed through one core, dividing the plurality of contents into a preset number unit to generate a plurality of contents list can In this case, in the performing of the analysis, a plurality of cores may analyze the generated plurality of content lists in a parallel processing manner.

In this case, the analysis method of the system may further include merging the analysis results for each of the plurality of content lists.

On the other hand, the analysis method of the system includes the steps of storing backup information for the plurality of content groups and the plurality of analyzed content in at least one storage, and when the system is booted, based on the backup information, the plurality of The method may further include loading a content group and the plurality of analyzed content into the memory.

The acquiring of the plurality of analysis contents includes: based on a period set according to the increase rate of the data acquired in real time, the number of cores included in the system, and the performance of each core, the data is divided into a plurality of Each of the content groups may be pre-processed according to at least one index keyword to obtain a plurality of analyzed content.

On the other hand, the analysis method of the system includes the steps of identifying at least one time section in which a search is performed through the system in which a search word and a condition are input, and based on the identified time section, The method may further include generating a pattern and selecting at least one time interval in which a search is most likely to be performed based on the generated pattern. In this case, the obtaining of the plurality of analysis contents may include preprocessing each of the plurality of contents groups in which the data is divided according to at least one index keyword during the selected time period to obtain the plurality of analysis contents.

An in-memory computing based system according to an embodiment of the present disclosure includes at least one memory and at least one processor connected to the at least one memory. The processor divides the data obtained in real time into a plurality of content groups according to time, stores them in the memory, and pre-processes each of the plurality of content groups according to at least one index keyword to obtain a plurality of analyzed content, When at least one search word and condition are input, a search is performed on the plurality of content groups in a parallel processing manner through a plurality of cores to identify at least one content, and at least one analysis content matching the identified content to perform the analysis.

In the in-memory computing-based system and analysis method according to the present disclosure, by classifying and pre-processing data obtained in real time to match the parallel processing method and storing it in the memory, there is an effect that the search and analysis process of the parallel processing method can be accelerated .

1 is a diagram for schematically explaining the operation of a system according to an embodiment of the present disclosure;
2 is a block diagram for explaining the configuration of a system according to an embodiment of the present disclosure;
3 is an algorithm for explaining the operation of the system according to an embodiment of the present disclosure;
4 is a diagram for explaining an operation of a system classifying and pre-processing data according to an embodiment of the present disclosure;
5 is a diagram for explaining an operation of a system performing a search in a parallel processing manner through a plurality of cores according to an embodiment of the present disclosure;
6 is a view for explaining an operation in which the system performs analysis in a parallel processing manner through a plurality of cores according to an embodiment of the present disclosure; and
7 is a block diagram illustrating a functional configuration of a system according to an embodiment of the present disclosure.

Before describing the present disclosure in detail, a description will be given of the description of the present specification and drawings.

First, terms used in the present specification and claims have been selected in consideration of functions in various embodiments of the present disclosure. However, these terms may vary depending on the intention or legal or technical interpretation of a person skilled in the art, and the emergence of new technology. Also, some terms are arbitrarily selected by the applicant. These terms may be interpreted in the meaning defined in the present specification, and if there is no specific term definition, it may be interpreted based on the general content of the present specification and common technical knowledge in the art.

Also, the same reference numerals or reference numerals in each drawing attached to this specification indicate parts or components that perform substantially the same functions. For convenience of description and understanding, the same reference numerals or reference numerals are used in different embodiments. That is, even though all components having the same reference number are illustrated in a plurality of drawings, the plurality of drawings do not mean one embodiment.

In addition, in this specification and claims, terms including an ordinal number such as “first” and “second” may be used to distinguish between elements. This ordinal number is used to distinguish the same or similar elements from each other, and the meaning of the term should not be construed as limited due to the use of the ordinal number. For example, components combined with such an ordinal number should not be limited in the order of use or arrangement by the number. If necessary, each ordinal number may be used interchangeably.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It is to be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment of the present disclosure, terms such as “module”, “unit”, “part”, etc. are terms for designating a component that performs at least one function or operation, and such component is hardware or software. It may be implemented or implemented as a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except when each needs to be implemented as individual specific hardware, and thus at least one processor. can be implemented as

In addition, in an embodiment of the present disclosure, when a part is connected to another part, this includes not only direct connection but also indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that other components may be further included without excluding other components unless otherwise stated.

1 is a diagram schematically illustrating an operation of a system according to an embodiment of the present disclosure.

Referring to FIG. 1 , the system according to the present disclosure may acquire various external data.

Specifically, the system may acquire various data such as SNS content, news content, public data, and other texts through various types of wired/wireless networks.

In addition, the system can build large-capacity big data that is expanded in real time by collecting the data in real time.

And, while the system stores data in the memory, the system may perform various processes such as preprocessing, classifying, and indexing the corresponding data using the memory. As the data is acquired in real time, the corresponding processing may also be performed in real time (or according to a specific period/condition).

Here, the system may store data on which processing such as pre-processing, classification, or indexing has been performed, in the memory.

In addition, the system may perform a search by treating the corresponding data in a parallel processing method. Specifically, the system may perform a search using each core for each of a plurality of sub data into which data is divided.

In this case, the system may use various search technologies such as keyword-based search, classification-based search, and pattern-based search.

In addition, the system may perform various analyzes on the content obtained as a result of the search. To this end, the system may use modules that provide various functions/services.

At this time, the cores executing each of the modules may analyze the corresponding content in a parallel processing manner, and the results of the search or analysis may be shared between the modules. In this case, the analysis results may be stored in the system's memory.

And, the system may provide search results and/or analysis results. In this case, the search result and/or analysis result may be provided through an output device connected to the system, or the search result and/or analysis result may be provided through at least one user terminal included in a network to which the system is connected.

With reference to the drawings below, the configuration and operation of the system according to the present disclosure will be described in more detail.

2 is a block diagram illustrating the configuration of a system according to an embodiment of the present disclosure.

Referring to FIG. 2 , the system 100 includes at least one memory 110 and at least one processor 120 .

The system 100 may include one or more electronic devices or computers, and for example, may be implemented as a plurality of server devices that provide a search or analysis function (Application, Program, etc.), but is not limited thereto.

The memory 110 is a configuration for storing various data. In the system 100 of the present disclosure using the in-memory computing method, the memory 110 may be configured as one or more random access memory (RAM).

Specifically, the memory 110 may be implemented as a volatile memory such as a dynamic RAM (DRAM) or a static RAM (SRAM). However, the memory 110 may include a dynamic RAM (MRAM), a phase-change RAM (PRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), or other flash memory.

The processor 120 is a configuration for controlling the overall operation of the system 100 , and may be connected to the memory 110 to perform various functions.

Specifically, the processor 120 may perform a search and analysis method to be described later by executing at least one instruction stored in the memory 110 .

The processor 120 may include a general-purpose processor such as a CPU, an AP, or a digital signal processor (DSP), a graphics-only processor such as a GPU, a vision processing unit (VPU), or the like, or an artificial intelligence-only processor such as an NPU. In addition, the processor 120 may include SRAM.

The processor 120 may include a plurality of cores. Each core may perform the same function in parallel or may perform different functions.

Specifically, the processor 120 may classify and pre-process various types of data stored in the memory 110 using a plurality of cores, and perform various functions (eg, search, analysis, etc.) using the classified and pre-processed data. can be performed.

3 is an algorithm for explaining the operation of the system according to an embodiment of the present disclosure.

Referring to FIG. 3 , the system 100 may classify data acquired in real time into a plurality of content groups according to time and store the data in the memory 110 ( S310 ).

Specifically, the system 100 may define a plurality of content groups by classifying data (including various contents) stored in the memory 110 based on items such as date and Media ID.

In particular, when various (contents) data is received from the outside in real time, the system 100 may store a plurality of content groups in which the received data is divided by date in the memory 110 . In this case, the content data classified by date may be stored in the memory 110 in the form of a queue, but in addition, various forms such as a stack, a linked list, an array, and a tree are possible.

As a result, the content data may be built in the memory 110 in a structure of a plurality of content groups in a form that is easy to search in the future.

Then, the system 100 may obtain a plurality of analyzed content by preprocessing each of the plurality of content groups according to at least one index keyword ( S320 ).

The index keyword may be a criterion for indexing each content group.

As a specific example, the system 100 preprocesses each of the plurality of content groups based on at least one of the frequency of appearance of the index keyword, the order of appearance of one or more index keywords, classification information according to the index keyword, and pattern information based on the index keyword. can do. Also, the system 100 may pre-process each content group based on an ID or a target matching the index keyword.

As a result, a plurality of analysis contents included in each content group may be defined and stored in the memory 110 .

The analysis content includes content including specific index keywords, content including specific index keywords in a specific order, content classified according to index keywords classified according to a specific criterion, content including specific index keywords in a preset pattern, and specific It may be defined in various ways, such as an ID matching an index keyword or content generated by a user.

In addition, the analysis content may include statistical analysis-related data (numerical values, indicators, classifications, etc.) calculated for a specific content group based on a specific index keyword.

In this way, the content data may be built in the memory 110 in the form of analysis contents that are easy for various analysis such as statistics.

4 is a diagram for explaining an operation of classifying and preprocessing data by a system according to an embodiment of the present disclosure.

Referring to FIG. 4 , the system 100 classifies data based on date and Media ID to identify a plurality of content groups 410, 420, 430, ..., and stores information about the identified plurality of content groups in memory. (110) can be stored.

In addition, the system 100 may perform pre-processing for each content group 410 , 420 , 430 , ... .

Specifically, referring to FIG. 4 , the system 100 performs pre-processing/classification based on various index keywords (environment, corona, fund, IBK, etc.) for the content group 410, thereby performing a plurality of analysis contents ( 411, 412, 413, 414, ...) can be obtained.

For example, in the case of the analyzed content 411 according to the “environment” keyword, information on the frequency of appearance of “environment” in the content group 410 and the content group 410 sequentially listed according to the frequency of appearance of “environment” It includes information about my plurality of contents, information on the appearance pattern of the keyword “environment” in the content group 410, and information on ID/users who use the word “environment” in the content group 410 with high frequency. can, but is not limited thereto.

In a state in which data in the memory 110 is constructed to facilitate search and analysis according to the above-described embodiments, the system 100 may perform search and analysis based on the constructed data.

Specifically, referring to FIG. 3 , when at least one search word and condition are input, a plurality of cores in the system 100 searches for a plurality of content groups (eg. 410, 420, 430, ...) in a parallel processing manner. may be performed to identify at least one content (S330).

Here, the conditions input together with the search term include a condition that includes or not include at least one keyword, a search target time (period), a stop word, and when the search term is a compound word, the words included in the compound word are at a distance of less than a certain number of words from each other. It may correspond to a variety of conditions, such as adjacent conditions.

And, according to an embodiment, each of the plurality of cores in the system 100 may perform a search for each of the plurality of content groups (parallel processing).

If a plurality of processors and/or a plurality of servers are included in the system 100 , the cores included in the plurality of processors/servers may perform a search for each of a plurality of content groups in a parallel processing manner.

As a result of performing a search for a plurality of content groups through the parallel processing method, the search speed may be increased, and the core usage efficiency may be increased. In addition, when data is aggregated or duplicated, simultaneous access is impossible, whereas a search for each of a plurality of divided content groups is performed, so that the synchronization period is minimized, and parallel processing efficiency can be guaranteed.

However, when the data size of at least one content group among the plurality of content groups is larger than the data size preset according to the search performance of each of the plurality of cores, the system 100 divides the at least one content group in a parallel processing method. You can also do a search. In this case, parallel processing performance of the cores may be maintained without abnormality.

5 is a diagram for explaining an operation in which a system performs a search in a parallel processing method through a plurality of cores according to an embodiment of the present disclosure.

Referring to FIG. 5 , each core (thread) may perform a search for one content group (shard contents group).

Here, each content group may be one of the plurality of content groups (ex. 410, 420, 430, ...) divided above, and a portion of one of the plurality of content groups (when the size of the content group is large) It can also be a content group.

Referring to FIG. 5 , each core performs a search on another content group (shard contents group) after a search for one content group (shard contents group) is completed, and this process is applied to all contents group (shard contents group). It can be repeated until performed for

Each core may identify at least one piece of content by performing a search. Here, each core can remove duplicate results.

When a plurality of contents are identified according to a search performed through one core, the core may generate a plurality of contents list by dividing the plurality of contents into a unit of a predetermined number.

As a specific example, each core may generate a search result in the form of List<Contents[]>. In this case, the size of the generated search result may be set to be less than or equal to a predetermined number of contents.

In the analysis process, when Contents[] is retrieved from List<Contents[]>, synchronization occurs. To minimize this section, the contents are loaded in an array form (if one is imported, a synchronization section occurs for each section to be imported). Therefore, the speed is greatly affected).

As such, as a result of generating the search result in the form of a List rather than a Queue, in a later analysis process ( S340 ), the plurality of cores may analyze the plurality of content lists in a parallel processing manner.

And, when the search is performed as in the above-described embodiment, the system 100 may perform the analysis using at least one analysis content matching the identified content (according to the search) (S340).

As an embodiment, each core in the system 100 may analyze each of a plurality of content lists that are search results.

Specifically, each core may analyze each of the plurality of content lists in a parallel processing manner using the above-described analyzed content (: preprocessed data) that matches each of the plurality of content lists.

As such, since the preprocessed data structure (analysis content) in the memory 110 is processed only by the operation between the processor 120 and the memory 110, processes such as memory data exchange and data conversion (ex. JSON-> Memory Object) are performed. This is not necessary, and unlike the prior art, the processing speed of the analysis process can be increased.

FIG. 6 is a diagram for explaining an operation in which a system performs analysis in a parallel processing method through a plurality of cores according to an embodiment of the present disclosure.

Referring to FIG. 6 , each core (thread) can quickly and parallelly process analysis (eg, statistical analysis) using preprocessing contents.

Here, the pre-processed content may be at least a portion of the analyzed content matched to the content list identified as a result of the search.

As such, the data is pre-classified and pre-processed, built and maintained or updated in the memory 110, so that a plurality of cores in the system 100 can quickly perform a search and analysis of data in a parallel processing manner. can be guaranteed.

Meanwhile, the analysis contents of each core may be analysis according to the same function/module or analysis according to different functions/modules.

As an embodiment, when a plurality of cores perform analysis according to the same module/function, the system 100 may merge the analysis results of the plurality of cores for each of the plurality of content lists.

As an example, it is assumed that the first to fourth statistical data are calculated according to the analysis performed through each of the first to fourth cores.

In this case, the processor 120 is configured to obtain first merge statistical data by merging the first statistical data and the second statistical data, and obtain the second merge statistical data by merging the third statistical data and the fourth statistical data. can

In addition, the processor 120 may calculate final statistical data by merging the first merged statistical data and the second merged statistical data.

Meanwhile, according to an embodiment, the system 100 may perform the above-described processes S310 to S320 according to a predetermined period.

That is, as external data is added in real time, the system 100 may periodically perform classification or pre-processing of the corresponding data to build a data environment in the memory 110 .

As a specific example, the system 100 is configured for each of a plurality of content groups in which data is divided based on a period set according to the increase rate of data acquired in real time, the number of cores included in the system 100, and the performance of each core. may be pre-processed according to at least one index keyword to obtain a plurality of analysis contents.

As an example, the speed at which the system 100 classifies/pre-processes data to be added in real time suitable for search and analysis (S310 to S320) should be faster than the rate of increase of data added in real time. In this case, a result obtained by dividing a value obtained by multiplying the data processing speed of each core by the number of cores by a period may be faster than an increase rate of data acquired in real time.

In addition, as an embodiment, the system 100 may perform the above-described classification and pre-processing (S310 to S320) during a time period in which the search (S330) and the analysis (S340), which will be described later, are not expected to be performed.

As a specific example, the system 100 may identify at least one time period in which a search was performed through the system 100 by inputting a search word and a condition. Then, based on the identified time interval, the system 100 may generate a pattern (eg, average start/end time of the time interval) for the time interval in which the search is performed.

In this case, based on the generated pattern, the system 100 may select at least one time interval in which a search is most likely to be performed.

Then, the system 100 may perform processes S310 to S320 for the added/updated data during the selected time period. Specifically, the system 100 may obtain a plurality of analyzed content by preprocessing each of a plurality of content groups in which data is divided according to at least one index keyword.

For example, when the search and analysis according to the request of the server administrator or user is mainly performed between 8:00 am and 7:00 pm, the system 100 performs the above-described processes S310 to S320 from 11:00 pm to 2:00 am the next day. can be performed

In this case, pre-processing may be performed on external data (eg, content group) that is additionally acquired/collected every day and built on the memory 110 , and as a result, the speed of daily search and analysis can be guaranteed. .

Meanwhile, according to an embodiment, the system 100 may store, in at least one storage, backup information for a plurality of content groups and a plurality of analyzed content obtained through processes S310 to S320.

The storage may include various configurations such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory, but is not limited thereto.

And, when the system 100 is booted (eg, rebooted after termination), the system 100 may load a plurality of content groups and a plurality of analysis content into the memory 110 based on the backup information. In this case, the system 100 may load a plurality of content groups and a plurality of analyzed content in a parallel processing method through a plurality of cores.

Meanwhile, the system 100 may store the backup information on at least one non-volatile memory included in the memory 110 . In this case, the rebooted system 100 may load a plurality of content groups and a plurality of analyzed content onto the volatile memory of the memory 110 by using the backup information stored in the non-volatile memory.

On the other hand, in one embodiment, the system 100 uses the data for the search term used in the search of step S330 (eg, the frequency of each search term, the search term listing according to the search frequency), the memory building process (S320) performed later can be done

As a specific example, the system 100 may obtain a plurality of analysis contents by setting each search word as a sequential index keyword according to the input frequency, and performing the pre-processing of step S320 using each set index keyword.

In addition, as an embodiment, the system 100 may use at least one (input) condition used for the search in step S330 in a pre-processing process ( S320 ) according to the index keyword later.

Meanwhile, FIG. 7 is a block diagram illustrating a functional configuration of a system according to an embodiment of the present disclosure.

Referring to FIG. 7 , the system 100 may include various modules for performing analysis of various forms/functions. Once retrieved, the results can be shared and used by multiple modules.

Specifically, the system 100 may include a Classify Task Force (TF) for performing various classification-related processing, a Word TF for performing word-related analysis, a Word Term Frequency-Inverse Document Frequency (TFIDF), and the like. System 100 may also include Media TFs related to the creation/processing of media. Also, the system 100 may include a Topic Modeling module (eg, Latent Dirichlet Allocation (LDA)) for performing language processing for discovering topics such as word detection and word pattern detection for various documents. In addition, the system 100 may include a Social Network Analysis (SNA) module for acquiring various external data.

Each of the above-described modules may be implemented through software and/or hardware, and may be executed through at least one core included in the processor 120 .

Also, referring to FIG. 7 , the system 100 may include a Natural Language Processing (NLP) module for natural language processing and other modules for performing various real-time data processing.

Specifically, the system 100 includes at least one module for collecting media content from the outside in real time, a module for changing/adding/deleting data in the memory 110 in real time using the collected content, and adding/modified It may include a module that performs indexing (pre-processing) in real time using the index keyword, and the like.

As an example, the system 100 may add/change index keywords in real time based on removal of stopwords, management of various words, learning of new words, and the like.

In addition, the system 100 may perform pre-processing for each of the plurality of content groups by using the added/changed index keyword. As a result, a plurality of analysis contents built in the memory 110 may be updated in real time according to changes in times and languages.

Meanwhile, Table 1 below is a comparison table of search and analysis times for explaining the effects of an embodiment according to the present disclosure.

standard Search Results
ratio in-memory
Unapplied Elastic Search + Spark Spark RDD + Spark SQL Search and analysis according to the present disclosure time taken
(candle) 5% 72.2 4.76 2.5+α 0.65 11% 258 6.5 3.5+α 1.2 15% 314 8.1 5.5+α 1.45 20% 404 9.3 6.5+α 1.8

Table 1 calculates the search and analysis time of each example for the same 1.7 million data. In particular, SparkRdd + SQL is a value that does not calculate the query time of SQL, and it can be seen that it takes longer time in reality. It shows a performance difference of 10 to 100 times or more until the time to transmit the analysis. And, in the search and analysis according to an embodiment of the present disclosure, compared with Spark, which receives data through a search engine (ex. Elastic Search), 4 More than double the performance difference.

Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described in the present disclosure are ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays) ), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing other functions may be implemented using at least one.

In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the above-described software modules may perform one or more functions and operations described herein.

On the other hand, computer instructions or computer programs for performing a processing operation in at least one electronic device in the system 100 according to various embodiments of the present disclosure described above are non-transitory computer readable media (non-transitory). may be stored in a computer-readable medium). When the computer instructions or computer program stored in such a non-transitory computer-readable medium are executed by the processor of the specific device, the specific device performs the processing operation in the system 100 according to various embodiments described above.

The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field pertaining to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

100: system 110: memory
120: processor

Claims (1)

In the system analysis method,
storing the data acquired in real time in a memory as a plurality of content groups classified according to the acquired date;
obtaining a plurality of analyzed content by preprocessing each of the plurality of content groups according to at least one index keyword;
identifying at least one content by performing, by a plurality of cores, a parallel processing method on the plurality of content groups when at least one search word and condition are input; and
performing statistical analysis on the identified content using at least one analyzed content matching the identified content;
The step of obtaining the plurality of analysis contents includes:
Pre-processing each of the plurality of content groups based on at least one of the frequency of appearance of the index keyword, the order of appearance of the index keyword, classification information according to the index keyword, and pattern information based on the index keyword,
Based on a period set according to the rate of increase of the data acquired in real time, the number of cores included in the system, and the performance of each core, each of the plurality of content groups in which the data is divided according to at least one index keyword Pre-processing to obtain a plurality of analysis contents,
The analysis method of the system is,
When the system performs at least one of classification and pre-processing of data added in real time, a result obtained by dividing a value obtained by multiplying the data processing speed of each core by the number of cores included in the system by a period set according to the performance of each core to exceed the rate of increase of the data acquired in real time, the analysis method of the system.

Images