KR20220090358A - Device and method for analyzing and visualizing big data - Google Patents

Abstract

The present invention relates to a big data analysis visualization apparatus and method, and more particularly, to a big data analysis visualization apparatus and method for analyzing big data with various analysis models in a GUI-based block connection method and providing visualization information suitable for the results. . According to an embodiment of the present invention, a big data analysis visualization apparatus and method can analyze big data using various analysis models without program coding.

Description

Big data analysis visualization device and method {DEVICE AND METHOD FOR ANALYZING AND VISUALIZING BIG DATA}

The present invention is a big data analysis visualization apparatus and method, and more specifically, a big data analysis visualization apparatus and method that analyzes big data into various analysis models without coding in a cloud-based block connection method and provides visualization information suitable for the results. is about

Big data is data generated in a digital environment. As cloud services are commercialized due to the development of information and communication technologies, interest in technologies for efficiently processing large amounts of big data is increasing. In particular, thanks to the revival of the Internet of Things, an unimaginably large amount and various types of data are being generated every moment.

Such big data needs to be processed through a new algorithm or paradigm that is different from the conventional data processing method, and various values can be created through big data through processing and analysis processes that meet the needs of consumers.

Recently, with the advent of high-performance portable devices such as tablets and smart phones in addition to PCs, the number of people who enjoy mobile shopping, search, and e-mail checking through mobile connections as well as Internet access through desktop PCs has increased significantly. are doing With the spread of these portable devices and the development of mobile Internet technology, a lot of data existing on the Internet is being collected through web robots, web crawlers, spiders, etc., and the collected big data is analyzed and used according to a desired purpose.

Existing data analysis systems analyzed big data based on data analysis codes written using programming languages such as scala and python. In other words, users who have learned a programming language such as Scala or Python can write data analysis codes, but users who have not learned the programming language have difficulty writing data analysis codes, so the data analysis flow written by other users can be intuitive. As a result, there are difficulties in understanding and maintenance becomes difficult.

Accordingly, even if a user has not learned a programming language such as Python or Scala, data analysis technology that helps them easily write a data analysis flow to control or modify a specific data file is required.

Background art of the present invention is disclosed in Korean Patent Laid-Open Publication No. 10-2013-0155808.

The present invention provides a big data analysis visualization apparatus and method for analyzing big data into various analysis models without program coding in a cloud-based block connection method.

The present invention provides a big data analysis visualization apparatus and method that can easily create a workflow by connecting functional blocks in an appropriate position and order.

The present invention provides a big data analysis visualization apparatus and method for providing a big data analysis workflow or analysis scenario that can be reused for the same type of big data analysis as a template.

The present invention provides a big data analysis visualization apparatus and method capable of executing and checking each process in block units based on a GUI.

The present invention provides a big data analysis visualization apparatus and method for recommending and expressing a suitable visualization method for various analysis results.

The present invention provides a big data analysis visualization apparatus and method for confirming errors in input data and performing an analysis model by recommending an alternative value.

The present invention provides a big data analysis visualization apparatus and method for performing an analysis model with optimal performance by recommending hyperparameters.

The present invention provides a big data analysis visualization apparatus and method for improving the efficiency and accuracy of an analysis process by recommending a functional block for the next step when creating a workflow for setting steps for big data analysis.

The present invention provides a big data analysis visualization apparatus and method for improving the efficiency of an analysis process and the accuracy of analysis by recommending an analysis model, workflow, or template suitable for a dataset.

The present invention provides a big data analysis visualization apparatus and method for arranging recommended blocks by analyzing an input dataset or a selected functional block, and completing a workflow.

According to one aspect of the present invention, there is provided a big data analysis visualization apparatus.

A big data analysis visualization apparatus according to an embodiment of the present invention includes a collection unit for collecting data to be analyzed, a preprocessing unit for preprocessing data to fit an analysis model, an analysis unit for analyzing data into an analysis model, and performing the analysis model. It may include a visualization unit that visualizes the result as a suitable graph and an execution unit that performs data collection, pre-processing, analysis, and visualization processes by matching them with functional blocks.

According to another aspect of the present invention, there is provided a computer-readable recording medium in which a big data analysis visualization method and a computer program executing the same are recorded.

The big data analysis visualization method according to an embodiment of the present invention includes the steps of collecting data to be analyzed, preprocessing the data to fit the analysis model, analyzing the data using the analysis model, visualizing the analyzed data, and It may include performing the steps of collecting, pre-processing, analyzing, and visualizing data by matching them with functional blocks.

According to an embodiment of the present invention, big data can be analyzed using various analysis models without program coding in a cloud-based block connection method.

According to an embodiment of the present invention, it is possible to easily create an analysis workflow by connecting the recommended blocks in an appropriate position and order.

According to an embodiment of the present invention, big data can be easily analyzed by providing a big data analysis workflow or analysis scenario that can be reused for analysis of the same type of big data as a template.

According to an embodiment of the present invention, each process can be executed and confirmed in block units based on the GUI.

According to an embodiment of the present invention, a suitable visualization method for various analysis results can be recommended and expressed.

According to an embodiment of the present invention, an analysis model may be performed by checking an error in input data and recommending an alternative value.

According to an embodiment of the present invention, it is possible to perform an analysis model with optimal performance by recommending hyperparameters.

According to an embodiment of the present invention, it is possible to improve the efficiency of the analysis process and the accuracy of the analysis by recommending an analysis model, workflow, or template suitable for a dataset.

According to an embodiment of the present invention, it is possible to improve the efficiency of the analysis process and the accuracy of the analysis by recommending the functional block of the next step when creating a workflow for setting the steps for big data analysis.

According to an embodiment of the present invention, it is possible to analyze the work progress stage and complete the workflow through appropriate recommendation and arrangement of a plurality of blocks.

1 to 20 are diagrams for explaining a big data analysis visualization apparatus according to an embodiment of the present invention.
21 to 31 are diagrams for explaining a big data analysis visualization method according to an embodiment of the present invention.
32 to 36 are exemplary screens of a big data analysis visualization apparatus according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail through detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Also, the expressions "a" and "a", "a" and "a", as used in this specification and claims, should generally be construed to mean "one or more" unless stated otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. do it with

1 to 20 are diagrams for explaining a big data analysis and visualization apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the big data analysis and visualization apparatus 10 can easily use data in various formats by dragging and dropping it with a mouse without a special change process. For example, the big data analysis visualization apparatus 10 may collect data to be analyzed by connecting an Excel file format, a CSV file format, an RDS file format, a TXT file format, and a database by drag and drop. In addition, the big data analysis visualization apparatus 10 may directly collect data using the provided OpenAPI.

The big data analysis visualization apparatus 10 performs pre-processing of the collected data in various formats. The big data analysis visualization apparatus 10 performs various data preprocessing in a required data format according to the analysis model. The big data analysis visualization device 10 easily processes the pre-processing process that occupies 70% of the big data analysis tasks such as column name change, type setting, dataset merging, sampling, and group by operation (group-by operation) with a web page manipulation method. It can be saved as a file. For example, the big data analysis visualization device 10 may store in various data formats, such as a CSV file format, an xml file format, a yml file format, a json file format, a txt file format, a log file format, or an input data format.

The big data analysis visualization apparatus 10 may visualize the analysis data through various graphs. The big data analysis visualization apparatus 10 may visualize the analysis data in the form of time, distribution, relationship, comparison, space, or the like. For example, the big data analysis visualization apparatus 10 may visualize the analysis data in various forms using various graphs, such as a line, circle, bar, histogram, bubble chart, scatter plot, box plot, and word cloud.

The big data analysis visualization apparatus 10 may provide the visualization result in a file format. For example, the big data analysis visualization apparatus 10 may provide the analysis result in various formats, such as a web page, PDF, MS-Word, and CSV format.

The big data analysis visualization apparatus 10 may analyze the preprocessed data using various analysis models. For example, the big data analysis visualization apparatus 10 is a supervised learning or association analysis such as correlation analysis, regression analysis, decision tree, kNN, random forest, MLP, etc., unsupervised learning such as K-means, PCA, hierarchical clustering Various analysis models such as R or Python can be applied and analyzed without writing program codes such as R or Python.

The big data analysis visualization apparatus 10 creates a workflow by matching functions with blocks. The big data analysis visualization apparatus 10 adds desired functional blocks to the workflow by clicking or dragging and dropping, and creates an analysis and visualization workflow by connecting them. The big data analysis visualization device 10 can easily create a workflow by dragging and dropping blocks and connecting the data collection, pre-processing, analysis and visualization processes based on GUI. The big data analysis visualization apparatus 10 may perform a workflow for each block step.

The big data analysis visualization apparatus 10 may reuse the generated workflow for homogeneous analysis.

The big data analysis visualization apparatus 10 provides a frequently used analysis scenario as a template to perform big data analysis without writing program code.

The big data analysis and visualization apparatus 10 may be performed in a cloud platform environment.

Referring to FIG. 2 , the big data analysis and visualization apparatus 10 includes a collection unit 100 , a preprocessor 200 , an analysis unit 300 , and a visualization unit 400 .

Referring to FIG. 3 , the collection unit 100 may collect data to be analyzed from various types of data. For example, the collection unit 100 may upload and collect files in Excel, TEXT, and CSV format. In addition, the collection unit 100 may collect data to be analyzed by directly connecting to the relational database. The collection unit 100 may directly input or paste data to directly generate the data. The collection unit 100 may directly collect data using the provided OpenAPI.

Referring to FIG. 4 , the preprocessor 200 preprocesses the collected data in various ways to fit the analysis model. The preprocessor 200 may perform preprocessing suitable for the input data set to increase the accuracy of analysis. The preprocessor 200 may be recommended a preprocessing method according to the analysis of the input data set.

Referring to FIG. 5 , the big data analysis visualization apparatus 10 analyzes data using an analysis model. The analysis unit 300 performs various analysis models without coding. The analysis unit 300 may receive a recommendation for an analysis model according to the input data set.

Referring to FIG. 6 , the visualization unit 400 visualizes analysis data using various visualization graphs. The visualization unit 400 may be executed before or after data analysis.

The visualization unit 400 may be mainly used before learning the data analysis model when checking the form of data. For example, the visualization unit 400 is a histogram visualization function block that can check the change in time series data, a scatterplot or heat map visualization function block that can check the correlation between variables, a word cloud visualization function block that can check the data frequency, A boxplot visualization block that can check outliers in the input data column can be used before training the analysis model.

Referring to FIG. 7 , the visualization unit 400 may express an analysis result report using visualization function blocks after learning the analysis model. For example, the visualization unit 400 may create an analysis result report on a classification hit value after applying the test data when classifying data using a pie chart visualization block.

The visualization unit 400 pre-processes the case where the actual value and the predicted value match as a hit value and executes it as a pie chart visualization function block to write an analysis visualization report that the model hit with a probability of 88.9%.

Referring to FIG. 8 , the performing unit 500 matches each process of the collecting unit 100 , the preprocessing unit 200 , the analyzing unit 300 , and the visualization unit 400 on a block basis, and matching each process Function blocks can be selected. The performing unit 500 selects, arranges, and connects functional blocks to analyze big data and perform visualization without program coding. The performing unit 500 may select and connect each function block by a drag-and-drop method. The execution unit 500 may generate a workflow by selecting, arranging, and connecting functional blocks. When the function blocks are selected, the execution unit 500 minimizes errors when generating a workflow by displaying whether the block is located in the middle or the block located at the beginning or the end as a connection point. For example, the starting function block has no connection points on the left side of the block and exists only on the right side. If there is one pre-work, the left connection point is one, and if there are two, the connection point is displayed as two. Subsequent tasks are displayed in the same way by matching the number of connection points and the number of tasks. For the last function block, there is no connection point on the right. The performing unit 500 distinguishes each stage by differentiating the color of the functional block for each stage.

The performing unit 500 may recommend a preprocessing function block to determine whether there are missing values and outliers in the dataset and correct them to normal values or normal categories, predict normal values or normal categories for missing values and outliers, can provide

The performing unit 500 analyzes and learns the dataset, the learned analysis model, and the learning result, and recommends hyperparameters suitable for the dataset and the analysis model. For example, the execution unit 500 may recommend a hyperparameter capable of improving the performance of the analysis model according to the input data set.

The performing unit 500 may analyze the dataset collected by the collection unit 100 to recommend an analysis model functional block or a next-stage functional block, and may recommend a template composed of a plurality of recommended functional blocks. For example, the execution unit 500 may recommend a regression analysis function block in case of univariate analysis through independent variable analysis, and may recommend a cluster analysis function block in case of multivariate analysis. Alternatively, the execution unit 500 may recommend a chi-square test and logistic regression analysis function block for the categorical type, and a Pearson correlation analysis and a linear regression analysis function block for the continuous type, by determining the type of the dependent variable.

When the data is date-type time series data, the performing unit 500 may recommend a histogram visualization function block or a line graph visualization function block for confirming the amount of change.

The execution unit 500 may select functional blocks to create a desired workflow or use a provided template. Alternatively, the execution unit 500 may store the generated workflow and reuse it for analysis of the same or similar data, and may refer to it when creating a workflow or selecting a function block.

Referring to FIG. 9 , the execution unit 500 includes an error correction unit 510 , a performance improvement unit 520 , a block recommendation unit 530 , a model recommendation unit 540 , and a workflow generation unit 550 . .

The error correction unit 510 determines whether the preprocessed data is suitable for the selected analysis model. For example, an error occurs if the preprocessed data is of a different data type than the data used in the learning model, outliers that exceed the range of learned values, or missing values are not processed. The error correction unit 510 may recommend a matching normal value or a normal category value or a preprocessing function block for the generated error.

Referring to FIG. 10 , the error correction unit 510 uses the entire database including metadata of input values, metadata of blocks currently being executed, metadata of input values to be used for model formation, and block metadata.

The error correction unit 510 may include the size of input data, the number of input fields, data type, and main characteristics analyzed by itself as a type of metadata field of the input value.

The error correction unit 510 may include a block ID, types and values of input parameters, types and values of parameters in which an error occurs, etc. as types of block metadata fields.

When an error occurs during execution of the function block, the error correction unit 510 collects the function block metadata and collects the generated error.

The error correction unit 510 corrects the error by recommending an alternative value when an error occurs, such as that the input data or the preprocessed data is different from the data used in the learning model or the range of the learned value is exceeded. For example, the error correction unit 510 may determine whether there is a problem with the missing value arbitrarily input by the user, and may determine whether there is a problem with the missing value arbitrarily input by the user when the user's judgment arbitrarily substitutes the missing value because the standard is uncertain in processing the missing value. It can provide guidelines for , and can recommend alternative values for missing values.

Referring to FIG. 11 , the error correction unit 510 includes a determination unit 5101 , a learning unit 5102 , and a recommendation unit 5103 .

The determination unit 5101 may analyze and track the point where the error occurred by synthesizing the function block metadata and information such as the error occurrence and the type and value of the parameter.

The determination unit 5101 determines whether the input data is suitable for the selected analysis model. Also, the determination unit 5101 may determine whether the preprocessed data is suitable for the selected analysis model.

The determination unit 5101 determines as an error when the preprocessed data is a different data type from the data used in the learning model, an outlier exceeding the range of the learned value, or a missing value is not processed. For example, the determination unit 5101 may determine whether input data is normal or abnormal through a classification algorithm such as SVM or random forest.

The learning unit 5102 performs normal process matching through a predictive learning model that has learned the normal process and the abnormal process. More specifically, the learning unit 5102 generates a predictive learning model that predicts a normal value or a normal category value of a dataset by learning the normal process and the error process of the selected analysis model. For example, the predictive learning model may be an xgboost learning model that learns and classifies a normal process and an erroneous process .

The learning unit 5102 determines the type of target variable, the distribution of the dataset, the ratio of existing missing values, the input missing information replacement value, and the analysis model performance and error occurrence when processing the missing value replacement value in determining the normal category. Available.

The learning unit 5102 determines whether the replacement value of the input missing value is a normal value or a value within a normal range. If the replacement value input by the user does not fall within the normal range, the learning unit 5102 determines and recommends either the guideline within the normal range or the value within the normal range as the recommended replacement value. At this time, when the user selects a recommended replacement value, the learning unit 5102 uses the predictive learning model to re-learn with data to increase the accuracy of determining the normal range of the data.

The learning unit 5102 uses the data in which an error has occurred as an input value and performs a predictive learning model to predict a normal value or a normal category value of the error value.

The recommendation unit 5103 recommends a replacement value of a parameter at a point where an error occurs according to an algorithm that has learned the normal execution process or matches the normal execution process. In more detail, the recommender 5103 recommends a value predicted through the predictive learning model as a substitute value for the error value. The recommendation unit 5103 performs the predictive learning model generated by the learning unit 5102 to automatically correct or recommend a normal value for a missing value or an outlier, or any one value within the normal range or normal range as a replacement value. have.

12 is an example of an error value generated when performing a random forest analysis model in the big data analysis visualization apparatus according to an embodiment of the present invention.

Referring to the example of FIG. 12 , the determination unit 5101 determines whether input data is an outlier or a missing value. When the input data belongs to an abnormal value or an abnormal range, the learning unit 5102 predicts a normal value or a normal category value through the predictive learning model for an error value generated while performing the random forest analysis model. The recommender 5103 recommends a value predicted through the predictive learning model as a substitute value for the error value.

The big data analysis visualization apparatus 10 uses an analysis model that has learned the normal process and the error process to determine an error, identify an error occurrence point and the content of the error, and predict a normal value or a normal category. For example, the determination unit 5101 may use a classification algorithm such as random forest or SVM to identify error information. When the input value corresponds to an outlier or a missing value, the learner 5102 may predict the replacement value through an analysis model such as the xgboost algorithm.

Referring back to FIG. 9 , the performance improving unit 520 recommends an optimal hyperparameter value for evaluating an analysis model and improving performance. In big data analysis, hyperparameter optimization is one of the most important tasks because the performance of the analysis model varies greatly depending on the hyperparameter setting value.

Referring to FIG. 13 , the performance enhancing unit 520 recommends a hyperparameter suitable for a data set and an analysis model. For example, the performance enhancing unit 520 may recommend a hyperparameter capable of deriving an optimal performance. The performance improvement unit 520 may suggest performance improvement of the analysis model by recommending a hyperparameter using a hyperparameter cross-validation algorithm. The performance enhancer 520 may provide a performance evaluation value predicted when learning is performed using the recommended hyperparameter value.

Referring to FIG. 14 , the performance enhancing unit 520 includes an input unit 5201 , an adjusting unit 5202 , and a performance comparing unit 5203 .

The input unit 5201 receives a hyperparameter including a data set learning/verification ratio from a user to build an analysis model. For example, analysis models that require a dataset training/validation ratio are models such as random forest, regression analysis, decision tree, and logistic regression analysis. Hyperparameters that can be input include setting independent dependent variables, training/validation dataset ratio, and number of trees. For a random forest analysis model, the proportion of training data, the proportion of validation data for model performance validation, and the number of trees. The depth of the tree, the minimum number of data of each leaf, the minimum number of data of non-leaf nodes, etc. are used as hyper parameters. The adjustment unit 5202 recommends an adjustment value of the hyperparameter for optimal performance of the selected analysis model. The adjustment unit 5202 recommends an optimal adjustment value by analyzing the type of data set and the selected analysis model. The adjustment unit 5202 predicts a hyperparameter value suitable for a dataset and an analysis model without performing analysis model learning through direct hyperparameter adjustment using a hyperparameter prediction algorithm. For example, the adjuster 5202 recommends an adjustment value of a hyperparameter capable of deriving an optimal performance by adjusting a data rate according to whether the performance is under-performance or over-fitting. The hyperparameter algorithm is an algorithm that predicts hyperparameters that can derive optimal performance by analyzing and learning the input data set, the performed analysis model, and the performance results.

Existing hyperparameter tracking algorithms such as gridsearch and randomsearch consume a lot of time and money because of the large amount of computation. That is, when the user inputs the initial hyperparameter, the performance is derived by checking the model performance value index, and to improve the performance, the ratio between the verification dataset and the actual dataset is repeatedly adjusted and learned to find the adjustment value. In order to improve the performance of the analysis model, time and money are consumed because it is necessary to repeatedly learn by adjusting the hyperparameter values until the desired performance is obtained. However, the big data analysis visualization device 10 that corrects the error value predicts and recommends hyperparameter values for performance improvement by learning the type of the existing dataset, the selected analysis model, and the performance result, thereby reducing time and cost. .

When the performance of the previous model is low, the adjustment unit 5202 repeats the recommendation of another adjustment value, when the performance is high, stops the recommendation, and stops the recommendation even when the preset threshold value or more is reached.

The performance comparison unit 5203 may provide a performance evaluation value predicted when learning is performed using the recommended hyperparameter value.

The performance comparison unit 5203 visualizes the performance when the hyperparameter input by the user is applied and the performance when the hyperparameter adjustment value for optimal performance is applied. For example, the performance comparison unit 5203 provides a result when the hyperparameter value input by the user as visual data of the out of bag error graph, the correlation matrix graph, and the verification index is applied to the analysis model. can In a way that the performance comparison unit 5203 provides the recommended value to the user as visual data, the user input value and the recommended adjustment value are displayed together, and then the screen of the before and after change of the out of bag error graph indicator is displayed. Displays and provides changes in performance and result indicators.

Referring back to FIG. 9 , the block recommendation unit 530 recommends the next functional block through the data set analysis and the current function block stage analysis.

The block recommendation unit 530 may improve the efficiency of the analysis process and the accuracy of the analysis by recommending a functional block of the next stage in the big data analysis process.

Referring to FIG. 15 , the block recommendation unit 530 analyzes the input data set and analyzes the functional blocks performed so far, and recommends the next functional block with reference to a workflow or template having a high degree of similarity. For example, the block recommendation unit 530 may recommend a regression analysis function block in case of univariate analysis through independent variable analysis, and may recommend a cluster analysis function block in case of multivariate analysis. Alternatively, the block recommendation unit 530 may recommend a chi-square test and a logistic regression analysis function block for the categorical type, and a Pearson correlation analysis and a linear regression analysis function block for the continuous type, by determining the type of the dependent variable.

When the data is date-type time series data, the block recommendation unit 530 may recommend a histogram visualization function block or a line graph visualization function block for confirming a change amount.

The block recommendation unit 530 may use the generated workflow or workflow template big data in which workflow information of the provided template is stored for analysis.

Referring to FIG. 16 , the block recommendation unit 530 includes a block analysis unit 5301 and a data set analysis unit 5302 . It includes a generation unit 5303 , a clustering unit 5304 , a similarity analysis unit 5305 , and a block step recommendation unit 5306 .

The block recommendation unit 530 may receive a request for recommending a functional block in the next step or may recognize a situation in which a functional block recommendation is required.

The block analysis unit 5301 performs a block analysis algorithm capable of classifying the steps of the current functional block in detail. For example, the block analyzer 5301 extracts metadata of the current functional block in order to identify the current progress stage. The function block metadata includes basic information of the function block, such as a function block identification ID, parameter list, and parameter input value, and may further include a function block usage history list for redundancy check of the used function block.

In addition, the dataset analysis unit 5302 performs a dataset analysis algorithm for discriminating the input dataset. For example, the dataset analysis unit 5302 extracts metadata of the input dataset. The metadata of the dataset includes the data type of the dataset, data size, number of characteristics, whether or not there are missing values, and whether or not there is an outlier, and whether or not there is duplicate data.

The generator 5303 collects the result value of the block analyzer 5301 and the result value of the dataset analyzer 5302 to generate detailed step analysis metadata that can classify detailed processes. For example, detailed step analysis metadata is generated using the extracted metadata of the current function block and metadata of the input dataset.

The clustering unit 5304 clusters the workflow template big data based on the feature value of the detailed stage analysis metadata. For example, the clustering unit 5304 may use a clustering technique such as K-means (K-MEANS), K-mode (K-MODE), or DBSCAN. Workflow Template Big data includes a generated workflow or a workflow of a template provided as an analysis scenario.

The similarity analysis unit 5305 analyzes the clustered workflow and the similarity based on the feature value of the detailed stage analysis metadata to extract a workflow having a higher ranking with a high similarity value. For example, the similarity analyzer 5305 may analyze the similarity by using a similarity analysis technique such as the Euclidean distance, the Manhattan distance, and the Spearman correlation score. The similarity analyzer 5305 may recommend workflows having a similarity higher ranking.

The block step recommendation unit 5306 may search for a process of a current functional block in workflows having a higher similarity ranking, and recommend a functional block corresponding to a next step in the workflows ranked higher in similarity.

Referring back to FIG. 9 , the model recommendation unit 540 recommends an analysis model most suitable for the input data set and performs analysis. For example, the model recommendation unit 540 may recommend an analysis model that has obtained a score greater than or equal to a standard by calculating the suitability of the dataset and the analysis model as a score.

Referring to FIG. 17 , the model recommendation unit 540 recommends a suitable analysis model by analyzing an input dataset, and recommends a workflow or a template by analyzing the dataset and the recommended analysis model. In more detail, the model recommendation unit 540 recommends an analysis model suitable for the input dataset through the analysis model recommendation algorithm. The model recommendation unit 540 recommends analysis models having a high simulation evaluation score. The model recommendation unit 540 analyzes the similarity with the workflow template big data using the dataset metadata and the analysis model metadata. Workflow Template Big data includes a generated workflow or a workflow of a template provided as an analysis scenario. The model recommendation unit 540 may provide a guide when a user selects an analysis model by recommending a template including a workflow having a high similarity.

Referring to FIG. 18 , the model recommendation unit 540 includes a simulation evaluation unit 5401 , an analysis model recommendation unit 5402 , a similarity analysis unit 5403 , and a recommendation unit 5404 .

The simulation evaluation unit 5401 performs a suitable model evaluation algorithm to simulate and apply various types of analysis models to the input dataset, and calculates scores of the dataset and the analysis model. For example, the simulation evaluation unit 5401 calculates an AUC score by simulating random forest, correlation analysis, multilayer perceptron, naive Bayes, k means model, and the like, and extracts a fit score.

The analysis model recommendation unit 5402 recommends the top n analysis models or an analysis model greater than or equal to a preset threshold based on the calculated score.

The similarity analysis unit 5403 analyzes the similarity with the workflow template big data based on the selected recommended analysis model and dataset metadata. In this case, the similarity analysis unit 5403 may calculate the similarity with the workflow template big data by selecting the model that has obtained the highest fit score or the analysis model selected by the user from among the recommended analysis models.

The similarity analysis unit 5403 may calculate a similarity with the workflow template big data based on the selected recommended analysis model and dataset metadata.

The recommendation unit 5404 may recommend top n workflows having a high similarity through the similarity analysis unit 5403 .

The recommendation unit 5404 may recommend a template including the top n workflows having a high similarity through the similarity analysis unit 5403 .

Referring back to FIG. 9 , the workflow generating unit 550 generates a workflow combined with the recommended blocks.

Referring to FIG. 19 , the workflow generating unit 550 analyzes input data and selected functional blocks and arranges the recommended blocks in an appropriate position and order in the workflow. The workflow generator 550 may generate, store, and provide a workflow in which recommended blocks are arranged.

The workflow generating unit 550 may recommend and arrange respective functional blocks according to the degree of creation of the workflow to complete the incomplete workflow.

The workflow generating unit 550 extracts an upper workflow having a high similarity based on the input data set and the selected blocks and uses it to generate the workflow. The workflow generator 550 may analyze the selected blocks to generate a workflow based on the actually progressed task.

Referring to FIG. 20 , the workflow generating unit 550 includes a data extraction unit 5501 , a pre-processing recommendation unit 5502 , an analysis model recommendation unit 5503 , a visualization recommendation unit 5504 , and a completion unit 5505 . do.

The data extraction unit 5501 analyzes a dataset using a dataset analysis algorithm and a block analysis algorithm, and analyzes a block. In more detail, the data extraction unit 5501 extracts the dataset metadata of the input dataset through the dataset analysis algorithm, and extracts the block metadata of the blocks already selected through the block analysis algorithm.

The preprocessing recommendation unit 5502 analyzes necessary preprocessing based on the dataset metadata. For example, the preprocessing recommendation unit 5502 may recommend an outlier processing block when an outlier is found, a missing value processing block when a missing value is found, and a derived variable block and a preprocessing block required for the PCA block dataset if the number of columns is unnecessarily large. can

The analysis model recommendation unit 5503 recommends an analysis model using block metadata and dataset metadata.

The visualization recommendation unit 5504 recommends a visualization block through a visualization recommendation algorithm. For example, the visualization recommendation unit 5504 recommends a boxplot or scatterplot visualization block if the preprocessing block being used is an outlier processing block, excluding duplicate visualization, and a pie chart that can check the ratio if the dataset is categorical. A visualization block can be recommended.

The completion unit 5505 arranges the recommended blocks and completes the workflow to generate them. The completion unit 5504 extracts a workflow with a high degree of similarity through a block arrangement algorithm, arranges each recommended block, and creates a workflow. The block arrangement algorithm extracts the top n cases with high similarity to the workflow during work, and recommends the functional block and its location by identifying the location of the functional block.

21 to 31 are diagrams for explaining a big data analysis visualization method according to an embodiment of the present invention. Each process described below is a process performed by each functional unit constituting the big data analysis and visualization apparatus, but for the sake of concise and clear explanation of the present invention, the subject of each step is collectively referred to as a big data analysis and visualization apparatus.

Referring to FIG. 21 , the big data analysis visualization apparatus 10 collects data to be analyzed in step S2101. The big data analysis visualization apparatus 10 may collect data to be analyzed from various types of data. For example, the big data analysis visualization apparatus 10 may upload and collect files in Excel, TEXT, and CSV format. In addition, the big data analysis visualization apparatus 10 may collect data to be analyzed by directly connecting to the relational database. The big data analysis visualization apparatus 10 may directly input or paste data to directly generate the data. The big data analysis visualization apparatus 10 may directly collect data using the provided OpenAPI.

In step S2102, the big data analysis visualization apparatus 10 pre-processes the collected data according to a desired analysis model.

In step S2103, the big data analysis visualization apparatus 10 analyzes data using the analysis model. The big data analysis visualization apparatus 10 may analyze a dataset and recommend an analysis model. The big data analysis visualization apparatus 10 may recommend a preprocessing method by analyzing the dataset and the analysis model to be used.

In step S2104, the big data analysis visualization apparatus 10 visualizes and expresses the analyzed data. The big data analysis visualization apparatus 10 may recommend a visualization block suitable for the analysis result.

22 is an exemplary diagram for explaining a method for a big data analysis visualization apparatus for recommending a block according to an embodiment of the present invention to determine an error point when an error occurs and to recommend a normal value.

Referring to FIG. 22 , when an error occurs due to an input parameter while performing an analysis model function block, the big data analysis visualization apparatus 10 finds an exact error point and corrects the value to proceed with the analysis process without error.

In step S2201, the big data analysis and visualization apparatus 10 collects metadata of functional blocks and information on errors.

In step S2202, the big data analysis visualization apparatus 10 analyzes the information on the error occurrence to determine the error occurrence point.

In step S2203, the big data analysis and visualization apparatus 10 analyzes the metadata of the block, the error occurrence, the type and value of the parameter, and the like, and determines whether the data set and the analysis model are suitable for matching. The big data analysis visualization apparatus 10 may recommend an analysis model suitable for the data set when the data set and the analysis model do not match. In detail, the big data analysis visualization apparatus 10 extracts a normal value of the normal analysis model performing process and recommends a normal value if it does not match the selected analysis model.

In step S2204, the big data analysis visualization apparatus 10 determines whether an outlier or a missing value in the data. The big data analysis and visualization apparatus 10 may recommend and automatically apply a normal value or a normal range in case of an error due to an outlier or a missing value of the input data.

The big data analysis visualization apparatus 10 may self-diagnose an error point and replace it with a predicted normal value to continue analysis without error.

23 is an example of determining and correcting an error by the big data analysis visualization apparatus according to a temporary example of the present invention.

Referring to FIG. 23 , in step S2301 , the big data analysis and visualization apparatus 10 receives data to be analyzed from the user.

In step S2302, the big data analysis visualization apparatus 10 creates a data analysis workflow using any one of a newly created workflow, a provided workflow template, or a stored workflow by dragging and dropping blocks.

In step S2303, when an error occurs, the big data analysis visualization apparatus 10 determines an error type by referring to information in the error DB and identifies an error point. The big data analysis visualization apparatus 10 uses an analysis model that has learned the normal process and the error process to determine the error occurrence point and the content of the error when an error occurs. For example, the big data analysis visualization apparatus 10 may use a classification algorithm such as a random forest or SVM to identify error information.

In step S2304, the big data analysis visualization apparatus 10 predicts a normal value or a normal range value using a predictive learning model for an error value. When the input value corresponds to an outlier or a missing value, the big data analysis visualization apparatus 10 may recommend an alternative value of the error value through an analysis model such as the xgboost algorithm. The replacement value of the error value may be a normal value or a normal category in which no error occurs and big data analysis can be performed normally.

In step S2305, the big data analysis visualization apparatus 10 recommends a predicted value, that is, a result value of the predictive learning model to the user. Alternatively, the big data analysis visualization apparatus 10 may automatically apply a normal value or a normal category to continue the analysis.

In step S2306, the big data analysis visualization apparatus 10 performs data analysis by applying the recommended value.

24 is a diagram for explaining a method of recommending an alternative value when an error occurs by the big data analysis and visualization apparatus according to an embodiment of the present invention.

The error correction unit 510 corrects the error by recommending an alternative value when an error occurs, such as that the input data or the preprocessed data is different from the data used in the learning model or the range of the learned value is exceeded.

Referring to FIG. 24 , in step S2401 , the error correction unit 510 builds a learning model learned based on the normal value of the performed workflow. For example, the error correction unit 510 generates a predictive learning model for predicting a normal value or a normal category value of a dataset by learning a normal process and an error process of the selected analysis model.

In step S2402, when an error occurs in the input data set, the error correction unit 510 collects error value information and determines an error point.

In step S2403, the error correction unit 510 predicts a normal value or a normal categorical value of the error value by performing the predictive learning model built in step S1501.

In step S2404, the error correction unit 510 recommends the predicted value as a normal value or performs automatic input processing.

In step S2405, when the recommended value is applied, the error correction unit 510 adds it to the predictive learning model as a normal value or a normal range value.

In step S2406, the error correction unit 510 supports the user to correct the error by correcting the error value based on the recommended normal value or the normal range value.

25 is a diagram for explaining a method of recommending and adjusting a hyperparameter capable of deriving an optimal performance of an analysis model, by the apparatus for visualizing big data analysis according to an embodiment of the present invention.

Referring to FIG. 25 , the performance enhancing unit 520 recommends a hyperparameter suitable for a data set and an analysis model. For example, the performance enhancing unit 520 may recommend and adjust a hyperparameter capable of deriving an optimal performance.

In step S2501, the big data analysis visualization apparatus 10 receives a data set that requires analysis.

In step S2502, the big data analysis visualization apparatus 10 selects an analysis model that requires hyperparameters such as a learning/verification ratio. For example, analysis models that require hyperparameters include random forest, regression analysis, decision tree, and logistic regression analysis models. do.

In step S2503, the big data analysis visualization apparatus 10 performs an analysis model with the hyperparameter input by the user.

In step S2504, the big data analysis visualization apparatus 10 visualizes the performed performance result and verification index, such as a graph or chart, and determines the necessity of improving the performance of the analysis model. For example, the big data analysis visualization apparatus 10 displays the performance results when the user input hyperparameter values are applied to the analysis model as visual data of the out of bag error graph, the correlation matrix graph, and the verification index. can provide

When it is necessary to improve the performance of the analysis model, the big data analysis visualization apparatus 10 recommends hyperparameters for optimal performance of the selected analysis model in step S2405. The big data analysis visualization apparatus 10 predicts a hyperparameter value suitable for a dataset and an analysis model without performing analysis model learning through direct hyperparameter adjustment using a hyperparameter prediction algorithm. The hyperparameter algorithm is an algorithm that predicts hyperparameters that can derive optimal performance by analyzing and learning the input data set, the performed analysis model, and the performance results. The big data analysis visualization apparatus 10 recommends an adjustment value of a hyperparameter capable of deriving an optimal performance by adjusting the data ratio according to whether the performance is under-performance or over-fitting.

In step S2506, the big data analysis visualization device 10 visualizes and displays the change by comparing the performance result and verification index of the analysis model to which the recommended adjustment value is applied and the performance result and verification index of the analysis model to which the user input hyperparameter is applied. .

In step S2507, the big data analysis visualization apparatus 10 determines the analysis model and hyperparameters and stops the hyperparameter adjustment operation if it is determined that the performance value of the analysis model is optimal or the performance value target is achieved.

26 is a diagram for explaining a method of recommending, by a big data analysis visualization apparatus, a function block of a next step according to an embodiment of the present invention.

Referring to FIG. 26 , the block recommendation unit 530 recommends the next functional block through data set analysis and current block stage analysis.

In step S2601, the big data analysis visualization apparatus 10 receives a request for recommendation of the next functional block while performing the data analysis workflow.

In step S2602, the big data analysis visualization apparatus 10 analyzes the progress step of the current functional block by performing a block analysis algorithm for classifying the progress steps of the selected functional block.

In step S2603, the big data analysis visualization apparatus 10 analyzes the input dataset using the dataset analysis algorithm.

In step S2604, the big data analysis visualization apparatus 10 collects the block analysis algorithm result value and the dataset analysis algorithm result value to generate detailed stage analysis metadata capable of classifying the detailed process.

In step S2605, the big data analysis visualization apparatus 10 performs similarity analysis between the detailed step analysis metadata and the workflow analysis data based on the detailed step analysis metadata and the existing workflow analysis data.

In step S2606, the big data analysis visualization apparatus 10 recommends a functional block used in a workflow ranked higher in the similarity analysis. In other words, the big data analysis visualization apparatus 10 extracts workflow analysis data with high similarity to detailed stage analysis metadata and recommends functional blocks in the corresponding workflow.

27 is an exemplary screen in which the apparatus for visualizing big data analysis according to an embodiment of the present invention recommends a functional block of a next step.

Referring to FIG. 27 , in step S2701 , the big data analysis visualization apparatus 10 receives a request for recommendation of a next function block while performing a data analysis workflow.

In step S2702, the big data analysis visualization apparatus 10 extracts the metadata of the current functional block by performing a block analysis algorithm for classifying the progress steps of the selected functional block. For example, block metadata includes basic block information such as block ID, parameter ID, parameter input value, and parameter list, and a history list of blocks that have already been used.

In step S2703, the big data analysis visualization apparatus 10 extracts metadata of the input dataset using a dataset analysis algorithm. For example, the input dataset metadata includes information such as data type, data size, number of characteristics, whether or not there are missing values, and whether or not there is an outlier, and whether or not there is duplicate data.

In step S2704, the big data analysis visualization apparatus 10 generates detailed step analysis metadata using block metadata and dataset metadata.

In step S2705, the big data analysis visualization apparatus 10 performs the generated workflow or the workflow and the clustering model of the provided template as the feature value of the detailed stage analysis metadata. In this case, clustering models include k-means (k-means), k-mode (k-mode), and DBSCAN (density-based) clustering techniques.

In step S2706, the big data analysis visualization apparatus 10 extracts an upper rank workflow similar to the feature value of the detailed stage analysis metadata within the clustered workflow through similarity analysis. For example, the big data analysis visualization apparatus 10 may analyze a similarity such as a dataset pattern, a designated dependent variable, a used functional block, and a connection relationship between the functional blocks. The similarity analysis model may use a similarity analysis technique of Euclidean distance, Manhattan distance, Spearman correlation score, and the like.

In step S2707, the big data analysis visualization apparatus 10 recommends a functional block used in the corresponding step in the workflow ranked higher in the similarity analysis.

28 is a diagram for explaining a method for a big data analysis visualization apparatus to recommend an analysis model according to an embodiment of the present invention.

Referring to FIG. 28 , the model recommendation unit 540 recommends an analysis model most suitable for the input data set and performs analysis.

In step S2801, the big data analysis visualization apparatus 10 may receive a data set that requires analysis, and may designate a predicted target value (Y value) when a target variable is designated.

In step S2802, the big data analysis visualization apparatus 10 performs a fitting model evaluation algorithm to simulate and apply various types of analysis models to the input dataset, and calculates a fitness score of the dataset and the analysis model.

In step S2803, the big data analysis visualization apparatus 10 recommends an analysis model having a top n fit score calculated by using the fitting model evaluation algorithm or applies the analysis model that has obtained the highest fitting score to the workflow.

In step S2804, the big data analysis visualization apparatus 10 performs similarity analysis using the dataset metadata and block metadata of the recommendation analysis model. In detail, the big data analysis visualization apparatus 10 analyzes the block metadata of the recommended analysis model with the block analysis algorithm, analyzes the input dataset metadata with the dataset analysis algorithm, and collects the result values to classify the detailed process. Generate detailed step-by-step analysis metadata that can be The big data analysis visualization apparatus 10 may perform similarity analysis between the detailed step analysis metadata and the workflow analysis data based on the detailed step analysis metadata and the existing workflow analysis data.

In step S2805, the big data analysis visualization apparatus 10 recommends a template based on a workflow with a high degree of similarity.

29 is an exemplary screen in which the big data analysis visualization apparatus recommends an analysis model according to an embodiment of the present invention.

Referring to FIG. 29 , in step S2901, the big data analysis and visualization apparatus 10 receives a data set.

In step S2902, the big data analysis and visualization apparatus 10 performs a fitting model recommendation algorithm for calculating a fit score by simulating the input dataset to the analysis model. For example, the big data analysis visualization apparatus 10 calculates the AUC score by applying a random forest, correlation analysis, multi-layer perceptron, naive Bayes, k-means (k-means) model, etc. to the input data set by simulating, and Calculate the fit score.

In step S2903, the big data analysis and visualization apparatus 10 recommends an analysis model whose calculated fit score corresponds to the top n cases.

In step S2904, the big data analysis visualization apparatus 10 analyzes the block of the analysis model selected from among the recommended analysis models or the analysis model that has obtained the highest AUC score with a block analysis algorithm, and analyzes the dataset through the dataset analysis algorithm.

In step S2905, the big data analysis visualization apparatus 10 combines the block metadata obtained as a result value of the block analysis algorithm and the dataset analysis algorithm with the dataset metadata to perform a similarity analysis with the workflow of the provided template.

In step S2906, the big data analysis visualization apparatus 10 recommends a template including the top n workflows with high similarity.

30 is a diagram for explaining a method of generating a workflow using blocks recommended by the apparatus for visualizing big data analysis according to an embodiment of the present invention.

Referring to FIG. 30 , the workflow generating unit 550 may generate a new workflow by generating recommended blocks at appropriate locations within the workflow. The workflow generator 550 may store the generated workflow as a user's workflow or as a workflow template.

In step S3001, the big data analysis visualization apparatus 10 is requested to recommend an analysis process during workflow creation.

In step S3002, the big data analysis visualization apparatus 10 analyzes the dataset through the input dataset analysis algorithm and recommends a preprocessing block.

In step S3003, the big data analysis visualization apparatus 10 analyzes the block of the workflow during creation through the block analysis algorithm.

In step S3004, the big data analysis visualization apparatus 10 recommends a block of the analysis model using the result values of the block analysis algorithm and the dataset analysis algorithm.

In step S3005, the big data analysis visualization apparatus 10 recommends a visualization block suitable for effective visualization through a visualization recommendation algorithm.

In step S3006, the big data analysis visualization apparatus 10 connects the recommended blocks in a suitable position and order through a block arrangement algorithm to generate a workflow.

31 is an exemplary screen for generating a workflow using blocks recommended by the big data analysis visualization apparatus according to an embodiment of the present invention.

In step S3101, the big data analysis visualization apparatus 10 is requested to recommend a block-to-block analysis process. The big data analysis visualization apparatus 10 checks the input data set and the selected function block for the analysis process recommendation.

In step S3102, the big data analysis visualization apparatus 10 extracts the dataset metadata from the input dataset through the dataset analysis algorithm.

In step S3103, the big data analysis visualization apparatus 10 analyzes necessary pre-processing based on the dataset metadata and recommends a pre-processing block. For example, the big data analysis visualization apparatus 10 recommends an outlier processing block when an outlier is found, a missing value processing block when a missing value is found, and a preprocessing block required for a derived variable block and PCA blocks dataset when the number of columns is unnecessarily large. can

In step S3104, the big data analysis visualization apparatus 10 extracts block metadata of the selected functional block through a block analysis algorithm.

In step S3105, the big data analysis visualization apparatus 10 recommends a block of the analysis model using block metadata and dataset metadata.

In step S3106, the big data analysis visualization apparatus 10 recommends a visualization block through a visualization recommendation algorithm. For example, the big data analysis visualization device 10 recommends a boxplot or scatterplot visualization block if the preprocessing block being used is an outlier processing block, except for duplicate visualization, and if the dataset is categorical, it is possible to check the ratio. A pie chart visualization block can be recommended.

In step S3107, the big data analysis visualization apparatus 10 creates a workflow by arranging the recommended blocks in a suitable position and order through a block context analysis algorithm. For example, in the big data analysis visualization apparatus 10, the visualization block is disposed after the associated block, and the preprocessing block is disposed between the data input and the target data analysis block.

32 to 36 are exemplary screens of a big data analysis visualization apparatus according to an embodiment of the present invention.

32 is an exemplary screen in which the big data analysis visualization apparatus 10 collects data.

Referring to FIG. 32 , the big data analysis visualization apparatus 10 may collect data desired to be analyzed by connecting an Excel file format, a CSV file format, an RDS file format, a TXT file format, and a database by drag and drop. In addition, the big data analysis visualization apparatus 10 may directly collect data using the provided OpenAPI.

33 is an exemplary screen in which the big data analysis visualization apparatus 10 performs data pre-processing.

Referring to FIG. 33 , the big data analysis visualization apparatus 10 may change column information and check data for pre-processing.

34 is an exemplary screen in which the big data analysis visualization apparatus 10 analyzes big data using an analysis model.

Referring to FIG. 34 , the big data analysis visualization apparatus 10 designates hyperparameters and target variable values to analyze using the random forest analysis model, and may also check the performance index of the analysis model.

35 is an example screen on which the big data analysis visualization apparatus 10 performs visualization.

Referring to FIG. 35 , the big data analysis and visualization apparatus 10 may analyze big data without program coding in a web page manipulation method and visualize the result.

36 is an exemplary screen of a functional block used by the big data analysis and visualization apparatus 10 .

Referring to FIG. 36 , the big data analysis visualization apparatus 10 may select and move a function block by dragging and dropping or clicking. The big data analysis visualization apparatus 10 expresses the functional block in different colors at each stage, and includes an internal point or an external point of the functional block according to a request. In addition, the color of the inner point or the outer point of the function block is different depending on the state, so it is intuitive. The big data analysis visualization apparatus 10 may connect functional blocks with external points using a block pipeline.

The above-described big data analysis visualization method may be implemented as a computer-readable code on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). can The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet and installed in the other computing device, thereby being used in the other computing device.

In the above, even though it has been described that all components constituting the embodiment of the present invention are combined or operated as one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more.

Although acts are shown in a particular order in the drawings, it should not be understood that the acts must be performed in the specific order or sequential order shown, or that all illustrated acts must be performed to obtain a desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the embodiments described above should not be construed as necessarily requiring such separation, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is

So far, the present invention has been looked at focusing on the embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

10: Big data analysis visualization device
100: collection unit
200: preprocessor
300: analysis unit
400: visualization unit
500: attendant
510: error correction unit
520: performance enhancement unit
530: block recommendation
540: model recommendation
550: workflow creation unit

Claims (9)

In the big data analysis visualization device,
a collection unit for collecting data to be analyzed;
a pre-processing unit pre-processing the data to fit the analysis model;
an analysis unit that analyzes the data as an analysis model;
a visualization unit that visualizes the result of performing the analysis model as a suitable graph; and
Big data analysis and visualization apparatus including a performing unit for matching the data collection, pre-processing, analysis and visualization processes with functional blocks.
According to claim 1,
The preprocessor
A big data analysis visualization device that generates and provides the data that has undergone a pre-processing process as a file.
According to claim 1,
the performing unit
A big data analysis visualization device using a workflow for selecting, arranging and linking the functional blocks.
According to claim 1,
A big data analysis visualization device that provides a dataset analysis scenario as a template.
In the method for big data analysis visualization device to analyze big data
collecting data to be analyzed;
preprocessing the data to fit an analysis model;
analyzing the data using an analysis model;
Visualizing the analyzed data; and
Big data analysis visualization method comprising the step of performing the data collection, pre-processing, analysis, and visualization steps by matching them with functional blocks.
6. The method of claim 5,
The step of preprocessing the data to fit the analysis model is
A big data analysis visualization method for generating and providing the data that has undergone a pre-processing process as a file.
6. The method of claim 5,
A big data analysis visualization method using a workflow for selecting, placing and linking the functional blocks.
6. The method of claim 5,
A big data analysis visualization method that provides a dataset analysis scenario as a template.
A computer program recorded in a computer-readable recording medium executing the big data analysis and visualization method of any one of claims 5 to 8.

Images