KR102533708B1 - Method and server for generating dataset relationship automatically - Google Patents

Abstract

A method for automatically generating dataset relationships is provided. The method may include extracting data level feature information included in a dataset having a first description for a predetermined dataset service; generating at least one second description in which the data level characteristic information is reflected in the first description of the dataset; calculating a similarity between a plurality of datasets based on the first and second descriptions; generating a similarity matrix based on the calculated similarity; calculating association information between an entity corresponding to the service and the dataset; generating a relationship matrix between the entity and a dataset based on the relationship information; and generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix.

Description

Dataset relationship automatic creation method and server {METHOD AND SERVER FOR GENERATING DATASET RELATIONSHIP AUTOMATICALLY}

The present invention relates to a method and server for automatically generating dataset relationships, and more particularly to a method and server for automatically generating relationships between datasets when providing multiple types of datasets.

Conventional services that provide a dataset to users mainly use a method of providing a dataset that the user wants through matching the search keyword, category, etc. requested by the user based on the description explicitly described by the provider when registering the dataset. there is.

In the case of such a prior art, there are problems such as lack of data set description information, lack of information on the data set configuration data level, and inability to apply a variable weight reflecting the data set usage situation.

Domestic Patent Publication No. 10-2017-0136976 (2017.12.12)

Embodiments of the present invention provide information for effective data set search and selection by providing not only specified description information but also data sets with a high degree of relevance to the data set in providing various types of data sets to the user. Provides a method and server for automatically generating multi-type dataset relationships that can be used.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, the method for automatically generating a dataset relationship according to the first aspect of the present invention includes data level characteristic information included in a dataset having a first description for a predetermined dataset service. extracting; generating at least one second description in which the data level characteristic information is reflected in the first description of the dataset; calculating a similarity between a plurality of datasets based on the first and second descriptions; generating a similarity matrix based on the calculated similarity; calculating association information between an entity corresponding to the service and the dataset; generating a relationship matrix between the entity and a dataset based on the relationship information; and generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix.

In some embodiments of the present invention, generating a second description in which the data level characteristic information is reflected in the first description of the dataset includes the second description in a different combination corresponding to the type of data included in the dataset. A description can be created.

In some embodiments of the present invention, generating a similarity matrix based on the calculated similarity may generate a relative similarity matrix for a second dataset based on the first dataset among the plurality of datasets. there is.

In some embodiments of the present invention, the entity corresponding to the service may include at least one of a component constituting the first description corresponding to the dataset, a user search keyword of the dataset, a user profile component, and user preference information. can include

In some embodiments of the present invention, the generating of the association matrix between the entity and the dataset based on the association information may generate a plurality of association matrices corresponding to the number of entities.

In some embodiments of the present invention, the generating of the association matrix between the entity and the dataset based on the association information may include generating the association matrix by assigning a weight to each entity according to the association information. can

In some embodiments of the present invention, the step of generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix may include applying the relevance matrix to the similarity matrix and relative dataset relevance matrix for each entity. can create

In some embodiments of the present invention, generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix may include first relevance information corresponding to a first dataset constituting the relevance matrix and summing up second association information corresponding to a second dataset; and generating the relative dataset relevance matrix by multiplying the summation result by a similarity corresponding to the first dataset and the second dataset.

In some embodiments of the present invention, generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix further includes applying at least one of filtering and weighting to a specific entity among the entities. can do.

In addition, the server for automatically generating a dataset relationship for a predetermined dataset service according to the second aspect of the present invention includes a communication module for providing service information corresponding to a dataset service request through data transmission and reception with a user terminal, and the dataset service It includes a memory storing a program for automatically generating a dataset relationship for providing, and a processor for executing the program stored in the memory. At this time, as the program is executed, the processor extracts data level feature information included in a dataset having a first description for the dataset service and reflects it to generate at least one second description, Based on the first and second descriptions, a similarity matrix is created by calculating and reflecting similarity between a plurality of datasets, and calculating and reflecting association information between an entity corresponding to the service and the dataset to obtain a association matrix. and create a relative dataset relevance matrix based on the similarity matrix and relevance matrix.

In some embodiments of the present invention, the processor adds up first association information corresponding to a first dataset constituting the association matrix and second association information corresponding to a second dataset, and The relative dataset relevance matrix may be generated by multiplying the summation result by similarities corresponding to the dataset and the second dataset.

In some embodiments of the present invention, the processor may generate the relative dataset relevance matrix by applying at least one of filtering and weighting to a specific entity among the entities.

In addition to this, another method for implementing the present invention, another system, and a computer readable recording medium recording a computer program for executing the method may be further provided.

According to the present invention as described above, when a data user searches for data, it is possible to overcome the limitations of the limitedly provided dataset description and to provide an efficient method for searching for and using a dataset.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram for explaining a dataset description structure in a general dataset providing service.
2 is a flowchart of a method for automatically generating dataset relationships according to an embodiment of the present invention.
3 is a diagram illustrating a process of generating a second description.
4 is a diagram illustrating a connection structure of first and second descriptions.
5 is a diagram for explaining similarity calculation and similarity matrix.
6 is a diagram for explaining contents of generating an association matrix.
7 is a diagram for explaining a relative dataset relevance matrix.
8 is a diagram for explaining a general application process of the present invention.
9 is a diagram for explaining a server for automatically generating dataset relationships according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

1 is a diagram for explaining a dataset description structure in a general dataset providing service.

Referring to FIG. 1, the prior art mainly uses a method in which a separate description format is configured for each dataset and a dataset registrant explicitly describes corresponding information in accordance with the description format. At this time, information included in the description format includes dataset name, provider, registration date, keyword, brief description, etc., and additional information such as dataset classification and keyword may be added depending on the service.

As shown in FIG. 1, the conventional service providing a data set to the user was provided based only on the description explicitly described by the data set provider. In this prior art, the search limit is determined according to the components of the data set description, and information not included in the components of the description format cannot be searched, resulting in significantly reduced search efficiency.

For example, if the dataset description format consists of only the dataset name, provider, and upload date, data users can only search for the data they want through keyword searches for the dataset name and provider. In this case, the data It took a lot of time and effort to search the set user's desired data.

In addition, the dataset search method based on the dataset description according to the prior art, due to the absence of implicit feature information included in the data included in the dataset, is useful for detailed dataset searches and searches for high-quality datasets desired by users. There are difficulties. Because of this, dataset users have no choice but to select the dataset they want by checking the data of the dataset one by one after a description-based search.

In addition, in the case of the prior art, there is a problem that it is impossible to apply a variable weight reflecting the dataset usage situation.

Data set users may have different desired datasets depending on the purpose and situation of data use. For example, a dataset uploaded by a specific organization may be preferred, or a dataset containing characteristics of specific data may be preferred.

The search and use conditions of such data can change from time to time, and conventional dataset providing services provide a filtering function for specific items for this purpose, but this only provides a function to exclude specific items when searching for a dataset description. . As a result, the prior art has a problem in that it is impossible to apply variable weights to frequently changing data retrieval and use conditions.

As such, the prior art allows a data user to search for a desired dataset through explicit dataset description, and mainly supports search through keyword search or field unit search.

As described above, this prior art has a limitation in the amount of information and cannot reflect implicit information about the data itself included in the dataset.

In contrast, in one embodiment of the present invention, in providing various types of datasets to the user, not only the specified description information, but also a dataset having a high correlation with the dataset is provided together, so that the search and selection of the dataset is effective. There are advantages to providing information.

Hereinafter, a method for automatically generating dataset relationships according to an embodiment of the present invention will be described with reference to FIGS. 2 to 8 .

2 is a flowchart of a method for automatically generating dataset relationships according to an embodiment of the present invention.

Meanwhile, each of the steps shown in FIG. 2 may be understood to be performed by the server 100 (hereinafter referred to as the server) for automatically generating dataset relationships, but is not necessarily limited thereto.

First, the server 100 extracts data level feature information included in a dataset having a first description for a predetermined dataset service (S110). At this time, the server 100 may apply different feature extraction methods for each characteristic of the data.

In the present invention, in addition to the first description, which is an existing dataset description, the data included in the dataset is analyzed to generate a second description, which is a data-level description that can express the characteristics of the dataset, through which the existing first description reinforce Here, the data level feature information individually extracts information implicitly included in the data included in the dataset, and is described as a feature that can represent the dataset. Such implicit data-level characteristic information may be defined differently according to data types, and in the case of a dataset composed of complex data through a combination of various types of data, a plurality of second descriptions may be generated.

Next, the server 100 generates at least one second description in which the extracted data level feature information is reflected, targeting the first description of the dataset (S120). That is, the server 100 generates a second description by reflecting additional data level characteristic information to the first description, which is an existing dataset description.

As an example, the server 100 may generate the second description in a different combination corresponding to the type of data included in the dataset. In other words, the server 100 may generate the second description of the data level by combining different items according to the type of data included in the dataset. For example, the server 100 may generate second descriptions of several data levels for one dataset according to the type of data included in the dataset.

3 is a diagram illustrating a process of generating a second description. 4 is a diagram illustrating a connection structure of first and second descriptions.

For example, if an image is included in the data included in the dataset, the size of the image (average, minimum, maximum, resolution), format, location analysis (EXIF information statistics), information on included objects, etc. It contains additional information that can define a dataset based on information that can be extracted from individual data.

Such data level characteristic information may be generated as several descriptions (second descriptions) according to the type of data included in the dataset. And, the format of the second description can be extended through additional definition.

The server 100 may process information extracted through various methods such as image object labeling, classification, and statistical analysis according to the type of data in the process of extracting data level feature information and additionally reflect it in the first description. It is also possible to extract and configure specialized information to apply to the service.

Next, when the creation of the first explicit description for each dataset and the second description of the data level are completed, the server 100 calculates a similarity between a plurality of datasets based on the first and second descriptions (S130). , a similarity matrix is generated based on the calculated similarity (S140).

5 is a diagram for explaining similarity calculation and similarity matrix.

As an embodiment, the server 100 may apply various algorithms when calculating the similarity between the first and second descriptions, and the present invention is not limited to a specific similarity calculation and comparison algorithm.

The similarity between the plurality of datasets can be expressed as in Equation 1 below.

[Equation 1]

의 연산 방식은 서비스의 필요와 목적에 따라 다양한 유사도 산출 알고리즘을 이용할 수 있으며, 일 예로 두 데이터셋 간의 유사도 값은 다음 식 2와 같이 정의할 수 있다.At this time, the similarity between the two datasets

The calculation method of can use various similarity calculation algorithms depending on the need and purpose of the service. As an example, the similarity value between two datasets can be defined as in Equation 2 below.

[Equation 2]

The server 100 may generate a relative similarity matrix for a second dataset based on a first dataset among a plurality of datasets, and based on this, the server 100 may generate a dataset having a high similarity with the corresponding dataset. It is possible to configure and provide services based on similarity, such as recommending

Next, the server 100 calculates association information between the entity corresponding to the service and the dataset (S150), and creates a association matrix between the entity and the dataset based on the calculated association information (S150). S160).

6 is a diagram for explaining contents of generating an association matrix.

In the present invention, apart from generating a similarity matrix between datasets, a relational relationship between entities constituting a service and a dataset is calculated to construct a relational relationship matrix between entities and datasets.

Here, the entity corresponding to the service is a component constituting the first description corresponding to the dataset, such as the dataset provider and the date and time of organization registration (Factor), user search keywords of the dataset, and user profile components (affiliation, organization, region, age, etc.) and user preference information (preferred keyword, search record, etc.).

As an embodiment, the server 100 may generate a plurality of correlation matrices to correspond to the number of entities. That is, according to an embodiment of the present invention, a plurality of correlation matrices can be created according to the number of entities to be created. In this case, the entity and scale to be generated for the relationship matrix can be determined by the service operator, and can be appropriately created according to the characteristics and operational purpose of the service.

과 데이터셋의 관계는 다음 식 3과 같이 나타낼 수 있다.entity

The relationship between and the dataset can be expressed as Equation 3 below.

[Equation 3]

과

이 관계가 있는 경우 "1", 관계가 없는 경우 "0"으로 표현하는 것이며, 이는 식 4와 같이 나타낼 수 있다.In addition, the association matrix between entities and datasets can be created according to various methods, and the easiest way to express the relationship between each entity and the dataset based on the connection relationship is

class

If there is this relationship, it is expressed as "1", and if there is no relationship, it is expressed as "0", which can be expressed as in Equation 4.

[Equation 4]

As an embodiment, the server 100 may generate an association matrix by assigning a weight to each entity according to association information. Since association information is information that is frequently changed according to occurrence of an event, the server 100 may assign a differential value to each entity, and may reflect a weight for each entity through a method of assigning a differential value to each entity. For example, if a user has a preference for a specific organization, a high weight may be applied only to the specific organization when constructing relative association information. This leads to a different result from conventional filtering for a specific item. Such correlation information can be expressed as in Equation 5.

[Equation 5]

The server 100 may differently express the relationship matrix between the entity and the dataset according to the reference entity. Meanwhile, the creation and update timing of the association matrix may be determined according to a service operation policy.

As such, one embodiment of the present invention is characterized by utilizing not only the relationship between the dataset provider and the dataset, but also the relationship between various entities constituting the service. For example, the server 100 may configure a relation matrix using relation information (similarity matrix) between search keywords and datasets.

Next, the server 100 generates a relative dataset relevance matrix based on the similarity matrix and the relevance matrix (S170).

7 is a diagram for explaining a relative dataset relevance matrix.

When the entity-dataset association matrix is configured for each entity, the server 100 may construct a relative dataset association matrix using the previously calculated similarity matrix between datasets and the entity-data association matrix. there is.

As an embodiment, the server 100 may generate a relative dataset relevance matrix for each entity by applying a relevance matrix configured based on a corresponding entity to a pre-generated similarity matrix between datasets.

The relative dataset relevance matrix can be expressed as Equation 6.

[Equation 6]

을 기준으로 데이터셋간 상대적인 연관관계 정보 값

또한 다양한 방법을 적용하여 산출할 수 있다.The server 100 may generate this relative dataset relevance matrix by applying various methods, and the entity

Relative correlation information value between datasets based on

It can also be calculated by applying various methods.

The following Equation 7 shows the easiest example of calculating the relative relation information value between datasets.

[Equation 7]

That is, the server 100 sums the first association information corresponding to the first dataset constituting the association matrix and the second association information corresponding to the second dataset, and the first and second datasets. A relative dataset relevance matrix may be generated by multiplying the summation result by the similarity corresponding to .

In addition to this, the server 100 may apply at least one of filtering and weighting to a specific entity among entities, and this may use the above-described method applied when calculating correlation information between entities and datasets.

A series of processes related to the process of generating the second description in which the data level feature information proposed in the present invention is reflected, the process of generating an entity-dataset relation matrix, and the process of generating a relative dataset relatedness matrix are as shown in FIG. 8. .

8 shows a general application process of the technology proposed in the present invention, and it is possible to modify the process and change the update timing according to the service application and operation method, through which adaptive services are provided to users using the dataset. this is possible

As shown in FIG. 8 , when a first description corresponding to an explicit dataset related to an initial dataset is registered, a second description of a data level is extracted. Also, based on the first description and the second description, a similarity matrix between datasets and a relation matrix between entities and datasets are generated. Thereafter, a relative dataset relevance matrix is created using the generated similarity matrix and relevance matrix, and is configured into a structure that can be utilized for a dataset service.

In addition, in one embodiment of the present invention, when a new event (search, log, etc.) in which an entity is changed occurs, the entity-dataset association matrix can be updated, and the relative dataset association matrix is also updated. do. In this case, when configuring the relative dataset relevance matrix, weight conditions according to circumstances may be reflected as shown in FIG. 8, which may also be reflected in the aforementioned relevance matrix configuration.

Meanwhile, in the above description, steps S110 to S170 may be further divided into additional steps, or combined into fewer steps, according to an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order of steps may be changed. In addition, even if other omitted contents, the contents described in FIGS. 2 to 8 are also applied to the data set relationship automatic generation server 100 of FIG. 9 .

Hereinafter, a dataset relationship automatic generation server 100 for a predetermined dataset service according to an embodiment of the present invention will be described.

9 is a diagram for explaining a dataset relation automatically generating server 100 according to an embodiment of the present invention.

Referring to FIG. 9 , the server 100 for automatically generating dataset relationships includes a communication module 110 , a memory 120 and a processor 130 .

The communication module 110 provides service information corresponding to a dataset service request through data transmission and reception with a predetermined user terminal.

A program for automatically generating a dataset relationship for providing a dataset service is stored in the memory 120, and the processor 130 executes the program stored in the memory 120.

The processor 130 generates at least one second description by extracting data level characteristic information included in a dataset having a first description for a dataset service and reflecting it. And based on the first and second descriptions, a similarity matrix is created by calculating and reflecting the similarity between a plurality of datasets, and a correlation matrix is created by calculating and reflecting association information between an entity corresponding to a service and a dataset. Then, a relative dataset relevance matrix is created based on the similarity matrix and the relevance matrix, and based on this, the dataset service is provided to the user terminal.

The above-described method for automatically generating dataset relationships according to an embodiment of the present invention may be implemented as a program (or application) to be executed in combination with a server, which is hardware, and stored in a medium.

The aforementioned program is C, C++, JAVA, machine language, etc. It may include a code coded in a computer language of. These codes may include functional codes related to functions defining necessary functions for executing the methods, and include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, these codes may further include memory reference related codes for which location (address address) of the computer's internal or external memory should be referenced for additional information or media required for the computer's processor to execute the functions. there is. In addition, when the processor of the computer needs to communicate with any other remote computer or server in order to execute the functions, the code uses the computer's communication module to determine how to communicate with any other remote computer or server. It may further include communication-related codes for whether to communicate, what kind of information or media to transmit/receive during communication, and the like.

The storage medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and is readable by a device. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers accessible by the computer or various recording media on the user's computer. In addition, the medium may be distributed to computer systems connected through a network, and computer readable codes may be stored in a distributed manner.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: Dataset relationship auto-generation server
110: communication module
120: memory
130: processor

Claims (12)

In a method performed by a computer,
extracting data level feature information included in a dataset having a first description for a predetermined dataset service;
generating at least one second description in which the data level characteristic information is reflected in the first description of the dataset;
calculating a similarity between a plurality of datasets based on the first and second descriptions;
generating a similarity matrix based on the calculated similarity;
calculating association information between an entity corresponding to the service and the dataset;
generating a relationship matrix between the entity and a dataset based on the relationship information; and
Generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix,
Generating a relation matrix between the entity and the dataset based on the relation information,
Generating the association matrix by assigning a weight to each entity according to the association information;
How to automatically create dataset relationships.
According to claim 1,
Generating a second description in which the data level characteristic information is reflected in the first description of the dataset,
Generating the second description with a combination of different items corresponding to the type of data included in the dataset,
How to automatically create dataset relationships.
According to claim 1,
Generating a similarity matrix based on the calculated similarity,
Generating a relative similarity matrix for a second dataset based on the first dataset among the plurality of datasets,
How to automatically create dataset relationships.
According to claim 1,
The entity corresponding to the service includes at least one of a component constituting the first description corresponding to the dataset, a user search keyword of the dataset, a user profile component, and user preference information,
How to automatically create dataset relationships.
According to claim 1,
Generating a relation matrix between the entity and the dataset based on the relation information,
Generating a plurality of association matrices to correspond to the number of entities,
How to automatically create dataset relationships.
delete According to claim 1,
Generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix,
Generating a relative dataset relevance matrix for each entity by applying the association matrix to the similarity matrix,
How to automatically create dataset relationships.
According to claim 7,
Generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix,
adding up first association information corresponding to a first dataset constituting the association matrix and second association information corresponding to a second dataset; and
Generating the relative dataset relevance matrix by multiplying the summation result by similarities corresponding to the first dataset and the second dataset,
How to automatically create dataset relationships.
According to claim 7,
Generating a relative dataset relevance matrix based on the similarity matrix and the relevance matrix,
Further comprising applying at least one of filtering and weighting to a specific entity among the entities.
How to automatically create dataset relationships.
In the server for automatically generating dataset relationships for a given dataset service,
A communication module that provides service information corresponding to a dataset service request through data transmission and reception with a predetermined user terminal;
A memory storing a program for automatically generating a dataset relationship for providing the dataset service; and
Including a processor that executes the program stored in the memory,
As the program is executed, the processor extracts data level characteristic information included in a dataset having a first description for the dataset service and generates at least one second description by reflecting it, and Based on the first and second descriptions, a similarity matrix is created by calculating and reflecting similarity between a plurality of datasets, and a correlation matrix is created by calculating and reflecting association information between an entity corresponding to the service and the dataset, , Create a relative dataset relevance matrix based on the similarity matrix and relevance matrix,
wherein the processor generates the relative dataset relevance matrix by applying at least one of filtering and weighting to a specific entity among the entities;
Dataset Relationship Auto-Generation Server.
According to claim 10,
The processor adds up first association information corresponding to a first dataset constituting the association matrix and second association information corresponding to a second dataset, and Generating the relative dataset relevance matrix by multiplying the summation result by the corresponding degree of similarity,
Dataset Relationship Auto-Generation Server.
delete

Images