KR20210083007A - Cloud storage system and method of generating contents metadata in the cloud storage system - Google Patents

Abstract

The present invention relates to a cloud storage system. The cloud storage system includes: a relay server, when a file is uploaded from an external client terminal, requesting a virtual file system to create an index for the uploaded file; the virtual file system generating and storing first metadata which is file attribute information about the uploaded file in accordance with the request from the relay server, and extracting index information which is unique file identification information from the first metadata to transmit the same to the relay server; a file storage server including a cloud file storage to store the file provided from the relay server in the cloud file storage, providing storage position information to the relay server, and extracting second metadata which is content attribute information about the stored file to provide the same to the relay server; and a metadata server receiving, storing, and managing the index information about the file and the second metadata from the relay server. Therefore, the present invention is capable of enabling a search for an unstructured file as well as a structured file.

Description

Cloud storage system and method of generating contents metadata in the cloud storage system

The present invention relates to a cloud storage system and a method for generating metadata for unstructured data in the system, and more particularly, to a structured data file stored in a cloud storage system as well as metadata for content of unstructured data files. to a cloud storage system and a method for generating metadata that enable various searches for content by providing

As IoT technology develops and types of devices become diversified, the types of data generated from each device are also diversifying, and in processing such data, metadata for each data is an important indicator. .

Metadata, also called attribute information, is data that is given to content according to a certain rule in order to efficiently find information to be found among information that exists in a large amount. The metadata provided in this way includes the creator of the content, conditions of use, history of use, rights conditions, location and content of the content, and the like.

On the other hand, although general files have metadata set according to each file property, the properties are different and the metadata is insufficient in many cases. In particular, unstructured files do not sufficiently provide metadata because file attributes are not clear.

Korean Patent Publication No. 10-2015-0048260 Korean Patent Laid-Open Publication No. 10-2009-0037704 Korean Patent Publication No. 10-1285654

SUMMARY OF THE INVENTION An object of the present invention for solving the above-described problems is to generate and manage content-based metadata by analyzing content for unstructured data files uploaded to cloud storage to create and manage content-based metadata. To provide a managed cloud storage system.

Another object of the present invention is to provide a metadata generating method that enables content-based metadata to be generated with respect to unstructured data.

According to a first aspect of the present invention, there is provided a cloud storage system, comprising: a relay server that requests index creation for the uploaded file to a virtual file system when a file is uploaded from an external client terminal; In response to a request from the relay server, first metadata that is file attribute information for the uploaded file is generated and stored, and index information that is file unique identification information is extracted from the first metadata and transmitted to the relay server. file system; A cloud file storage is provided to store the file provided from the relay server in the cloud file storage, the storage location information is provided to the relay server, and second metadata that is content attribute information for the stored file is extracted to the relay server. File storage server provided by; and a metadata server that receives, stores, and manages index information and second metadata on a file from the relay server;

The relay server receives index information on a file from the virtual file system, provides and stores the file together with the index information to the file storage server, and stores the file storage location information and the second information on the file from the file storage server. The metadata is provided, and index information and second metadata for the file are provided to the metadata server, and stored and managed.

In the cloud storage system according to the first aspect of the present invention, the file storage server further comprises a second metadata generator,

The second metadata generating unit extracts a file signature from a binary file for a file stored in the cloud file storage, extracts information about a file category from the file signature, and the file Preferably, the second metadata is obtained according to a preset method according to the category of , and provided to the relay server.

In the cloud storage system according to the first aspect of the present invention, the second metadata generator reads a binary file for a file stored in the cloud file storage and extracts a file signature, a file category detection module for detecting information on a category of a file from the file signature; and acquiring second metadata, which is content attribute information for the file, using a preset module according to the information on the category of the file, storing the obtained second metadata, and providing second metadata to the relay server module; and

The second metadata acquisition module may include: an image recognition module for acquiring and providing second metadata for an image file; a text recognition module that obtains and provides second metadata for the text file; a medical data recognition module that obtains and provides second metadata for the medical data file; It is preferable to include a general-purpose recognition module that obtains and provides second metadata by analyzing a data structure format for a file whose format cannot be recognized normally.

In the cloud storage system according to the first aspect of the present invention, the image recognition module is a module configured by learning based on a CNN (Convolution Neural Network) algorithm, and the second metadata by the image recognition module is at least in an image file. It is desirable to include information about the included object.

In the cloud storage system according to the first aspect of the present invention, it is preferable that the text recognition module detects at least word and sentence information included in a text file and provides it as second metadata.

In the cloud storage system according to the first aspect of the present invention, in the relay server, when search condition information is input from the client terminal and a search is requested,

The input search condition information is analyzed, and if the search condition information relates to first metadata that is attribute information of a file, a VFS search is requested, and the search condition information is added to the second metadata that is content attribute information of the file. In this case, it is preferable that a search request is made to the metadata server, and when result information including index information is received according to the search request, a file corresponding to the index information is provided from the file storage server and provided to the client terminal.

A method for generating metadata according to a second aspect of the present invention relates to a method for generating metadata by a file storage server in a cloud storage system including a relay server, a virtual file system, a file storage server, and a metadata server, (a) extracting a file signature (File Signature) by reading a binary file (Binary File) for a file stored in the cloud file storage; (b) detecting information on a category of a file from the file signature; (c) acquiring second metadata, which is content attribute information for the file, using a preset module according to the information on the category of the file; and (d) storing the obtained second metadata and providing the obtained second metadata to the relay server.

In the method of generating metadata according to the second aspect of the present invention, in step (c), if the category of the file is an image file, the type of object included in the image is determined and at least information about the object is included in the second metadata and, if the category of the file is a text file, extracts words and sentences included in the text file, and provides at least information about the words and sentences as second metadata, and if the category of the file is medical data, medical data It is preferable to extract the imaging information and the patient information included in the , and provide at least the patient information as the second metadata.

The cloud storage system according to the present invention generates and provides first metadata, which is file attribute information for a file, and second metadata, which is content attribute information for a file, so that not only regular files but also non-standard files can be searched. do.

In addition, the cloud storage system according to the present invention identifies a category of a file through file signature extraction, and generates and provides second metadata that is content attribute information for a file using a method corresponding to the category of the file, thereby providing a file. It is possible to accurately extract the content attribute information of

1 is a configuration diagram illustrating a cloud storage system according to a preferred embodiment of the present invention as a whole.
2 is a block diagram illustrating a file system database of a virtual file system in a cloud storage system according to a preferred embodiment of the present invention.
3 is a block diagram schematically illustrating a configuration of a file storage server in a cloud storage system according to a preferred embodiment of the present invention.
4 is a diagram exemplarily showing second metadata for the contents of a file stored in the metadata server 40 in the cloud storage system according to the preferred embodiment of the present invention.
5 exemplarily shows a binary data file for a file in a cloud storage system according to a preferred embodiment of the present invention.
6 is a configuration diagram schematically illustrating an image recognition module in a cloud storage system according to a preferred embodiment of the present invention.
7 is a configuration diagram schematically illustrating a text recognition module in a cloud storage system according to a preferred embodiment of the present invention.
8 is a configuration diagram schematically illustrating a medical data recognition module in a cloud storage system according to a preferred embodiment of the present invention.
9 exemplarily illustrates a medical image data file (DICOM FILE) and second metadata obtainable therefrom in a cloud storage system according to a preferred embodiment of the present invention.
10 is a configuration diagram schematically illustrating a general-purpose recognition module in a cloud storage system according to a preferred embodiment of the present invention.
11 is a system flow diagram sequentially illustrating a file search and download process in a cloud storage system according to a preferred embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, a structure of a cloud storage system for generating and managing metadata for files and file contents according to a preferred embodiment of the present invention and first and second metadata in the system The creation and management method will be described in detail.

1 is a configuration diagram illustrating a cloud storage system according to a preferred embodiment of the present invention as a whole. 1, the cloud storage system 1 according to the present invention, a relay server (Resource Broker; 'RB', 10), a virtual file system (Virtual File System; 'VFS', 20), a file storage server ( Storage Element; 'SE', 30) and a metadata server (Metadata Server; 'MS', 40), and when a file is uploaded from an external client terminal 2, permission is checked and the cloud file storage 32 is stored and managed in the , and when a specific file is requested from the client terminal 2, the file is retrieved and provided to the client terminal, and various search services are provided to external client terminals. Hereinafter, each component constituting the cloud storage system according to the present invention will be described in more detail.

The relay server 10 is a server serving as a window for connecting the virtual file system 20, the file storage server 30, and the metadata server 40 to each other, and directly receives a request from an external client terminal. It is configured to receive and respond to requests from client terminals through communication with VFS, file storage servers, and metadata servers. Each of the servers and terminals according to the present invention is preferably designed as an API (Application Programming Interface) using HTTP (Hyper Text Transfer Protocol), which is a basic communication protocol.

The cloud storage system according to the present invention provides a file upload function, a file search function, and a file download function to client terminals through the relay server. In the file upload function, when a file is uploaded from the client terminal to the relay server, first metadata including attribute information of the uploaded file is generated and stored in the VFS, and the uploaded file is stored in the cloud of the file storage server. The second metadata stored in the file storage and including information on the contents of the uploaded file is stored in the metadata server. The file download function, when a download of a specific file is requested from the client terminal to the relay server, checks the first metadata and index information of the VFS, and obtains the file from the cloud file storage of the file storage server to the client terminal will be downloaded to The file search function analyzes search information input from the client terminal, searches the first metadata and the second metadata, and provides search result information. A detailed description of each of these functions will be described later.

The virtual file system (Virtual File System; 'VFS', 20) generates first metadata for the uploaded file according to a request from the relay server and stores it in a file system database, Index information that is file identification information among the first meta data is provided to the relay server. The first metadata includes a Universally Unique IDentifier (UUID), a file name, a path, user information attempting to upload, access right information, group access right information, file creation date information, and file size information. 2 is a block diagram illustrating a file system database of a virtual file system in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 2 , in the file system database of the VFS, first metadata, which is attribute information for each file, is stored on the basis of a UUID for each file. Meanwhile, the index information is information for identifying a file, and includes a PID, an ID, and a UUID among the first metadata. Here, the ID is a unique identifier in the virtual file system, and is a unique identifier for the virtual file folder that is never duplicated. The PID is a Parent ID, which is an ID having a higher correlation. If the PID value is null, it does not exist except for the top-level path. UUID is a unique identifier for the actual file data, even if the same file is uploaded multiple times, it is not the same and is created at the moment of upload. do.

3 is a block diagram schematically illustrating a configuration of a file storage server in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 3 , the file storage server 30 includes a cloud file storage 32 to store a file provided from the relay server together with index information in the cloud file storage, and store location information for the file. provided to the relay server. In addition, the file storage server 30 further includes a second metadata generation unit 34, and the second metadata generation unit 34 analyzes the contents of the files stored in the cloud file storage to The second metadata for the content is extracted, and the second metadata together with the index information for the file is provided to the relay server. On the other hand, when the relay server receives index information and second metadata for a file from the file storage server, the relay server updates the second metadata by providing it to the metadata server 40 .

The metadata server 40 receives, stores and manages the index information and second metadata on the file from the relay server. 4 is a diagram exemplarily showing second metadata for the contents of a file stored in the metadata server 40 in the cloud storage system according to the preferred embodiment of the present invention. Referring to FIG. 4 , the second metadata is attribute information on the contents of a file, and may be set in various ways according to the category of the file.

Referring to FIG. 3 , the second metadata generator of the file storage server includes a file category detection module 340 and a second metadata acquisition module 350 . The file category detection module 340 extracts a file signature from a binary data file (Binary File) for a file stored in the cloud file storage, and detects information about a file category from the file signature. . 5 exemplarily shows a binary data file for a file in a cloud storage system according to a preferred embodiment of the present invention. In FIG. 5, (a) is a part of a header of a binary data file for a PDF file, and (b) is a part of a header (Header) of a binary data file for a DCM file, which is a medical data file. Referring to FIG. 5 , from the header of the binary data file, it is possible to extract file signatures for determining the category of each file, respectively.

The second metadata obtaining module 350 obtains second metadata for the content of the file according to a method preset according to the category of the file and provides it to the relay server. The second metadata acquisition module 350 includes an image recognition module 352 , a text recognition module 354 , a medical data recognition module 356 , and a general-purpose recognition module 358 that are applied according to the category of the file. .

The image recognition module 352 is a module that obtains and provides second metadata for an image file. 6 is a configuration diagram schematically illustrating an image recognition module in a cloud storage system according to a preferred embodiment of the present invention. 6 , (a) shows a process of generating a second metadata extraction model by a learning process, and (b) is a process of defining second metadata for an arbitrary image using the extraction model will show Referring to FIG. 6 , the image recognition module according to the present invention is a model capable of generating second metadata from images by pre-learning images for training using deep learning technology as shown in (a). and, as shown in (b), by applying the image file to the model generated by the training, second metadata for the image file is defined and output. The above-described image recognition technology is a technology that has already been widely used using deep learning technology. Through a learning process, an object is detected from an image for each image, and the region including the detected objects is segmented (Segmentation). ) and classifying the object class is included. Accordingly, the image recognition module 352 extracts information on the type, characteristic, attribute, color, and background of an object included in the image and provides it as second metadata.

The text recognition module 354 is a module that obtains and provides second meta data for a file. 7 is a configuration diagram schematically illustrating a text recognition module in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 7 , the text recognition module extracts each keyword from a specific sentence included in a text file, detects the frequency and similarity of keywords in the sentence, and determines a rank using the frequency and similarity. calculation, and even a text summary is generated using this rank, and the generated information is acquired as second metadata. Accordingly, the text recognition module 354 extracts various keywords and summaries included in the text file and provides them as second metadata.

The medical data recognition module 356 is a module that obtains and provides second metadata for a medical data file. 8 is a configuration diagram schematically illustrating a medical data recognition module in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 8 , standard tags are extracted from a DCM file, which is a medical image data file, and standard tags are defined and obtained as second metadata. In general, the medical data file includes medical image data (DICOM File and DCM File), and related tags, ie, metadata, are already defined as standards for these DCM files as medical image data standards. Accordingly, the medical data recognition module may extract standard tags included in the medical image data file, select some and use them as second metadata.

9 exemplarily illustrates a medical image data file (DICOM FILE) and second metadata obtainable therefrom in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 9 , there are an image area and a tag area in medical image data, and second metadata can be selected and extracted from them. The second metadata that can be extracted from the medical image data may include a measurement date, measurement time, patient information, and the like.

The general-purpose recognition module 358 is a module that obtains and provides second metadata by analyzing a data structure format for a file whose format cannot be recognized normally. When the file stored in the cloud file storage is not the image file, text file, or medical data file described above, the second metadata for the contents of the file is extracted using the universal recognition module 358 . 10 is a configuration diagram schematically illustrating a general-purpose recognition module in a cloud storage system according to a preferred embodiment of the present invention. A JSON file is a data type in which key values are mapped, and is formed in a text-based data format. Referring to FIG. 10 , the general-purpose recognition module extracts a key and a key value from the JSON FILE, extracts information from the key and the key value, and defines it as second metadata. In addition, when the key included in the file is text, the second metadata may be extracted by applying the text rank technique.

Hereinafter, a file upload process in the cloud storage system according to a preferred embodiment of the present invention will be described in detail with reference to FIG. 10 .

10 is a system flowchart sequentially illustrating a file upload and data update process in a cloud storage system according to a preferred embodiment of the present invention.

Referring to FIG. 10 , first, a user uploads an arbitrary file to the relay server through an external client terminal (step 500). At this time, the file name, the path information to which the file is to be uploaded, the file size, and information about the user attempting to upload are uploaded together.

When a file is uploaded from the client terminal, the relay server requests permission check, index verification, and creation from the virtual file system (step 510).

In response to the request, the virtual file system checks whether or not there is an authority to write a file to the corresponding path. In this case, the virtual file system determines whether the user is the owner of the corresponding path, whether it is included in the corresponding group, and whether the file can be recorded without the ownership right, in order to check the authority. Next, for index verification, the virtual file system determines whether a path to upload a file exists and whether a space to record a file exists in the path. Next, the virtual file system randomly generates 128-bit UUIDs so that they are not duplicated, and extracts and stores first metadata, which is standardized file attribute information, from the file (step 512). The first metadata includes a file name, file size, user information, group information, creation date and time, authority, directory status, UUID, and the like.

Also, the virtual file system extracts index information from the first metadata according to the request of the relay server and transmits the extracted index information to the relay server (step 520). The index information is information for identifying each file, and includes an ID, PID, and UUID for each file.

When the index information is transmitted from the VFS, the relay server stores the file together with the index information in the cloud file storage of the file storage server (step 530), and updates the file storage information of the virtual file system (step 532), Notifies the client terminal that the file upload has been completed (step 534). If an error occurs when storing the file in step 530, the first metadata generated in step 512 is deleted without updating the file storage information, and the file upload process is terminated.

On the other hand, the file storage server generates second metadata that is attribute information of the contents of the file with respect to the file stored in the cloud file storage (step 540), and transmits the generated second metadata together with the index information of the file to the relay server to (step 542). Here, the second metadata generation of the file storage server may be implemented by the above-described second metadata generation unit of the file storage server.

The relay server receiving the index information of the file and the second metadata for the file transmits and stores the index information and the second metadata information to the metadata server, thereby updating the metadata server (step 550).

Hereinafter, a file search and download process in the cloud storage system according to a preferred embodiment of the present invention will be described in detail with reference to FIG. 11 .

11 is a system flow diagram sequentially illustrating a file search and download process in a cloud storage system according to a preferred embodiment of the present invention. Referring to FIG. 11 , when a search is requested along with input of search condition information from the client terminal (step 600), the relay server checks the input search condition information, and the search condition information is a first meta information that is file attribute information. It is determined whether the data corresponds to any of the data and the second metadata that is content attribute information (step 610).

If the search condition information relates to the first metadata that is the file attribute information, the first metadata of the virtual file system is searched and the authority for the searched file is checked (step 620). Next, the VFS transmits the index information on the file searched according to the search result and the storage location information of the cloud file storage for the file to the relay server (step 622).

If the search condition information relates to the second metadata that is the content attribute information, the second metadata of the metadata server is searched (step 624), and the metadata server transmits matching data, which is the search result information, to the relay server. do (step 626). After receiving the matching data from the metadata server, the relay server requests an inquiry and permission check to the virtual file system based on the matching data (step 628), and the virtual file system responds to the matching data after the inquiry and permission check. The index information and the storage location information of the cloud file storage are transmitted to the relay server (step 629).

Next, the relay server requests the file to the file storage server based on the index information and the file storage location information received from the virtual file system (step 630), and accordingly, the file storage server provides the file to the relay server, (Step 640), the relay server downloads the file to the client terminal (Step 650).

In the above, the present invention has been described with respect to the preferred embodiment thereof, but this is merely an example and does not limit the present invention, and those of ordinary skill in the art to which the present invention pertains without departing from the essential characteristics of the present invention. It will be appreciated that various modifications and applications not exemplified above in the scope are possible. In addition, differences related to such modifications and applications should be construed as being included in the scope of the present invention defined in the appended claims.

1: Cloud storage system
10: Relay Server (Resource Broker ; 'RB')
20: Virtual File System ('VFS')
30: File storage server (Storage Element; 'SE')
32 : Cloud File Storage
34: second metadata generating unit
340: file category detection module
350: second metadata acquisition module
40: Metadata Server ( Metadata Server ; 'MS')

Claims (8)

When a file is uploaded from an external client terminal, a relay server for requesting creation of an index for the uploaded file to the virtual file system;
Virtual data for generating and storing first metadata that is file attribute information for the uploaded file according to a request from the relay server, extracting index information that is file unique identification information from the first metadata and transmitting it to the relay server file system;
A cloud file storage is provided to store the file provided from the relay server in the cloud file storage, the storage location information is provided to the relay server, and second metadata that is content attribute information for the stored file is extracted to the relay server. File storage server provided by;
a metadata server that receives, stores, and manages index information and second metadata on a file from the relay server;
comprising, the relay server,
Receive index information about the file from the virtual file system,
Provide and store the file together with the index information to the file storage server, and receive file storage location information and second metadata for the file from the file storage server,
A cloud storage system, characterized in that it stores and manages by providing index information and second metadata on the file to the metadata server. The method of claim 1, wherein the file storage server further comprises a second metadata generator,
The second metadata generation unit,
Extracting a file signature from a binary file for a file stored in the cloud file storage, extracting information about a category of a file from the file signature, and a method preset according to the category of the file The cloud storage system according to claim 1, wherein the second metadata is obtained and provided to the relay server. The method of claim 2, wherein the second metadata generator comprises:
a file category detection module that reads a binary file for a file stored in the cloud file storage, extracts a file signature, and detects information about a file category from the file signature; and
A second metadata acquisition module for acquiring second metadata that is content attribute information for a file using a preset module according to information on a category of a file, and storing the acquired second metadata and providing it to the relay server ; and
The second metadata acquisition module includes:
an image recognition module that obtains and provides second metadata for the image file;
a text recognition module that obtains and provides second metadata for the text file;
a medical data recognition module that obtains and provides second metadata for the medical data file;
a general-purpose recognition module for obtaining and providing second metadata by analyzing a data structure format for a file whose format cannot be recognized normally;
A cloud storage system comprising a. The method of claim 3, wherein the image recognition module
A module configured by learning based on a Convolution Neural Network (CNN) algorithm, wherein the second metadata by the image recognition module includes information about an object included in an image file. The method of claim 3, wherein the text recognition module
A cloud storage system, comprising detecting word and sentence information included in a text file and providing it as second metadata. According to claim 1, wherein the relay server,
When search condition information is input from the client terminal and a search is requested,
Analyze the input search condition information,
If the search condition information relates to first metadata that is attribute information of a file, a VFS search is requested, and if the search condition information relates to second metadata that is content attribute information of a file, a search is requested to a metadata server, ,
When result information including index information is received according to a search request, a file corresponding to the index information is provided from the file storage server and provided to the client terminal. A method for generating metadata by a file storage server in a cloud storage system including a relay server, a virtual file system, a file storage server, and a metadata server, the method comprising:
(a) extracting a file signature (File Signature) by reading a binary file (Binary File) for a file stored in the cloud file storage;
(b) detecting information on a category of a file from the file signature;
(c) acquiring second metadata, which is content attribute information for the file, using a preset module according to the information on the category of the file; and
(d) storing the obtained second metadata and providing it to the relay server;
A method for generating metadata in a cloud storage system, comprising: generating and providing second metadata that is content attribute information for a file stored in the cloud file storage. The method of claim 7, wherein the step (c) comprises:
If the category of the file is an image file, the type of object included in the image is determined and at least information about the object is provided as second metadata,
If the category of the file is a text file, extracts words and sentences included in the text file, and provides at least information about the words and sentences as second metadata;
If the category of the file is medical data, the method for generating metadata in a cloud storage system, comprising extracting photographing information and patient information included in the medical data, and providing at least patient information as second metadata.

Images