KR100817250B1 - Data transfer method and the system, and the store device which records a method - Google Patents

Abstract

The present invention relates to a data uploading method and system using distributed processing, and a recording medium on which the method is recorded.

More specifically, in a method and system for uploading data via the Internet, (a) reading data to be uploaded and storing it in a memory, (b) dividing the data and generating each temporary file Storing in the cache, transmitting and storing each temporary file having been divided into a temporary memory, and (c) reading and distributing each temporary file stored in the temporary memory and transferring the temporary file to the Internet. A data uploading method using distributed processing is provided,

A user that stores a variety of data that the user wants to upload in its own database, and provides a program for automatically uploading and managing the data by distributing the data to web servers of various shopping mall companies or content companies by the user's selection. Data uploading using distributed processing including a web server of a shopping mall company or a content company that performs internet sales or promotion using data automatically distributed and processed by the user's selection in conjunction with a terminal and the user terminal. System is provided,

(a) reading the data to be uploaded and storing it in the memory, (b) reading the data to be uploaded and loading it into the temporary memory, and dividing the data according to a predetermined size of capacity, and (c) Generating the divided data into respective temporary files, and then transferring each of the divided temporary files to a temporary memory for distributed processing, (d) reading each temporary file stored in the temporary memory, Duplicate each temporary file to generate a child module for distributed processing as many as the number of temporary files, and then adjust the order; and (e) using the child modules generated as many as the number of temporary files as described above. Provides a recording medium that records a program for realizing the function of controlling the speed and completion of temporary files and transmitting them to the network.

Distributed Processing, Threads, Temporary Files, Cache, Uploading

Description

Data transfer method and the system, and the store device which records a method}

1 is a view showing a schematic configuration of a data uploading system using distributed processing according to an embodiment of the present invention.

2 is a block diagram showing the detailed block configuration of the file cache processing unit of the program according to the present invention.

3 is a block diagram illustrating the detailed block configuration of the distributed processing unit in the program according to the present invention.

4 is a diagram illustrating a data partitioning method using distributed processing according to an exemplary embodiment of the present invention.

5 is a view showing a dispersion processing method using the dispersion treatment according to an embodiment of the present invention.

*** Explanation of symbols for main parts of drawing ***

100: main server 200: network

300: user terminal 310: distributed processing main module

315: thread division module 320: thread control module

325: thread sink module 330: replicated distributed processing module

340: memory queue 345: file division module

350: temporary file generation module 355: distributed processing module

400: Web server (shopping mall or content)

The present invention relates to a data uploading method and system using distributed processing, and a recording medium on which the method is recorded.

More specifically, when a user uploads a large amount of data such as product information or content information to the Internet using a user terminal, the data to be uploaded is divided and distributed to one or more web servers connected to the Internet through distributed processing. A data uploading method and system using distributed processing which can be uploaded, and a recording medium on which the method is recorded.

In general, in an Internet environment through a communication service such as the Internet using a computer, a user registers as a member of his or her computer or a specific web server, and then uses various solutions such as text, voice, image, and video using the solution provided by the web server. Is transferring data.

The most widely used method of data transmission in the Internet is packet switching. A packet is a "packet" originally handled by a post office, which is divided into pieces of appropriate size and tagged with a destination label. A packet in the Internet communication network refers to a binary number, or group of bits, containing data and call control signals. In particular, when transmitting data in a packet-switched system, the packet is decomposed into a basic transmission unit called a packet, and then transmitted again. Reassemble into data and process.

When you send an email, html file, gif file, or any other kind of file over the Internet from one place to another, the TCP layer of TCP / IP is cut to an efficient size for sending this file. Each divided packet is assigned a separate number and includes the destination Internet address, and each packet can be transmitted through a different path through the Internet. When all sent packets arrive, the TCP layer's receiver reassembles the packets back into the original file.

On the other hand, sellers or shoppers who want to sell product information or content information to various shopping mall companies such as clothing, food, cosmetics, and various content companies such as music, movies, and videos may use the product information or content information for the shopping mall companies or content companies. I'm using the Internet to upload to a web server.

However, despite the improvement of the transmission speed due to the Internet infrastructure, the sellers and shoppers upload a lot of large-capacity or product information to sell to a web server of a number of shopping mall servers or contents companies. There was a problem that the transmission speed is lowered due to product information or a large amount of content information.

The present invention has been made to solve the problems described above, when the user uploads various large-capacity data such as product information or content information to the Internet using a user terminal, the data to be uploaded is divided and then distributed processing The present invention relates to a data uploading method and system using a distributed process that can be uploaded to at least one web server connected to the Internet through the Internet, and a recording medium on which the method is recorded.

Another object of the present invention is to provide a recording medium on which a method for automatically uploading data to the Internet through a distributed process after dividing the data to be uploaded is recorded.

As described above, a system and method for uploading data through the Internet, and a recording medium recording the method, store various data to be uploaded by the user in its own database, A user terminal that provides a program for distributing the data to a web server of a shopping mall company or a content company to automatically upload and manage the data and the user terminal that is linked to the user terminal and distributed and automatically uploaded by the user's selection. There is provided a data uploading system using a distributed process including a web server of a shopping mall company or a content company that performs Internet sales or promotions,

(a) reading the data to be uploaded and storing it in a memory, and (b) dividing the data, creating each temporary file, storing the data in a cache, and transferring each temporary file having been partitioned to the temporary memory. And (c) reading and distributing each temporary file stored in the temporary memory and transmitting the data to the Internet, wherein the data uploading method using the distributed processing is provided.

(a) reading the data to be uploaded and storing it in the memory, (b) reading the data to be uploaded and loading it into the temporary memory, and dividing the data according to a predetermined size of capacity, and (c) Generating the divided data into respective temporary files, and then transferring each of the divided temporary files to a temporary memory for distributed processing, (d) reading each temporary file stored in the temporary memory, Duplicate each temporary file to generate a child module for distributed processing as many as the number of temporary files, and then adjust the order; and (e) using the child modules generated as many as the number of temporary files as described above. This is achieved by providing a recording medium that records a program for realizing the function of controlling the speed and completion of temporary files and transmitting them to the network.

First of all, the term "thread" used in the present invention means that a plurality of execution paths (child processes) exist in one process and are executed at the same time. One process uses two or more child processes at the same time. It is meant to be used to perform a task.

Hereinafter, a data uploading method and system using distributed processing according to an exemplary embodiment of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a view showing a schematic configuration of a data uploading system using distributed processing according to an embodiment of the present invention. In more detail with reference to Figure 1,

The system of the present invention is composed of a user terminal 300, a network 200, a main server 100, a web server 400 and the like.

The user terminal 300 may store various data that the user wants to upload in its own database, and automatically upload and manage the data by distributing the data to web servers of various shopping mall companies or content companies by the user's selection. Provide a program to make it work.

Of course, the main server 100 is also connected to the user terminal 300, the user receives a variety of data to be uploaded and stored in a database divided by the user or company, etc., according to the user's selection In the web server of various shopping mall companies such as food, accessories, toys, health food, cosmetics, or content companies such as music, movies, and videos, the stored data is distributed and automatically uploaded and managed.

The web server 400 is a server of a shopping mall company that sells various products on the Internet, or a server of a company that provides various contents such as music, movies, and videos. The web server 400 is linked to the user terminal and distributed by the user's selection. Internet sales or promotions are performed using the automatically uploaded data.

The program mounted on the user terminal 300 or the main server 100 is a control unit (not shown) for controlling the overall function of transmitting the data to the web server 400 using the network after partitioning and distributing the data; It further comprises a file cache processing unit for dividing the data to be uploaded into one or more temporary files stored in the memory and a distributed processing unit for copying and transmitting the one or more temporary files stored in the memory after distribution.

2 is a block diagram showing the detailed block configuration of the file cache processing unit of the program according to the present invention. 3 is a block diagram illustrating the detailed block configuration of the distributed processing unit in the program according to the present invention. 2 and 3 will be described in more detail.

As shown in FIG. 2, the file cache processor of the present invention further includes a memory 360, a memory queue 340, a file division module 345, a temporary file generation module 350, and a distributed processing module 355. It is composed.

The memory 360 includes a hard disk or a removable memory in the case of the user terminal 300, and an internal memory or a separate external memory in the case of the main server 100. Content information such as various music, movies, and videos to be uploaded or clothing, food, cosmetics, health food, and accessories to be sold through the shopping mall are stored by selection of the information.

The memory queue 340 is a function of temporarily loading or storing data to be searched, extracted, copied, and transmitted when the user selects data to be uploaded from various content information or product information stored in the memory 360. Do this.

The file division module 345 performs a function of dividing the content information or the product information data stored in the memory queue 340 according to the capacity to transmit the data of the content information or the product information to the shopping mall or the web server 400 of the content. Alternatively, the data of the product information can be divided into pieces between 2 and 16 according to their capacity, so that they can be easily transmitted and fast.

The temporary file generation module 350 performs a function of generating the data divided according to the capacity in the file division module 345 into each temporary file.

The distributed processing module 355 performs a function of transmitting the divided newly created temporary files to a memory which is a separate storage space for distributed processing in order to transmit them to the Internet.

As shown in FIG. 3, the distributed processing unit of the present invention includes a distributed processing main module 310, a memory 335, a thread splitting module 315, a thread sinking module 325, a thread control module 320, and a duplicated distribution. It further comprises a processing module 330 and the like.

The memory 335 receives and temporarily stores each temporary file divided by the file cache unit.

The distributed processing main module 310 reads the temporary files stored in the memory 335 and stores them as data for distributed processing.

The thread division module 315 performs a function of replicating itself by one of the data of each temporary file and generating a child module for distributed processing to handle uploading.

The thread sync module 325 performs a function of adjusting the order so that each divided child module is not duplicated or previously processed to upload the data to the Internet.

The thread control module 320 controls the generation and destruction of each divided child module and adjusts speed and completion so that the data can be received by the web server.

The replicated distributed processing module 330 is generated by the thread splitting module 315 to perform uploading of one piece of data and to notify the completion of processing, and then extinguish.

Hereinafter, a method of dividing data using the data uploading system configured as described above and uploading to the Internet using distributed processing will be described in more detail.

4 is a diagram illustrating a data partitioning method using distributed processing according to an exemplary embodiment of the present invention. 5 is a view showing a dispersion processing method using the dispersion treatment according to an embodiment of the present invention. In more detail with reference to Figures 4 and 5,

First, a user who wants to upload product information or content information to various shopping mall companies or content companies can upload the product information or content information using a user terminal or a main server. In an embodiment of the present invention, a main server is used. Shall be.

The user stores the product information or the content information in the database of the main server using the user terminal, and then the web server of various shopping mall companies such as clothing, food, cosmetics, or content companies such as music, movies, and videos. If the product information or content information is to be uploaded to the main server, at least one data to be uploaded is selected from among pre-stored product information and content information.

The main server reads each original data sequentially according to the order selected in the database of the main server (S402), and after determining whether there is an error in the contents of the original data (S404), if there is no error, the original data is stored in a memory queue ( Load and store according to the order of the original data selected in 340 (S406),

The original data stored in the memory queue 406 is divided into one or more temporary files before being transmitted to the web server 400 through the Internet. The first data stored in the memory queue 406 is first divided and stored.

First, it is determined whether the same file in which the original data to be split is previously divided exists. When the same file is previously split (S408), the original data does not need to go through a separate division process. The existing divided file is output to the memory for distributed processing (S420).

If there is no previously divided file (S408), the file division module 345 is operated to divide the original data (S410). If the determination is not sufficient (S412) without dividing the original data is output to the memory for distributed processing as it is (S422),

If there is enough memory to store and load when dividing the original data (S412), the original data is started to be divided, and the partitioning is 2 to 16 temporary pieces depending on the capacity of the data (for example, divided into 1024 KB units). The data is divided into data, optimized according to the file system of the network and the web server, and the original data is divided (S414).

If the temporary data divided in the above step does not match the original data divided (S416), the original data is output to the memory for distributed processing as it is (S422), if the original data (S416), the temporary file The temporary data divided by the generation module 350 are generated as respective temporary files, and loaded and stored in a separate memory (for example, loaded in a cache) so that the divided temporary files can be used later. (S418).

Each temporary file generated in the above step is output to the memory for distributed processing by the distributed processing module 355 (S420).

Hereinafter, a method of distributing the temporary file generated as described above and transmitting it to the Internet will be described in more detail.

Each temporary file generated by dividing one original data in the step (S502) is subjected to the total number check and data integrity check by the distributed processing main module 310 (S504), if there is an error in the content (S506) When the problem is displayed, retrying is performed (S508). If there is no problem (S506), each temporary file is loaded and stored in the memory queue 335 for distributed processing (S510).

It is determined whether the temporary files stored in the memory queue 335 can be distributed (S512). If the number of temporary files is one or less, the temporary files having one or less are transmitted directly through the Internet without the need for distributed processing. The product or the content is registered in the shopping mall or the content company (S524).

Of course, it is obvious that the number of temporary files for determining whether the distributed processing is possible may be changed to 10 or as necessary, and may also vary depending on a network and a system of a corresponding web server.

If the number of temporary files is one or more (S512), a thread (child module) which is a distributed distributed processing module 330 for copying each temporary file and transmitting it to the Internet by operating the thread division module 315. Or a child process), and first determine whether there is a currently created thread (S516), and if not, create a thread for a new distributed processing (S522).

In addition, if there are currently created threads, it is determined whether the total number of created threads exceeds the specified number (for example, 10), and if less, wait for a while and retry the thread splitting module 315 again. It is made (S520).

If the total number of created threads exceeds the specified number (for example, 10) (S518), the threads that are distributed processing modules 330 replicated by the thread sync module 325 are not duplicated or processed earlier. The order is adjusted so as to avoid the step (S522), thereby transmitting the corresponding product information or content information to the Internet (S524).

As described above, by dividing a large amount of product information or content information into one or more temporary files, and transmitting the divided plurality of temporary files to the Internet through distributed processing, product information and content information can be transmitted more quickly.

Although one preferred embodiment of the present invention has been described above, it is clear that the present invention may use various changes, modifications, and equivalents, and may be applied in the same manner by appropriately modifying the embodiment. Accordingly, the above description does not limit the scope of the invention as defined by the limits of the following claims.

As described above, the data uploading method and system using the distributed processing of the present invention, after the user divides a large amount of data, such as product information or content information on the Internet using a user terminal, one or more connected to the Internet through the distributed processing There is an effect that can be uploaded to the shopping mall server or content server more quickly.

Claims (11)

In a method of uploading data using a user terminal or main server connected to the Internet and a web server of a content company, (a) reading data to be uploaded and storing it in a memory of a user terminal or a main server; (b) dividing the data, generating each temporary file and storing the temporary file in a cache of a user terminal or a main server, and transferring each temporary file of the splitting to a temporary memory of the user terminal or a main server; (c) reading and distributing each temporary file stored in a temporary memory of the user terminal or may server and transmitting the distributed temporary file to a corresponding web server through the Internet; Data uploading method using a distributed process comprising a. The method of claim 1, wherein step (b) (b-1) reading data to be uploaded from the user terminal and loading the data into a memory of the user terminal or the main server; (b-2) dividing the data according to a predetermined size of capacity; (b-3) generating the divided data into respective temporary files; And (b-4) transmitting the divided temporary files to a memory of a user terminal or a main server for distributed processing; Data uploading method using a distributed process further comprises. The method of claim 2, wherein in step (b-2) The data uploading method using distributed processing, wherein the data to be uploaded is divided into 2 to 16 pieces according to capacity. The method of claim 1, wherein step (c) (c-1) reading each temporary file stored in the memory of the user terminal or the main server; (c-2) copying the temporary files, generating child modules for distributed processing as many as the number of temporary files, and then adjusting the order; And (c-3) transmitting the temporary files to a web server of a content company by using a child module generated as many as the number of temporary files; Data uploading method using a distributed process further comprises. In a system for uploading data using a user terminal or main server connected to the Internet and a web server of a content company, A user that stores a variety of data that the user wants to upload in its own database, and provides a program for automatically uploading and managing the data by distributing the data to web servers of various shopping mall companies or content companies by the user's selection. terminal; And A web server of a shopping mall company or a content company linked to the user terminal to perform internet sales or promotion using data uploaded and distributed by the user's selection; Data uploading system using distributed processing, characterized in that comprises a. In a system for uploading data using a user terminal or main server connected to the Internet and a web server of a content company, After the user connects to the user terminal and stores a variety of data in the database, and by the user's choice to provide a program that can automatically upload and manage the data distributed to the web server of various shopping mall companies or content companies Main server; And A web server of a shopping mall company or a content company interworking with the main server to perform internet sales or promotions using data automatically distributed and processed by the user's selection; Data uploading system using distributed processing, characterized in that comprises a. 7. The method of claim 5 or 6, wherein the program is A controller which divides and distributes the data and transmits the data to the web server through a network; A file cache processor for dividing data to be uploaded into one or more temporary files and storing the data in a memory; And A distribution processing unit for distributing and transmitting one or more temporary files stored in the memory after copying; Data uploading system using a distributed process, characterized in that further comprises. The method of claim 7, wherein the file cache processing unit A memory storing various data according to a user's selection; A memory queue for selecting, retrieving, extracting, loading or storing data to be uploaded from the memory; A file division module for dividing data stored in the memory queue according to capacity; A temporary file generation module for generating the divided file as each temporary file; And A distributed processing module for transmitting the divided temporary files to a memory for distributed processing; Data uploading system using a distributed process, characterized in that further comprises. The method of claim 7, wherein A data uploading system using distributed processing, characterized by dividing the data into pieces between 2 and 16 according to capacity. The method of claim 7, wherein the dispersion processing unit A memory for storing each divided temporary file; A distributed processing main module which reads the temporary files and stores the temporary files as data for distributed processing; A thread splitting module for replicating itself by one of the data and generating a child module for distributed processing to handle uploading; A thread sink module for adjusting the order so that the divided child modules are not duplicated or processed in advance; A thread control module for controlling generation and destruction of each divided child module and adjusting speed and completion to receive the data from a web server; And A distributed distributed processing module which is a child module generated by the thread division module and responsible for uploading one data, and then destroyed after notifying the end of processing; Data uploading system using a distributed process that further comprises. In the method and system for uploading data using a user terminal or main server connected to the Internet and a web server of a content company, (a) reading data to be uploaded and storing it in a memory; (b) reading data to be uploaded, loading the data into temporary memory, and dividing the data according to a predetermined size; (c) generating the divided data into respective temporary files and then transferring each of the divided temporary files to a temporary memory for distributed processing; (d) reading each temporary file stored in the temporary memory, duplicating each temporary file, generating a child module for distributed processing by the number of each temporary file, and adjusting the order; And (e) a function of controlling the speed and completion of the temporary files and transmitting them to a network by using child modules generated as many as the number of temporary files; A recording medium having recorded thereon a program for realizing the same, wherein the recording medium can be processed by a computer.

Images