CN111104372A - Mobile internet big data processing system and processing method thereof - Google Patents
Mobile internet big data processing system and processing method thereof Download PDFInfo
- Publication number
- CN111104372A CN111104372A CN201911335628.6A CN201911335628A CN111104372A CN 111104372 A CN111104372 A CN 111104372A CN 201911335628 A CN201911335628 A CN 201911335628A CN 111104372 A CN111104372 A CN 111104372A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- module
- original
- file data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 8
- 238000006243 chemical reaction Methods 0.000 claims abstract description 34
- 238000007726 management method Methods 0.000 claims abstract description 11
- 238000013075 data extraction Methods 0.000 claims description 13
- 238000013500 data storage Methods 0.000 claims description 6
- 238000013523 data management Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000000034 method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/116—Details of conversion of file system types or formats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a mobile internet big data processing system and a processing method thereof in the technical field of big data management, wherein the mobile internet big data processing system comprises a data file processing system, a data file management system and an original file data cloud storage system; the data file processing system is used for acquiring a file to be processed, encoding the file data, storing the encoded file data in the original file data cloud storage system, and performing format conversion on the encoded file data; the original file data cloud storage system is used for storing the coding information of the file data before processing and the original file data in a classified mode.
Description
Technical Field
The invention relates to the technical field of big data management, in particular to a mobile internet big data processing system and a processing method thereof.
Background
Big data refers to a collection of data whose content cannot be captured, managed, and processed within a certain time using conventional software tools. Big data has five characteristics, namely large quantity, high speed, diversity, low value density and authenticity. It has no statistical sampling method, but only observes and tracks what happens.
When the format conversion processing is carried out on the existing big data, once the conversion is carried out and after the conversion is finished, the original file data information cannot be traced through the converted data information, so that much inconvenience is brought to the conversion processing of the data.
Disclosure of Invention
The invention aims to provide a mobile internet big data processing system and a processing method thereof, which are used for solving the problem that the original file data information cannot be traced through the converted data information once the conversion is performed and after the conversion is completed when the format conversion processing is performed on the big data in the prior art.
In order to achieve the purpose, the invention provides the following technical scheme: a mobile internet big data processing system comprises a data file processing system, a data file management system and an original file data cloud storage system;
the data file processing system is used for acquiring a file to be processed, encoding the file data, storing the encoded file data in the original file data cloud storage system, and performing format conversion on the encoded file data;
the original file data cloud storage system is used for storing the encoding information of the file data before processing and the original file data in a classified manner;
and the data file management system is used for inputting the converted file data to inquire the original file data information of the original file data cloud storage system.
Preferably, the data file processing system comprises a data file acquisition module, a data file coding module and a format conversion module;
the data file acquisition module is used for acquiring file data of which the format is required to be converted by a user;
the data file coding module is used for coding the acquired file to be converted and storing the file to be converted into the original file data cloud storage system;
and the format conversion module is used for carrying out format conversion on the coding file passing through the data file coding module and reserving coding information.
Preferably, the data file coding module comprises a file format type obtaining module, a format channel identification module and an automatic file uploading module;
the file format type acquisition module is used for acquiring the format type of the encoded file data;
the format channel identification module is used for identifying a corresponding format uploading channel uploaded to the original file data cloud storage system;
and the file automatic uploading module is used for uploading data of the original file data cloud storage system through the identification channel of the format channel identification module.
Preferably, the original file data cloud storage system comprises a storage classification module and a data storage library module which correspond to the file automatic uploading module;
the storage classification module is used for carrying out cloud classification on the files uploaded through different channels;
and the data storage library module is used for permanently storing the classified files.
Preferably, the data file management system comprises a file query module and an encoded data extraction module;
the file query module is used for inputting the format conversion file data information with the coding information;
the coded data extraction module is used for extracting coded information in a file to be queried and querying the original file of the corresponding code by the original file data cloud storage system.
Preferably, the original file data cloud storage system further comprises a code query module, a code storage module and a file data download module corresponding to the code data extraction module;
the code query module is used for querying the corresponding file codes in the code storage module through the code information extracted by the code data extraction module;
the code storage module is used for storing the coded data corresponding to the stored original file data;
and the file data downloading module is used for downloading the stored code file in a cloud after the corresponding code is inquired.
The invention also provides a processing method of the mobile internet big data processing system, which comprises the following steps:
acquiring original file data, encoding the original file data and then carrying out cloud storage;
step two, carrying out format conversion on the file data coded in the step one to obtain file data with a converted format, wherein the converted file data retains coded data;
and thirdly, extracting the coded data of the converted file data with the coded data from the coded data, and inquiring the cloud storage library to find out the original file data corresponding to the code.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, by utilizing the file coding technology, coding information can be added when the format of the big data file is converted, and then after the conversion is completed, the fast query of the original file data is realized, so that the tracing difficulty of the converted file when the original file is lost is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of the overall system architecture of the present invention;
FIG. 2 is a system diagram of a data file encoding module and an original file data cloud storage system according to the present invention;
FIG. 3 is a system diagram of an original file data cloud storage system box encoding data extraction module according to the present invention;
FIG. 4 is a schematic flow chart of the method of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 to fig. 3, the present invention provides a technical solution: a mobile internet big data processing system comprises a data file processing system, a data file management system and an original file data cloud storage system;
the data file processing system is used for acquiring a file to be processed, encoding the file data, storing the encoded file data in the original file data cloud storage system, and performing format conversion on the encoded file data;
the original file data cloud storage system is used for storing the encoding information of the file data before processing and the original file data in a classified manner;
and the data file management system is used for inputting the converted file data to inquire the original file data information of the original file data cloud storage system.
It should be noted that, when format conversion is performed on a big data file, in order to realize query of the converted file on original file information, the original file information may be input through the data file processing system, the unique code is compiled on the file, the file is stored in the cloud storage system, format conversion is performed on the original file added with the code information to obtain a converted file, and when the original file information corresponding to the converted file needs to be queried, the converted file may be directly input into the data file management system to realize query of the specified code, and then the specified code is searched for the source file data cloud storage system, so that the original file information corresponding to the code information is found, and loss of the original file data information after format conversion is prevented.
In a further embodiment, the data file processing system comprises a data file acquisition module, a data file encoding module and a format conversion module;
the data file acquisition module is used for acquiring file data of which the format is required to be converted by a user;
the data file coding module is used for coding the acquired file to be converted and storing the file to be converted into the original file data cloud storage system;
the format conversion module is used for carrying out format conversion on the coding file passing through the data file coding module and reserving coding information;
after the data file is acquired and the file data is encoded, one copy of the file data is stored in a cloud end for storage, the other copy of the file data is subjected to format conversion, and the encoded data of the converted file is still reserved.
In a further implementation mode, the data file coding module comprises a file format type obtaining module, a format channel identification module and an automatic file uploading module;
the file format type acquisition module is used for acquiring the format type of the encoded file data;
the format channel identification module is used for identifying a corresponding format uploading channel uploaded to the original file data cloud storage system;
the file automatic uploading module is used for uploading data of the original file data cloud storage system through the identification channel of the format channel identification module;
by acquiring the format type of the file and uploading the file through the file format channel corresponding to the file, the file uploading efficiency can be effectively improved.
In a further embodiment, the original file data cloud storage system comprises a storage classification module and a data storage library module which correspond to the file automatic uploading module;
the storage classification module is used for carrying out cloud classification on the files uploaded through different channels;
the data storage library module is used for permanently storing the classified files;
the uploaded files are sorted and then stored, so that the data can be stored orderly.
In a further embodiment, the data file management system comprises a file query module and an encoded data extraction module;
the file query module is used for inputting the format conversion file data information with the coding information;
the coded data extraction module is used for extracting coded information in a file to be queried and querying an original file of a corresponding code by the original file data cloud storage system;
by inputting the converted file and inquiring the file coding information, the corresponding coding file stored in the cloud can be quickly searched.
In a further implementation manner, the original file data cloud storage system further comprises a code query module, a code storage module and a file data download module corresponding to the code data extraction module;
the code query module is used for querying the corresponding file codes in the code storage module through the code information extracted by the code data extraction module;
the code storage module is used for storing the coded data corresponding to the stored original file data;
the file data downloading module is used for performing cloud downloading on the stored coding file after the corresponding codes are inquired;
the corresponding file coding information can be inquired at the cloud end, file data of the specified codes can be searched and downloaded, and rapid conversion between the converted file and the original file can be conveniently realized.
Referring to fig. 4, as shown in fig. 4, the present invention further provides a processing method of a mobile internet big data processing system, including the following steps:
acquiring original file data, encoding the original file data and then carrying out cloud storage;
step two, carrying out format conversion on the file data coded in the step one to obtain file data with a converted format, wherein the converted file data retains coded data;
extracting the coded data of the converted file data with the coded data to query a cloud storage library, and finding out original file data corresponding to the code;
the file conversion processing after the file is coded by acquiring the original file data can be convenient for realizing the interconversion between the converted file and the corresponding original file directly through the coding information when the original file information is searched after the file conversion processing is finished, and the traceability efficiency of the converted file is improved.
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (7)
1. A mobile internet big data processing system is characterized in that: the system comprises a data file processing system, a data file management system and an original file data cloud storage system;
the data file processing system is used for acquiring a file to be processed, encoding the file data, storing the encoded file data in the original file data cloud storage system, and performing format conversion on the encoded file data;
the original file data cloud storage system is used for storing the encoding information of the file data before processing and the original file data in a classified manner;
and the data file management system is used for inputting the converted file data to inquire the original file data information of the original file data cloud storage system.
2. The mobile internet big data processing system according to claim 1, wherein: the data file processing system comprises a data file acquisition module, a data file coding module and a format conversion module;
the data file acquisition module is used for acquiring file data of which the format is required to be converted by a user;
the data file coding module is used for coding the acquired file to be converted and storing the file to be converted into the original file data cloud storage system;
and the format conversion module is used for carrying out format conversion on the coding file passing through the data file coding module and reserving coding information.
3. The mobile internet big data processing system according to claim 2, wherein: the data file coding module comprises a file format type obtaining module, a format channel identification module and an automatic file uploading module;
the file format type acquisition module is used for acquiring the format type of the encoded file data;
the format channel identification module is used for identifying a corresponding format uploading channel uploaded to the original file data cloud storage system;
and the file automatic uploading module is used for uploading data of the original file data cloud storage system through the identification channel of the format channel identification module.
4. The mobile internet big data processing system according to claim 3, wherein: the original file data cloud storage system comprises a storage classification module and a data storage base module which correspond to the file automatic uploading module;
the storage classification module is used for carrying out cloud classification on the files uploaded through different channels;
and the data storage library module is used for permanently storing the classified files.
5. The mobile internet big data processing system according to claim 1, wherein: the data file management system comprises a file query module and an encoded data extraction module;
the file query module is used for inputting the format conversion file data information with the coding information;
the coded data extraction module is used for extracting coded information in a file to be queried and querying the original file of the corresponding code by the original file data cloud storage system.
6. The mobile internet big data processing system according to claim 5, wherein: the original file data cloud storage system also comprises a code query module, a code storage module and a file data downloading module which correspond to the code data extraction module;
the code query module is used for querying the corresponding file codes in the code storage module through the code information extracted by the code data extraction module;
the code storage module is used for storing the coded data corresponding to the stored original file data;
and the file data downloading module is used for downloading the stored code file in a cloud after the corresponding code is inquired.
7. A processing method of a mobile internet big data processing system is characterized by comprising the following steps:
acquiring original file data, encoding the original file data and then carrying out cloud storage;
step two, carrying out format conversion on the file data coded in the step one to obtain file data with a converted format, wherein the converted file data retains coded data;
and thirdly, extracting the coded data of the converted file data with the coded data from the coded data, and inquiring the cloud storage library to find out the original file data corresponding to the code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911335628.6A CN111104372A (en) | 2019-12-23 | 2019-12-23 | Mobile internet big data processing system and processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911335628.6A CN111104372A (en) | 2019-12-23 | 2019-12-23 | Mobile internet big data processing system and processing method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111104372A true CN111104372A (en) | 2020-05-05 |
Family
ID=70422913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911335628.6A Pending CN111104372A (en) | 2019-12-23 | 2019-12-23 | Mobile internet big data processing system and processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104372A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112100416A (en) * | 2020-11-09 | 2020-12-18 | 南京诺源医疗器械有限公司 | Medical fluorescence imaging image data classification cloud storage system and storage method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473372A (en) * | 2013-09-29 | 2013-12-25 | 方正国际软件有限公司 | Management server, multi-dimensional mirror image service system and multi-dimensional mirror image data management method |
| CN105787280A (en) * | 2016-03-23 | 2016-07-20 | 武汉联影医疗科技有限公司 | Patient data processing method of medical cloud gateway, cloud gateway and medical cloud system |
| CN109657110A (en) * | 2018-12-13 | 2019-04-19 | 上海达梦数据技术有限公司 | A kind of data source tracing method and corresponding data are traced to the source device |
-
2019
- 2019-12-23 CN CN201911335628.6A patent/CN111104372A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473372A (en) * | 2013-09-29 | 2013-12-25 | 方正国际软件有限公司 | Management server, multi-dimensional mirror image service system and multi-dimensional mirror image data management method |
| CN105787280A (en) * | 2016-03-23 | 2016-07-20 | 武汉联影医疗科技有限公司 | Patient data processing method of medical cloud gateway, cloud gateway and medical cloud system |
| CN109657110A (en) * | 2018-12-13 | 2019-04-19 | 上海达梦数据技术有限公司 | A kind of data source tracing method and corresponding data are traced to the source device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112100416A (en) * | 2020-11-09 | 2020-12-18 | 南京诺源医疗器械有限公司 | Medical fluorescence imaging image data classification cloud storage system and storage method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109741517A (en) | A kind of invoice checking method, device and system | |
| CN107291949B (en) | Information searching method and device | |
| CN101807208A (en) | Method for quickly retrieving video fingerprints | |
| CN101320372A (en) | Compression method for repeated data | |
| CN115630343B (en) | Electronic document information processing method, device and equipment | |
| CN113626444B (en) | Table query method, device, equipment and medium based on bitmap algorithm | |
| CN105243027A (en) | Method for storing data in storage device and memory controller | |
| CN108762979B (en) | Terminal information backup method and backup device based on matching tree | |
| CN111104372A (en) | Mobile internet big data processing system and processing method thereof | |
| CN106651972A (en) | Binary image coding and decoding methods and devices | |
| CN106202501A (en) | A kind of information analysis system | |
| CN117176177B (en) | Data sharing method and system for forestry information | |
| CN113391817B (en) | ANTLR 4-based header file replacement method and device | |
| CN104645610A (en) | Game object identification code coding method and system | |
| CN118013364A (en) | Multidimensional data intelligent identification method | |
| CN112131202B (en) | Distributed file storage and reading method, terminal device and storage medium | |
| CN113179265A (en) | Log compression encryption method, device, equipment and storage medium | |
| CN116701300B (en) | Computer file management system based on Internet | |
| CN111026736B (en) | Data blood margin management method and device and data blood margin analysis method and device | |
| CN105160013A (en) | Communication message storage method for data acquisition system | |
| CN112886967B (en) | Data compression coding processing method and device | |
| CN106777260A (en) | A kind of querying method and system based on index binary tree | |
| CN106789232B (en) | efficient information-flow processing control system | |
| CN104572730A (en) | Method and device for importing and exporting digital resources | |
| CN103220274B (en) | A kind of network message pattern matching process for operator's network outlet and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200505 |