CN115422126A - Method, system and device for rapidly transferring certificate OFD format file to picture - Google Patents
Method, system and device for rapidly transferring certificate OFD format file to picture Download PDFInfo
- Publication number
- CN115422126A CN115422126A CN202211373109.0A CN202211373109A CN115422126A CN 115422126 A CN115422126 A CN 115422126A CN 202211373109 A CN202211373109 A CN 202211373109A CN 115422126 A CN115422126 A CN 115422126A
- Authority
- CN
- China
- Prior art keywords
- data
- license
- file
- ofd
- format file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/116—Details of conversion of file system types or formats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/176—Support for shared access to files; File sharing support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a system and a device for fast converting a license OFD format file into a picture, belonging to the technical field of electronic license OFD picture conversion.A MapReduce module is used for processing a large-scale data set in parallel, a MapReduce frame is adopted for realizing multi-task conversion of the license OFD format file, definition is defined according to different license types, the input end is different license type OFD format files, and the output end correspondingly outputs license picture files of corresponding license types; the method is realized by the following steps: the data input module and the data processing module comprise a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting an OFD format file into a lightweight picture; and a data storage module. The invention can complete the effect of converting a large number of OFD files into lightweight pictures in a short time, and promotes the application service level of the electronic license system.
Description
Technical Field
The invention relates to the technical field of electronic license OFD image conversion, in particular to a method, a system and a device for quickly converting a license OFD format file into an image.
Background
At present, the electronic license OFD picture is basically provided when the picture is used, the picture is generated in real time through a program, certain time is needed for waiting for conversion when a large license attachment is met, time is consumed for loading at an APP end, and customer experience is influenced.
Part electron license system adopts third party conversion interface, and conversion efficiency is low, and unable control conversion definition and size moreover to go out the problem easily, the investigation problem is more complicated, needs both sides system personnel to investigate jointly, and consuming time is hard, influences work efficiency.
The OFD layout file needs plug-ins for presentation on a third-party browser, and is not compatible with all browsers, and in addition, presentation of the OFD layout file can be completed by purchasing related services, but this greatly increases application costs.
Disclosure of Invention
The technical task of the invention is to provide a method, a system and a device for quickly converting the OFD format file into the picture, which can complete the effect of converting a large number of OFD files into light-weight pictures in a short time, and can more conveniently and quickly analyze the picture file of the format of the certificate by a docking system, thereby promoting the application service level of an electronic certificate system.
The technical scheme adopted by the invention for solving the technical problem is as follows:
a method for rapidly transferring a certificate OFD format file to a picture is characterized in that a large-scale data set is processed in parallel based on a MapReduce module, a MapReduce frame is adopted to realize multi-task conversion of the certificate OFD format file, definition is defined according to different certificate types, the condition that the input end is OFD format files of different certificate types is realized, and the output end correspondingly outputs certificate picture files of corresponding certificate types; the method is realized by the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
and the data storage module is used for storing the converted data, including data storage and method evaluation.
The method is based on a mechanism that a MapReduce module of Hadoop completes multitask parallel processing conversion, realizes quick conversion of format files of the license, supports definition parameters defined according to different license types, quickly completes data conversion of OFD format files, can greatly improve the response speed of relevant interfaces of the format files, and improves the user experience.
The data processing module is mainly divided into two parts: the first part is a data conversion submodule, namely, the OFD format file is converted into a lightweight picture on the premise of not influencing the reading and use of a user, so that not only is the space saved, but also the use of the OFD format file is not influenced. The other part is a Hadoop MapReduce parallel processing submodule, most of the conventional electronic license systems rarely consider the display problem of OFD format files in the butt joint of third parties, so that most of the conventional license format files exist in an OFD format, the license data of each city project is generally more than 500 ten thousand, provincial projects are more than several tens of millions, and some projects are hundreds of millions. To complete the fast conversion of the license OFD format file, the corresponding format file data processing must be completed fast in multiple tasks and multiple processes by means of a powerful data processing model.
By using the method, the OFD format file is quickly converted into the picture, and the final picture result can be used for license OFD format file sharing and application, so that the butt joint of other systems or APP is facilitated. The license attachment exists in a light-weight picture form, so that the response speed of a sharing interface of the license layout file can be effectively improved, and the sharing and application of the license layout file can be promoted.
Preferably, the data entry module completes the data entry process as follows:
(1.1) acquiring a license type code of the license type;
(1.2) acquiring a license identifier of the license according to the license type code;
(1.3) acquiring a table name and a table field for storing MONGDB data, and acquiring index information of a license layout file of the license according to the license identifier;
(1.4) reading in a data file stream through an index field in a table structure;
and (1.5) converting the file stream into a license format OFD file and storing the license format OFD file in a server.
Because the quantity of the license OFD files is large and the size of each file and the total storage size are uncertain, and because MongoDB uses a fragmentation technology and has the advantages of easy expandability, high-performance real-time insertion property, storage dynamics and the like, a plurality of electronic license systems select Mongdb to store the electronic license OFD format files. The electronic license OFD format file entry module constructed in the method can directly read the data in the Mongdb and complete the entry of the data.
Preferably, the data conversion sub-module converts the OFD format file into the lightweight picture as follows:
(2.1) constructing an OFDReader class to finish reading the OFD format file;
(2.2) constructing an Ofd2Img class and finishing the initialization of the data conversion class;
(2.3) traversing each page of file of the Ofd format file, completing the conversion of the data file by using the Ofd2Img class, and setting different dpi according to different license types, thereby not only ensuring definition, but also reducing the disk space occupied by the converted picture as much as possible;
and (2.4) constructing an image output class ImageIO, finishing the storage of the output file, and closing the related file stream.
And converting the OFD format file on the server, and completing the conversion of data from the OFD format file to the lightweight picture file. The output file not only contains all information of the OFD format file, but also the converted picture file has the file size as small as possible on the premise of ensuring the definition, and the storage cost can be reduced as much as possible.
Preferably, the MapReduce parallel processing submodule comprises a map phase and a Reduce phase,
the Map stage is a task disassembling stage and comprises a plurality of Map tasks, a plurality of data blocks can be input in the Map stage, namely, the Map stage is divided, a plurality of electronic license format files are simultaneously used as one data block, and each Map task is responsible for processing the data block of the electronic license format file containing a plurality of license types: after receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data;
the Reduce stage is to recombine the results, complete data re-output according to the requirements of a set person, process each Reduce task and can process the key value pair binding the electronic certificate layout file and outputting the light-weight picture data, and finally only output one result for the same type of key value Reduce stage;
before entering into the Reduce stage, the data generated by the map task is further processed by shuffle to realize partitioning, sorting and merging operations; after the Reduce stage is completed, outputting the result to a distributed file storage system for storage;
the existence of the reduce task phase is determined by project requirements, one or more than one exists, or in some special scenes, the phase is omitted.
Furthermore, since the map task divides the OFD electronic license format file data of a plurality of different license types into a plurality of records, each map task may output OFD electronic license format file data of a plurality of different license types and key value pairs of a lightweight electronic license picture corresponding thereto, and the key value pairs are divided into different electronic license types according to whether the keys are the same: the keys are the same and belong to the same electronic license format file type, and the keys are different and belong to another electronic license format file type.
Because a plurality of key value pairs of different electronic license types may be generated when the map task is finished, different key value pairs are processed by partitioning and matched with the reduce task, one type of electronic license layout file corresponds to one type of partition, one type of partition corresponds to one reduce task, the electronic license types are different among different partitions, each reduce task corresponds to different license types, the license type codes of the electronic license layout files are different, different license type data enter different reduce tasks, the same type of key value pair is ensured to be sent to the same reduce task, and the data processing is completed.
The number of map tasks is determined by the number of data fragments of the input OFD electronic license format file, and the number of reduce tasks can be specified according to the number of input license types.
The fragment is a flexible quantifier, which can be defined as a file or part of data of the file, and the method refers to an OFD electronic license format data file composed of a plurality of different license types.
Preferably, the process of storing the data is as follows:
(3.1) acquiring result data after the reduce step;
(3.2) acquiring the mongdb configuration information of the stored layout file pictures;
(3.3) sequentially storing the result data into the mongdb, and recording the index of the data in the mongdb picture library;
(3.4) storing the index into a database, so that the query operation of the data is facilitated;
and (3.5) deleting the generated temporary files, including the source license layout file and the result file generated by the reduce, and releasing the space.
Further, the method evaluates result files generated in the reduce stage, each license type detects result data according to a sampling detection method, the result data comprise data size, whether content conversion of the data is normal or not and whether pictures of the converted layout files are clear or not, and the qualification rate, the conversion efficiency and the response speed are used for evaluating data conversion results.
The qualification rate is an evaluation index of the accuracy rate of the OFD format file data conversion lightweight picture result, and represents a ratio of qualified samples (namely, the data size is proper, the data content is converted normally, the converted format file pictures are clear, and the like) to the total number of sampling samples; when the ratio is low, the three problems of proper data size, normal data content conversion and clear picture of the converted layout file need to be detected, and corresponding logics need to be adjusted to improve corresponding yield aiming at different problems.
The conversion efficiency is an evaluation index of the conversion speed of the OFD format file data conversion lightweight picture result, namely, the ratio of the data conversion time to the conversion time of the common method of the electronic license system is finished by using the method aiming at 100 ten thousand electronic license format file data. The smaller the ratio, the faster the switching speed.
And when the response speed is higher, the response speed of the accessed system or APP is higher.
Preferably, the method is implemented as follows:
(1) The data are acquired, the data table is indexed according to the electronic license attachment, the electronic license OFD format file can be read from the MONGDB library and stored on a server, an electronic license type code corresponding to the electronic license OFD format file is read at the same time, and basic data support is provided for MapReduce to construct key value pairs;
(2) The data conversion program is constructed, and different DPI settings are completed according to different license types, so that the definition of the conversion result is ensured, and the size of the disk space occupied by the data is also ensured to be reduced;
(3) Fusing the conversion program into a MapReduce multitask processing program to complete data processing;
(4) Storing the OFD format file after conversion into mongdb, storing index data returned by the mongdb into the warehouse, and deleting the temporary file before and after conversion.
The invention also claims a system for fast transferring the license OFD format file to the picture, which is based on the parallel processing of a MapReduce module on a large scale data set, adopts a MapReduce frame to realize the multi-task conversion of the license OFD format file, realizes the input end of different license type OFD format files according to the self-defined definition of different license types, and correspondingly outputs the license picture file of the corresponding license type at the output end; the method comprises the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
the data storage module is used for storing the converted data, including data storage and method evaluation;
the system realizes the rapid image conversion of the license OFD format file by the method.
The invention also claims a device for fast transferring the license OFD format file to the picture, which comprises the following steps: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program to perform the method described above.
Compared with the prior art, the method, the system and the device for rapidly converting the license OFD format file into the picture have the following beneficial effects:
1. supports the output definition of the self-defined electronic license format file,
in the method, a user can dynamically adjust the dpi value according to the output effect of each license type, so that the memory space occupation is reduced as much as possible and the cost is reduced under the condition of ensuring the definition of the converted pictures.
2. Is friendly to other application systems or APP accessing the license system, has low cost,
the certificate attachment content is displayed in a picture mode, data butt joint can be completed in a short time, and in addition, the cost for purchasing an OFD reader to check the OFD format file is saved.
3. Improve the response speed of other applications or APP, expand the application range of the electronic license,
the certificate accessories stored in the form of pictures are advantageous in size, and due to the fact that conversion time is saved, the response speed after the interface is adapted is high, and the use experience of a user is improved.
4. Can quickly complete the conversion of thousands of millions of data,
for a large number of electronic certificates, the method can utilize the parallel processing capacity of MapReduce to quickly complete the data conversion task.
Drawings
Fig. 1 is a flowchart of acquiring a data set in a method for fast converting a license OFD format file into a picture according to an embodiment of the present invention;
fig. 2 is a data flow diagram of MapReduce operation in the method for fast converting a license OFD format file into a picture according to the embodiment of the present invention;
fig. 3 is a flowchart of method evaluation in the method for quickly converting an identification OFD format file into a picture according to the embodiment of the present invention.
Detailed Description
The opinion on accelerating the expansion of the application field of the electronic certificate and the national mutual authentication proposes that a national integrated government service platform electronic certificate sharing service system is basically established at the end of the year, the electronization of the certificate making and issuing and the application of enterprises and masses is realized, and the standard unification and the mutual authentication of the certificate making and issuing are achieved nationwide. The application that will constantly strengthen the electronic license, the electronic license format file in the electronic license system also corresponding can dock with more and more systems or APP, for the management and the application of better realization electronic license, for the system of docking provide more high-efficient, swift OFD format picture file's service, the format file of a lot of electronic license systems all exists with the OFD mode, just need can accomplish the effect that a large amount of OFD files change light-weight picture in the short time, make the docking system also can make things convenient for fast analysis license format picture file. Further promoting the application service level of the electronic license system.
Based on the above, the embodiment of the invention provides a method for rapidly transferring a certificate OFD format file to a picture, which is based on a MapReduce module to process a large-scale data set in parallel, adopts a MapReduce frame to realize multi-task conversion of the certificate OFD format file, defines definition according to different certificate types, realizes that the input end is the OFD format file of different certificate types, and outputs the certificate picture file of the corresponding certificate type correspondingly at the output end; the method is based on a mechanism that a MapReduce module of Hadoop completes multi-task parallel processing conversion, realizes quick conversion of format files of the license, supports definition parameter customization according to different license types, quickly completes data conversion of OFD format files, can greatly improve response speed of relevant interfaces of the format files, and improves user experience.
The method is realized by the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
and the data storage module is used for storing the converted data, including data storage and method evaluation.
The specific implementation mode is as follows:
1. data entry module
The electronic license OFD format file entry module is constructed, because the number of the license OFD files is large and the size and the total storage size of each file are uncertain, and because MongoDB has the advantages of easy expandability, high-performance real-time insertion property, storage dynamics and the like by using a fragmentation technology, many electronic license systems select Mongdb to store the electronic license OFD format files. The electronic certificate OFD format file entry module constructed in the method can directly read the data in the Mongdb and complete the entry of the data.
The data entry module completes the data entry process as follows:
(1.1) acquiring a license type code of the license type;
(1.2) acquiring a license identifier of the license according to the license type code;
(1.3) acquiring a table name and a table field for storing MONGDB data, and acquiring index information of a license layout file of the license according to the license identifier;
(1.4) reading in the data file stream through an index field in the table structure;
and (1.5) converting the file stream into an license format OFD file and storing the license format OFD file in a server.
2. Data processing module
The data processing module is mainly divided into two parts: the first part is a data conversion submodule, namely, the OFD format file is converted into a lightweight picture on the premise of not influencing the reading and use of a user, so that not only is the space saved, but also the use of the OFD format file is not influenced. The other part is a Hadoop MapReduce parallel processing submodule, most of the conventional electronic license systems rarely consider the problem of displaying OFD format files in a third-party butt joint mode, so that the conventional license format files mostly exist in an OFD format, the license data of each city project is generally more than 500 ten thousand, provincial projects are more than several thousand, and some projects are even hundreds of millions. To complete the fast conversion of the license OFD format file, the corresponding format file data processing must be completed fast in multiple tasks and multiple processes by means of a powerful data processing model.
2.1 data conversion submodule
And converting the OFD format file on the server, and completing the conversion of data from the OFD format file to the lightweight picture file. The output file not only contains all information of the OFD format file, but also has the smallest file size on the premise of ensuring definition of the converted picture file, and the storage cost can be reduced as much as possible.
The data conversion submodule realizes the process of converting the OFD format file into the lightweight picture as follows:
(2.1.1) constructing an OFDReader class to finish reading the OFD format file;
(2.1.2) constructing an Ofd2Img class and finishing the initialization of the data conversion class;
(2.1.3) traversing each page of file of the Ofd format file, completing conversion of the data file by using the Ofd2Img class, and setting different dpi according to different license types, thereby not only ensuring definition, but also reducing the disk space occupied by the converted picture as much as possible;
and (2.1.4) constructing an image output class ImageIO, finishing the storage of the output file, and closing the related file stream.
2.2 MapReduce parallel processing submodule
The electronic certificate library format file amount and storage capacity are large, in order to quickly process and complete conversion from a large amount of ofd format files to light-weight pictures, a Hadoop MapReduce parallel processing module is used in the method, and MapReduce (MR for short) is a software framework of distributed computing and consists of a Resource Manager (RM), a node manager (NodeManager, NM) and a MapReduce application Master (MRAM). The address exposed by ResourceManager to the client. The client submits the application program to the RM through the address, kills the application program and the like. NodeManager total available physical memory. Note that this parameter is not modifiable, and once set, is not dynamically modifiable throughout the run. In addition, the default value of this parameter is 8192MB. MRAM is a specific example of an application server ApplicationMaster (AM).
The YARN is a universal resource management system, completes unified scheduling of resource management and adjustment, and realizes unified support of upper-layer application. The system consists of three components, namely a Resource Manager (RM), a Node Manager (NM) and an application server (AM). The design concept of the YARN is to separate cluster resource management and job scheduling/monitoring, RM completes resource management and scheduling of the whole cluster, NM realizes resource management of a single node, AM realizes monitoring tasks and scheduling jobs, and the three are closely related and inseparable.
In summary, MR is only one application form of YARN, and thus MR and YARN work flow are the same. The starting of MapReduce is also accompanied with the starting of the MRAM process, the process further combines with the resource manager to perform resource allocation, and complete the starting, monitoring and real-time returning of each subtask in the resource manager, it is noted that the MR is also unique, the subtask executed in the container is one of map and reduce, and the order cannot be reversed, and the map task is executed preferentially over the reduce task.
The MapReduce parallel processing submodule comprises a map phase and a Reduce phase.
The Map stage is a task disassembling stage and comprises a plurality of Map tasks, a plurality of data blocks (fragments) can be input into the Map stage, a plurality of electronic license format files can be simultaneously used as one data block, each Map task is responsible for processing one data block containing a plurality of electronic license format files of license types, and the processing method is only according to the program 2.1. After receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data;
the Reduce stage is to recombine the results, complete data re-output according to the requirements of the setting person, the existence of the Reduce task stage is determined by project requirements, and one or more than one stage can exist or in some special scenes, the stage is omitted.
Each reduce task processing can process key value pairs for binding electronic license layout files and outputting light-weight picture data, and only one result is finally output for the same type of key value reduce stage;
before entering the Reduce stage, the data generated by the map task is further processed by shuffle to realize operations such as partitioning, sorting, merging and the like; after the Reduce stage is completed, the result is output to a Distributed File storage System (HDFS) for storage.
The data flow of the MapReduce job is shown in FIG. 2:
in the figure: the Map stage is formed by inputting three pieces of fragment data, each piece of fragment data is formed by inputting a plurality of OFD (office automation system) electronic certificate format files, each Map subtask performs conversion from the OFD electronic certificate format files to light-weight picture data in respective partition, and reduces tasks are executed after partition, sequencing and combination in the shuffle stage, the three reduce tasks are respectively responsible for data of each partition, and part1, part2 and part3 are output of final results.
The map task divides OFD format file fragments containing a plurality of license types into a plurality of records, and because the map task divides OFD electronic license format file data of a plurality of different license types into a plurality of records, each map task can output OFD electronic license format file data of a plurality of different license types and key value pairs of light-weight electronic license pictures corresponding to the OFD electronic license format file data, and according to whether the keys are the same or not, the key value pairs can be divided into different electronic license types, the keys are the same, belong to the same electronic license format file type, and belong to another electronic license format file type if the keys are different. Because a plurality of key value pairs of different electronic license types may be generated when the map task is finished, different key value pairs are processed by partitioning and matched with the reduce task, one type of electronic license layout file corresponds to one type of partition, one type of partition corresponds to one reduce task, the electronic license types are different among different partitions, each reduce task corresponds to different license types, the license type codes of the electronic license layout files are different, different license type data enter different reduce tasks, the same type of key value pair is ensured to be sent to the same reduce task, and the data processing is completed.
The number of map tasks is determined by the number of data fragments of the input OFD electronic license format file, and the number of reduce tasks can be specified according to the number of input license types.
The fragment is a flexible quantifier, which can be defined as a file or part of data of the file, but in the method, the fragment refers to an OFD electronic license format data file composed of a plurality of different license types.
3. Data storage module
3.1, data storage, comprising the following steps:
(3.1.1) obtaining result data after the reduce step in the step 2;
(3.1.2) acquiring the mongdb configuration information for storing the layout file pictures;
(3.1.3) sequentially storing the result data into the mongdb, and recording the index of the data in the mongdb picture library;
(3.1.4) storing the index into a database, so as to facilitate the query operation of the data;
and (3.1.5) deleting the generated temporary files, including the source license layout file and the result file generated by the reduce, and releasing the space.
3.2 evaluation of method
Aiming at a result file generated in a reduce stage, each license type detects the data size, whether the content conversion of the data is normal or not and whether the picture of the converted layout file is clear or not according to a sampling detection method, and the qualification rate, the conversion efficiency and the response speed are used for evaluating the data conversion result. As shown with reference to fig. 3.
The Percent of Pass (Percent of Pass) is denoted by P. The method is an evaluation index for the accuracy of the OFD format file data conversion lightweight picture result, and represents the ratio of qualified samples (namely, the data size is proper, the data content is converted normally, the converted format file pictures are clear, and the like) to the total number of sampling samples. The definition formula is shown as (3-1).
P=TP/(TP+FP) (3-1)
Where TP is the number of qualified sample data and FP is the number of unqualified samples.
If the value P is lower, the three problems of proper data size, normal data content conversion and clear picture of the converted layout file need to be detected, and corresponding logics need to be adjusted to improve corresponding yield aiming at different problems.
Conversion Efficiency (Efficiency), denoted by E. The method is an evaluation index of the conversion speed of the OFD format file data conversion lightweight picture result, namely the ratio of the data conversion time to the conversion time of the common method of the electronic license system is finished by using the method aiming at 100 ten thousand electronic license format file data. The definition formula is shown as (3-2).
E=T1/T2 (3-2)
Wherein T1 is the time for completing data conversion by the method, and T2 is the time for converting the electronic license system by a common method. The smaller the ratio, the faster the switching speed.
The response speed (speed of response) is represented by S, for the converted data, the response speed of the interface is tested by using post tman, and if the speed is higher, the response speed of the accessed system or APP is higher. The definition formula is shown as (3-3).
E=T3/T4 (3-3)
Wherein T3 is the time of interface access after the data is finished by the method, and T4 is the time of real-time conversion interface access of the electronic license system. The smaller the ratio is, the more obvious the conversion effect is, and the more meaningful the representation of the conversion is.
The method is realized as follows:
(1) Firstly, reading an electronic license OFD format file from an MONGDB library and storing the file on a server according to an electronic license attachment index data table, and simultaneously reading an electronic license type code corresponding to the electronic license OFD format file to provide basic data support for a MapReduce constructed key value pair;
(2) The data conversion program is constructed, and the setting of different DPI is completed according to different license types, so that the definition of the conversion result is ensured, and the reduction of the disk space occupied by the data is also ensured;
(3) And fusing the conversion program into a MapReduce multitask processing program to complete the data processing. The method comprises the steps of carrying out map tasks on temporarily stored files and corresponding license type codes, completing disassembling of all acquired license OFD format file data, and completing the formulation of the number of the map tasks according to the number of the license types of actual license format files, wherein in the method, a plurality of electronic license OFD format files of different license types are simultaneously used as a data block, each map task is responsible for processing a data sheet containing a plurality of electronic license format files of different license types, and the processing method is only according to a program introduced by 2.1.
After receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data; and the Reduce stage is to recombine the converted results of the license layout files, each Reduce task can process the key value pairs which bind the electronic license layout files and output light-weight picture data, and only one calculation result is finally output in the Reduce stage of the same type of key values.
(4) And warehousing the data conversion result and deleting the temporary file, storing the converted OFD format file into mongdb, warehousing index data returned by the mongdb, and deleting the temporary file before and after conversion.
By using the method to rapidly convert the OFD format file into the picture, the final picture result can be used for license OFD format file sharing and application, and the butt joint of other systems or APP is facilitated. The license attachment exists in a lightweight picture form, so that the response speed of a sharing interface of the license layout file can be effectively improved, and the sharing and application of the license layout file can be promoted.
The embodiment of the invention also provides a system for rapidly transferring the license OFD format file to the picture, which is characterized in that a large-scale data set is processed in parallel based on a MapReduce module, a MapReduce frame is adopted to realize multi-task conversion of the license OFD format file, the definition is defined according to different license types, the input end is the OFD format file with different license types, and the output end correspondingly outputs the license picture file with the corresponding license type; the method comprises the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
the data storage module is used for storing the converted data, including data storage and method evaluation;
the system realizes the rapid image conversion of the license OFD format file through the method for rapidly converting the license OFD format file into the image.
1. The data entry module completes the data entry process as follows:
(1.1) acquiring a license type code of the license type;
(1.2) acquiring a license identifier of the license according to the license type code;
(1.3) acquiring a table name and a table field for storing MONGDB data, and acquiring index information of a license layout file of the license according to the license identifier;
(1.4) reading in a data file stream through an index field in a table structure;
and (1.5) converting the file stream into a license format OFD file and storing the license format OFD file in a server.
2. The data processing module comprises a data conversion sub-module and a Hadoop MapReduce parallel processing sub-module:
2.1 data conversion submodule
And converting the OFD format file on the server, and completing the conversion of data from the OFD format file to the lightweight picture file. The output file not only contains all information of the OFD format file, but also has the smallest file size on the premise of ensuring definition of the converted picture file, and the storage cost can be reduced as much as possible. The data conversion submodule realizes the process of converting the OFD format file into the lightweight picture as follows:
(2.1.1) constructing an OFDReader class to finish reading the OFD format file;
(2.1.2) constructing an Ofd2Img class and finishing the initialization of the data conversion class;
(2.1.3) traversing each page of file of the Ofd format file, completing the conversion of the data file by using the Ofd2Img class, and setting different dpi according to different license types, thereby not only ensuring the definition, but also reducing the disk space occupied by the converted picture as much as possible;
and (2.1.4) constructing an image output class ImageIO, finishing the storage of the output file, and closing the related file stream.
2.2 MapReduce parallel processing submodule
The MapReduce parallel processing submodule comprises a map phase and a Reduce phase.
The Map stage is a task disassembling stage and comprises a plurality of Map tasks, a plurality of data blocks (fragments) can be input into the Map stage, a plurality of electronic license format files can be simultaneously used as one data block, each Map task is responsible for processing one data block containing a plurality of electronic license format files of license types, and the processing method is only according to the program 2.1. After receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data;
the Reduce stage is to recombine the results, complete data re-output according to the requirements of the setting person, the existence of the Reduce task stage is determined by project requirements, and one or more than one stage can exist or in some special scenes, the stage is omitted.
Each reduce task processing can process key value pairs for binding electronic license layout files and outputting light-weight picture data, and only one result is finally output for the same type of key value reduce stage;
before entering the Reduce stage, the data generated by the map task is further processed by shuffle to realize operations such as partitioning, sorting, merging and the like; after the Reduce stage is completed, the result is output to a Distributed File storage System (HDFS) for storage.
The data flow of the MapReduce job is shown in FIG. 2:
in the figure: the Map stage is formed by inputting three pieces of fragment data, each piece of fragment data is formed by inputting a plurality of OFD electronic certificate layout files, each Map subtask performs conversion from the OFD electronic certificate layout files to light-weight picture data in respective partition, a reduce task is executed after partition, sequencing and combination in the shuffle stage, the three reduce tasks are respectively responsible for data of each partition, and part1, part2 and part3 are output of a final result.
The map task divides OFD format file containing a plurality of license types into a plurality of records, and the map task divides OFD electronic license format file data of a plurality of different license types into a plurality of records, so that each map task can output OFD electronic license format file data of a plurality of different license types and key value pairs of light-weight electronic license pictures corresponding to the OFD electronic license format file data, and the key value pairs can be divided into different electronic license types according to whether the keys are the same or not, are the same, belong to the same electronic license format file type, and belong to another electronic license format file type if the keys are different. Because a plurality of key value pairs of different electronic license types may be generated when the map task is finished, different key value pairs are processed by partitioning and matched with the reduce task, one type of electronic license layout file corresponds to one type of partition, one type of partition corresponds to one reduce task, the electronic license types are different among different partitions, each reduce task corresponds to different license types, the license type codes of the electronic license layout files are different, different license type data enter different reduce tasks, the same type of key value pair is ensured to be sent to the same reduce task, and the data processing is completed.
The number of map tasks is determined by the number of data fragments of the input OFD electronic license format file, and the number of reduce tasks can be specified according to the number of input license types.
The fragment is a flexible quantifier, which can be defined as a file or part of data of the file, but in the method, the fragment refers to an OFD electronic license format data file composed of a plurality of different license types.
3. The data storage module comprises data storage and method evaluation:
3.1, data storage, comprising the following steps:
(3.1.1) obtaining result data after the reduce step in the step 2;
(3.1.2) acquiring the mongdb configuration information for storing the layout file pictures;
(3.1.3) sequentially storing the result data into the mongdb, and recording the index of the data in the mongdb picture library;
(3.1.4) storing the index into a database, so as to facilitate the query operation of the data;
and (3.1.5) deleting the generated temporary files, including the source license layout file and the result file generated by the reduce, and releasing the space.
3.2 evaluation of method
Aiming at a result file generated in a reduce stage, each license type detects the data size, whether the content conversion of the data is normal and whether the picture of the converted layout file is clear according to a sampling detection method, and the qualification rate, the conversion efficiency and the response speed are used for evaluating the data conversion result. As shown with reference to fig. 3.
The Percent of Pass (Percent of Pass) is denoted by P. The method is an evaluation index for the accuracy of the OFD format file data conversion lightweight picture result, and represents the ratio of qualified samples (namely, the data size is proper, the data content is converted normally, the converted format file pictures are clear, and the like) to the total number of sampling samples. The definition formula is shown as (3-1).
P=TP/(TP+FP) (3-1)
Where TP is the number of qualified sample data and FP is the number of unqualified samples.
If the value of P is low, the three problems of proper data size, normal data content conversion and clear picture of the converted layout file need to be detected, and corresponding logics need to be adjusted to improve corresponding qualification rate aiming at different problems.
Conversion Efficiency (Efficiency), denoted by E. The method is an evaluation index of the conversion speed of the OFD format file data conversion lightweight picture result, namely, the ratio of the data conversion time to the conversion time of the electronic license system by using the method is finished for 100 ten thousand electronic license format file data. The definition formula is shown as (3-2).
E=T1/T2 (3-2)
Wherein T1 is the time for completing data conversion by the method, and T2 is the time for converting the electronic license system by a common method. The smaller the ratio, the faster the switching speed.
And response speed (speed of response), denoted by S, for the converted data, using POSTMAN to test the interface response speed, wherein if the speed is faster, the response speed of the accessed system or APP is faster. The definition formula is shown as (3-3).
E=T3/T4 (3-3)
Wherein T3 is the time of interface access after the data is finished by the method, and T4 is the time of real-time conversion interface access of the electronic license system. The smaller the ratio is, the more obvious the conversion effect is, and the more meaningful the representation of the conversion is.
The specific implementation mode is as follows:
(1) The data are acquired, the data table is indexed according to the electronic license attachment, the electronic license OFD format file can be read from the MONGDB library and stored on a server, an electronic license type code corresponding to the electronic license OFD format file is read at the same time, and basic data support is provided for MapReduce to construct key value pairs;
(2) The data conversion program is constructed, and different DPI settings are completed according to different license types, so that the definition of the conversion result is ensured, and the size of the disk space occupied by the data is also ensured to be reduced;
(3) And fusing the conversion program into a MapReduce multitask processing program to complete the data processing. The method comprises the steps of carrying out map tasks on temporarily stored files and corresponding license type codes, completing disassembling of all acquired license OFD format file data, and completing the formulation of the number of the map tasks according to the number of the license types of actual license format files, wherein in the method, a plurality of electronic license OFD format files of different license types are simultaneously used as a data block, each map task is responsible for processing a data sheet containing a plurality of electronic license format files of different license types, and the processing method is only according to a program introduced by 2.1.
After receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data; and the Reduce stage is to recombine the converted results of the license layout files, each Reduce task can process the key value pairs which bind the electronic license layout files and output light-weight picture data, and only one calculation result is finally output in the Reduce stage of the same type of key values.
(4) And warehousing the data conversion result and deleting the temporary file, storing the converted OFD format file into mongdb, warehousing index data returned by the mongdb, and deleting the temporary file before and after conversion.
The system completes a mechanism of multi-task parallel processing conversion based on a MapReduce module of Hadoop, realizes quick conversion of format files of license, supports definition parameters customized according to different license types, quickly completes data conversion of OFD format files, can greatly improve the response speed of relevant interfaces of the format files, and improves the user experience.
Hadoop is a distributed system infrastructure developed by Apache. A user can develop distributed programs under the framework. And high-speed operation and storage using the cluster is possible. Hadoop implements a Distributed File System (Distributed File System), represented by HDFS (Hadoop Distributed File System). The HDFS has the characteristic of high fault tolerance and can be deployed on low-cost hardware; and can provide high-throughput access to application data, the most core design of the framework of Hadoop is as follows: HDFS and MapReduce. HDFS provides storage for massive data, while MapReduce provides computation for massive data.
The embodiment of the invention also provides a device for rapidly converting the license OFD format file into the picture, which comprises the following steps: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program, and execute the method for quickly converting the license OFD format file into the picture in the embodiment.
The present invention can be easily implemented by those skilled in the art from the above detailed description. It should be understood, however, that the intention is not to limit the invention to the particular embodiments described. On the basis of the disclosed embodiments, a person skilled in the art can combine different technical features at will, thereby implementing different technical solutions.
In addition to the technical features described in the specification, the technology is known to those skilled in the art.
Claims (10)
1. A method for rapidly transferring a certificate OFD format file to a picture is characterized in that a large-scale data set is processed in parallel based on a MapReduce module, a MapReduce frame is adopted to realize multi-task conversion of the certificate OFD format file, definition is defined according to different certificate types, the fact that the input end is the OFD format file of different certificate types is realized, and the output end correspondingly outputs the certificate picture file of the corresponding certificate type; the method is realized by the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
and the data storage module is used for storing the converted data, including data storage and method evaluation.
2. The method for rapidly converting the license OFD format file into the picture according to claim 1, wherein the data entry module completes the data entry process as follows:
(1.1) acquiring a license type code of the license type;
(1.2) acquiring a license identifier of the license according to the license type code;
(1.3) acquiring a table name and a table field for storing MONGDB data, and acquiring index information of a license layout file of the license according to the license identifier;
(1.4) reading in the data file stream through an index field in the table structure;
and (1.5) converting the file stream into an license format OFD file and storing the license format OFD file in a server.
3. The method for rapidly converting the license OFD format file into the picture according to claim 1 or 2, wherein the data conversion sub-module is used for converting the OFD format file into the lightweight picture as follows:
(2.1) constructing an OFDReader class to finish reading the OFD format file;
(2.2) constructing an Ofd2Img class and finishing the initialization of the data conversion class;
(2.3) traversing each page of file of the Ofd format file, completing conversion of the data file by using the Ofd2Img class, and setting different dpi according to different license types;
and (2.4) constructing an image output class ImageIO, finishing the storage of the output file, and closing the related file stream.
4. The method for fast converting picture of certificate OFD format file as claimed in claim 3, wherein said MapReduce parallel processing submodule includes a map phase and a Reduce phase,
the Map stage is a task disassembling stage and comprises a plurality of Map tasks, a plurality of data blocks can be input into the Map stage, namely, the Map stage is divided, a plurality of electronic license format files are simultaneously used as one data block, and each Map task is responsible for processing the data block of the electronic license format file containing a plurality of license types: after receiving the fragment data containing the electronic license format file, the map task defaults to take each electronic license format data file of the fragment as a record, completes the sequential processing of each record, and finally outputs a plurality of key value pairs containing the input electronic license format file and the output light-weight picture data;
the Reduce stage is to recombine the results, complete data re-output according to the requirements of a set person, process each Reduce task and can process the key value pair binding the electronic certificate layout file and outputting the light-weight picture data, and finally only output one result for the same type of key value Reduce stage;
before entering into Reduce stage, the data generated by map task is further processed by shuffle to realize operations of partitioning, sorting and merging; after the Reduce stage is completed, outputting the result to a distributed file storage system for storage;
the reduce task phase exists according to project requirements, and one or more than one reduce task phase exists or the reduce task phase is omitted in some special scenes.
5. The method for fast transferring a license OFD format file to a picture according to claim 4, wherein since the map task divides OFD electronic license format file data of a plurality of different license types into a plurality of records, each map task may output OFD electronic license format file data of a plurality of different license types and key value pairs of a light-weight electronic license picture corresponding thereto, and the key value pairs are divided into different electronic license types according to whether the keys are the same or not: the keys are the same and belong to the same electronic license format file type, and the keys are different and belong to another electronic license format file type;
the electronic license layout files of one type correspond to one type of subarea, one type of subarea corresponds to one reduce task, the electronic license types are different among different subareas, each reduce task corresponds to different license types, the license type codes are different, different license type data enter different reduce tasks, the same type of key value pair is guaranteed to be sent to the same reduce task, and data processing is completed.
6. The method for rapidly transferring the license OFD format file to the picture according to claim 4, wherein the data storage process comprises the following steps:
(3.1) acquiring result data after the reduce step;
(3.2) acquiring the mongdb configuration information for storing the layout file pictures;
(3.3) sequentially storing the result data into the mongdb, and recording the index of the data in the mongdb picture library;
(3.4) storing the index into a database, so as to facilitate the query operation of the data;
and (3.5) deleting the generated temporary files, including the source license layout file and the result file generated by the reduce, and releasing the space.
7. The method for fast image conversion of license OFD format file as claimed in claim 6, wherein the method evaluates the result file generated in reduce stage, each license type detects the result data according to sampling detection method, including data size, whether the content conversion of data is normal and whether the image of the converted format file is clear, and evaluates the data conversion result by using qualification rate, conversion efficiency and response speed.
8. The method for fast converting the license OFD format file into the picture according to claim 4, wherein the method is implemented as follows:
(1) Firstly, reading an electronic license OFD format file from an MONGDB library and storing the file on a server according to an electronic license attachment index data table, and simultaneously reading an electronic license type code corresponding to the electronic license OFD format file to provide basic data support for a MapReduce constructed key value pair;
(2) The data conversion program is constructed, and the setting of different DPI is completed according to different license types, so that the definition of the conversion result is ensured, and the reduction of the disk space occupied by the data is also ensured;
(3) Fusing the conversion program to a MapReduce multi-task processing program to complete the processing of data;
(4) And warehousing the data conversion result and deleting the temporary file, storing the converted OFD format file into mongdb, warehousing index data returned by the mongdb, and deleting the temporary file before and after conversion.
9. A system for rapidly transferring a certificate OFD format file to a picture is characterized in that a large-scale data set is processed in parallel based on a MapReduce module, multitask conversion of the certificate OFD format file is realized by adopting a MapReduce framework, definition is defined according to different certificate types, the condition that the input end is the different certificate OFD format files is realized, and the output end correspondingly outputs the certificate picture files of the corresponding certificate types; the method comprises the following steps:
the data input module is used for completing the input of certificate OFD format file data;
the data processing module comprises a MapReduce parallel processing submodule for processing data in parallel and a data conversion submodule for converting the OFD format file into a lightweight picture;
the data storage module is used for storing the converted data, including data storage and method evaluation;
the system realizes the rapid picture conversion of the license OFD format file through the method of any one of claims 1 to 8.
10. The utility model provides a device that certificate OFD format file changes picture fast which characterized in that includes: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor, configured to invoke the machine readable program to perform the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211373109.0A CN115422126B (en) | 2022-11-04 | 2022-11-04 | Method, system and device for rapidly transferring certificate OFD format file to picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211373109.0A CN115422126B (en) | 2022-11-04 | 2022-11-04 | Method, system and device for rapidly transferring certificate OFD format file to picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115422126A true CN115422126A (en) | 2022-12-02 |
| CN115422126B CN115422126B (en) | 2023-03-24 |
Family
ID=84207849
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211373109.0A Active CN115422126B (en) | 2022-11-04 | 2022-11-04 | Method, system and device for rapidly transferring certificate OFD format file to picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115422126B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693102A (en) * | 2011-02-09 | 2012-09-26 | 佳能株式会社 | Data processing apparatus, and data processing method |
| CN104391748A (en) * | 2014-11-21 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | Mapreduce computation process optimization method |
| CN104750937A (en) * | 2015-04-08 | 2015-07-01 | 西安电子科技大学 | Integrated circuit layout conversion method based on Hadoop |
| CN105677763A (en) * | 2015-12-29 | 2016-06-15 | 华南理工大学 | Image quality evaluating system based on Hadoop |
| CN106598930A (en) * | 2016-12-29 | 2017-04-26 | 南威软件股份有限公司 | Electronic certificate processing method based on layout file |
| CN107315805A (en) * | 2017-06-26 | 2017-11-03 | 福建亿榕信息技术有限公司 | A kind of license e-file sharing method and system |
| CN108415887A (en) * | 2018-02-09 | 2018-08-17 | 武汉大学 | A kind of method that pdf document is converted to OFD files |
| CN109754356A (en) * | 2018-12-26 | 2019-05-14 | 广州市中智软件开发有限公司 | Checking method, system and the storage medium of electronics license based on layout files |
| CN109920033A (en) * | 2019-02-28 | 2019-06-21 | 山东浪潮云信息技术有限公司 | A kind of license filling part generation method based on format document |
| CN110532812A (en) * | 2019-09-02 | 2019-12-03 | 江西金格科技股份有限公司 | A kind of electronics license warehouse-out method based on OFD form format |
| CN110597929A (en) * | 2019-09-18 | 2019-12-20 | 广东省智能机器人研究院 | Parallel data cube construction method based on MapReduce |
| CN110716897A (en) * | 2019-10-15 | 2020-01-21 | 北部湾大学 | Cloud computing-based marine archive database parallelization construction method and device |
| CN111753500A (en) * | 2020-07-07 | 2020-10-09 | 江苏中威科技软件系统有限公司 | Method for merging and displaying formatted electronic form and OFD (office file format) and generating catalog |
| CN111813973A (en) * | 2020-05-18 | 2020-10-23 | 冠群信息技术(南京)有限公司 | License conversion method and system |
| CN113407782A (en) * | 2021-07-23 | 2021-09-17 | 重庆交通大学 | MapReduce-based distributed XSLT processing method and system |
| CN113742284A (en) * | 2021-07-29 | 2021-12-03 | 航天信息股份有限公司 | Method and system for converting OFD file into picture based on Java |
-
2022
- 2022-11-04 CN CN202211373109.0A patent/CN115422126B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693102A (en) * | 2011-02-09 | 2012-09-26 | 佳能株式会社 | Data processing apparatus, and data processing method |
| CN104391748A (en) * | 2014-11-21 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | Mapreduce computation process optimization method |
| CN104750937A (en) * | 2015-04-08 | 2015-07-01 | 西安电子科技大学 | Integrated circuit layout conversion method based on Hadoop |
| CN105677763A (en) * | 2015-12-29 | 2016-06-15 | 华南理工大学 | Image quality evaluating system based on Hadoop |
| CN106598930A (en) * | 2016-12-29 | 2017-04-26 | 南威软件股份有限公司 | Electronic certificate processing method based on layout file |
| CN107315805A (en) * | 2017-06-26 | 2017-11-03 | 福建亿榕信息技术有限公司 | A kind of license e-file sharing method and system |
| CN108415887A (en) * | 2018-02-09 | 2018-08-17 | 武汉大学 | A kind of method that pdf document is converted to OFD files |
| CN109754356A (en) * | 2018-12-26 | 2019-05-14 | 广州市中智软件开发有限公司 | Checking method, system and the storage medium of electronics license based on layout files |
| CN109920033A (en) * | 2019-02-28 | 2019-06-21 | 山东浪潮云信息技术有限公司 | A kind of license filling part generation method based on format document |
| CN110532812A (en) * | 2019-09-02 | 2019-12-03 | 江西金格科技股份有限公司 | A kind of electronics license warehouse-out method based on OFD form format |
| CN110597929A (en) * | 2019-09-18 | 2019-12-20 | 广东省智能机器人研究院 | Parallel data cube construction method based on MapReduce |
| CN110716897A (en) * | 2019-10-15 | 2020-01-21 | 北部湾大学 | Cloud computing-based marine archive database parallelization construction method and device |
| CN111813973A (en) * | 2020-05-18 | 2020-10-23 | 冠群信息技术(南京)有限公司 | License conversion method and system |
| CN111753500A (en) * | 2020-07-07 | 2020-10-09 | 江苏中威科技软件系统有限公司 | Method for merging and displaying formatted electronic form and OFD (office file format) and generating catalog |
| CN113407782A (en) * | 2021-07-23 | 2021-09-17 | 重庆交通大学 | MapReduce-based distributed XSLT processing method and system |
| CN113742284A (en) * | 2021-07-29 | 2021-12-03 | 航天信息股份有限公司 | Method and system for converting OFD file into picture based on Java |
Non-Patent Citations (3)
| Title |
|---|
| PAYAM EZATPOOR ET AL.: "Finding Top- k Dominance on Incomplete Big Data Using MapReduce Framework", 《IEEE ACCESS》 * |
| 胥香宇: "面向科技咨询大数据服务平台的研究与实现", 《中国优秀硕士学位论文全文数据库》 * |
| 陈亚军等: "版式文档处理技术在电子证照系统中的应用", 《信息技术与标准化》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115422126B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| EP3889774A1 (en) | Heterogeneous computing-based task processing method and software-hardware framework system | |
| US9063992B2 (en) | Column based data transfer in extract, transform and load (ETL) systems | |
| CN111209352B (en) | Data processing method and device, electronic equipment and storage medium | |
| US20200210481A1 (en) | Parallel graph events processing | |
| US10496659B2 (en) | Database grouping set query | |
| CN111475564A (en) | Streaming data processing method, system, computer equipment and storage medium | |
| CN112860777B (en) | Data processing method, device and equipment | |
| CN112115113B (en) | Data storage system, method, device, equipment and storage medium | |
| CN112379884A (en) | Spark and parallel memory computing-based process engine implementation method and system | |
| CN110975293A (en) | Method, device, server and medium for establishing resource reference relation table | |
| CN111143359A (en) | Query statement generation method and device | |
| CN114756629B (en) | Multi-source heterogeneous data interaction analysis engine and method based on SQL | |
| CN111367953A (en) | Streaming processing method and device for information data | |
| US20130086124A1 (en) | Mapping Data Structures | |
| CN116719622A (en) | Service flow arranging method and service flow arranging system | |
| CN115774552A (en) | Configurated algorithm design method and device, electronic equipment and readable storage medium | |
| Kuderu et al. | Relational database to NoSQL conversion by schema migration and mapping | |
| CN115422126B (en) | Method, system and device for rapidly transferring certificate OFD format file to picture | |
| CN113010542A (en) | Service data processing method and device, computer equipment and storage medium | |
| CN107493205B (en) | Method and device for predicting capacity expansion performance of equipment cluster | |
| Davidson et al. | Technical review of apache flink for big data | |
| CN112506944B (en) | Data standard conversion access method, device, equipment and medium between service systems | |
| CN117591035B (en) | Data set processing method, device and computer readable storage medium | |
| CN117435367A (en) | User behavior processing method, device, equipment, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |