CN111708541A - Method and device for compiling mirror image file - Google Patents
Method and device for compiling mirror image file Download PDFInfo
- Publication number
- CN111708541A CN111708541A CN202010571814.6A CN202010571814A CN111708541A CN 111708541 A CN111708541 A CN 111708541A CN 202010571814 A CN202010571814 A CN 202010571814A CN 111708541 A CN111708541 A CN 111708541A
- Authority
- CN
- China
- Prior art keywords
- image file
- file
- mirror image
- compiling
- files
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 29
- 230000001360 synchronised effect Effects 0.000 claims abstract description 18
- 241000282326 Felis catus Species 0.000 claims description 8
- 238000000926 separation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002079 cooperative effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Stored Programmes (AREA)
Abstract
The invention relates to a compiling method and a device of an image file, wherein the method comprises the following steps: acquiring a mirror image file; dynamically acquiring logic core resources of a current compiling server; cutting the mirror image file into small files with specified byte number, and performing multi-core multi-thread synchronous processing on the small files; and outputting the raw image mirror file. The invention utilizes hardware resources of the compiling server and multi-thread and multi-core synchronous processing, and solves the problem of long time consumption of the current mainstream technology.
Description
Technical Field
The invention relates to the technical field of computer software, in particular to a compiling method and device of an image file.
Background
At present, consumer electronic products are more and more commonly applied to daily life of people, and the storage media used by electronic products are various, wherein flash memory images are very commonly applied to embedded devices, such as nor flash, nand flash, spi nand flash, cfi flash and the like.
The current mainstream technology mainly reads single page data and calculates each page of ECC (Error Correcting Code) by using BCH16 or BCH8 coding algorithm in a single thread sequence, and each page takes about 0.13 s. For example, a binary file of 74MByte is burned into a nand flash memory of 512MByte, and the time for generating the raw image file is about 1.5 hours, which is calculated according to about 4W page image. Especially, under the condition that the time complexity and the space complexity of the BCH coding algorithm cannot exceed, if the size multiple of the binary file is increased, the consumed time is longer, and the output efficiency is greatly influenced.
Disclosure of Invention
Therefore, it is necessary to provide a method and an apparatus for compiling an image file, which utilize hardware resources of a compiling server and perform multi-thread and multi-core synchronous processing, and solve the problem of long time consumption of the current mainstream technology.
In order to achieve the above purpose, the invention adopts the following technical scheme.
The invention provides a compiling method of an image file, which comprises the following steps:
acquiring a mirror image file;
dynamically acquiring logic core resources of a current compiling server;
cutting the mirror image file into small files with specified byte number, and performing multi-core multi-thread synchronous processing on the small files;
and outputting the raw image mirror file.
In the compiling method, the image file is an uboot image file, a config image file, an env image file and/or an ubi image file.
In the compiling method, the step of dynamically acquiring the logical core resource of the current compiling server specifically includes:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
In the above compiling method, the step of cutting the image file into small files with specified byte number specifically includes:
specifying the number of bytes of the small file;
the binary bin file is cut into small files of the same size using split tools.
Correspondingly, the invention also provides a compiling device of the image file, which comprises the following components:
the file acquisition module is used for acquiring a mirror image file;
the resource acquisition module is used for dynamically acquiring the logic core resources of the current compiling server;
the file processing module is used for cutting the mirror image file into small files with specified byte number and carrying out multi-core multi-thread synchronous processing on the small files;
and the output module is used for outputting the raw image mirror image file.
In the compiling device, the image file is an uboot image file, a config image file, an env image file and/or an ubi image file.
In the compiling apparatus, the resource obtaining module is specifically configured to:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
In the compiling device, the file processing module is specifically configured to:
specifying the number of bytes of the small file;
the binary bin file is cut into small files of the same size using split tools.
In addition, the invention further provides a compiling method of the image file, which comprises the following steps:
acquiring mirror image files, wherein the mirror image files are uboot mirror image files, config mirror image files, env mirror image files and/or ubi mirror image files;
dynamically acquiring logic core resources of a current compiling server;
adopting a BCH16 or BCH8 coding algorithm, reading ECC data by pages through a Bin2nand tool, and performing multi-core multi-thread synchronous processing on the ECC data of each page;
and outputting the raw image mirror file.
Correspondingly, the invention further provides a compiling device of the image file, which comprises:
the file acquisition module is used for acquiring a mirror image file, wherein the mirror image file is an uboot mirror image file, a config mirror image file, an env mirror image file and/or an ubi mirror image file;
the resource acquisition module is used for dynamically acquiring the logic core resources of the current compiling server;
the file processing module reads ECC data according to pages through a Bin2nand tool by adopting a BCH16 or BCH8 coding algorithm, and performs multi-core multi-thread synchronous processing on the ECC data of each page;
and the output module is used for outputting the raw image mirror image file.
The invention utilizes the hardware resources of the multi-core compiling server, adopts split to cut large size files into small files, and realizes read-write separation when carrying out multi-thread synchronous processing on the small files, thereby greatly shortening the processing time length only by slightly improving the mainstream technology, having very obvious improvement effect, and effectively solving the technical bottleneck that the logic is executed in sequence by a single thread in the prior art and has extremely long time consumption.
Drawings
FIG. 1 is a flowchart illustrating a method for compiling a mirror image file according to an embodiment of the present invention;
FIG. 2 is a block diagram of a compiling device of the second embodiment of the mirror device;
fig. 3 is a flowchart illustrating a compiling method of the mirroring device in the third embodiment.
The implementation of the objects of the present invention and their functions and principles will be further explained in the detailed description with reference to the attached drawings.
Detailed Description
The following further description is made with reference to the drawings and specific embodiments.
The first embodiment is as follows:
as shown in fig. 1, this embodiment provides a method for compiling an image file, including the following steps:
s11: acquiring a mirror image file;
s12: dynamically acquiring logic core resources of a current compiling server;
s13: cutting the mirror image file into small files with specified byte number, and performing multi-core multi-thread synchronous processing on the small files;
s14: and outputting the raw image mirror file.
The image files are uboot image files, config image files, env image files and/or ubi image files, and the image files can be input into the compiling server in a file transmission mode.
Step S12 specifically includes:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
The cat, proc, and cpuinfo files are typically used to characterize the CPU, the core number of the CPU, and whether the CPU enables hyper-threading. In the Linux system, the software and hardware information of a proc file system display system is provided. If the provider of the CPU and the related configuration information in the system are to be known, they can be obtained through the cat/proc/cpuinfo file.
Subsequently, the image file needs to be cut into small files with specified byte number, and step S13 is as follows:
firstly, appointing the byte number of the small file;
then, the binary bin file is cut into small files of the same size using split tools.
After each set of actions from S11 to S13 is completed, a raw image file can be output through the step S14 until the entire uboot image file, the config image file, the env image file and/or the ubi image file are completely compiled.
Through measurement and calculation, if a compiling server with 40 logical cores is adopted to generate a raw image file with the size of 74Mbytes, the conventional method needs 72 minutes to process the file, but the method of the embodiment can be completed in less than 4 minutes, the required time is obviously shortened, and the compiling efficiency is greatly increased.
Example two:
referring to fig. 2, the present invention provides a compiling apparatus 100 for an image file, which mainly includes:
the file acquisition module 10 is used for acquiring a mirror image file;
a resource obtaining module 20, configured to dynamically obtain a logic core resource of a current compiling server;
the file processing module 30 is configured to cut the image file into small files with specified byte numbers, and perform multi-core and multi-thread synchronous processing on the small files;
and the output module 40 is used for outputting the raw image mirror file.
In the compiling apparatus 100 of this embodiment, the image file is an uboot image file, a config image file, an env image file, and/or an ubi image file.
The resource obtaining module 20 is specifically configured to:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
Meanwhile, the file processing module 30 is specifically configured to:
specifying the number of bytes of the small file;
the binary bin file is cut into small files of the same size using split tools.
Through the cooperative action of the modules, the image file with large size can be quickly output into a raw image file, and efficient compiling output is realized.
Example three:
referring to fig. 3, the present invention provides another method for compiling an image file, including the following steps:
s21: acquiring mirror image files, wherein the mirror image files are uboot mirror image files, config mirror image files, env mirror image files and/or ubi mirror image files;
s22: dynamically acquiring logic core resources of a current compiling server;
s23: adopting a BCH16 or BCH8 coding algorithm, reading ECC data by pages through a Bin2nand tool, and performing multi-core multi-thread synchronous processing on the ECC data of each page;
s24: and outputting the raw image mirror file.
The implementation code of the Bin2nand tool is referred to as follows:
the BCH code uses domain theory and polynomial in finite field, and a detection polynomial can be constructed for detecting errors. If one wants to construct a BCH code that is able to detect and correct two errors, a finite field GF (16) is used. The detailed algorithm has large space complexity and time complexity, and please refer to the BCH encoding and decoding related documents for specific implementation.
In summary, the compiling method of this embodiment is basically similar to that in the first and second embodiments, and the difference is mainly that the embodiment does not cut the image file, but uses a Bin2nand tool to read ECC data per page, and then performs multi-core multi-thread synchronous processing on the ECC data of each page, so as to achieve read-write separation, thereby improving the compiling efficiency.
Example four:
based on the third embodiment, the present invention provides a compiling apparatus 200 for an image file, including:
the file obtaining module 210 is configured to obtain an image file, where the image file is an uboot image file, a config image file, an env image file, and/or an ubi image file;
a resource obtaining module 220, configured to dynamically obtain a logic core resource of a current compiling server;
the file processing module 230 reads the ECC data page by page through a Bin2nand tool by adopting a BCH16 or BCH8 coding algorithm, and performs multi-core multi-thread synchronous processing on the ECC data of each page;
and the output module 240 is configured to output a raw image file.
The functions of the above modules can refer to the second embodiment and the third embodiment, and are not described herein again.
In summary, the present invention utilizes hardware resources of a multi-core compiling server, splits large size files into small files, and implements read-write separation when performing multi-thread synchronous processing on the small files, so that processing time can be greatly shortened only by slightly improving the main stream technology, the improvement effect is very obvious, and the technical bottleneck that logic is executed in sequence by a single thread in the prior art and the time is extremely long is effectively solved.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (10)
1. A compilation method of an image file, characterized by comprising the steps of:
acquiring a mirror image file;
dynamically acquiring logic core resources of a current compiling server;
cutting the mirror image file into small files with specified byte number, and performing multi-core multi-thread synchronous processing on the small files;
and outputting the raw image mirror file.
2. The compiling method according to claim 1, wherein the image file is an uboot image file, a config image file, an env image file, and/or an ubi image file.
3. The compilation method according to claim 1, wherein the step of dynamically acquiring the logical core resources of the current compilation server specifically comprises:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
4. The compiling method of claim 1, wherein the step of cutting the image file into small files of a specified number of bytes specifically comprises:
specifying the number of bytes of the small file;
the binary bin file is cut into small files of the same size using split tools.
5. An apparatus for compiling an image file, comprising:
the file acquisition module is used for acquiring a mirror image file;
the resource acquisition module is used for dynamically acquiring the logic core resources of the current compiling server;
the file processing module is used for cutting the mirror image file into small files with specified byte number and carrying out multi-core multi-thread synchronous processing on the small files;
and the output module is used for outputting the raw image mirror image file.
6. The compiling device of claim 5, wherein the image file is an uboot image file, a config image file, an env image file, and/or an ubi image file.
7. The compiling device of claim 5, wherein the resource obtaining module is specifically configured to:
and dynamically acquiring the current logical core resources of the compiling server through cat, proc or cpuinfo.
8. The compiling device of claim 5 wherein the file processing module is specifically configured to:
specifying the number of bytes of the small file;
the binary bin file is cut into small files of the same size using split tools.
9. A compilation method of an image file, characterized by comprising the steps of:
acquiring mirror image files, wherein the mirror image files are uboot mirror image files, config mirror image files, env mirror image files and/or ubi mirror image files;
dynamically acquiring logic core resources of a current compiling server;
adopting a BCH16 or BCH8 coding algorithm, reading ECC data by pages through a Bin2nand tool, and performing multi-core multi-thread synchronous processing on the ECC data of each page;
and outputting the raw image mirror file.
10. An apparatus for compiling an image file, comprising:
the file acquisition module is used for acquiring a mirror image file, wherein the mirror image file is an uboot mirror image file, a config mirror image file, an env mirror image file and/or an ubi mirror image file;
the resource acquisition module is used for dynamically acquiring the logic core resources of the current compiling server;
the file processing module reads ECC data according to pages through a Bin2nand tool by adopting a BCH16 or BCH8 coding algorithm, and performs multi-core multi-thread synchronous processing on the ECC data of each page;
and the output module is used for outputting the raw image mirror image file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571814.6A CN111708541A (en) | 2020-06-22 | 2020-06-22 | Method and device for compiling mirror image file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571814.6A CN111708541A (en) | 2020-06-22 | 2020-06-22 | Method and device for compiling mirror image file |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111708541A true CN111708541A (en) | 2020-09-25 |
Family
ID=72542393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010571814.6A Pending CN111708541A (en) | 2020-06-22 | 2020-06-22 | Method and device for compiling mirror image file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708541A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112486514A (en) * | 2020-11-27 | 2021-03-12 | 广州视源电子科技股份有限公司 | Method and device for manufacturing eMMC burning file and computer equipment |
| CN116069504A (en) * | 2023-01-28 | 2023-05-05 | 广汽埃安新能源汽车股份有限公司 | Scheduling method and device for multi-core processor in automatic driving simulation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102135904A (en) * | 2011-03-11 | 2011-07-27 | 华为技术有限公司 | Multi-core target system oriented mapping method and device |
| US20150160946A1 (en) * | 2013-12-06 | 2015-06-11 | Huawei Technologies Co., Ltd. | Chip and Starting Method Thereof |
| CN107977233A (en) * | 2016-10-19 | 2018-05-01 | 华为技术有限公司 | The quick loading method of kernel mirror image file and device |
| CN110673853A (en) * | 2019-09-25 | 2020-01-10 | 中国工商银行股份有限公司 | Compiling method, device and system |
-
2020
- 2020-06-22 CN CN202010571814.6A patent/CN111708541A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102135904A (en) * | 2011-03-11 | 2011-07-27 | 华为技术有限公司 | Multi-core target system oriented mapping method and device |
| US20150160946A1 (en) * | 2013-12-06 | 2015-06-11 | Huawei Technologies Co., Ltd. | Chip and Starting Method Thereof |
| CN107977233A (en) * | 2016-10-19 | 2018-05-01 | 华为技术有限公司 | The quick loading method of kernel mirror image file and device |
| CN110673853A (en) * | 2019-09-25 | 2020-01-10 | 中国工商银行股份有限公司 | Compiling method, device and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112486514A (en) * | 2020-11-27 | 2021-03-12 | 广州视源电子科技股份有限公司 | Method and device for manufacturing eMMC burning file and computer equipment |
| CN112486514B (en) * | 2020-11-27 | 2024-01-16 | 广州视源电子科技股份有限公司 | Manufacturing method and device of eMMC burning file and computer equipment |
| CN116069504A (en) * | 2023-01-28 | 2023-05-05 | 广汽埃安新能源汽车股份有限公司 | Scheduling method and device for multi-core processor in automatic driving simulation |
| CN116069504B (en) * | 2023-01-28 | 2023-11-10 | 广汽埃安新能源汽车股份有限公司 | Scheduling method and device for multi-core processor in automatic driving simulation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360938B2 (en) | Files having unallocated portions within content addressable storage | |
| US8055633B2 (en) | Method, system and computer program product for duplicate detection | |
| US7924183B2 (en) | Method and system for reducing required storage during decompression of a compressed file | |
| US20150161153A1 (en) | File versions within content addressable storage | |
| US9864542B2 (en) | Data deduplication using a solid state drive controller | |
| US8933824B1 (en) | Hardware decompression of deflate encoded data with multiple blocks | |
| US20130103982A1 (en) | Log file compression | |
| CN111708541A (en) | Method and device for compiling mirror image file | |
| CN111131403A (en) | Message coding and decoding method and device for Internet of things equipment | |
| CN111723059B (en) | Data compression method and device, terminal equipment and storage medium | |
| CN111880916B (en) | Method, device, terminal, medium and host for processing multiple drawing tasks in GPU | |
| CN109213703B (en) | Data detection method and data detection device | |
| US10019311B2 (en) | Validation of a symbol response memory | |
| US10007456B1 (en) | Efficient scrubbing of mirrored memory | |
| US9344120B2 (en) | Adjusting redundancy in an erasure code object store to account for varying data value | |
| CN112115105A (en) | Service processing method, device and equipment | |
| JP2016511901A (en) | Data protection method, apparatus and equipment | |
| TW202019100A (en) | Flash memory controller and encoding circuit and decoding circuit within flash memory controller | |
| US9098446B1 (en) | Recovery of corrupted erasure-coded data files | |
| US20160335154A1 (en) | Error correction coding redundancy based data hashing | |
| CN115454343A (en) | Data processing method, device and medium based on RAID chip | |
| CN112820343B (en) | Data protection method, device, computer equipment and storage medium | |
| CN105354107A (en) | Data transmission method and system for NOR Flash | |
| US11237729B1 (en) | Fast bus inversion for non-volatile memory | |
| US11226740B2 (en) | Selectively performing inline compression based on data entropy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200925 |
|
| RJ01 | Rejection of invention patent application after publication |