CN113704207B - High-altitude meteorological data compression and decoding method based on video compression technology - Google Patents
High-altitude meteorological data compression and decoding method based on video compression technology Download PDFInfo
- Publication number
- CN113704207B CN113704207B CN202110912060.0A CN202110912060A CN113704207B CN 113704207 B CN113704207 B CN 113704207B CN 202110912060 A CN202110912060 A CN 202110912060A CN 113704207 B CN113704207 B CN 113704207B
- Authority
- CN
- China
- Prior art keywords
- data
- altitude
- altitude image
- compression
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention discloses a high-altitude meteorological data compression and decoding method based on a video compression technology, which compresses high-altitude detected data, so that the detected data is more timely in the process of being transmitted to the ground, certain errors can be correspondingly reduced, and the accuracy and stability in the data transmission process are improved. At the same time, the method is suitable for more application environments, for example, data transmission in high air can be influenced by a plurality of environmental factors, and the data is more timely and effective to transmit by compressing the data with a higher compression ratio.
Description
Technical Field
The invention belongs to the technical field of data compression, and particularly relates to a high-altitude meteorological data compression and decoding method.
Background
The high-altitude meteorological data mainly come from meteorological elements such as temperature, air pressure, humidity, wind direction, wind speed and the like from the ground of high-altitude meteorological observation to 3 ten thousand meters high altitude, and timely and accurate high-altitude meteorological data are provided for weather forecast, climate analysis, scientific research and international exchange. In the meteorological observation field, high-altitude meteorological detection provides important first-hand data, and the obtained information intuitively and conveniently reflects the environment of high-altitude meteorological, so that the conventional high-altitude meteorological detection data is very important, the data must be timely and stable during transmission, and higher requirements are put forward on data transmission. High altitude image detection data analysis and quality control System design and implementation [ D ] university of electronics and technology 2014 (03) ] for example, the west Anjing river sounding station, 7 a.m. per day: 15. night 19:15 and early morning 1:15 are performed three times of timing high-altitude meteorological detection, and each time the detected data reach thousands of groups. According to the requirements of the China weather department, the timeliness of the high-altitude meteorological detection uploading data requires that the time from the detection data to the uploading data is not more than 30 minutes, the actual time is different according to the rising speed of the released balloon, and the actual time is only 10 minutes in some cases. How to efficiently compress meteorological data in a limited time so as to reduce transmission time has become an urgent problem to be solved.
Currently, [ Lv Guo Ying, ren Ruizheng, qian Yuhua: the data compression application mentioned in the university of Qinghua press, 2006 is more lossy compression and lossless compression, lossy compression means that the compressed data is used for reconstruction, the reconstructed data is different from the original data, and the original data cannot be completely recovered, but the lost part has less influence on understanding the original image; lossless compression refers to reconstruction by using compressed data, wherein the reconstructed data is identical to the original data, and is compressed by using statistical redundancy of the data, and the theoretical limit of the statistical redundancy of the data is 2:1 to 5:1. However, the compression ratio is relatively low for high-altitude image data which needs to be transmitted in time and stably, and compression of the high-altitude image data is not suitable.
Disclosure of Invention
In order to solve the technical problems mentioned in the background art, the invention provides a high-altitude image data compression and decoding method based on a video compression technology, which improves timeliness, accuracy and stability in the high-altitude image data transmission process.
In order to achieve the technical purpose, the technical scheme of the invention is as follows:
a high-altitude meteorological data compression and decoding method based on video compression technology comprises the following steps:
(1) Reading data: reading original high-altitude image data, and extracting required high-altitude image element data;
(2) Data preprocessing: determining the range of the high-altitude image element data, and carrying out mapping processing on the high-altitude image element data;
(3) And (3) data storage: storing the mapped data according to the three-dimensional lattice point data, and forming a high-altitude meteorological data graph corresponding to three color channels of the picture;
(4) Data compression: performing lossless compression coding on the high-altitude meteorological data graph;
(5) And (3) decoding data: decompressing the compressed high-altitude image data, and restoring the decompressed high-altitude image data into original high-altitude image data through inverse mapping.
Further, in step (1), the same high-altitude image element data of each site at the same time is stored in a two-dimensional matrix.
Further, in step (2), the high-altitude image element data is mapped into a 10-bit image in the range of 0 to 1024.
Further, the mapping is performed by:
wherein y represents data after mapping, k represents data before mapping, a represents the accuracy to be reserved for the data, max represents the maximum value of the data, min represents the minimum value of the data, and n represents dividing the data into n channels.
Further, in step (5), inverse mapping is performed as follows:
in the above formula, k 'is the data after inverse mapping, sqrt (mse (k-k')) is the error formula, and error is the maximum error allowed.
Further, in the step (3), the mapped data is stored as three-dimensional lattice data by using a Matlab tool.
Further, in step (4), the HM tool of HEVC is used to perform 10bit lossless compression, open TransformQuantizer Bypass in the encoding configuration file, and modify parameters in the encoding configuration file:
-TransquantBypassEnable＝1
-CUTransquantBypassFlagForce＝1。
the beneficial effects brought by adopting the technical scheme are that:
1. the conventional data compression cannot read the content of the original data and is only responsible for compression, and the invention can read the original data in a decompression and data reduction mode, further utilize useful data and provide a premise for the next operation;
2. the invention pre-processes the read high-altitude meteorological data, and carries out 10bit lossless data compression, firstly carries out integer processing on the original data, and can intercept according to the observation precision of the elements in the process of storing the high-altitude meteorological data.
3. The invention maps the data range of the high-altitude image elements into the 10bit image in the range of 0-1024 by carrying out data mapping on the high-altitude image data, thereby carrying out 10bit video lossless compression, and being capable of being higher than the compression ratio of the common data compression.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The technical scheme of the present invention will be described in detail below with reference to the accompanying drawings.
The invention designs a high-altitude meteorological data compression and decoding method based on a video compression technology, which comprises the following steps as shown in figure 1:
step 1, data reading: original high-altitude image data is read, and required high-altitude image element data is extracted.
In this embodiment, a preferable technical scheme is adopted, and the same high-altitude meteorological element data of each site at the same time is stored in a two-dimensional matrix form.
Step 2, data preprocessing: and determining the range of the high-altitude image element data, and carrying out mapping processing on the high-altitude image element data.
The aeronautical element data ranges are shown in table 1 below:
TABLE 1
In this embodiment, a preferable technical scheme is adopted to map the high-altitude image element data into a 10-bit image in the range of 0-1024.
Further, the mapping is performed by:
wherein y represents data after mapping, k represents data before mapping, a represents the accuracy to be reserved for the data, max represents the maximum value of the data, min represents the minimum value of the data, and n represents dividing the data into n channels.
Step 3, data storage: and storing the mapped data according to the three-dimensional lattice point data, and forming a high-altitude meteorological data graph corresponding to three color channels of the picture.
In this embodiment, a preferred technical scheme is adopted, and three different high-altitude meteorological elements are formed into a three-dimensional weather forecast data image by using a Matlab tool. Suppose meteorological data 1: x1, meteorological data 2: x2, meteorological data 3: x3, the matlab command that constitutes the three-dimensional weather forecast data is: cat (3, X1, X2, X3).
Step 4, data compression: the high-altitude image data map is subjected to lossless compression coding.
In this embodiment, a preferred technical scheme is adopted, the HM tool of HEVC is adopted to perform 10bit lossless compression, and Transform Quantizer Bypass is opened in the encoding configuration file, so as to modify parameters in the encoding configuration file:
-TransquantBypassEnable＝1
-CUTransquantBypassFlagForce＝1。
step 5, data decoding: decompressing the compressed high-altitude image data, and restoring the decompressed high-altitude image data into original high-altitude image data through inverse mapping.
In this embodiment, a preferred technical solution is adopted, and inverse mapping is performed according to the following formula:
in the above formula, k 'is the data after inverse mapping, sqrt (mse (k-k')) is the error formula, and error is the maximum error allowed.
The embodiments are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited by the embodiments, and any modification made on the basis of the technical scheme according to the technical idea of the present invention falls within the protection scope of the present invention.
Claims (4)
1. The high-altitude meteorological data compression and decoding method based on the video compression technology is characterized by comprising the following steps of:
(1) Reading data: reading original high-altitude image data, and extracting required high-altitude image element data;
(2) Data preprocessing: determining the range of the high-altitude image element data, and carrying out mapping processing on the high-altitude image element data;
(3) And (3) data storage: storing the mapped data according to the three-dimensional lattice point data, and forming a high-altitude meteorological data graph corresponding to three color channels of the picture;
(4) Data compression: performing lossless compression coding on the high-altitude meteorological data graph;
(5) And (3) decoding data: decompressing the compressed high-altitude image data, and restoring the decompressed high-altitude image data into original high-altitude image data through inverse mapping;
in the step (2), the high-altitude image element data are mapped into 10bit images in the range of 0-1024;
mapping is performed by:
wherein y represents mapped data, k represents data before mapping, a represents the accuracy to be reserved for the data, max represents the maximum value of the data, min represents the minimum value of the data, and n represents the division of the data into n channels;
in step (4), using the HM tool of HEVC to perform 10bit lossless compression, opening Transform Quantizer Bypass in the encoding configuration file, and modifying parameters in the encoding configuration file:
-TransquantBypassEnable＝1
-CUTransquantBypassFlagForce＝1。
2. the method for compressing and decoding high-altitude image data based on video compression technique as recited in claim 1, wherein in step (1), the same high-altitude image element data of each site at the same time is stored in the form of a two-dimensional matrix.
3. The method for compressing and decoding high-altitude image data based on video compression technique as recited in claim 1, wherein in step (5), inverse mapping is performed as follows:
in the above formula, k 'is the data after inverse mapping, sqrt (mse (k-k')) is the error formula, and error is the maximum error allowed.
4. The method for compressing and decoding high-altitude image data based on video compression technique as recited in claim 1, wherein in step (3), the mapped data is stored as three-dimensional lattice data using Matlab tool.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912060.0A CN113704207B (en) | 2021-08-10 | 2021-08-10 | High-altitude meteorological data compression and decoding method based on video compression technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912060.0A CN113704207B (en) | 2021-08-10 | 2021-08-10 | High-altitude meteorological data compression and decoding method based on video compression technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113704207A CN113704207A (en) | 2021-11-26 |
| CN113704207B true CN113704207B (en) | 2023-05-09 |
Family
ID=78652068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110912060.0A Active CN113704207B (en) | 2021-08-10 | 2021-08-10 | High-altitude meteorological data compression and decoding method based on video compression technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113704207B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523446A (en) * | 2011-12-26 | 2012-06-27 | 南京鹏力系统工程研究所 | Adaptive compression method of radar video in vessel traffic navigation system |
| CN105871382A (en) * | 2016-03-29 | 2016-08-17 | 中海油信息科技有限公司 | Micaps fourth-class lattice point data lossless compression algorithm |
| CN108513136A (en) * | 2018-04-08 | 2018-09-07 | 华中科技大学苏州脑空间信息研究院 | The compression method and read method of magnanimity space three-dimensional image data |
| CN110188586A (en) * | 2018-04-13 | 2019-08-30 | 山东百世通大数据科技有限公司 | System and application method based on meteorological observation, road camera shooting visibility identification |
| CN112468154A (en) * | 2019-09-06 | 2021-03-09 | 广东华风海洋信息系统服务有限公司 | Data compression method suitable for visualization of oceanographic weather |
| CN112887722A (en) * | 2020-12-09 | 2021-06-01 | 清华大学 | Lossless image compression method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060023782A1 (en) * | 2004-07-27 | 2006-02-02 | Microsoft Corporation | System and method for off-line multi-view video compression |
-
2021
- 2021-08-10 CN CN202110912060.0A patent/CN113704207B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523446A (en) * | 2011-12-26 | 2012-06-27 | 南京鹏力系统工程研究所 | Adaptive compression method of radar video in vessel traffic navigation system |
| CN105871382A (en) * | 2016-03-29 | 2016-08-17 | 中海油信息科技有限公司 | Micaps fourth-class lattice point data lossless compression algorithm |
| CN108513136A (en) * | 2018-04-08 | 2018-09-07 | 华中科技大学苏州脑空间信息研究院 | The compression method and read method of magnanimity space three-dimensional image data |
| CN110188586A (en) * | 2018-04-13 | 2019-08-30 | 山东百世通大数据科技有限公司 | System and application method based on meteorological observation, road camera shooting visibility identification |
| CN112468154A (en) * | 2019-09-06 | 2021-03-09 | 广东华风海洋信息系统服务有限公司 | Data compression method suitable for visualization of oceanographic weather |
| CN112887722A (en) * | 2020-12-09 | 2021-06-01 | 清华大学 | Lossless image compression method |
Non-Patent Citations (1)
| Title |
|---|
| 利用JPEG-LS高效无损压缩气象卫星云图数据;曹青 等;《应用气象学报》;380-382 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113704207A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105554501B (en) | Image acquisition and compression method and device | |
| CN100524452C (en) | Digital image storage, displaying method and device | |
| CN102036059A (en) | Method, device and system for compressing and decompressing transparent images | |
| CN110728725B (en) | Hardware-friendly real-time system-oriented lossless texture compression method | |
| CN112767423B (en) | Remote sensing image building segmentation method based on improved SegNet | |
| CN110109896A (en) | UAV flight parameter intelligence restoring method | |
| CN102724500B (en) | A kind of compressing/decompressing method of video data and system | |
| CN105024702A (en) | Floating-point-type data lossless compression method for scientific calculation | |
| CN113258934A (en) | Data compression method, system and equipment | |
| CN112069141A (en) | Special compression method for meteorological forecast lattice point data | |
| CN102323192B (en) | Cloud particle measuring system and method | |
| CN113704207B (en) | High-altitude meteorological data compression and decoding method based on video compression technology | |
| CN101546434A (en) | Method and system for compressing and decompressing BMP picture file | |
| CN102523446A (en) | Adaptive compression method of radar video in vessel traffic navigation system | |
| CN110191345B (en) | Incremental lossless compression method for separating foreground and background based on Huffman coding | |
| CN111787321A (en) | Image compression and decompression method and system for edge end based on deep learning | |
| CN111479286A (en) | Data processing method for reducing communication flow of edge computing system | |
| CN2777644Y (en) | Magazine data recorder | |
| CN116663739A (en) | Complex terrain fan output prediction method, system, equipment and storage medium | |
| CN103975593A (en) | Method and apparatus for generating a bitstream of repetitive structure discovery based 3d model compression | |
| CN102740075B (en) | Video data compressing/decompressing method and system | |
| CN103581675A (en) | Video data compression or decompression method and system | |
| CN114024952A (en) | File compression transmission method based on DL/T698.45-2007 protocol | |
| CN102647597A (en) | Joint photographic experts group (JPEG) image compression method based on polygon clipping discrete cosine transformation (DCT) | |
| CN116683915B (en) | Meteorological data compression method, system and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: No.219, ningliu Road, Jiangbei new district, Nanjing, Jiangsu Province, 210032 Patentee after: Nanjing University of Information Science and Technology Address before: 214032 No. 219, ningliu Road, Jiangbei new district, Wuxi City, Jiangsu Province Patentee before: Nanjing University of Information Science and Technology |
|
| CP02 | Change in the address of a patent holder |