CN114187369A - Weather forecast data compression method based on video compression technology - Google Patents
Weather forecast data compression method based on video compression technology Download PDFInfo
- Publication number
- CN114187369A CN114187369A CN202111420160.8A CN202111420160A CN114187369A CN 114187369 A CN114187369 A CN 114187369A CN 202111420160 A CN202111420160 A CN 202111420160A CN 114187369 A CN114187369 A CN 114187369A
- Authority
- CN
- China
- Prior art keywords
- data
- meteorological
- weather forecast
- forecast data
- weather
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a weather forecast data compression method based on a video compression technology, and belongs to the technical field of data compression. The method comprises the following steps: s1, acquiring original weather forecast data, and extracting time slices and weather element data; s2, preprocessing weather forecast data: mapping the same meteorological element data of each station at the same time to an interval [ 0-1023 x n ], wherein n is the number of mapped channels; s3, meteorological data three-dimensional lattice point reconstruction: storing the weather forecast data according to the three-dimensional lattice point data, and forming a weather data graph corresponding to the three color channel data formats of the picture; s4, carrying out lossless compression coding on weather forecast data with 10bit depth and 4:4:4 chroma brightness ratio by using a video compression tool; and S5, decoding the meteorological data of the coded file generated in the S4 by using a video compression tool, and performing meteorological prediction data inverse transformation. The invention reduces the storage space of weather forecast data, and the final compression ratio can reach 10: 1.
Description
Technical Field
The invention belongs to the technical field of data compression, and particularly relates to a weather forecast data compression method based on a video compression technology.
Background
The meteorological forecast data has the characteristics of large data volume, various meteorological elements, high timeliness and the like. Because a large amount of data in weather forecast data are space-time data, observed quantities of various physical quantities in time and space ranges are recorded, and a large amount of storage resources and network transmission resources of people are consumed by the large amount of weather data. Therefore, a good meteorological forecast data compression algorithm is selected, the occupation of disk space can be greatly reduced, the load of a network is smaller, and the user experience satisfaction is improved. The weather forecast data includes 6 weather elements:
| storage channel | Meteorological element | Unit of |
| PRE | Total precipitation over 1 hour | mm |
| TMP | 2 m temperature | K |
| PRS | Surface pressure | Pa |
| RH | Relative humidity | ％ |
| U10 | 10metre U wind component | m/s |
| V10 | 10metre v wind component | m/s |
Video compression has a long history of development, and a set of very mature algorithms with high compression rate is formed. The meteorological data is similar to video data in structure, and the meteorological data generated by the operation of a numerical forecasting model generally adopts an organization mode of lattice point data, and one lattice point forecasts one numerical value. Therefore, each lattice point data in the meteorological data corresponds to each pixel in the video, and the meteorological data is reconstructed into a frame in the three-dimensional lattice point data corresponding to the video sequence, namely, the meteorological forecast data can be compressed by using a video compression tool.
The meteorological lattice data lossless compression method based on BP neural network [ J ] Earth science progress, 2008, 23(2): 637-. Lossless compression can retain all information in data, but the compression ratio is generally not high, and is generally 2:1 to 5: 1; for a large amount of weather forecast data, it is necessary that the compression ratio is large enough, but data loss has an immeasurable impact on the accuracy and quality of data prediction.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the problems, the invention provides a weather forecast data compression method based on a video compression technology, which processes weather forecast data into data in a video format, thereby realizing the compression of the weather forecast data and improving the compression ratio of the weather forecast data.
The technical scheme is as follows: in order to realize the purpose of the invention, the technical scheme adopted by the invention is as follows: a weather forecast data compression method based on a video compression technology comprises the following steps:
s1, acquiring original weather forecast data, and extracting time slices and weather element data;
s2, preprocessing weather forecast data: mapping the same meteorological element data of each station at the same time to an interval [ 0-1023 x n ], wherein n is the number of mapped channels;
s3, meteorological data three-dimensional lattice point reconstruction: storing the weather forecast data according to the three-dimensional lattice point data, and forming a weather data graph corresponding to the three color channel data formats of the picture;
s4, carrying out lossless compression coding on weather forecast data with 10bit depth and 4:4:4 chroma brightness ratio by using a video compression tool;
and S5, decoding the meteorological data of the coded file generated in the S4 by using a video compression tool, and performing meteorological prediction data inverse transformation.
Preferably, the mapping manner in S2 is as follows:
setting data values before and after mapping of certain meteorological element data as x and y respectively, setting the precision of the meteorological element required to be reserved as a, setting the maximum value of the meteorological element as max, and setting the minimum value as min, wherein the mapping expression is as follows:
and obtaining a weather forecast data frame sequence after the mapping of all the weather element data is completed.
Preferably, the S3 stores the weather forecast data as three-dimensional grid data, and the method includes:
a mat file of a mat tool is adopted, three different meteorological elements are combined into a three-dimensional meteorological forecast data image according to three color channel data formats of a picture;
assuming that the meteorological data 1 is recorded as X1, the meteorological data 2 is recorded as X2, and the meteorological data 3 is recorded as X3, the matlab command composing the three-dimensional meteorological forecast data is: cat (3, X1, X2, X3).
Preferably, the S4 performs 10-bit lossless compression coding on the weather forecast data map by using the HM tool of the HEVC standard; the command parameters in the encoding configuration file are set as follows:
InputChromaFormat:444；
TransquantBypassEnable＝1；
CUTransquantBypassFlagForce＝1。
preferably, the inverse transformation method of the S5 data is as follows:
and if the data of a certain meteorological element after decoding and inverse transformation is k', the mapped value is y, the maximum value of the meteorological element is max, a is the precision which needs to be reserved by the meteorological element, n is the number of channels to be mapped, and the allowed maximum error is error, the inverse transformation formula is as follows:
sqrt(mse(y-k′))≤error
wherein sqrt is a square root function and mse is a mean square error function.
Has the advantages that: compared with the prior art, the technical scheme of the invention has the following beneficial technical effects:
the method for compressing the weather forecast data based on video compression, which is provided by the invention, maps the weather forecast data of each station in the same time period to the range of 10bit image storage, forms weather forecast data frames into a weather forecast data sequence according to the time dimension, and then performs coding compression on the weather forecast data frame sequence by using the video compression technology, thereby reducing the storage space of the weather forecast data, greatly facilitating the transmission of the weather forecast data, simultaneously keeping the weather forecast data information as much as possible, and enabling the compression ratio to reach 10: 1.
Drawings
FIG. 1 is a flow chart of weather forecast data compression according to the present invention.
Detailed Description
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
As shown in fig. 1, the present invention provides a method for compressing weather forecast data based on video compression, which specifically includes:
(1) and reading original weather forecast data. Reading original ground meteorological data, extracting time slice and meteorological element data, and storing the same meteorological element data of each station at the same time in a two-dimensional matrix form to obtain a meteorological forecast data frame.
(2) And (4) preprocessing a weather forecast data frame. And (3) mapping the same meteorological data of each site at the same time to an interval [ 0-1023 x n ] (10bit depth corresponds to a data range of 0-1023) according to a formula (1), wherein n is the number of mapped channels. In order to reduce errors caused by data mapping, when the data range is too large, the data range can be divided into n channels. The specific mapping mode is as follows:
if a certain meteorological element data k is set, the accuracy that the meteorological element needs to be kept is a (if the keeping accuracy is 0.1, the value is multiplied by 10 and then rounded, that is, a is 10), the maximum value of the meteorological element is max (the keeping accuracy), the minimum value is min (the keeping accuracy), and the data values before and after mapping are x and y respectively, the mapping mode is as follows:
and obtaining a weather forecast data frame sequence after the mapping of all the weather element data is completed.
(3) And (3) meteorological data three-dimensional lattice point reconstruction: storing the weather forecast data according to the three-dimensional lattice point data, and forming a weather data graph corresponding to the three color channel data formats of the picture; namely, three different meteorological elements are combined into a three-dimensional meteorological forecast data image. Specifically, the method comprises the following steps:
a mat file of a mat tool is adopted, three different meteorological elements are combined into a three-dimensional meteorological forecast data image according to three color channel data formats of a picture;
assuming that the meteorological data 1 is X1, the meteorological data 2 is X2, and the meteorological data 3 is X3, for example: when the total precipitation in the past 1 hour is recorded as X1, the surface pressure is recorded as X2, and the relative humidity is recorded as X3, the matlab command composing the three-dimensional weather forecast data is: cat (3, X1, X2, X3).
(4) Carrying out lossless compression coding with 10bit depth and a chroma-luminance ratio of 4:4:4 on a weather forecast data graph by using an HM tool of an HEVC (high Efficiency Video coding) standard (a new Video compression standard); a Transform Quantizer Bypass (TQB) is opened in the encoding configuration file. Setting command parameters in the encoding configuration file:
–InputChromaFormat:444；
–TransquantBypassEnable＝1；
–CUTransquantBypassFlagForce＝1。
the coding process implementation command in this embodiment is as follows:
TAppEncoder[--help][-c config.cfg][--parameter＝value]
where tappenencoder is the encoder, hellp is the help specification parameter, cfg is the specified encoding profile, and parameter value is the specific parameter value in the set encoding profile, the process outputs the bin-suffixed binary stream file.
(5) And (3) decoding the coded file (compressed data stream (bin file)) generated in the step (4) by using a video compression tool, and performing inverse data transformation.
The decoding process in this embodiment implements the following commands:
TAppDecoder-b str.bin-o dec.yuv
wherein TAppDecoder is a decoder, b str. bin is an output result of the step (4), and o dec.yuv is a decoded meteorological data map.
And if the data of a certain meteorological element after decoding and inverse transformation is k', the mapped value is y, the maximum value of the meteorological element is max, a is the precision which needs to be reserved by the meteorological element, n is the number of channels to be mapped, and the allowed maximum error is error, the inverse transformation formula is as follows:
sqrt(mse(y-k′))≤error
wherein sqrt is a square root function and mse is a mean square error function.
The following table is a calculation chart of the weather forecast data compression error of the present invention. The storage channel represents variable names of different meteorological elements corresponding to the mat two-dimensional matrix storage file, the meteorological elements represent a specific meteorological element, the error represents the error between compressed data and uncompressed data, and the average compression ratio is the size of the compressed data/original data.
The foregoing is a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (5)
1. A weather forecast data compression method based on a video compression technology is characterized by comprising the following steps: the method comprises the following steps:
s1, acquiring original weather forecast data, and extracting time slices and weather element data;
s2, preprocessing weather forecast data: mapping the same meteorological element data of each station at the same time to an interval [ 0-1023 x n ], wherein n is the number of mapped channels;
s3, meteorological data three-dimensional lattice point reconstruction: storing the weather forecast data according to the three-dimensional lattice point data, and forming a weather data graph corresponding to the three color channel data formats of the picture;
s4, carrying out lossless compression coding on weather forecast data with 10bit depth and 4:4:4 chroma brightness ratio by using a video compression tool;
and S5, decoding the meteorological data of the coded file generated in the S4 by using a video compression tool, and performing meteorological prediction data inverse transformation.
2. The weather forecast data compression method based on the video compression technology as claimed in claim 1, wherein: the mapping manner in S2 is as follows:
setting data values before and after mapping of certain meteorological element data as x and y respectively, setting the precision of the meteorological element required to be reserved as a, setting the maximum value of the meteorological element as max, and setting the minimum value as min, wherein the mapping expression is as follows:
and obtaining a weather forecast data frame sequence after the mapping of all the weather element data is completed.
3. The weather forecast data compression method based on the video compression technology as claimed in claim 1 or 2, wherein: and S3, storing the weather forecast data according to the three-dimensional lattice point data, wherein the method comprises the following steps:
a mat file of a mat tool is adopted, three different meteorological elements are combined into a three-dimensional meteorological forecast data image according to three color channel data formats of a picture;
assuming that the meteorological data 1 is recorded as X1, the meteorological data 2 is recorded as X2, and the meteorological data 3 is recorded as X3, the matlab command composing the three-dimensional meteorological forecast data is: cat (3, X1, X2, X3).
4. The weather forecast data compression method based on the video compression technology as claimed in claim 1 or 2, wherein: the S4 carries out 10-bit lossless compression coding on the weather forecast data map by using an HM tool of an HEVC standard; the command parameters in the encoding configuration file are set as follows:
InputChromaFormat:444；
TransquantBypassEnable＝1；
CUTransquantBypassFlagForce＝1。
5. the weather forecast data compression method based on the video compression technology as claimed in claim 1 or 2, wherein: the inverse transformation method of the S5 data is as follows:
if the data after decoding and inverse transformation of a certain meteorological element is k', the mapped value is y, the maximum value of the meteorological element is max, a is the precision which needs to be reserved by the meteorological element, n is the number of channels to be mapped, and the allowed maximum error is error, the inverse transformation formula is as follows:
sqrt(mse(y-k′))≤error
wherein sqrt is a square root function and mse is a mean square error function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111420160.8A CN114187369A (en) | 2021-11-26 | 2021-11-26 | Weather forecast data compression method based on video compression technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111420160.8A CN114187369A (en) | 2021-11-26 | 2021-11-26 | Weather forecast data compression method based on video compression technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114187369A true CN114187369A (en) | 2022-03-15 |
Family
ID=80602721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111420160.8A Pending CN114187369A (en) | 2021-11-26 | 2021-11-26 | Weather forecast data compression method based on video compression technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114187369A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116244265A (en) * | 2023-03-07 | 2023-06-09 | 国家海洋环境预报中心 | Processing method and device for marine weather numerical forecasting product and electronic equipment |
| CN116683915A (en) * | 2023-06-14 | 2023-09-01 | 上海海洋中心气象台 | Meteorological data compression method, system and medium |
-
2021
- 2021-11-26 CN CN202111420160.8A patent/CN114187369A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116244265A (en) * | 2023-03-07 | 2023-06-09 | 国家海洋环境预报中心 | Processing method and device for marine weather numerical forecasting product and electronic equipment |
| CN116244265B (en) * | 2023-03-07 | 2023-08-18 | 国家海洋环境预报中心 | Processing method and device for marine weather numerical forecasting product and electronic equipment |
| CN116683915A (en) * | 2023-06-14 | 2023-09-01 | 上海海洋中心气象台 | Meteorological data compression method, system and medium |
| CN116683915B (en) * | 2023-06-14 | 2024-02-13 | 上海海洋中心气象台 | Meteorological data compression method, system and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114187369A (en) | Weather forecast data compression method based on video compression technology | |
| CN101018336B (en) | Quality based image compression | |
| CN1843039B (en) | System and method for encoding and decoding enhancement layer data using descriptive model parameters | |
| CN109842803B (en) | Image compression method and device | |
| CN102523367A (en) | Real-time image compression and reduction method based on plurality of palettes | |
| CN110933438B (en) | JPEG image reversible information hiding method | |
| JP2016226001A (en) | Decoder and decoding method | |
| CN112995662B (en) | Method and device for attribute entropy coding and entropy decoding of point cloud | |
| CN106658004B (en) | A kind of compression method and device based on image flat site feature | |
| CN106303521A (en) | A kind of HEVC Rate-distortion optimization method based on sensitivity of awareness | |
| CN103702133A (en) | Image compression display method and image compression display device | |
| CN113051237A (en) | Compression storage method of time series remote sensing data | |
| CN114143557B (en) | Low-complexity coding method for wavelet transformation high-frequency coefficient of video image | |
| CN103763558A (en) | Texture image compression method based on image similarities | |
| CN116828210A (en) | Intelligent transmission method and system for submerged video acquisition | |
| CN101860753B (en) | Fractal-based video compression and decompression method | |
| CN101472182B (en) | Virtually lossless video data compression | |
| CN105120276A (en) | Adaptive Motion JPEG coding method and system | |
| CN112468154A (en) | Data compression method suitable for visualization of oceanographic weather | |
| CN111711511B (en) | Method for lossy compression of frequency domain data | |
| CN103002282B (en) | A kind of number of color adaptive decision-making method and method for compressing image | |
| CN104869426A (en) | JPEG coding method lowering image diamond effect under low compression code rate | |
| CN103701468A (en) | Data compression and decompression method on basis of orthogonal wavelet packet transform and rotating door algorithm | |
| CN106559668A (en) | A kind of low code rate image compression method based on intelligent quantization technology | |
| CN116567269A (en) | Spectrum monitoring data compression method based on signal-to-noise separation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |