CN114625992A - Visualization method and system for hundred-megametric meteorological network data - Google Patents
Visualization method and system for hundred-megametric meteorological network data Download PDFInfo
- Publication number
- CN114625992A CN114625992A CN202210249428.4A CN202210249428A CN114625992A CN 114625992 A CN114625992 A CN 114625992A CN 202210249428 A CN202210249428 A CN 202210249428A CN 114625992 A CN114625992 A CN 114625992A
- Authority
- CN
- China
- Prior art keywords
- data
- image
- hundred
- visualization
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007794 visualization technique Methods 0.000 title claims abstract description 10
- 238000012800 visualization Methods 0.000 claims abstract description 44
- 238000009877 rendering Methods 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000007906 compression Methods 0.000 claims abstract description 37
- 230000006835 compression Effects 0.000 claims abstract description 37
- 230000005540 biological transmission Effects 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 29
- 238000006243 chemical reaction Methods 0.000 claims abstract description 16
- 238000007405 data analysis Methods 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 claims description 14
- 230000009467 reduction Effects 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 16
- 238000011161 development Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 8
- 238000001556 precipitation Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000004927 fusion Effects 0.000 description 5
- 238000013079 data visualisation Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 101001094649 Homo sapiens Popeye domain-containing protein 3 Proteins 0.000 description 1
- 101000608234 Homo sapiens Pyrin domain-containing protein 5 Proteins 0.000 description 1
- 101000578693 Homo sapiens Target of rapamycin complex subunit LST8 Proteins 0.000 description 1
- 102100027802 Target of rapamycin complex subunit LST8 Human genes 0.000 description 1
- 241000727732 Tokoyo Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910002056 binary alloy Inorganic materials 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011900 installation process Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9577—Optimising the visualization of content, e.g. distillation of HTML documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/88—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/32—Image data format
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Library & Information Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
Abstract
The invention provides a visualization method and system for hundred-megabyte-level meteorological grid data, and belongs to the field of meteorological data processing. The visualization method comprises the steps of carrying out data analysis on the hundred-megabyte-level meteorological grid data, carrying out high-8-bit and low-8-bit compression storage and storing the data as a PNG (portable network generator) format image; then, the PNG format images are reduced by different multiples by adopting a pyramid model, and all the images are sliced and stored; then, a distributed network is adopted to carry out multithreading transmission on all slices; the client requests slicing according to visualization requirements, splicing the obtained slices, and performing grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte meteorological grid data. The method and the device realize the rapid visualization of the hundred-megabyte-level meteorological grid data, can efficiently support the efficient rendering of large data files of various spatial grid points, improve the visualization efficiency, avoid the loss of characteristic values caused by sparse rendering, and reduce the pressure of a server side.
Description
Technical Field
The invention belongs to the field of meteorological data processing, and particularly relates to a method and a system for visualizing hundred-megawatt meteorological grid data.
Background
With the development of information collection and storage technology, various industries around the world enter a big data era, and the data volume is continuously increased. Among them, weather data plays an important role in various aspects such as environment, natural resources, economy, and life as one of data information necessary for daily use. The meteorological data is usually stored in a grid mode, and the storage and expression modes comprise grib and nc formats.
The meteorological grid data has strong specialization, and can be used after being decompressed, analyzed and processed by specialized tools, and the data expression and service forms comprise point-based extraction of meteorological element values, meteorological element value domain rendering of area surfaces and the like. However, meteorological grid data is used not only to serve professionals, but also to serve the public of society. Professionals generally use meteorological grid data through professional software or development kits, for example, the American NASA (network discovery and development kit) based on JAVA, puboply software of a client side for displaying the meteorological grid data is issued, grid files in Grib2 and NC format are supported, simultaneously, World Wind of geography science popularization software is also issued, and images and data provided by various large service providers are used for displaying the meteorological data through a three-dimensional earth model; in China, desktop version software such as MeteoInfo, traps and the like provides tools such as visual analysis and the like aiming at network data such as NetCDF, GRIB and the like; the applications are all localized applications, and a data transmission process is omitted by directly reading the local disk file, so that the visualization efficiency is improved. However, the local installation process of the specialized development kit or software has certain requirements on the environment of a computer, and is not suitable for services for the social public.
Page-side applications based on B/S structures are gradually being developed for the social public to also use weather grid data. By comprehensively utilizing rich browser application designs such as WebGIS, HTML5 and the like, the non-installation visual application of meteorological grid data is realized. The implementation mode comprises the steps that the background processes images of the grids, the GEOServer technology is used for realizing the release of the dynamic slices, and the front end of the page is directly calculated and displayed. For example, RainViewer has the largest coverage of weather radar on the market, covers a single image of more than 1000 doppler radars of 90 countries, forms an image file with a resolution of 8000 × 4000 and a data size of about 1M, and performs 18-layer slicing on the image, where the resolution of a single slice is 256 × 256 and the size is about 10 k. In the mode of displaying by overlapping pictures, the size of the original lattice point file is not considered, and the original lattice point file is directly converted into a proper picture, but the requirements of real-time interaction such as changing the color system range, filtering according to values and the like cannot be met; when the amplified resolution exceeds the maximum resolution of the image, layer blurring can occur, so that the user experience is poor; in the practical application of some special cases, the meteorological grid data are converted into transparent pictures which are directly superposed on a map, but the resolution effect is poor after the meteorological grid data are amplified; or thinning layer by layer into a sparse grid, which can cause loss of characteristic values (maximum value, minimum value and the like) and influence the real rendering result of the front end. In addition, page rendering based on a B/S architecture directly processes data, and currently, processing technologies such as Geojson or Topjson, binary system, and gray scale map are mainly used: geojson or Topjson, the data volume is multiplied after processing, and when the file is large, the transmission efficiency is low; after binary and gray-scale image processing, certain amount of compression is carried out on data, and the method is suitable for ten-megabyte-level file compression processing and transmission, but has low compression and transmission efficiency for hundred-megabyte-level files. Therefore, the page rendering based on the B/S framework cannot give consideration to both efficiency and accuracy.
Meanwhile, although the meteorological grid data is quickly visualized at the webpage end, the meteorological grid data still has the problem of incompatibility at the mobile end along with the application of platform technologies such as the mobile end, WeChat, Wap and the like. For example, the global visualization display system nullshool.net in the united states is a base map of a coastline profile with a resolution of 50km or 110km in a json format, and is difficult to be used for refined service application in service, and cannot be applied to platforms such as a mobile terminal, WeChat, and Wap.
Disclosure of Invention
In view of the above defects or shortcomings in the prior art, the present invention aims to provide a method and a system for visualizing hundred-megabyte-level meteorological grid data, which implement efficient compression of data in background, can be applied to Web at PC end, Wap and APP at mobile terminal, and the like, and improve data transmission efficiency, user experience and public service efficiency on the premise of guaranteeing content accuracy.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
in a first aspect, an embodiment of the present invention provides a method for visualizing hundred-megabyte-level meteorological grid data, including the following steps:
step S1, after the hundred-megametric weather grid data is subjected to data analysis, the data is subjected to high 8-bit and low 8-bit compression storage and is stored as a PNG format image;
step S2, carrying out reduction of different multiples on the PNG format image by adopting a pyramid model to obtain a PNG format image and each reduced image;
step S3, slicing and storing the PNG format image and each reduced image;
step S4, carrying out multi-thread transmission on all slices by adopting a distributed network;
and step S5, the client requests slices according to visualization requirements, splices the acquired slices, and performs grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte meteorological grid data.
As a preferred embodiment of the present invention, the compressing and storing the data with the upper 8 bits and the lower 8 bits and saving the data as the PNG format image includes:
setting the original data as O, the legend precision of the visual effect graph as N, multiplying the original data by the reciprocal of the legend precision, and then rounding to obtain conversion data C, as shown in formula (1):
and performing high 8-bit value storage on the conversion data C into RGB G data, performing low 8-bit value storage into RGB B data, wherein the formula (2) is a high 8-bit value formula, the formula (3) is a low 8-bit value formula, and the formulas (4) - (6) are storage formulas:
D＝C％256 (3)
G_N[：][：][：，：，1]＝U (4)
G_N[：][：][：，：，2]＝D (5)
and D is the intermediate value after the value of the lower 8 bits, and G _ N is a newly-built PNG image used for storing a corresponding compression value.
As a preferred embodiment of the present invention, the legend accuracy N is 0.1.
As a preferred embodiment of the present invention, the step S2 of performing reduction with different multiples on the PNG format image by using the pyramid model to obtain the PNG format image and each reduced image includes:
taking the PNG format image as the 0 th layer of the pyramid, and carrying out reduction by 2 times, 4 times, 8 times and 16 times by utilizing bicubic difference values to obtain the 1 st layer image, the 2 nd layer image, the 3 rd layer image and the 4 th layer image;
for any pixel point P in the layer 1 image, corresponding to a pixel point P' in the layer 0 image; assuming that the size of the 0-th layer image G is M x N, the size of the reduced 1-th layer target image G is M x N, (x, y) is the position coordinate of the point P, and (x ', y ') is the coordinate of the corresponding point P ' of the point P on G in the image G; wherein, the first and the second end of the pipe are connected with each other,
in the layer 0 image, 16 sampling points P00-P33 closest to P' are selected; assume position P11 as (x)11,y11) The position of P' can be represented as (x)11+u,y11+ v), u, v represent the difference between P' and P11 abscissa and ordinate, respectively; with the pixel pitch of 1, the pixel P' andthe distance of the sampling pixel point P00 is known to be (1+ u,1+ v); respectively taking the abscissa and the ordinate differences as independent variables, calculating the pixel value weighting coefficients in the respective directions by a polynomial interpolation function to obtain the abscissa weight of P00 to P
Weight on ordinate of
The pixel contribution of P00 to P' is Pix00xW (1+ u). xW (1+ v), wherein Pix00A pixel value representing a point P00; similarly, the pixel weight of 16 points to the point P ', i.e. the weighting coefficient, is obtained through an interpolation function, the pixel value of P' is obtained through calculation according to the weighting coefficient and the pixel values of 16 pixels, and finally the pixel value is mapped to the pixel point P in the zoomed image g.
As a preferred embodiment of the present invention, the pixel value Pix of P' is calculated according to the weighting factor and the pixel values of 16 pixels(x',y')The specific steps are as in formula (8):
wherein, PixijRepresenting the pixel values of the closest 16 sample points Pij,
representing the weight of the abscissa direction of the corresponding sample point,
representing the weight of the ordinate direction of the corresponding sample point.
As a preferred embodiment of the present invention, the polynomial interpolation function selects a BiCubic basis function, which is shown in equation (9):
wherein, a is-0.5.
As a preferred embodiment of the present invention, in the slicing operation in step S3, the pixel points of the layer 4 image are taken as the pixel size of a single slice, and the other four-layer data are sliced, and the storage directory is in the following format: filename > number of layers > number of rows > number of columns PNG.
As a preferred embodiment of the present invention, the step S5, the client requesting the slice according to the visualization requirement further includes:
step S51, the client acquires the display map magnification ZS for displaying the meteorological grid data;
step S52, according to ZS, requesting and obtaining a slice group CN from the server;
step S53, obtaining the projection size (lon) of the visible size of the local screen on the map according to ZSs,lats)-(lone,late)；
Step S54, according to the slice group CN and the projection size (lon)s,lats)-(lone,late) And the client requests the server through the browser and acquires the index of the slice to be displayed.
As a preferred embodiment of the present invention, the requesting and obtaining an index of the slice to be displayed in step S54 further includes:
according to the projection size (lon)s,lats)-(lone,late) The corresponding screen distance is found by equation (11):
Screen D＝Distance((lon1,lat1),(lon2,lat2))*scale (11)
determining a scaling layer of the selected slice according to the diagonal length of each layer of the pyramid picture; the scaled layer determination uses equation (12), as follows:
in equation (12), orgSZ represents a diagonal picture size of the compressed resource file; the division of "orgSZ" and "screen D" results in how many times the visible range on the screen is than the compressed original file;
after the zoom layer is determined, the latitude and longitude range of the acquired screen is the upper left corner (lu)x,luy) And the lower right corner (rd)x,rdy) The slice index is obtained according to equation (13):
disx＝tilex-lux
if disx>0 colstart＝ceil(disx/interval)；
else colstart＝0；
colend＝colstart+ceil((rdx-lux)/interval) (13)
in formula (13), tilexIs the upper left corner of the data in the x-direction, disx is the distance between the upper left corner of the screen and the upper left corner of the data in the x-direction, colstartFor initial slice column indexing, colendTo terminate the slice column index, interval is the slice length and width, ceil denotes the divisor rounded up.
As a preferred embodiment of the present invention, the longitude and latitude distances of the two points determined on the screen in step S53 are calculated as follows:
Distance((lons,lats),(lone,late))
＝R*cos-1[cos(lats)*cos(late)*cos(lons-lone)+sin(lats)*sin(late)] (10)
wherein, R is approximately equal to 6371.0km and is the radius of the earth.
In a second aspect, an embodiment of the present invention further provides a system for visualizing hundred-megabyte weather grid data, where the system includes: the system comprises an analysis compression module, a pyramid processing module, a slicing module, a transmission module and a splicing rendering module; wherein the content of the first and second substances,
the analysis compression module is provided with an input port for receiving data, is connected with the pyramid processing module, and is used for carrying out data analysis on the received hundred-megametric meteorological grid data, then carrying out high-8-bit and low-8-bit compression storage on the data, storing the data as a PNG format image and sending the image to the pyramid processing module;
the pyramid processing module is used for reducing the PNG format image by different times by adopting a pyramid model to obtain the PNG format image and each reduced image, and sending the PNG format image and each reduced image to the slicing module;
the slicing module is used for slicing and storing the PNG format image and each reduced image and sending the sliced and stored PNG format image and each reduced image to the transmission module;
the transmission module is used for carrying out multi-thread transmission on all the slices by adopting a distributed network and uploading the slices to a cloud or a server;
the splicing and rendering module is arranged at the client and used for requesting slices to the cloud or the server according to the visualization demand under the request of the client, splicing the obtained slices, and performing grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte-scale meteorological grid data.
The invention has the following beneficial effects:
the hundred-megametric meteorological network data visualization method and system provided by the embodiment of the invention respectively carry out specific processing on links such as data processing, compression and rendering, a multi-terminal rendering framework is constructed, rapid visualization of hundred-megametric grid point data is realized, WebGIS efficient rendering of various spatial grid point big data files can be efficiently supported, a solution is provided for rapid visualization of industrial data and fused spatial big data, and the installation-free visualization efficiency of the grid point big data based on a browser is further improved; the method has the advantages that the loss of characteristic values caused by rarefaction rendering is avoided to the greatest extent by adopting the separation of figures, meanwhile, the rendering capability of the browser end and the advantages of the cloud + end are fully played based on the WebGL technical framework in the rendering process, and the pressure of the server end is relieved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a method for visualizing meteorological grid data in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a bicubic difference principle in an embodiment of the present invention;
FIG. 3 is a flowchart illustrating the process of stitching and visually rendering slice images according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a screen visible map range and slice layers in an embodiment of the invention;
FIG. 5 is a comparison of compression before and after compressing slices in an embodiment of the present invention.
Detailed Description
After finding the above problems, the inventor of the present application has conducted an in-depth study on the existing meteorological grid data visualization method, and finds that the development of the vector slicing technology provides a feasible solution for transmitting large-scale vector data to a client for rendering under a limited bandwidth condition, and also provides a possibility for front-end real-time interaction. For example, the mapper supports the release of grid images and vector image slices, converts geographic data into slices compatible with Google Maps and Earth, and can directly upload the slices to any WEB server or cloud storage device (such as Amazon S3) for release, thereby providing dynamic visual display of elements such as wind, cloud, temperature and ocean waves in terms of meteorological support. With the development of the WebGIS technology, the NOAA develops the NWS Geographic Information Services based on pictures, vector maps and release Services provided by the ArcGIS StoryMaps platform, and comprehensively and visually displays various meteorological data such as meteorological radars, monitoring and alarming. However, since meteorological data is data with a relatively strong frequency of temporal-spatial dynamic change, the dynamic vector slicing scheme involves steps such as data preparation, dynamic vector slicing service, front-end visualization, and the like, and dynamic vector slicing has a high requirement on performance of a server where a spatial database (PostGIS) is located.
For example, a global visualization display system (nullshop. net) is designed by U.S. Nullschool aiming at ECMWF forecast data, a base map converts a map in Natural Earth into a TopoJSON format, meteorological grid data is compressed and transmitted by adopting an epak format, and a foreground is quickly rendered and displayed by using node. js; tokoyo Wind Map is developed by Tokyo weather bureau and the like based on a global visual display system framework. Although the software plays the advantage of quick visualization at the webpage end, the mobile end still has the problem of incompatibility, and the map is a base map of a 50km or 110km resolution coastline profile of a json format, so that the software is difficult to be used for refined service application. The China weather bureau releases 1km of weather live grid data in 2021, the data covers east Asia regions, the grid size is 7000x 4500, and the data volume reaches 106M. For the hundred-megabyte-level meteorological grid data, the page rendering based on the B/S framework is difficult to guarantee both the efficiency and the data accuracy.
It should be noted that the above prior art solutions have defects which are the results of practical and careful study by the inventors, and therefore, the discovery process of the above problems and the solutions proposed by the following embodiments of the present invention to the above problems should be the contribution of the inventors to the present invention in the course of the present invention.
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It should be apparent that the described embodiments are only some of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures. In the description of the present invention, the terms "first," "second," "third," "fourth," and the like are used merely to distinguish one description from another, and are not to be construed as merely or implying relative importance.
After the above deep analysis, the embodiment of the application provides a method for visualizing hundred-megametric meteorological grid data, which solves the following problems: in the visualization process of the front end of the hundred-megabyte-level grid data, the background compression processing cannot realize high-efficiency compression, so that the transmission efficiency is low, the response of a user to the data visualization operation is slow, and the user experience is poor; in the whole visualization process, if the precision requirement of the file data is high in the compression process, the file compression efficiency is low, the requirement of the file compression efficiency is high, and the precision loss of the file content is severe. The hundred million weather grid data here is in the geographic range of 70-140E, 15-60N, and includes 7000 latitude points and 4500 longitude points, with a single file containing over 3 hundred million points, e.g., a single file of precipitation fusion products per hour is 101.3M.
The method for visualizing the hundred-million-level meteorological grid data is applied to personal computer browsers, mobile end browsers, wap and wechat small programs or mobile end APPs and other clients, links such as compression, slicing and displaying are comprehensively considered on the basis of a MapBox open source WebGIS service platform, a hundred-million-level meteorological grid data fast transmission and fast viewing process is constructed, a middle platform support mode is provided for meteorological grid data visualization service, and the hundred-million-level meteorological grid data fast transmission and fast viewing process is published on meteorological visualization columns, mobile phone APPs and WeChat clients of a China meteorological data network. The method provides a quick and convenient solution for quick visual rendering of other grid data, and provides reference for visual application of grid data in other industries.
As shown in fig. 1, the method for visualizing the hundred-megabyte weather grid data includes the following steps:
and step S1, after data analysis is carried out on the hundred-megabyte-level meteorological grid data, the data is compressed and stored in a high 8-bit mode and a low 8-bit mode, and the data is stored as a PNG format image, so that the data precision is stored in the transmission process to the maximum extent.
In this step, data analysis and compression are performed, the meteorological data after data analysis is generally floating point type data, and the floating point type data is used as original data for compression. Setting original data as O, setting legend precision of the visual effect graph as N, multiplying the original data by the reciprocal of the legend precision, and then rounding to obtain conversion data C, as shown in formula (1):
and carrying out high 8-bit value storage on the conversion data C in G data of RGB, carrying out low 8-bit value storage in B data of RGB, wherein the formula (2) is a high 8-bit value formula, the formula (3) is a low 8-bit value formula, and the formulas (4) - (6) are storage formulas:
D＝C％256 (3)
G_N[：][：][：，：，1]＝U (4)
G_N[：][：][：，：，2]＝D (5)
and D is the intermediate value after the value of the lower 8 bits, and G _ N is a newly-built PNG image used for storing a corresponding compression value. The high-low 8-bit compression storage can meet the requirement of visual display of the accumulated value of the meteorological time periods such as year, month and day.
In this step, the legend precision is the precision of three parameters in RGB, and N in this embodiment is preferably 0.1.
The algorithm of this embodiment is applicable to a larger-scale accumulated value and a higher image precision requirement, for example, the visualization precision of three significant digits after the decimal point is retained, which affects the calculation and transmission speed, and belongs to a normal rule. Of course, the method of this embodiment may be similarly applied to the case of small-scale integrated values, but it is not obvious or necessary to improve the calculation and transmission speed.
Using hourly rainfall grid data as an example, the legend accuracy is set to 0.1. The maximum rainfall is below 250mm (rainstorm when the daily rainfall reaches 250 mm), and the value range of the grid data is-32768-32767. And after analyzing the network data, multiplying the network data by 10, rounding to obtain conversion data C, performing high 8-bit value storage on C as G, performing low 8-bit value storage as B, setting R as 0, and storing as a PNG format image in an RGB compression storage mode. The PNG image compresses the file based on the LZ77 derived algorithm, the generated file volume is small, the compression ratio is high, and data is not lost.
Preferably, the mesh data is parsed in this step based on python3.6 and calling the gribpy packet.
And step S2, carrying out reduction of different multiples on the PNG format image by adopting a pyramid model to obtain the PNG format image and each reduced image.
In this step, in order to improve the transmission efficiency, the data is processed according to the image pyramid model. And the compressed PNG file is reduced according to the image pyramid through bicubic interpolation. the tilemap pyramid model is a multi-resolution hierarchical model, and the resolution is reduced from the bottom to the top of the tiled pyramid, but the represented geographic range remains unchanged, i.e., the image is scaled and the squares are refilled. And performing downscaling scaling processing on the image pyramid on the compressed PNG file by using the bicubic difference value. The 7000x 4500 images are scaled first, and then the scaled images are square-filled according to the slice pyramid model.
Taking the PNG format image as the 0 th layer of the pyramid, and carrying out reduction by 2 times, 4 times, 8 times and 16 times by utilizing bicubic difference values to obtain the 1 st layer image, the 2 nd layer image, the 3 rd layer image and the 4 th layer image. In the pyramid model, the layer 1 image is calculated according to the layer 0 image, the layer 2 image is calculated according to the layer 1 image, the layer 3 image is calculated according to the layer 2 image, the layer 4 image is calculated according to the layer 3 image, and the logic process of calculation of each layer of image is the same.
The logic process of scaling the image of the pyramid model will be described below by taking the example of computing the layer 1 image from the layer 0 image.
For any pixel point P in the layer 1 image, it corresponds to a pixel point P' in the layer 0 image. Assuming that the size of the 0-th layer image G is M × N, the size of the reduced 1-th layer target image G is M × N, (x, y) is the position coordinate of the point P, and (x ', y ') is the coordinate of the corresponding point P ' of the point P on G in the image G. Wherein, the first and the second end of the pipe are connected with each other,
as shown in fig. 2, in the layer 0 image, 16 sampling points P00-P33 closest to P 'are selected, and the 16 sampling points have different weighting coefficients for the pixel influence of P', and the weighting coefficients are obtained by the position relationship and the interpolation function.
Assume position P11 as (x)11,y11) The position of P' can be represented as (x)11+u,y11+ v), u, v represent the difference between P' and P11 abscissa and ordinate, respectively. With the pixel pitch as 1, the distance between the pixel P' and the known sampling pixel P00 is (1+ u,1+ v). Respectively taking the abscissa and ordinate differences as independent variables, calculating the pixel value weighting coefficients in the respective directions by a polynomial interpolation function to obtain the abscissa weighting of P00 to P
Weight on ordinate of
The pixel contribution of P00 to P' is Pix00xW (1+ u). times.W (1+ v), wherein Pix00Indicating the pixel value at point P00. Similarly, the pixel weight of 16 points to the point P ', i.e. the weighting coefficient, is obtained through an interpolation function, the pixel value of P' is obtained through calculation according to the weighting coefficient and the pixel values of 16 pixels, and finally the pixel value is mapped to the pixel point P in the zoomed image g. In this step, the pixel value Pix of P' is calculated according to the weighting coefficient and the pixel values of 16 pixels(x',y')The specific steps are as formula (8):
wherein, PixijRepresenting the pixel values of the closest 16 sample points Pij,
representing the weight of the abscissa direction of the corresponding sample point,
representing the weight of the ordinate direction of the corresponding sample point.
Preferably, in this embodiment, the polynomial interpolation function selects a BiCubic basis function as shown in formula (9).
Wherein, a is-0.5.
In step S3, the PNG format image and each reduced image are sliced and stored.
In this step, during slicing operation, the pixel point of the 4 th layer image is taken as the pixel size of a single slice, and other four-layer data is sliced, and the storage directory is in the following format: filename > number of layers > number of rows > number of columns PNG.
And step S4, performing multi-thread transmission on all slices by adopting a distributed network.
In the step, when multi-thread transmission is performed, the number of concurrent requests of the browser is limited for the same domain name, and the commonly used browsers such as Chrome, firefox, IE11 and the like are all 6, so that the concurrent multi-request to slices in the page rendering process is limited, and request queuing or overtime is caused. In the embodiment, the concurrency number is increased by adding the domain dash of the second-level domain, and the concurrency of each second-level domain is controlled to be 2-4 by considering that the DNS resolution burden is increased by excessive domain names.
Meanwhile, the slices uploaded in multiple threads can be stored in the cloud, the cloud serves as a server, when the browser needs the slices, a request is sent to the cloud, the slices needed by the browser are directly downloaded and obtained, and all the slices do not need to be downloaded.
The embodiment is based on a B/S service framework, adopts a Nginx server for proxy, and Nginx is a lightweight Web server/reverse proxy server and an electronic mail (IMAP/POP3) proxy server, is issued under a BSD-like protocol, and has the advantages of less memory occupation and strong concurrency capability. In addition, the Nginx can start a Gzip compression function, so that css, js, xml and html files of the website can be compressed during transmission, the access speed is increased, and the performance of the Nginx is optimized.
And step S5, the client requests slices according to visualization requirements, splices the acquired slices, and performs grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte meteorological grid data.
In the step, the client requests slicing according to the visualization requirement, and the optimal number of slicing layers is adopted by comprehensively considering different screen resolutions and map magnification factors. And converting the map coordinate and the screen coordinate through the conversion of the map longitude and latitude coordinate and the screen coordinate. By solving the problem of scale, the space information of the original grib2 file on the earth corresponding to the longitude and latitude range is changed into a screen range, and finally the best rendered pyramid slice is confirmed according to the size of each layer of the image pyramid of the grib2 file.
As shown in fig. 3 and 4, the client requesting slicing according to visualization requirements further includes:
in step S51, the client obtains the display map enlargement size ZS locally used for displaying the weather grid data.
And step S52, requesting and acquiring the slice group CN from the server according to ZS.
Step S53, obtaining the projection size (lon) of the visible size of the local screen on the map according to ZSs,lats)-(lone,late)。
In this step, the longitude and latitude distances of two points determined on the screen are calculated as follows:
Distance((lons,lats),(lone,late))
＝R*cos-1[cos(lats)*cos(late)*cos(lons-lone)+sin(lats)*sin(late)] (10)
wherein, R is approximately equal to 6371.0km and is the radius of the earth.
Step S54, according to the slice group CN and the projection size (lon)s,lats)-(lone,late) And the client requests the server through the browser and acquires the index of the slice to be displayed.
In this step, the projection size (lon) is determineds,lats)-(lone,late) The corresponding screen distance is obtained according to the formula (11) by a mapbox scale scarle.
Screen D＝Distance((lon1,lat1),(lon2,lat2))*scale (11)
And finally, determining the scaling layer of the selected slice according to the diagonal length of each layer of the pyramid picture. The scaled layer is obtained by equation (12) as follows:
in equation (12), orgSZ represents a diagonal picture size of the compressed resource file; the division of "orgSZ" and "screen D" results in how many times the visible range on the screen is than the compressed file (layer 0). Since the sizes of adjacent layers differ by a factor of two, invoking the indication layer will round up.
After the zoom layer is determined, the latitude and longitude range of the acquired screen is the upper left corner (lu)x,luy) And the lower right corner (rd)x,rdy) The slice index is obtained according to equation (13):
disx＝tilex-lux
if disx>0 colstart＝ceil(disx/interval)；
else colstart＝0；
colend＝ colstart+ceil((rdx-lux)/interval) (13)
in formula (13), tilexIs the upper left corner of the data in the x-direction, disx is the distance between the upper left corner of the screen and the upper left corner of the data in the x-direction, colstartFor initial slice column indexing, colendTo terminate the slice column index, interval is the slice length and width, ceil denotes the divisor rounded up.
The same way the row indices of the start and end slices can be obtained.
The coordinate range of the slice must be larger than the coordinate range of the screen, so some offset is made for all slices. And taking the upper left corner of the screen coordinate as an original point, and calculating the difference value of the pixel coordinate of the upper left corner of each slice and the pixel coordinate of the slice of the upper left corner of the screen to obtain the deviation value on the page map.
After the request slicing is completed, the following steps are executed:
step S55, the client requests the browser to dynamically splice the slices according to the index to form a needed visual image;
and step S56, decompressing the visual image into the display map, and finishing rendering.
In the rendering process of the step, based on an Vue development framework, the space-time attribute of meteorological data and the requirement of high-speed rendering are comprehensively considered, mapbox is used as the service support of geographic information, and the front-end visual development of the system is completed by combining a WebGL high-performance front-end rendering technology. Vue is a progressive and high-performance front-end javascript framework of a page, which mainly takes view layer rendering as a core, and utilizes a component mechanism, a routing mechanism, state management and the like to quickly realize front-end high-frequency operation and realize efficient interaction on the page.
Preferably, the embodiment adopts nodejs service, and vue-cli (vue scaffold) is installed through npm, so that the rapid building of the frame is realized. Mapbox is a high-efficiency WEB GIS development framework aiming at GIS engines with responses of different platforms. In the embodiment, development of the web-side Html5 is realized by adopting a Mapbox GL JS (Mapbox-GL component), and the Mapbox-GL is a JavaScript library which can render a large number of map elements and has smooth interaction and animation effects.
By adopting the visualization method for the hundred-megabyte-level meteorological grid data, the hourly precipitation fusion data with the resolution of 1km x 1km at 16:00 days at 9 and 23 months in 2020 is visualized, wherein the hourly precipitation fusion data comprises 7000 latitude points and 4500 longitude points, a single file contains more than 3 hundred million points, and the size of the single file of an hourly precipitation fusion product is about 100M.
After analyzing the time-by-time precipitation fusion data, performing high-low 8-bit compression and storing the data as a PNG format image; and then, carrying out reduction on the PNG format image by different multiples by adopting a pyramid model to generate 2-time reduced G1(3500x2250), 4-time reduced G2(1750x1125), 8-time reduced G3(875x562) and 16-time reduced G4(437x281), and constructing 5-layer pyramid data including the original image. And (3) supplementing a square to the image file of each layer, taking the maximum length or width as the side length of the square, creating a new square canvas, overlapping the upper left corner of the original image and the upper left corner of the square, and filling the rest part with white. After the processing is finished, the data of each layer of the pyramid is as follows: the 5-layer pyramid determines the degree of the pixel interval corresponding to each scaling ratio according to the scaling factor, and the degree is shown in table 1, wherein the degree is G _ N _0(7000x7000), G _ N _1(3500x3500), G _ N _2(1750x1750), G _ N _3(875x875) and G _ N _4(437x437) are layers 1, 1 and 2, and G _ N _2(1750x1750) and G _ N _3(875x875) are layers 3. During slicing operation, G _ N _4 is used as the pixel size of a single slice to slice other four layers of data, and a storage directory is prepared for a page request according to a file name, a layer number, a row number and a column number.
TABLE 1 slice compression information
Through the visualization process of the hundred-megabyte weather grid data in the embodiment, as shown in fig. 5, the time for compressing all slices in the original file (layer 0) to complete the transmission progress from the server to the browser is 1000 times faster than the time for opening the original slices when 256 concurrent channels are opened; the maximum number of browser requests includes slices of maps and data, with the actual maximum request amount in this embodiment being less than 60, since the visual slices are determined from the map range; 64 concurrent channels are opened for requests, containing 8 subzones.
In this embodiment, an hour precipitation lattice point file with a size of 120M is processed by using the methods of geoJSON processing, binary compression, grayscale map conversion, and the like, which are commonly used in the webGIS, and the processing result is shown in table 2. As shown in table 2, the high-low eight-bit compression algorithm adopted in this embodiment is lossless, and compresses data to the maximum extent.
Table 2 background processing results during various webGIS rendering
According to the technical scheme, the visualization method of the hundred-megabyte-level meteorological network data is based on a VUE framework of HTML5, browsers such as a PC and a mobile terminal are used as carriers, links such as data processing, compression and rendering are respectively subjected to specific processing, a multi-terminal rendering framework is constructed, the rapid visualization of the hundred-megabyte-level lattice point data is realized, the WebGIS efficient rendering of various spatial lattice point big data files can be efficiently supported, a solution is provided for the rapid visualization of industrial data and fused spatial big data, and the installation-free visualization efficiency of the lattice point big data based on the browsers is further improved; the method of replacing original data with compressed images avoids loss of characteristic values caused by sparse rendering to a great extent, and meanwhile, based on a WebGL technical framework, rendering capability of a browser end and the advantages of a cloud end are fully played, and pressure of a server end is relieved.
Based on the same idea, an embodiment of the present invention further provides a system for visualizing hundred-megabyte-level meteorological grid data, where the system includes: the system comprises an analysis compression module, a pyramid processing module, a slicing module, a transmission module and a splicing rendering module;
the analysis compression module is provided with an input port for receiving data, is connected with the pyramid processing module, and is used for performing data analysis on the received hundred-megametric meteorological grid data, then performing high 8-bit and low 8-bit compression storage on the data, storing the data as a PNG format image and sending the PNG format image to the pyramid processing module;
the pyramid processing module is used for reducing the PNG format image by different times by adopting a pyramid model to obtain the PNG format image and each reduced image, and sending the PNG format image and each reduced image to the slicing module;
the slicing module is used for slicing and storing the PNG format image and each reduced image and sending the sliced and stored PNG format image and each reduced image to the transmission module;
the transmission module is used for performing multi-thread transmission on all slices by adopting a distributed network, wherein the transmission module is a process of uploading the slices to a cloud or a server;
the splicing and rendering module is arranged at the client and used for requesting slices to the cloud or the server according to the visualization demand under the request of the client, splicing the obtained slices, and performing grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte-scale meteorological grid data.
In the embodiment, each module is realized by a processor, and when the storage is needed, the storage is added appropriately. The Processor may be, but is not limited to, a microprocessor MPU, a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, and the like. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).
The visualization system of the hundred-megabyte-scale meteorological grid data of the embodiment corresponds to the visualization method of the hundred-megabyte-scale meteorological grid data, and the description of the method is also applicable to the system, and is not repeated herein.
The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features and the technical features (but not limited to) having similar functions disclosed in the present invention are mutually replaced to form the technical solution.
Claims (10)
1. A visualization method of hundred-megawatt meteorological grid data is characterized by comprising the following steps:
step S1, after data analysis is carried out on the hundred megametric meteorological grid data, high 8-bit and low 8-bit compression storage is carried out on the data, and the data are stored as PNG format images;
step S2, carrying out reduction of different multiples on the PNG format image by adopting a pyramid model to obtain a PNG format image and each reduced image;
step S3, slicing and storing the PNG format image and each reduced image;
step S4, carrying out multi-thread transmission on all slices by adopting a distributed network;
and step S5, the client requests slices according to visualization requirements, splices the acquired slices, and performs grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte meteorological grid data.
2. The method for visualizing the hundred megametric weather grid data as claimed in claim 1, wherein the compressed storing and saving of the data with the upper 8 bits and the lower 8 bits as the PNG format image comprises:
setting original data as O, setting legend precision of the visual effect graph as N, multiplying the original data by the reciprocal of the legend precision, and then rounding to obtain conversion data C, as shown in formula (1):
and performing high 8-bit value storage on the conversion data C into RGB G data, performing low 8-bit value storage into RGB B data, wherein the formula (2) is a high 8-bit value formula, the formula (3) is a low 8-bit value formula, and the formulas (4) - (6) are storage formulas:
D＝C％256 (3)
G_N[：][：][：，：，1]＝U (4)
G_N[：][：][：，：，2]＝D (5)
and D is the intermediate value after the value of the lower 8 bits, and G _ N is a newly-built PNG image used for storing a corresponding compression value.
3. The method for visualizing the hundred megametric weather grid data of claim 2, wherein the legend accuracy N is 0.1.
4. The method for visualizing the hundred-megabyte weather grid data as claimed in claim 1, wherein the step S2 is to perform reduction of the PNG format image by different multiples by using a pyramid model to obtain the PNG format image and each reduced image, and comprises:
taking the PNG format image as the 0 th layer of the pyramid, and carrying out reduction by 2 times, 4 times, 8 times and 16 times by utilizing bicubic difference values to obtain the 1 st layer image, the 2 nd layer image, the 3 rd layer image and the 4 th layer image;
for any pixel point P in the layer 1 image, corresponding to a pixel point P' in the layer 0 image; assuming that the size of the 0-th layer image G is M x N, the size of the reduced 1-th layer target image G is M x N, (x, y) is the position coordinate of the point P, and (x ', y ') is the coordinate of the corresponding point P ' of the point P on G in the image G; wherein the content of the first and second substances,
in the layer 0 image, 16 sampling points P00-P33 closest to P' are selected; assume position P11 as (x)11,y11) The position of P' can be represented as (x)11+u,y11+ v), u, v represent the difference between P' and P11 abscissa and ordinate, respectively; taking the pixel point distance as 1, and the distance between the pixel point P' and the known sampling pixel point P00 as (1+ u,1+ v); respectively taking the abscissa and the ordinate differences as independent variables, calculating the pixel value weighting coefficients in the respective directions by a polynomial interpolation function to obtain the abscissa weight of P00 to P
Weight on ordinate of
The pixel contribution of P00 to P' is Pix00xW (1+ u). xW (1+ v), wherein Pix00A pixel value representing a point P00; similarly, a weighting coefficient, which is the pixel weight of 16 points to the point P ', is obtained through an interpolation function, the pixel value of P' is obtained through calculation according to the weighting coefficient and the pixel values of 16 pixels, and the pixel value is mapped to the pixel point P in the zoomed image g.
5. The visualization method for the hundred megawatt-level meteorological grid data according to claim 4, wherein the pixel value Pix of P' is obtained through calculation according to the weighting coefficient and the pixel values of 16 pixels(x',y')The specific steps are as in formula (8):
wherein, PixijRepresenting the pixel values of the closest 16 sample points Pij,
representing the weight of the abscissa direction of the corresponding sample point,
representing the weight of the ordinate direction of the corresponding sample point.
6. The method for visualizing hundred megametric weather grid data as claimed in claim 4 or 5, wherein the polynomial interpolation function selects BiCubic basis functions, BiCubic basis functions are represented by formula (9):
wherein, a is-0.5.
7. The method for visualizing the hundred megametric weather grid data as claimed in claim 4, wherein in the slicing operation of step S3, the pixel points of the layer 4 image are taken as the pixel size of a single slice, and the other four-layer data are sliced, and the storage directory is in the following format: filename > number of levels > number of rows > number of columns PNG.
8. The visualization method for hundred megametric weather grid data according to claim 1, wherein the step S5, the client requesting the slice according to the visualization requirement further comprises:
step S51, the client acquires the display map magnification ZS for displaying the meteorological grid data;
step S52, requesting and acquiring a slice group CN from a server according to ZS;
step S53, obtaining the projection size (lon) of the visible size of the local screen on the map according to ZSs,lats)-(lone,late)；
Step S54, according to the slice group CN and the projection size (lon)s,lats)-(lone,late) And the client requests the server through the browser and acquires the index of the slice to be displayed.
9. The method for visualizing the hundred megametric weather grid data as claimed in claim 8, wherein the step of requesting and obtaining an index of the slice to be displayed in step S54 further comprises:
according to the projection size (lon)s,lats)-(lone,late) The corresponding screen distance is found by equation (11):
Screen D＝Distance((lon1,lat1),(lon2,lat2))*scale (11)
determining a scaling layer of the selected slice according to the diagonal length of each layer of the pyramid picture; the scaled layer determination uses equation (12), as follows:
in equation (12), orgSZ represents a diagonal picture size of the compressed resource file; the division of "orgSZ" and "screen D" results in how many times the visible range on the screen is than the compressed original file;
after the layer scaling is determined, the latitude and longitude range of the acquired screen, namely the upper left corner (lu)x,luy) And the lower right corner (rd)x,rdy) The slice index is obtained according to equation (13):
disx＝tilex-lux
if disx>0 colstart＝ceil(disx/interval)；
else colstart＝0；
colend＝colstart+ceil((rdx-lux)/interval) (13)
in formula (13), tilexIs the upper left corner of the data in the x-direction, disx is the distance between the upper left corner of the screen and the upper left corner of the data in the x-direction, colstartFor initial slice column indexing, colendTo terminate the slice column index, interval is the slice length and width, ceil denotes the divisor rounded up.
10. A system for visualizing hundred megametric weather grid data, the system comprising: the system comprises an analysis compression module, a pyramid processing module, a slicing module, a transmission module and a splicing rendering module; wherein the content of the first and second substances,
the analysis compression module is provided with an input port for receiving data, is connected with the pyramid processing module, and is used for carrying out data analysis on the received hundred-megametric meteorological grid data, then carrying out high-8-bit and low-8-bit compression storage on the data, storing the data as a PNG format image and sending the image to the pyramid processing module;
the pyramid processing module is used for reducing the PNG format image by different times by adopting a pyramid model to obtain the PNG format image and each reduced image, and sending the PNG format image and each reduced image to the slicing module;
the slicing module is used for slicing and storing the PNG format image and each reduced image and sending the sliced and stored PNG format image and each reduced image to the transmission module;
the transmission module is used for carrying out multi-thread transmission on all the slices by adopting a distributed network and uploading the slices to a cloud or a server;
the splicing and rendering module is used for requesting slices to a cloud end or a server according to visualization requirements under the request of a client, splicing the obtained slices, and performing grid conversion and visualization rendering on the spliced images to complete visualization of the hundred-megabyte meteorological grid data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210249428.4A CN114625992B (en) | 2022-03-14 | 2022-03-14 | Visualization method and system for hundred-megawatt meteorological network data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210249428.4A CN114625992B (en) | 2022-03-14 | 2022-03-14 | Visualization method and system for hundred-megawatt meteorological network data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114625992A true CN114625992A (en) | 2022-06-14 |
| CN114625992B CN114625992B (en) | 2022-11-22 |
Family
ID=81902436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210249428.4A Active CN114625992B (en) | 2022-03-14 | 2022-03-14 | Visualization method and system for hundred-megawatt meteorological network data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114625992B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115964037A (en) * | 2023-01-05 | 2023-04-14 | 三峡高科信息技术有限责任公司 | Engineering data visualization low code configuration method and system |
| CN117351163A (en) * | 2023-10-24 | 2024-01-05 | 中国科学院华南植物园 | Ecological pattern analysis visualization method, system, equipment and medium |
| CN117874301A (en) * | 2024-03-11 | 2024-04-12 | 浙江省气象台 | Method for processing, storing and calling pyramid slice based on grid data |
| CN117874301B (en) * | 2024-03-11 | 2024-05-31 | 浙江省气象台 | Method for processing, storing and calling pyramid slice based on grid data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455624A (en) * | 2013-09-16 | 2013-12-18 | 湖北文理学院 | Implement method of lightweight-class global multi-dimensional remote-sensing image network map service |
| CN103744997A (en) * | 2014-01-23 | 2014-04-23 | 广东中科遥感技术有限公司 | Network map service publication method and system |
| CN106383965A (en) * | 2016-10-13 | 2017-02-08 | 国家卫星气象中心 | Three-dimensional numerical atmospheric visual support system |
| CN108256127A (en) * | 2018-03-06 | 2018-07-06 | 北京工业大学 | A kind of weather-map system of customizable pattern |
-
2022
- 2022-03-14 CN CN202210249428.4A patent/CN114625992B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455624A (en) * | 2013-09-16 | 2013-12-18 | 湖北文理学院 | Implement method of lightweight-class global multi-dimensional remote-sensing image network map service |
| CN103744997A (en) * | 2014-01-23 | 2014-04-23 | 广东中科遥感技术有限公司 | Network map service publication method and system |
| CN106383965A (en) * | 2016-10-13 | 2017-02-08 | 国家卫星气象中心 | Three-dimensional numerical atmospheric visual support system |
| CN108256127A (en) * | 2018-03-06 | 2018-07-06 | 北京工业大学 | A kind of weather-map system of customizable pattern |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115964037A (en) * | 2023-01-05 | 2023-04-14 | 三峡高科信息技术有限责任公司 | Engineering data visualization low code configuration method and system |
| CN115964037B (en) * | 2023-01-05 | 2023-08-29 | 三峡高科信息技术有限责任公司 | Engineering data visualization low-code configuration method and system |
| CN117351163A (en) * | 2023-10-24 | 2024-01-05 | 中国科学院华南植物园 | Ecological pattern analysis visualization method, system, equipment and medium |
| CN117351163B (en) * | 2023-10-24 | 2024-05-03 | 中国科学院华南植物园 | Ecological pattern analysis visualization method, system, equipment and medium |
| CN117874301A (en) * | 2024-03-11 | 2024-04-12 | 浙江省气象台 | Method for processing, storing and calling pyramid slice based on grid data |
| CN117874301B (en) * | 2024-03-11 | 2024-05-31 | 浙江省气象台 | Method for processing, storing and calling pyramid slice based on grid data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114625992B (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114625992B (en) | Visualization method and system for hundred-megawatt meteorological network data | |
| CN109360153B (en) | Image processing method, super-resolution model generation method and device and electronic equipment | |
| Ruzinoor et al. | A review on 3D terrain visualization of GIS data: techniques and software | |
| CN108700988B (en) | Digital image presentation | |
| CN114491351B (en) | Lattice point data visualization method and system adaptable to WebGIS platform | |
| US20220074754A1 (en) | Dynamic scaling of geospatial data on maps | |
| WO2022057868A1 (en) | Image super-resolution method and electronic device | |
| Deibe et al. | Supporting multi-resolution out-of-core rendering of massive LiDAR point clouds through non-redundant data structures | |
| CN114723869B (en) | Image processing method and device | |
| Wang et al. | Capturing the dance of the earth: PolarGlobe: Real-time scientific visualization of vector field data to support climate science | |
| US9471959B2 (en) | Water color gradients on a digital map | |
| CN113077541B (en) | Virtual sky picture rendering method and related equipment | |
| CN112115226B (en) | Map rendering method and map rendering device | |
| Raposo | Variable DEM generalization using local entropy for terrain representation through scale | |
| US20230092057A1 (en) | Dynamic data tiling | |
| CN114996374A (en) | Online data visualization implementation method, system, device and medium | |
| CN113538468B (en) | Method and device for constructing high-definition satellite map vector slice based on svg format | |
| CN115689886A (en) | Distributed slicing method and distributed slicing device for framing images | |
| Yang et al. | Quick compression and transmission of meteorological big data in complicated visualization systems | |
| US20150154784A1 (en) | Use of Photo Animation Transitions to Mask Latency | |
| CN116737852A (en) | Vector tile data-based vector drawing method and device and electronic equipment | |
| Kim | Web-based geovisualization system of oceanographic information using dynamic particles and html5 | |
| CN115952244B (en) | On-line loading method, device and medium for vector slicing service in drawing software | |
| CN115984119B (en) | Meteorological data processing method and device, electronic equipment and storage medium | |
| CN114488070B (en) | Radar echo extrapolation method and device based on deep learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |