CN108366109A - A kind of meteorological data numerical forecast cloud shared platform and data sharing method - Google Patents
A kind of meteorological data numerical forecast cloud shared platform and data sharing method Download PDFInfo
- Publication number
- CN108366109A CN108366109A CN201810100754.2A CN201810100754A CN108366109A CN 108366109 A CN108366109 A CN 108366109A CN 201810100754 A CN201810100754 A CN 201810100754A CN 108366109 A CN108366109 A CN 108366109A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- cloud
- platform
- forecast
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/567—Integrating service provisioning from a plurality of service providers
Abstract
The present invention proposes a kind of meteorological data numerical forecast cloud shared platform, including：It is used for transmission, handles, storing, the sharing service platform of Shared value forecast mass data；It is researched and developed for realizing pattern and releases many intelligence platforms with technology；And for localization data processing and for the branch center platform of user's access process result.The invention also discloses a kind of data sharing methods.Meteorological data numerical forecast cloud shared platform proposed by the present invention and its data sharing method are around the core objective for improving numerical forecast accuracy rate and fining level, adhere to demand pull, strengthen innovation driving, with intensive, standardized development pattern, using cloud, big data, internet+, the information technologies such as intelligence, establish using high-resolution region Numerical Prediction System as the intensive numerical forecast Grid data applied business system of core.
Description
Technical field
The present invention relates to cloud and weather data analysis field more particularly to a kind of meteorological data numerical forecast cloud are shared
Platform and data sharing method.
Background technology
For a long time, numerical forecast mass data transfers are all the difficulties in service operation.This not only constrains height
The range of horizontal, high-resolution business model application so that national forecast personnel can not divide on the national platform of same level
Analysis and forecast, various regions prediction ability show larger difference；Also affect the state of numerical forecast intensivization development indirectly simultaneously
Family's strategy so that the development of region mode is in relatively unordered, small without the serious actual state of complete, low-level duplicate construction.
International and domestic Meteorological Science And Technology new development trend is made a general survey of, the new normality of national socio-economic development, whole nation meteorology thing are serviced
The pattern that industry is in the fast-developing phase is unchanged, and the pressure for facing international keen competition does not subtract.How high-resolution region mould is changed
The present situation of formula development, intensive, the modernization of realization service technique development, realizes that crowd discovers for the first time the good situation of exhibition, is put in pipe
The key problem of the line research staff of reason person.
Invention content
It is an object of the present invention to build and develop Chinese numerical forecast profession cloud, national numerical forecast core industry is realized
The shared and cooperation R ＆ D of business system.Core numerical forecast operation system Grid data, passes through the whole nation of numerical forecast profession cloud
It is shared, the enabling capabilities of high-resolution region numerical forecast operation system are greatly improved, the reduction whole nation is general to build numerical forecast system
System phenomenon, concentrating manpower improve numerical forecast ability；High-resolution Grid data, can meet the whole nation close in short-term, lattice
The demand of point forecast and fine forecast.
A kind of meteorological data numerical forecast cloud shared platform proposed by the present invention, including：It is used for transmission, handles, storing, altogether
Enjoy the sharing service platform of numerical forecast mass data；It is researched and developed for realizing pattern and releases many intelligence platforms with technology；And it is used for
Localization data processing and the branch center platform for user's access process result.
In the preferred technical solution of the present invention, the sharing service platform includes：For realizing application access system, Yun Cun
The basic system platform of storage system, data processing system, certification and callback module, business management system；With for realizing browsing,
The unified and open portal of data of the data and product Visualization Service such as inquiry.
In the preferred technical solution of the present invention, the basic platform system includes：
Using AM access module, the basic platform system and outer application system are realized for providing corresponding plug-in unit or module
Seamless connection and data customization；
Cloud storage module, for providing object storage, NoSQL storages, filing storage and database purchase；
Data processing module is obtained for logarithm forecast mass data, is sliced, is summarized and working process；
Messenger service module, for formulating messenger service Integrated Solution；
Certification and callback module, for creating user identity management, Service Privileges management, user management and callback mechanism；
And
Service management module for managing resource, data and permission, and has statistical analysis and log analysis function.
In the preferred technical solution of the present invention, the service management module includes：
Business intelligence display unit is used to rapidly and accurately generate dynamic chart using report tool；Dynamic chart packet
Include column diagram, line chart, bar chart, pie chart, radar map；
Operation management unit is used to manage cloud resource, realizes increase, deletion, change and dynamic adjustment computing resource, deposits
Store up resource, Internet resources and data resource and its state；
Service management unit, for realizing user identity management, Service Privileges management and user management；
Statistical analysis unit is used for resource analysis, data analysis, user behavior analysis and log analysis；
Security managing unit is used for real-time recorded data activity situation, and different grain size audit is carried out to data manipulation
Compliance management alerts the risk behavior that data are subjected to, is blocked to attack.
In the preferred technical solution of the present invention, further comprise：For being managed in sharing service platform described in backed up in synchronization
Information, user information and service information backup system.
In the preferred technical solution of the present invention, further comprise：Big data resource, calculating for being provided using cloud platform
The interface service of ability and algorithm cluster realizes the business aftertreatment products disposed on the cloud of weather service product after-treatment system
Cloud system.
In the preferred technical solution of the present invention, the meteorological data numerical forecast cloud shared platform is high using multizone deployment
Fast content distributing network carries out data distribution and transmission.
The invention also provides a kind of numerical forecast cloud data sharing methods, including：
Initial data uploads the object of numerical forecast cloud shared system from data source by internet or public cloud special line
In storage service；
After successfully by initial data upload object storage service after, it is automatic to send initial data upload message inbound message clothes
Business, the message informing that data conversion storage service subscription file uploads, while the message informing that data subscription service subscription file uploads；
After data conversion storage service receives file upload message, initial data is cut into the file of smaller particle size in advance
Subsequent batch download service is served, and is served in the data conversion storage in resolution file to NoSQL table storage services follow-up
Accurate reading；
Data conversion storage service write-in object storage after by original document slice, after being written successfully, automatic send disappears
Cease inbound message service；
NoSQL tables are written after parsing initial data, according to a variety of predefined granularities in data conversion storage service
In storage；
User terminal, which is adopted, to be got numerical forecast by forecast platform, data platform or webpage directly browsing and forecasts accordingly
Product result.
In the preferred technical solution of the present invention, the initial data passes through from data source on internet or public cloud special line
It passes in the object storage service of numerical forecast cloud shared system and further includes：
Initial data is migrated to NoSQL storage services from object storage service by Transformation Service；
The collective data collection of initial data is split as various dimensions record by pre-defined rule；Various dimensions are recorded as including by mould
Formula type, variable, space, time more element types data；
Various dimensions record is stored in by corresponding element type in NoSQL storage services.
In the preferred technical solution of the present invention, the user terminal is adopted through common forecast platform, data platform or net
Page directly browsing gets the corresponding forecast model products result of numerical forecast and further includes：
User submits personalized customization information, forms message event；
Message event is transmitted in messenger service, messenger service starts calling data processing work after detecting new information
Industry；
Data processing operation executes the acquisition of data, summarizes, handles, and obtains handling result and object is written from storage service；
Event message has been calculated into messenger service transmission, and returns to handling result and have access to or download for user to user and make
With.
Compared with prior art, beneficial effects of the present invention：Meteorological data numerical forecast cloud proposed by the present invention is shared flat
Platform and its data sharing method around numerical forecast accuracy rate and the horizontal core objective of fining is improved, adhere to demand pull,
Strengthen innovation driving, with intensive, standardized development pattern, using cloud, big data, internet+, the information such as intelligence
Technology is established using high-resolution region Numerical Prediction System as the intensive numerical forecast Grid data applied business body of core
System.By intensive many wound type numerical forecast profession cloud platforms of unified standard, uniform data and unified management, it is pre- to form numerical value
" cloud+end " application model of report business, scientific research etc., promotes weather information technology and modernization Meteorological Services are horizontal, accelerates to push away
It applies to seamless, precision, intelligent direction and develops into numerical forecast lattice point.
Description of the drawings
Fig. 1 shows the business structures of meteorological data numerical forecast cloud shared platform of the present invention.
Fig. 2 indicates the application architecture of meteorological data numerical forecast cloud shared platform of the present invention.
Fig. 3 indicates the feature list of application architecture.
Fig. 4 indicates the Technical Architecture of meteorological data numerical forecast cloud shared platform of the present invention.
Fig. 5 indicates the data framework of meteorological data numerical forecast cloud shared platform of the present invention.
Fig. 6 indicates the schematic diagram of high-speed content distribution network.
Fig. 7 indicates the stream compression figure of numerical forecast cloud data sharing method of the present invention.
Fig. 8 shows the flow charts of the stream compression of numerical forecast cloud data sharing method of the present invention.
Specific implementation mode
Below in conjunction with schematic diagram to meteorological data numerical forecast cloud shared platform proposed by the present invention and its data sharing
Method is described in more detail, and which show the preferred embodiment of the present invention, it should be appreciated that those skilled in the art can be with
Invention described herein is changed, and still realizes the advantageous effects of the present invention.Therefore, following description should be understood as
Those skilled in the art's is widely known, and is not intended as limitation of the present invention.
Technical solution of the present invention use TOGAF system architecture design method opinions, from own service feature carry out planning and
Problem to be solved is started with, and business demand is gradually disassembled, and then therefrom seeks the frame of information-based systems construction.TOGAF(The
Open Group Architecture Framework, open group architecture frame) by international standard authoritative organization The
Open Group are formulated.TOGAF is the process model based on an iteration (Iterative), support best practices and it is a set of can
The existing framework assets reused.The present invention, which designs, relies on business structure, application architecture, data framework, Technical Architecture level Four to carry out
Personalization and the practicability of numerical forecast profession cloud platform system Construction are realized in design.
The business structure of meteorological data numerical forecast cloud shared platform of the present invention is as shown in Figure 1, numerical forecast profession cloud master
To include numerical forecast data sharing, service and ability opening and innovation service.Wherein numerical forecast data sharing service according to
Service object is divided into user class service, system-level service and interface level service, and wherein user class service mainly passes through basic system
Platform externally provides data and service of goods, and user subscribes to data and the production of oneself care according to oneself demand by subscription service
Product are downloaded real-time informing booking reader after service is updated automatically, complete numerical forecast reprocessing, analysis and
It excavates.System-level service includes mainly calculating service, storage service and data conversion storage service；Interface level service mainly utilizes application
The standard interface that linking is built realizes the slitless connection with systems such as CIMISS, MICAPS, SWAN；Ability opening and innovation service
Cross many intelligence platform construction services ability is open and open source community, numerical forecast profession cloud handling capacity open service is by numerical value
Mode computation ability and NWP cloud abilitys are opened in a manner of servicing to be used to user, while will be carried for user by open source community
For pattern research and development and NWP product application function.
The application architecture of numerical forecast cloud of the present invention is shown in Fig. 2, which includes three platforms：Numerical value
The professional cloud sharing service platform of forecast, many intelligence platforms, branch center platform.Wherein sharing service platform include basic system platform,
The unified and open portal of data platform, many intelligence platforms include many intelligence basic platforms and the unified and open portal of many intelligence platforms, and branch center is flat
Platform is that the function localization of sharing service platform is realized, and is controlled by sharing service platform, and specific feature list is shown in Fig. 3.
Business structure and application architecture based on system have carried out technical framework design, and numerical value of the present invention is shown in Fig. 4
Forecast that the Technical Architecture figure of cloud, meteorological data numerical forecast cloud shared platform are built upon one under advanced cloud service frame
Integrated application platform is provided using required interface and service product, is uniformly opened by Business Management Platform and data platform
Portal is put, is realized to a data of data sharing and service of goods.
Technological frame includes mainly storage and computation layer, service layer and presentation layer, wherein storage includes mainly with computation layer
Storage capacity and computing capability, wherein storage capacity mainly by building cloud storage system, realize the difference for different services
Storage of type, including object storage, NoSQL storages etc., elastic Service Source, big number of the computing capability using public cloud offer
Cutting and the unloading service that data are realized in service are calculated according to analysis.Online service ability is main using data processing, analytical technology
There is provided batch download, subscribe to, browsing with customization etc. user classes service, it is system-level service and interface level service, wherein interface level and
System-level service relates generally to three big abilities and rents：Computing capability, storage capacity and network capabilities, platform are connect by data service
Mouthful, unified Data access standard is provided, and realize calculate service ability, storage service ability, network service capabilities can be horizontal
To extension, the seamless connection with operational forecast platform is completed.Presentation layer is mainly realized by the unified and open portal of data platform clear
It data and the product Visualization Service such as lookes at, inquire, big data analysis computing capability is utilized to realize Personalized service.
The Technical Architecture and application architecture of meteorological data numerical forecast cloud shared platform of the present invention carry out data prediction profession
Cloud data framework is as shown in figure 5, include mainly the data, numerical forecast profession cloud data and provincial meteorological inning in Grid data source
According to the data in wherein Grid data source depend in State Bureau, Beijing, Shanghai, the Numerical Prediction Models operation of four, Guangzhou
The Grid data of the heart, data source upload the data to sharing service platform by application accordingly.Sharing service platform mainly wraps
It includes：
Original domain：For storing initial data；
Normal data domain：By cutting, the processing such as summarize, store, user's specification product data are provided；
Self-defining data domain：Calculating analysis by big data reconfigures the result data of processing for personalized product
Data.
Branch center platform is localized data processing using the test data of branch center or other data, by handling result
It is deposited into the normal data domain of branch center platform, for authorized user's Lawful access, while also providing customized data production
Product.
Embodiment
Based on above-mentioned open group architecture frame, meteorological data numerical forecast cloud shared platform packet in this specific embodiment
Include three platforms：Sharing service platform, many intelligence platforms, branch center platform.Sharing service platform is used for transmission, handles, storing, altogether
Enjoy numerical forecast mass data.Many intelligence platforms, which are researched and developed and released for realizing pattern, uses technology.Branch center platform is for localizing number
According to processing and for user's access process result.
Data service of the sharing service platform based on unified standardization of the present invention is drawn using flattening design concept
Public cloud resource express network resource, smooth autgmentability, and the features such as diversification, standardized service product, based on first
Into cloud framework design and construction numerical forecast basic system platform, be responsible for multiple regions center numerical forecasting product it is timely on
Cloud is provided with data service for all types of user by basic system platform, is met from the short essence just before Short-term Weather numerical forecast
Refinement, quick numerical forecast demand for services.Wherein, sharing service platform is by basic system platform and the unified and open portal of data
It constitutes.
Basic system platform is for realizing application access system, cloud storage system, data processing system, certification and readjustment mould
Block, business management system.Basic system platform is the base support of sharing service platform, and basic system platform mainly uses cloud to take
Business rent mode carries out the integrated of system and integrates, realizes data and service of goods.This system is carried using domestic public cloud service
For the products such as the cloud storage service of quotient, calculating service, messenger service, database service, Cloud Server, network service, research clothes
The contents such as business provider properties of product, price, application scenarios, standard interface, technical strength, technical support ability, and to service
The technical solution of provider is compared, and is preferentially chosen.Referring specifically to Fig. 2, in this specific embodiment, basic system platform includes
Following function module：
(1) AM access module is applied
It the modern information technologies such as excavates based on cloud computing and big data, establishes country-province's two-stage weather monitoring analysis, short faces
Early warning, medium-short term prediction and professional forecast business module, in order to combine the softwares such as MICAPS and CIMISS of meteorological system maturation,
Core is done step-by-step for MICAPS system development ability open innovation platforms in research and development plug-in type application and interpretation intellectual technology
The seamless connection of heart business application system and numerical forecast profession cloud is realized intelligent, visualization and intensivization development, is constructed
Using AM access module.
It is the bridge between other application system and basic system platform using AM access module.It defines and unites using AM access module
One service standard externally provides data service, and data become reusable information resources service according to certain business rule, are convenient for
Data transmission and access interface are provided for platforms such as MICAPS and CIMISS；It supports to different technologies, language, using offer
Functional interface carries out standards service encapsulation, is flexibly selected convenient for different callers.It will be applied in this specific embodiment
AM access module is using the api interface opened to the outside world based on API gateway and existing open ability is rented, for MICAPS, CIMISS
Equal forecast platforms provide corresponding data customization service, and provide corresponding plug-in unit or module to realize seamless connection.
(2) cloud storage module
Cloud storage module uses the cloud storage service of public cloud, and the data or product design for different business demand are different
Cloud storage system, including the data-storage system of object-oriented, the data-storage system towards table structure, filing data storage
System, system R, the wherein data-storage system of object-oriented mainly store recent meteorological data file；Face
Nearest three days data are mainly stored to the data-storage system of structure, are serviced for the accurate quick search of user and generation；
Filing data-storage system mainly stores cold data, and the big data analysis in later stage, system R is convenient for mainly to store
Relational data provides data supporting for business, management platform.In this specific embodiment, cloud storage module using object storage,
NoSQL storages, filing storage and database purchase, the system architecture of more different service providers, price, are answered at properties of product
With scene etc., rational cloud storage Integrated Solution is formulated, ensures the on-demand optimal storage of different data and product.
(3) data processing module
Data processing module is mainly responsible for Grid data file and is converted into the list data that can accurately inquire, by list data
The forecast model products that user needs are generated, for forecaster's real-time query.It, can be real-time simultaneously according to users ' individualized requirement
Meet the personalized shared service demand of user.Major function includes that object storage is read, decoding, slice, storage, NoSQL tables
Storage, Data Integration.
In this specific embodiment, data processing module handles service or self-built application platform into line number using big data is rented
According to acquisition, be sliced, summarize and working process, and dump in object storage.Rational data processing scheme is formulated, is rationally made
With the cloud storage service of rental, business demand, R＆D costs and data-handling efficiency etc. are considered, ensure different data and production
Product are rapidly and efficiently handled on demand.
(4) messenger service module
Messenger service module is mainly responsible for the scheduling of message, including message queue is established, and is read, and subscribes to and notice services.
The user that this service is subscribed to according to subscription list real-time informing is subjected to data when system generates new numerical forecasting product
Or displaying live view, the download of product.
In the present embodiment, messenger service module is provided using the messenger service product for renting public cloud, more different services
System architecture, properties of product, price, application scenarios of quotient etc. formulate rational messenger service Integrated Solution, ensure the fast of message
Fast reliable transmission.
(5) certification and callback module
Certification and callback module are responsible for the uniform registration of user, certification, empowerment management, callback function, by certification with return
Mode transfer block is authenticated, authorizes to the user of base application, logging in and account management.Additionally by callback mechanism for system at
The result managed is adjusted back in real time, is further processed in time convenient for system.In the present embodiment, certification is adopted with callback module
On the basis of based on the original authentication mechanism of public cloud, exploitation creates user identity management, Service Privileges management, user management and returns
The system of tune mechanism, the readjustment processing function of main unified certification management, the rights management and event for realizing cloud platform.
(6) service management module
The professional cloud platform of service management module logarithm forecast and many intelligence platform base cloud service facility resources are managed,
It is managed for resource services such as Cloud Server, object storage, NoSQL storage services involved in public cloud service, together
When data analysis, log analysis, user behavior analysis are carried out to the service of offer, by analysis result in a manner of BI (business intelligence)
It is shown, the service of offer is optimized and improved convenient for system, provided for Meteorological Unit convenient for preferably more good
Service.
Service management module carries out integrated management, including resource, number for numerical forecast profession cloud platform and many intelligence platforms
According to the management with permission, and has the ability of statistical analysis and log analysis.In this specific embodiment, service management module has
Such as lower unit：
(a) business intelligence display unit uses flexible report tool rapidly and accurately to generate interactive and powerful
Dynamic chart, user can select column diagram, line chart, bar chart, pie chart, radar map etc. according to different business demands, be
Operation management personnel or policymaker propose decision-making foundation, and enterprise is helped to make wise business business decision.
(b) operation management unit realizes management to public cloud resource, for involved by increasing, delete, change
Computing resource, storage resource, Internet resources and data resource, and the state of various resources can be adjusted by tactful dynamic, with full
Sufficient demand；The utilization rate statistical report form of resource, such as computing resource (CPU, memory, network, IO etc.) are provided, (capacity, visit are stored
The amount of asking), the operation conditions of entire numerical forecast profession cloud platform can be monitored in real time, reference is provided for resource allocation；According to data
Demand could be adjusted to ensure the high availability and high efficiency of data to storage strategy.
(c) service management unit leads user identity management, Service Privileges management and user management to be offered, core
Function includes user identity and empowerment management, and Identity Management includes user identity management, is grouped management and role to user
Identity Management；Service Privileges management includes resource service and authorization service, is authorized to user or Role Identity, for business
System customizes required resource service (including resource, data, shared service etc.), ensures to optimize scheduling of resource, for
Authorization service provides standardized data service or processing method, and corresponding permission is arranged, to ensure its applicability.
(d) statistical analysis unit comprising resource analysis, data analysis, user behavior analysis and log analysis, resource point
Analysis includes mainly resource service condition, provides weekly, monthly magazine, annual report, utilization of resources trend is obtained, for the distribution in resource later stage
Foundation is provided；Data analysis Main Analysis data service condition includes upload, the shared situation remittance table of data.User behavior point
Analysis provides the hot spot region of data and the focus of user by the analysis of user's service condition (later stage precision inquiry), right
Research and development of products provides feedback opinion.User, which is obtained, by log analysis accesses the PV values of content, independent IP numbers (i.e. after duplicate removal
IP quantity) etc., service ranking list, user's residence time or the highest clothes of access times that user retrieves or browses are calculated
Business etc., convenient for being adjusted and optimizing for post-service.
(e) security managing unit, can real-time recorded data activity situation, to data manipulation carry out different grain size audit
Compliance management, the risk behavior that data are subjected to is alerted, attack is blocked.It is by visiting user
Record, analysis and the report for asking data behavior, user generates compliance report afterwards, accident is traced to its source for helping, simultaneously plus
Strong inside and outside data network behavior record improves data assets safety.
(the unified and open portal of data platform)
The unified and open portal of data platform specifically includes for realizing data and product Visualization Services such as browsing, inquiries
Service navigation, service customization and Visualization Service.Service navigation is to show open directory service, including data service and production
Product service, user can be navigated to by fuzzy query in the service of specific or certain class, while can carry out service subscription.System
Latest services real time push can be given to the user subscribed to according to user's subscribed content, be obtained at the first time convenient for user newest
Data information.Service customization is to provide varigrained subject product, and user can be fixed according to the type unrestricted choice of system
Make the service needed.System generates the product customized or data by quick search, processing, and having access to or downloading for user makes
With.User can comment on and mark for product simultaneously, and system will analyze the content of comment, utilize the knot of analysis
Fruit carries out the exploitation of products perfection or new product.Visualization Service is to provide the online Visualization Service function of data product, forecast
On-line Product can directly access browsing and download, show form by diversification, realize data visualization function.
(many intelligence platforms)
Many intelligence platforms encourage pattern research staff for formulating support policy, incentive mechanism and the system of intellectual property rights protection
Independent research or collaborative development.Many intelligence platforms establish the access of operation system new technology, new algorithm and new technological process and exit system,
Based on project web site, system version management system, defect tracking system and real-time AC system etc., iterative method numerical forecast
The opening of increasing income of pattern and operational forecast system software, implementation pattern are researched and developed and are released many wound environment with technology, it is pre- to build numerical value
Report the good ecology of development.Many intelligence platforms include mainly the increase income open policy and mechanism module, open source community module.
The open policy of increasing income provides the relevant policy guidance increased income, opened of numerical forecast with mechanism module, and works out phase
The specification and management method answered, protection software provider possess associated rights right.For technology in current numerical forecast flow
With code, determines the opening depth and diffusion range of the relevant technologies, ensure its increase income open feasibility and general favour of technology.It opens
The source open policy determines the Leakage in Value and appraisal procedure for open achievement of increasing income with mechanism module, formulates corresponding incentive mechanism,
Excite the enthusiasm of Numerical Forecast Technology more new development and innovation.
Open source community module is using ripe business software or open source projects, including 1) source code edition management system；2)
Defect tracking system；3) project web site；4) open source community united portal website；5) mail server；6) it exchanges and mutual in real time
Dynamic system；7) WIKI systems.Open source community module is by building and integrating project document management, code administration, Version Control, lack
It falls into tracking, real-time interactive and the functions such as exchanges, realize open source community service, iterative method Numerical Prediction Models and operational forecast system
System software is increased income opening, is built pattern research and development and is released many wound environment with technology, builds the good ecology of numerical forecast development.
In the preferred embodiment of the present invention, meteorological data numerical forecast cloud shared platform is also configured with for described in backed up in synchronization
Management information, user information and service information backup system in sharing service platform.Standby system in order to system High Availabitity
Property, build a set of strange land disaster recovery and backup systems.According to business demand, using different public cloud service providers, realize fully functional a set of
Numerical forecast profession cloud platform system.After the failure of main platform, efficiently started in time by manual intervention and serviced, completes numerical value
The service of the professional cloud platform of forecast.Furthermore, it is contemplated that the characteristics of business, standby system only smallest size is run, for managing letter
Breath, user information are synchronous with information on services.
In the preferred technical solution of the present invention, further comprise：Big data resource, calculating for being provided using cloud platform
The interface service of ability and algorithm cluster realizes the business aftertreatment products disposed on the cloud of weather service product after-treatment system
Cloud system.Business aftertreatment products cloud system utilizes big data resource, computing capability and the algorithm cluster that cloud platform provides
Interface service, realize and disposed on the cloud of critical products after-treatment system such as weather business model.After establishing plug-in type intelligence
The normalized interface service (API) of processing system and software development kit (SDK) standard realize that business model data are not landed, closed
The real-time generation of key product is shared and visualizes.
Meteorological data numerical forecast cloud shared platform of the present invention is using clothes such as multizone deployment, content distributing networks (CDN)
The multiple spot deployment of pragmatic existing data, makes user's downloading data that can be got with nearby principle, poor to reduce network operator
The different influence brought.Used high-speed content distribution network is as shown in fig. 6, all systems or module use the side of messaging bus
Formula realizes message trigger, the service timeliness of timing mode before being promoted.
Meteorological data numerical forecast cloud shared platform of the present invention realizes data sharing service, and the whole flow process of shared service is all
Respective service or interface will be provided by being directed to data flow, and formulate corresponding standards and norms, specific data spread signal
Figure is as shown in Fig. 7 and Fig. 8.
Referring particularly to Fig. 8, the data sharing method of meteorological data numerical forecast cloud shared platform of the present invention includes following step
Suddenly：
S1 initial data uploads pair of numerical forecast cloud shared system from data source by internet or public cloud special line
As in storage service.Corresponding object storage service is write direct with the initial data of State Bureau and wide three regions center of going up north
For, the mode data of China Meteorological Administration and Beijing, Shanghai, Guangzhou three regions center passes through the lattice of standardized GRIB2
Formula；By special APP and corresponding configuration information, realize that Grid data is uploaded to object storage.
More specifically, after initial data reaches object storage, by corresponding strategy, turned for corresponding data
It changes.It is migrated from object storage automatically by Transformation Service for the real-time prediction data for providing precision inquiry service are needed
Storage can be extremely inquired, it is more by mode type, variable, space, time etc. to realize that the data set convergence is split as by pre-defined rule
The various dimensions of element record, and are entered into NoSQL storages, provide precision data query service.For need for a long time it is retaining,
The data that need not be accessed in time, pass through archiving policies, it will automatically the file data collection being retained in object storage service,
It remains into corresponding archiving services storage, is moved back again to corresponding position when being needed for user.All operations of above-mentioned conversion
All by the way of unmanned intervene, various conversions and the data service of data mode are realized using Strategy Auto.
S2 after successfully by initial data upload object storage service after, it is automatic to send initial data upload message inbound message
The message that service, the message informing that data conversion storage service subscription file uploads, while data subscription service subscription file upload is logical
Know；
More specifically, data conversion storage service is the standardization resource provided for the making of product and data working process
With data call service all reprocessing and processing task canbe used on line are realized by merging and eliminating reprocessing operation
Extensive batch processing simultaneously stores the result into object storage service, the functions such as completes data slicer, summarize, store.Data
Subscription service be for the data or product of publication provide to the user subscribe to etc. mode, data subscription service by cloud platform number
Newest data information is pushed to user terminal by laying down a regulation according to messenger service and subscription service platform, in order to user, phase
It closes module and system obtains latest data information in time.
S3 is after data conversion storage service receives file upload message, according to message content by raw data associated pre-cut
It is divided into the file service of smaller particle size in subsequent batch download service, and the data conversion storage in resolution file is stored to NoSQL
In serve subsequent accurate reading.
More specifically, in reading the data, lower of most of scene need to be read in a certain weather forecast element
Same data message, conventional method need to read entire data file, thus can also read out most of garbage,
That is the computing resource of occupied bandwidth resource and resolution file also reduces the concurrent capability of system.Convert raw data into knot
Structure data are stored in after NoSQL storages, are accessed the data for all only accessing needs every time, are greatly reduced for hash
Reading, save bandwidth resources, and provide higher concurrent capability and lower access delay.
S4 data conversion storages service write-in object storage again after by original document slice, is written to using buffer technology
It is automatic after work(to send message inbound message service；
NoSQL is written after parsing initial data, according to a variety of predefined granularities in S5 data conversion storage services
In storage；
S6 user terminals adopt that get numerical forecast by forecast platform, data platform or webpage directly browsing pre- accordingly
Report product result.
More specifically, the personalized customization window services that user is provided by portal website submit personalized customization letter
Message event is transmitted in messenger service by breath, background application, and log-on data is handled after message service system detects new information
The operation of system is called, and is executed the acquisition of data, is summarized, handles, and returns to handling result to user, have access in order to user or
It downloads and uses.
The preferred embodiment of the present invention is above are only, does not play the role of any restrictions to the present invention.Belonging to any
Those skilled in the art, in the range of not departing from technical scheme of the present invention, to the invention discloses technical solution and
Technology contents make the variations such as any type of equivalent replacement or modification, belong to the content without departing from technical scheme of the present invention, still
Within belonging to the scope of protection of the present invention.
Claims (10)
1. a kind of meteorological data numerical forecast cloud shared platform, which is characterized in that including：
It is used for transmission, handles, storing, the sharing service platform of Shared value forecast mass data；
It is researched and developed for realizing pattern and releases many intelligence platforms with technology；And
For localization data processing and for the branch center platform of user's access process result.
2. meteorological data numerical forecast cloud shared platform according to claim 1, which is characterized in that the shared service is flat
Platform includes：
For realizing application access system, cloud storage system, data processing system, certification and callback module, business management system
Basic system platform；With
For realizing the unified and open portal of data of the data and product Visualization Service such as browsing, inquiry.
3. meteorological data numerical forecast cloud shared platform according to claim 2, which is characterized in that the basic platform system
System includes：
Using AM access module, the nothing of the basic platform system and outer application system is realized for providing corresponding plug-in unit or module
Seam linking and data customization；
Cloud storage module, for providing object storage, NoSQL storages, filing storage and database purchase；
Data processing module is obtained for logarithm forecast mass data, is sliced, is summarized and working process；
Messenger service module, for formulating messenger service Integrated Solution；
Certification and callback module, for creating user identity management, Service Privileges management, user management and callback mechanism；And
Service management module for managing resource, data and permission, and has statistical analysis and log analysis function.
4. meteorological data numerical forecast cloud shared platform according to claim 3, which is characterized in that the service management mould
Block includes：
Business intelligence display unit is used to rapidly and accurately generate dynamic chart using report tool；Dynamic chart includes column
Shape figure, line chart, bar chart, pie chart, radar map；
Operation management unit is used to manage cloud resource, and realization increases, deletes, change and dynamic adjust computing resource, storage money
Source, Internet resources and data resource and its state；
Service management unit, for realizing user identity management, Service Privileges management and user management；
Statistical analysis unit is used for resource analysis, data analysis, user behavior analysis and log analysis；
Security managing unit is used for real-time recorded data activity situation, and the conjunction that different grain size audit is carried out to data manipulation is advised
Property management, alerts the risk behavior that data are subjected to, is blocked to attack.
5. meteorological data numerical forecast cloud shared platform according to claim 1, which is characterized in that further comprise：With
Management information, user information and service information backup system in the sharing service platform described in backed up in synchronization.
6. meteorological data numerical forecast cloud shared platform according to claim 1, which is characterized in that further comprise：With
In the big data resource, the interface service of computing capability and algorithm cluster that are provided using cloud platform, after realizing weather service product
The business aftertreatment products cloud system disposed on the cloud of processing system.
7. meteorological data numerical forecast cloud shared platform according to claim 1, which is characterized in that the meteorological data number
Value forecast cloud shared platform carries out data distribution and transmission using multizone deployment high-speed content distribution network.
8. a kind of numerical forecast cloud data sharing method, which is characterized in that including：
The object that initial data uploads numerical forecast cloud shared system from data source by internet or public cloud special line stores
In service；
After successfully by initial data upload object storage service after, it is automatic to send initial data upload message inbound message service,
The message informing that data conversion storage service subscription file uploads, while the message informing that data subscription service subscription file uploads；
After data conversion storage service receives file upload message, initial data is cut into the file service of smaller particle size in advance
In subsequent batch download service, and subsequent essence is served in the data conversion storage in resolution file to NoSQL table storage services
Really read；
Data conversion storage service write-in object storage after by original document slice, the automatic message that sends enters after being written successfully
Messenger service；
Data conversion storage service is after parsing initial data, according to a variety of predefined granularity write-in NoSQL tables storages
In；
User terminal is adopted gets the corresponding forecast model products of numerical forecast by forecast platform, data platform or webpage directly browsing
As a result.
9. numerical forecast cloud data sharing method according to claim 8, which is characterized in that the initial data is from data
Source by further including in the object storage service of internet or public cloud special line upload numerical forecast cloud shared system：
Initial data is migrated to NoSQL storage services from object storage service by Transformation Service；
The collective data collection of initial data is split as various dimensions record by pre-defined rule；Various dimensions are recorded as including by pattern class
Type, variable, space, time more element types data；
Various dimensions record is stored in by corresponding element type in NoSQL storage services.
10. numerical forecast cloud data sharing method according to claim 8, which is characterized in that the user terminal, which is adopted, to be passed through
Common forecast platform, data platform or webpage directly browsing get the corresponding forecast model products result of numerical forecast and also wrap
It includes：
User submits personalized customization information, forms message event；
Message event is transmitted in messenger service, messenger service starts calling data processing operation after detecting new information；
Data processing operation executes the acquisition of data, summarizes, handles, and obtains handling result and object is written from storage service；
Event message has been calculated into messenger service transmission, and returns to handling result and have access to or download use for user to user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810100754.2A CN108366109A (en) | 2018-02-01 | 2018-02-01 | A kind of meteorological data numerical forecast cloud shared platform and data sharing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810100754.2A CN108366109A (en) | 2018-02-01 | 2018-02-01 | A kind of meteorological data numerical forecast cloud shared platform and data sharing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108366109A true CN108366109A (en) | 2018-08-03 |
Family
ID=63004235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810100754.2A Pending CN108366109A (en) | 2018-02-01 | 2018-02-01 | A kind of meteorological data numerical forecast cloud shared platform and data sharing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108366109A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325740A (en) * | 2018-09-19 | 2019-02-12 | 中国电力科学研究院有限公司 | A kind of artificial intelligence shared platform and management method for electric system |
| CN109377438A (en) * | 2018-11-27 | 2019-02-22 | 武汉华信联创技术工程有限公司 | A kind of hazard weather closes on early-warning and predicting business platform |
| CN110489836A (en) * | 2019-08-07 | 2019-11-22 | 成都市环境保护科学研究院 | Long-term prediction of air quality system and method in a kind of predrive |
| CN110750962A (en) * | 2019-10-14 | 2020-02-04 | 深圳旗鱼体育传播有限公司 | Weather data conversion method and system |
| CN110969829A (en) * | 2019-10-18 | 2020-04-07 | 国网浙江省电力有限公司电力科学研究院 | Power failure data sharing method based on cloud platform computing and message bus technology |
| CN112000311A (en) * | 2020-07-16 | 2020-11-27 | 广东省气象台(南海海洋气象预报中心) | Intelligent forecasting tool crowd creating platform and sharing method |
| CN112015786A (en) * | 2019-05-29 | 2020-12-01 | 国家卫星气象中心 | Extreme weather monitoring and early warning information processing system for external competition field |
| CN112765463A (en) * | 2021-01-12 | 2021-05-07 | 陈漩 | Data management method for big data and user requirements and cloud computing server |
| CN113010591A (en) * | 2021-03-22 | 2021-06-22 | 薪得付信息技术(上海)有限公司 | Report generation method, device and system based on application interface |
| CN113076361A (en) * | 2021-03-17 | 2021-07-06 | 国家气象信息中心(中国气象局气象数据中心) | Method for realizing massive meteorological data unified service interface based on big data |
| CN116108019A (en) * | 2023-04-03 | 2023-05-12 | 中国民用航空局空中交通管理局航空气象中心 | Meteorological big data sharing service system based on distributed technology |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521716A (en) * | 2011-12-31 | 2012-06-27 | 成都信息工程学院 | Integrative specialized weather service integrated system |
| CN103327088A (en) * | 2013-06-08 | 2013-09-25 | 东北大学 | Cloud computing service building system and method based on IMS signaling system |
| CN103577936A (en) * | 2013-11-15 | 2014-02-12 | 国家电网公司 | Distributed maintenance and global sharing system for power grid model and implementation method thereof |
| CN105391803A (en) * | 2015-12-15 | 2016-03-09 | 腾讯科技（深圳）有限公司 | Message pushing method and device |
| CN106506586A (en) * | 2016-09-21 | 2017-03-15 | 东软集团股份有限公司 | Distributed caching synchronous method, device, node and system |
-
2018
- 2018-02-01 CN CN201810100754.2A patent/CN108366109A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521716A (en) * | 2011-12-31 | 2012-06-27 | 成都信息工程学院 | Integrative specialized weather service integrated system |
| CN103327088A (en) * | 2013-06-08 | 2013-09-25 | 东北大学 | Cloud computing service building system and method based on IMS signaling system |
| CN103577936A (en) * | 2013-11-15 | 2014-02-12 | 国家电网公司 | Distributed maintenance and global sharing system for power grid model and implementation method thereof |
| CN105391803A (en) * | 2015-12-15 | 2016-03-09 | 腾讯科技（深圳）有限公司 | Message pushing method and device |
| CN106506586A (en) * | 2016-09-21 | 2017-03-15 | 东软集团股份有限公司 | Distributed caching synchronous method, device, node and system |
Non-Patent Citations (1)
| Title |
|---|
| 殷岳、王晓峰等: ""数值预报云"", 《气象科技进展》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325740A (en) * | 2018-09-19 | 2019-02-12 | 中国电力科学研究院有限公司 | A kind of artificial intelligence shared platform and management method for electric system |
| CN109377438A (en) * | 2018-11-27 | 2019-02-22 | 武汉华信联创技术工程有限公司 | A kind of hazard weather closes on early-warning and predicting business platform |
| CN112015786A (en) * | 2019-05-29 | 2020-12-01 | 国家卫星气象中心 | Extreme weather monitoring and early warning information processing system for external competition field |
| CN110489836A (en) * | 2019-08-07 | 2019-11-22 | 成都市环境保护科学研究院 | Long-term prediction of air quality system and method in a kind of predrive |
| CN110489836B (en) * | 2019-08-07 | 2023-01-24 | 成都市环境保护科学研究院 | Pre-driven medium-and-long-term air quality forecasting system and method |
| CN110750962A (en) * | 2019-10-14 | 2020-02-04 | 深圳旗鱼体育传播有限公司 | Weather data conversion method and system |
| CN110750962B (en) * | 2019-10-14 | 2020-08-28 | 深圳旗鱼体育传播有限公司 | Weather data conversion method and system |
| CN110969829A (en) * | 2019-10-18 | 2020-04-07 | 国网浙江省电力有限公司电力科学研究院 | Power failure data sharing method based on cloud platform computing and message bus technology |
| CN112000311B (en) * | 2020-07-16 | 2021-05-25 | 广东省气象台(南海海洋气象预报中心) | Intelligent forecasting tool crowd creating platform system and sharing method |
| CN112000311A (en) * | 2020-07-16 | 2020-11-27 | 广东省气象台(南海海洋气象预报中心) | Intelligent forecasting tool crowd creating platform and sharing method |
| CN112765463A (en) * | 2021-01-12 | 2021-05-07 | 陈漩 | Data management method for big data and user requirements and cloud computing server |
| CN113076361A (en) * | 2021-03-17 | 2021-07-06 | 国家气象信息中心(中国气象局气象数据中心) | Method for realizing massive meteorological data unified service interface based on big data |
| CN113010591A (en) * | 2021-03-22 | 2021-06-22 | 薪得付信息技术(上海)有限公司 | Report generation method, device and system based on application interface |
| CN113010591B (en) * | 2021-03-22 | 2024-04-19 | 薪得付信息技术(上海)有限公司 | Report generation method, device and system based on application interface |
| CN116108019A (en) * | 2023-04-03 | 2023-05-12 | 中国民用航空局空中交通管理局航空气象中心 | Meteorological big data sharing service system based on distributed technology |
| CN116108019B (en) * | 2023-04-03 | 2023-06-16 | 中国民用航空局空中交通管理局航空气象中心 | Meteorological big data sharing service system based on distributed technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108366109A (en) | A kind of meteorological data numerical forecast cloud shared platform and data sharing method | |
| CN106445556B (en) | A kind of visual code generation method and system | |
| CN101018259B (en) | Telecom integrated information system and method | |
| CN104809634B (en) | Tourism data is investigated and monitoring system | |
| AU2012238999B2 (en) | Reservation method and system with improved PNR handling | |
| CN102340529B (en) | Page generating system and page generating method based on WAP (Wireless Application Protocol) platform | |
| CN107977795A (en) | A kind of government procurement manages transaction system | |
| CN111694555B (en) | Service system construction method and device, electronic equipment and storage medium | |
| CN108197895A (en) | A kind of enterprise information system Rights Management System | |
| CN102968719A (en) | Payment server, terminal and payment channel access method | |
| CN111861140A (en) | Service processing method, device, storage medium and electronic device | |
| CN101552801A (en) | A method and system for on-line browsing and downloading the address-book of user group | |
| CN108491164A (en) | A kind of mixed cloud storage architecture system | |
| KR102221863B1 (en) | Server and system for it service management, and method thereof | |
| CN110780856B (en) | Electricity data release platform based on micro-service | |
| CN104933607A (en) | Satellite remote sensing data order service system | |
| CN109636307B (en) | River chang APP system | |
| CN112801607A (en) | Management service platform and construction method | |
| US9870542B2 (en) | Managing information technology solution centers | |
| CN108305190A (en) | Tourism cloud service platform and method | |
| CN104735104A (en) | Intelligent public platform system of WeChat | |
| CN103226779A (en) | Road logistics price index | |
| CN105207989A (en) | Multi-dimensional user workbench system and control method thereof | |
| CN109873720A (en) | A kind of cloud center intelligence O&M method and system based on enterprise's public platform | |
| US8839452B1 (en) | Access rights mechanism for corporate records |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180803 |