CN112650800A - Data intelligent platform based on globalization collection - Google Patents

Data intelligent platform based on globalization collection Download PDF

Info

Publication number
CN112650800A
CN112650800A CN202010886643.6A CN202010886643A CN112650800A CN 112650800 A CN112650800 A CN 112650800A CN 202010886643 A CN202010886643 A CN 202010886643A CN 112650800 A CN112650800 A CN 112650800A
Authority
CN
China
Prior art keywords
data
service
platform
intelligent platform
collection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010886643.6A
Other languages
Chinese (zh)
Inventor
赵晓丽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Gedou Technology Co ltd
Original Assignee
Chengdu Gedou Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Gedou Technology Co ltd filed Critical Chengdu Gedou Technology Co ltd
Priority to CN202010886643.6A priority Critical patent/CN112650800A/en
Publication of CN112650800A publication Critical patent/CN112650800A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2465—Query processing support for facilitating data mining operations in structured databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Fuzzy Systems (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data intelligent platform based on global acquisition, which comprises a service terminal system, a data server, a service internal data receiver, a content distribution network, a CDN resource monitoring and downloading conversion center for all services, a source-attached data distributed storage infrastructure and an online analysis processing data warehouse. By adopting the structure, the invention can change the traditional data application architecture foundation, is not a data platform with vertical service, but is a data middle platform for overall accessing and partitioning BI data, avoids repeated operation and maintenance of the data platform, is oriented to all applications in an enterprise, is not mutually isolated any more, solves the problems of global data acquisition, service data isolated island and the like from the architecture level, plans the existing upper buildings such as a data lake, a data middle platform, a refined commercial intelligent index platform and the like, integrates data access, analysis and display, and provides perfect data management and BI service.

Description

Data intelligent platform based on globalization collection
Technical Field
The invention relates to the technical field of digital information, in particular to a data intelligent platform based on global acquisition.
Background
At present, data acquisition schemes in the market are in various states, enough components can meet most of data pipeline requirements in terms of single acquisition function, even if the acquisition requirement logic is very complex, the complex requirements can be independently realized by depending on a middleware, but various middleware products cannot respectively and independently and completely support a perfect data management platform. The service lines crossing the machine rooms and the clouds are mutually vertical, the network delay is high, globalized data collection is difficult to carry out, so that the data islanding phenomenon mostly occurs, the data is scattered, the data sharing and analysis between the service lines and the data centers are very difficult, and the service lines are led to be independently developed and maintained for one set of data access and data business intelligent service platform, so that the coupling is high, the inconvenience is very high, and the cost is relatively high.
The invention aims to solve the problems of global data acquisition and data isolated island, realize the global multi-point double-end acquisition in a file stream form around a Content Distribution Network (CDN), realize the middle-stage intelligent data service in single-point convergence around a distributed file system and a data warehouse as a core, break through the mutually vertical relation between the original service lines, enable service data to flow to a data lake at the same time, and partition heterogeneous data so as to facilitate the extraction and analysis of upper-layer application. Therefore, those skilled in the art provide a globalized data-based intelligent platform to solve the problems mentioned in the background art.
Disclosure of Invention
The invention aims to provide a data intelligent platform based on global acquisition to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme:
a data intelligent platform based on global collection comprises a service terminal system, a data server, a service internal data receiver, a content distribution network, all service CDN resource monitoring and downloading conversion centers, a source-attached data distributed storage infrastructure and an online analysis processing data warehouse.
As a further scheme of the invention: the service terminal system is a data producer, and data produced by a user at the terminal is transmitted to the data server through various protocols, and corresponding data is output by dotting according to service requirements.
As a still further scheme of the invention: the service internal data receiver is responsible for uniformly accessing all local data, and the local data receiver drops the data onto a content distribution network so as to facilitate downstream services and global high-speed access.
As a still further scheme of the invention: and the CDN resource monitoring and downloading conversion center for all services is responsible for summarizing data of different services in different regions around the world.
As a still further scheme of the invention: the source data distributed storage infrastructure is used for uniformly planning and storing data sets from different service different areas.
As a still further scheme of the invention: and the online analysis processing data warehouse distinguishes and redundantly processes data according to different use scenes so as to facilitate the extraction and use of upper layer data service application.
As a still further scheme of the invention: the data intelligent platform executes code:
Figure BDA0002655756350000021
Figure BDA0002655756350000031
Figure BDA0002655756350000041
Figure BDA0002655756350000051
Figure BDA0002655756350000061
Figure BDA0002655756350000071
Figure BDA0002655756350000081
Figure BDA0002655756350000091
Figure BDA0002655756350000101
Figure BDA0002655756350000111
Figure BDA0002655756350000121
Figure BDA0002655756350000131
compared with the prior art, the invention has the beneficial effects that:
by adopting the structure, the traditional data application architecture foundation can be changed, the data platform with the vertical service is not a data platform, the data platform in the data is integrally accessed and partitioned BI data, the repeated operation and maintenance of the data platform is avoided, the data in the data platform faces all applications in an enterprise, and the data are not isolated from each other; the CDN-based solution for global access can effectively solve the problem of high data access delay caused by spanning continents, and a main-standby mode is adopted to ensure HA (high availability) of the middlebox data access.
Drawings
Fig. 1 is a schematic view of a workflow of a data intelligent platform based on global collection.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, in an embodiment of the present invention, a data intelligent platform based on global collection, a service terminal system, a data producer, data produced by a user at a terminal are transmitted to a data server through various protocols, dotting and outputting corresponding data according to service requirements, wherein a service internal data receiver is responsible for uniformly accessing all local data, the local data receiver is used for landing the data on a content distribution network, the data processing system is convenient for downstream service, global high-speed access, monitoring of all service CDN resources and downloading of a conversion center, and is responsible for summarizing data of different services in different regions of the world, pasting source data distributed storage infrastructure, uniformly planning and storing data sets from different regions of different services, analyzing and processing a data warehouse on line, distinguishing and redundant data according to different use scenes, and facilitating extraction and use of upper layer data service application;
the invention is centered on solving the problems of global data acquisition and data isolated island, collects data locally and at two ends of a data lake around a CDN, and finally land the data mined from commercial values produced by a user at a terminal to a middle station end for unified treatment through operations such as cleaning, conversion, partitioning and the like, so that the data is not only limited to high availability of a flow, but also has a good monitoring and alarming mechanism for the whole link, and further ensures the reliability of global data acquisition. In the upper-layer application, additional service platforms with uniform apertures and different scenes are planned.
The data intelligent platform executes the codes as follows:
Figure BDA0002655756350000141
Figure BDA0002655756350000151
Figure BDA0002655756350000161
Figure BDA0002655756350000171
Figure BDA0002655756350000181
Figure BDA0002655756350000191
Figure BDA0002655756350000201
Figure BDA0002655756350000211
Figure BDA0002655756350000221
Figure BDA0002655756350000231
Figure BDA0002655756350000241
Figure BDA0002655756350000251
the invention covers the whole data life cycle, acquires different application data indiscriminately from data birth to data value realization, and realizes global data cocurrent based on the CDN. And accessing a unified data lake, and realizing a layered and multi-scenario data warehouse OLAP mechanism according to requirements, so that the upper-layer commercial intelligent data service has diversified use scenarios. The invention can positively solve the problems of global data acquisition, data link monitoring and data isolated island.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.

Claims (7)

1. A data intelligent platform based on global collection is characterized by comprising a service terminal system, a data server, a service internal data receiver, a content distribution network, a CDN resource monitoring and downloading conversion center for all services, a source-attached data distributed storage infrastructure and an online analysis processing data warehouse.
2. The intelligent platform for data based on globalization collection as recited in claim 1, wherein said business terminal system is a data producer, and the data produced by the user at the terminal is transmitted to the data server through various protocols, and the corresponding data is output according to the business requirement.
3. The intelligent platform for data based on globalization collection as recited in claim 1, wherein said internal data receiver of the service is responsible for unified access to all local data, and the local data receiver drops the data onto the content distribution network for downstream services, globalization and high-speed access.
4. The intelligent platform for data based on global collection according to claim 1, wherein S is a CDN resource monitoring and download switching center for all services, which is responsible for summarizing data of different services in different regions around the world.
5. The intelligent platform for data based on global collection according to claim 1, wherein the distributed storage infrastructure for source data is configured to store data sets from different areas served by different services in a unified manner.
6. The intelligent platform for data based on global collection according to claim 1, wherein the online analysis processes the data warehouse, and distinguishes and redundantly data according to different usage scenarios, so as to facilitate the extraction and use of upper layer data service application.
7. The globalization collection based data intelligence platform of claim 1, wherein the data intelligence platform executes code to:
Figure FDA0002655756340000011
Figure FDA0002655756340000021
Figure FDA0002655756340000031
Figure FDA0002655756340000041
Figure FDA0002655756340000051
Figure FDA0002655756340000061
Figure FDA0002655756340000071
Figure FDA0002655756340000081
Figure FDA0002655756340000091
Figure FDA0002655756340000101
Figure FDA0002655756340000111
Figure FDA0002655756340000121
CN202010886643.6A 2020-08-28 2020-08-28 Data intelligent platform based on globalization collection Pending CN112650800A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010886643.6A CN112650800A (en) 2020-08-28 2020-08-28 Data intelligent platform based on globalization collection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010886643.6A CN112650800A (en) 2020-08-28 2020-08-28 Data intelligent platform based on globalization collection

Publications (1)

Publication Number Publication Date
CN112650800A true CN112650800A (en) 2021-04-13

Family

ID=75346083

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010886643.6A Pending CN112650800A (en) 2020-08-28 2020-08-28 Data intelligent platform based on globalization collection

Country Status (1)

Country Link
CN (1) CN112650800A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104794150A (en) * 2015-01-30 2015-07-22 北京东方泰坦科技股份有限公司 Cloud storage model and management method based on space knowledge cloud environment
CN105389766A (en) * 2015-12-17 2016-03-09 北京中科云集科技有限公司 Smart city management method and system based on cloud platform
CN109146372A (en) * 2018-04-26 2019-01-04 宏泰集团（厦门）有限公司 Global wisdom manufactures sales service system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104794150A (en) * 2015-01-30 2015-07-22 北京东方泰坦科技股份有限公司 Cloud storage model and management method based on space knowledge cloud environment
CN105389766A (en) * 2015-12-17 2016-03-09 北京中科云集科技有限公司 Smart city management method and system based on cloud platform
CN109146372A (en) * 2018-04-26 2019-01-04 宏泰集团（厦门）有限公司 Global wisdom manufactures sales service system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘飞: "商务智能框架中的数据体系结构研究和设计", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Similar Documents

Publication Publication Date Title
CA3087309C (en) Blockchain-based data processing method and device
CN107590182B (en) Distributed log collection method
CN108335075B (en) Logistics big data oriented processing system and method
Malecki The economic geography of the Internet’s infrastructure
CN106651633A (en) Power utilization information acquisition system and method based on big data technology
CN102710785B (en) Cloud service node architecture in self-service tourism system, and service collaborating and balancing module and method among service nodes in self-service tourism system
WO2017175071A1 (en) Virtual infrastructure
CN102929961A (en) Data processing method and device thereof based on building quick data staging channel
CN109074287B (en) Infrastructure resource status
CN107612984B (en) Big data platform based on internet
CN104281456B (en) A kind of Oil-Gas Field Development Engineering application software system based on cloud computing
CN102412990B (en) Remote sensing satellite raw data recording system with centralized management and real-time transmission functions
CN111885439B (en) Optical network integrated management and duty management system
CN101860078A (en) Electric power integrated communication resource management system
CN112291728A (en) Private industry application platform implementation architecture based on 5G network
CN111641671A (en) Resource matrix type elastic computing desktop cloud system
CN102427470A (en) Architecture method of cloud monitoring center
CN103488697B (en) System and mobile terminal capable of automatically collecting and exchanging fragmented commercial information
CN110086647A (en) A kind of big data information centre network architecture method
CN112650800A (en) Data intelligent platform based on globalization collection
CN104035775A (en) Comprehensive front-end system of bank
CN215298210U (en) Multistage edge computing system of electric power thing networking
CN115412553A (en) CMDB automatic configuration method based on distributed cloud platform
CN114466039A (en) OPCUA communication system and method based on 5G private network
CN113467885A (en) Equipment operation and maintenance monitoring system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210413

WD01 Invention patent application deemed withdrawn after publication