CN104536965B - A kind of data query display systems under the conditions of big data and method - Google Patents
A kind of data query display systems under the conditions of big data and method Download PDFInfo
- Publication number
- CN104536965B CN104536965B CN201410662697.9A CN201410662697A CN104536965B CN 104536965 B CN104536965 B CN 104536965B CN 201410662697 A CN201410662697 A CN 201410662697A CN 104536965 B CN104536965 B CN 104536965B
- Authority
- CN
- China
- Prior art keywords
- data
- inquiry request
- rule
- client
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The present invention relates to the data query display systems and method under the conditions of a kind of big data, including client creation data inquiry request and it is sent to Web server；The data inquiry request of reception is sent to background server by Web server；The data inquiry request of reception is put into queue by background server, and data are obtained from database server according to queue mechanism, and storage management is carried out to the data of acquisition according to data configuration rule；Task completion prompt message is notified into client by message queue；Client obtains required data file according to obtained prompt message.The present invention solves the problems, such as that data query displaying success rate is low under the conditions of big data, it is big to be occupied to client rs PC host resource, the present invention ensures the quasi real time characteristic of efficiency data query using the data processing method of dynamic allocation process and efficient message queue mechanism, reliable and stable file management facilities can ensure the safety of sensitive information.
Description
Technical field
The present invention relates under the conditions of field of computer technology more particularly to a kind of big data data query display systems and
Method.
Background technology
Becoming increasingly popular for mobile communication equipment brings huge user group to telecom operators, based on telecommunication service
Complex characteristics can record a large amount of business record data in business operation support system.The staff of operator is in daily work
In work, in order to understand business development situation, provide the user with accurate data, it may be desirable to real-time business statistics information.For example,
User, the account information etc. for the signing member and member that certain government and enterprise customers needs this enterprise all in operator.
Signing member of some larger government and enterprise customers in provincial operator will reach tens thousand of, be based on Web frames
The business operation support system of frame is received from server-side asks progress data statistics, data organization, data to be transferred to browser
The process of client, the case where being frequently encountered session timeout, and client browser can occupy client in this process
The resource for holding PC more influences other operations of staff.
Invention content
The technical problem to be solved by the present invention is in view of the deficiencies of the prior art, provide the number under the conditions of a kind of big data
It is investigated that asking display systems and method.
The technical solution that the present invention solves above-mentioned technical problem is as follows：A kind of data query displaying system under the conditions of big data
System, which is characterized in that including client, Web server, background server and database server；
The client is used for creation data inquiry request and is sent to Web server, and receives the inquiry number of return
According to；
The Web server is used to the data inquiry request of reception being sent to background server, and by background service
The inquiry data that device returns are sent to client；
The background server is used to the data inquiry request of reception being put into message queue, according to queue mechanism
Data are obtained from database server, and storage management is carried out to the data of acquisition according to data configuration rule；
The database server is used to store required data.
The beneficial effects of the invention are as follows：
1. the inquiry request for the processing high-volume data that can stablize changes former mode high-volume Data Statistics Inquiry Through success
The low situation of rate；
2. the high-volume business datum statistics of a variety of inquiry bores can be supported；
3. after client submits inquiry request, data organization and Generating Data File process stage are not take up client
The resource of PC；
4. using message queue mechanism, the data inquiry request that rational layout, processing client are sent；
5. dynamically distribute server-side generate data program into number of passes, efficient process " data organization " and " data file life
At " task；
It is low, big to client rs PC host resource occupancy that the present invention solves data query displaying success rate under the conditions of big data
The problem of, the present invention ensures data query effect using the data processing method of dynamic allocation process and efficient message queue mechanism
The quasi real time characteristic of rate, reliable and stable file management facilities can ensure the safety of sensitive information.
Based on the above technical solution, the present invention can also be improved as follows.
Further, the background server includes message queue module, data processing module, rule configuration module and file
Management module；
The message queue module is used to receive data inquiry request, and carries out layout to data inquiry request, successively
Data inquiry request is sent to data processing module, is additionally operable to send task completion prompt message to client, makes client
Required data are obtained from document management module according to prompt message；
The data processing module is used to receive the data inquiry request in message queue, according to data inquiry request
Type of service data are obtained from database server, from configuration configuration module in read data configuration rule, according to data
The data composition of acquisition is generated big data and inquires file by configuration rule, and big data inquiry file is stored in file management mould
In block；
The rule configuration module is used for based on different service request types, configuration data configuration rule；
The document management module is used to manage the big data inquiry file generated.
Further, the data configuration rule includes data organization rule, data file naming rule and data file pipe
Reason rule.
Another technical solution that the present invention solves above-mentioned technical problem is as follows：A kind of data query exhibition under the conditions of big data
Show method, includes the following steps：
Step 1, client creation data inquiry request and it is sent to Web server；
Step 2, the data inquiry request of reception is sent to background server by Web server；
Step 3, the data inquiry request of reception is put into message queue by background server, according to queue mechanism from data
Data are obtained in the server of library, storage management is carried out to the data of acquisition according to data configuration rule；
Step 4, task completion prompt message is notified by client by message queue；
Step 5, client obtains required data file according to obtained prompt message.
Based on the above technical solution, the present invention can also be improved as follows.
Further, the step 3 is implemented as：
Step 31, data inquiry request is received, and layout is carried out to data inquiry request, successively sends out data inquiry request
Give data processing module；
Step 32, the data inquiry request that message queue is sent is received, according to the type of service of data inquiry request from number
According to obtaining data in the server of library；
Step 33, data configuration rule is read, the data composition of acquisition, which is generated big data, according to data configuration rule looks into
File is ask, big data inquiry file is stored in document management module.
Further, the data configuration rule includes data organization rule, data file naming rule and data file pipe
Reason rule.
Description of the drawings
Fig. 1 is the data query display systems block diagram under the conditions of a kind of big data of the present invention；
Fig. 2 is that the data query under the conditions of a kind of big data of the present invention shows flow chart.
In attached drawing, parts list represented by the reference numerals are as follows：
1, client, 2, Web server, 3, background server, 4, database server, 31, message queue module, 32,
Data processing module, 33, rule configuration module, 34, document management module.
Specific implementation mode
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the present invention.
As shown in Figure 1, the data query display systems under the conditions of a kind of big data, including client 1, Web server 2,
Background server 3 and database server 4；The client 1 is used for creation data inquiry request and is sent to Web service
Device, and receive the inquiry data of return；The Web server 2 is used to the data inquiry request of reception being sent to backstage clothes
Business device, and the inquiry data that background server returns are sent to client；The background server 3 is used to receive
Data inquiry request is put into queue, and data are obtained from database server according to queue mechanism, according to data configuration rule
Storage management is carried out to the data of acquisition；The database server 4 is used to store required data.
Wherein, the background server 3 includes message queue module 31, data processing module 32, rule configuration module 33
With document management module 34；The message queue module 31, is used to receive data inquiry request, and to data inquiry request into
Row layout, is sent to data processing module by data inquiry request successively, is additionally operable to send task completion prompt letter to client
Breath makes client obtain required data from document management module according to prompt message；The data processing module 32, is used for
The data inquiry request in message queue is received, number is obtained from database server according to the type of service of data inquiry request
According to the data composition of acquisition is generated big count by the reading data configuration rule from configuration configuration module according to data configuration rule
It is investigated that asking file, and big data inquiry file is stored in document management module；The rule configuration module 33, is used for base
In different service request types, configuration data configuration rule；The document management module 34 is used to manage the big number generated
It is investigated that asking file.
Wherein, the data configuration rule includes data organization rule, data file naming rule and data file management
Rule.
As shown in Fig. 2, the data query methods of exhibiting under the conditions of a kind of big data, includes the following steps：
Step 1, client creation data inquiry request and it is sent to Web server；
Step 2, the data inquiry request of reception is sent to background server by Web server；
Step 3, the data inquiry request of reception is put into message queue by background server, according to queue mechanism from data
Data are obtained in the server of library, storage management is carried out to the data of acquisition according to data configuration rule；
Step 4, task completion prompt message is notified by client by message queue；
Step 5, client obtains required data file according to obtained prompt message.
Based on the above technical solution, the present invention can also be improved as follows.
Wherein, the step 3 is implemented as：
Step 31, data inquiry request is received, and layout is carried out to data inquiry request, successively sends out data inquiry request
Give data processing module；
Step 32, the data inquiry request that message queue is sent is received, according to the type of service of data inquiry request from number
According to obtaining data in the server of library；
Step 33, data configuration rule is read, the data composition of acquisition, which is generated big data, according to data configuration rule looks into
File is ask, big data inquiry file is stored in document management module.
The data configuration rule includes data organization rule, data file naming rule and data file management rule.
The function module in background server is briefly introduced below.
1. message queue module (Message Queuing)
By program combination persistent object system, it is big mainly to receive the client passed over by Web server for this module
Data inquiry request, and this request is sent to data processing module.
It is main to realize the functions such as request message reception, message storage, message layout, message transmission, message informing.
2. data processing module (Data Processing)
This module mainly receives the solicited message in message queue by program, according to service request type from data source
Data are obtained, data file are generated according to rule, by file record in file server.
It is main to realize the functions such as data acquisition, data organization, Generating Data File, file transmission.
3. rule configuration module (Rule Configuration)
This module is mainly by dispensing containers, based on different service request types, the organization regulation of configuration data, data
File designation rule, data file management rule etc..
It is main to realize data collecting rule configuration, the configuration of data organization rule, the configuration of file designation rule, file management rule
Then configuration etc..
4. document management module (File Manager)
This function module is mainly used for the big data inquiry file that management generates, including the life cycle of file, file
The functions such as access, the operating right control of file.
It is main to realize file life cycle management, management of file access, file permission control management etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.
Claims (4)
1. the data query display systems under the conditions of a kind of big data, which is characterized in that including client, Web server, backstage
Server and database server；
The client is used for creation data inquiry request and is sent to Web server, and receives the inquiry data of return；
The Web server is used to the data inquiry request of reception being sent to background server, and background server is returned
The inquiry data returned are sent to client；
The background server is used to the data inquiry request of reception being put into message queue, according to queue mechanism from number
According to data are obtained in the server of library, storage management is carried out to the data of acquisition according to data configuration rule；
The database server is used to store required data；
Wherein, the background server includes message queue module, data processing module, rule configuration module and file management mould
Block；
The message queue module is used to receive data inquiry request, and carries out layout to data inquiry request, will count successively
Be sent to data processing module according to inquiry request, be additionally operable to client send task complete prompt message, make client according to
Prompt message obtains required data from document management module；
The data processing module is used to receive the data inquiry request in message queue, according to the industry of data inquiry request
Service type obtains data from database server, and data configuration rule is read from configuration module, according to data configuration rule
The data composition of acquisition is generated into big data and inquires file, and big data inquiry file is stored in document management module；
The rule configuration module is used for based on different service request types, configuration data configuration rule；
The document management module is used to manage the big data inquiry file generated.
2. the data query display systems under the conditions of a kind of big data according to claim 1, which is characterized in that the data
Configuration rule includes data organization rule, data file naming rule and data file management rule.
3. the data query methods of exhibiting under the conditions of a kind of big data, which is characterized in that include the following steps：
Step 1, client creation data inquiry request and it is sent to Web server；
Step 2, the data inquiry request of reception is sent to background server by Web server；
Step 3, the data inquiry request of reception is put into message queue by background server, is taken from database according to queue mechanism
Data are obtained in business device, storage management is carried out to the data of acquisition according to data configuration rule；
Step 4, task completion prompt message is notified by client by message queue；
Step 5, client obtains required data file according to obtained prompt message；
Wherein, the step 3 is implemented as：
Step 31, data inquiry request is received, and layout is carried out to data inquiry request, is successively sent to data inquiry request
Data processing module；
Step 32, the data inquiry request that message queue is sent is received, according to the type of service of data inquiry request from database
Data are obtained in server；
Step 33, data configuration rule is read, the data composition of acquisition is generated by big data inquiry text according to data configuration rule
Big data inquiry file is stored in document management module by part.
4. the data query methods of exhibiting under the conditions of a kind of big data according to claim 3, which is characterized in that the data
Configuration rule includes data organization rule, data file naming rule and data file management rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410662697.9A CN104536965B (en) | 2014-11-19 | 2014-11-19 | A kind of data query display systems under the conditions of big data and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410662697.9A CN104536965B (en) | 2014-11-19 | 2014-11-19 | A kind of data query display systems under the conditions of big data and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104536965A CN104536965A (en) | 2015-04-22 |
| CN104536965B true CN104536965B (en) | 2018-10-19 |
Family
ID=52852493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410662697.9A Active CN104536965B (en) | 2014-11-19 | 2014-11-19 | A kind of data query display systems under the conditions of big data and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104536965B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106445936A (en) * | 2015-08-05 | 2017-02-22 | 中国移动通信集团福建有限公司 | Data processing method and equipment |
| CN105763363B (en) * | 2016-01-20 | 2019-05-28 | 曾戟 | A kind of method and apparatus of data configuration that realizing user resources |
| CN107391695A (en) * | 2017-07-26 | 2017-11-24 | 温州市鹿城区中津先进科技研究院 | A kind of information extracting method based on big data |
| CN110071951A (en) * | 2018-01-24 | 2019-07-30 | 江苏迪纳数字科技股份有限公司 | Data query display systems and method under the conditions of a kind of big data |
| TWI736832B (en) * | 2019-01-29 | 2021-08-21 | 台灣房屋仲介股份有限公司 | System for interactively providing real estate agency information |
| CN110704415A (en) * | 2019-10-08 | 2020-01-17 | 加和数康(北京)信息科技有限公司 | Lightweight data acquisition method and system |
| CN111782399B (en) * | 2020-07-03 | 2023-12-01 | 北京思特奇信息技术股份有限公司 | UDP-based efficient realization method for configuration server |
| CN112181688A (en) * | 2020-09-29 | 2021-01-05 | 银盛支付服务股份有限公司 | Large-batch data processing method and system based on Rockettmq middleware |
| CN112650767B (en) * | 2020-11-30 | 2023-03-10 | 中国科学院信息工程研究所 | Data exchange method and system with preposed data filtering |
| CN114070831B (en) * | 2021-11-15 | 2024-08-02 | 中交第二航务工程勘察设计院有限公司 | REVIT-based BIM cloud service system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101267299A (en) * | 2007-03-14 | 2008-09-17 | 阿里巴巴集团控股有限公司 | A method and system for securely display data on the webpage |
| US7493311B1 (en) * | 2002-08-01 | 2009-02-17 | Microsoft Corporation | Information server and pluggable data sources |
| US7565394B1 (en) * | 2000-09-07 | 2009-07-21 | ReportEdge, LLC | Distributed report processing system and methods |
| CN101741896A (en) * | 2009-11-24 | 2010-06-16 | 中兴通讯股份有限公司 | Data processing method, device and system in WEB system |
| CN103067396A (en) * | 2012-12-29 | 2013-04-24 | 深圳先进技术研究院 | Three-dimensional (3d) geographic information system (GIS) asynchronous communication transmission system and method of asynchronous communication transmission |
| CN103294786A (en) * | 2013-05-17 | 2013-09-11 | 华中科技大学 | Metadata organization and management method and system of distributed file system |
-
2014
- 2014-11-19 CN CN201410662697.9A patent/CN104536965B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7565394B1 (en) * | 2000-09-07 | 2009-07-21 | ReportEdge, LLC | Distributed report processing system and methods |
| US7493311B1 (en) * | 2002-08-01 | 2009-02-17 | Microsoft Corporation | Information server and pluggable data sources |
| CN101267299A (en) * | 2007-03-14 | 2008-09-17 | 阿里巴巴集团控股有限公司 | A method and system for securely display data on the webpage |
| CN101741896A (en) * | 2009-11-24 | 2010-06-16 | 中兴通讯股份有限公司 | Data processing method, device and system in WEB system |
| CN103067396A (en) * | 2012-12-29 | 2013-04-24 | 深圳先进技术研究院 | Three-dimensional (3d) geographic information system (GIS) asynchronous communication transmission system and method of asynchronous communication transmission |
| CN103294786A (en) * | 2013-05-17 | 2013-09-11 | 华中科技大学 | Metadata organization and management method and system of distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104536965A (en) | 2015-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104536965B (en) | A kind of data query display systems under the conditions of big data and method | |
| CN109726074A (en) | Log processing method, device, computer equipment and storage medium | |
| CN109446274B (en) | Method and device for managing BI metadata of big data platform | |
| CN108572963A (en) | Information acquisition method and device | |
| CN109254901B (en) | A kind of Monitoring Indexes method and system | |
| CN103684874A (en) | Method and device for automatically distributing online customer service executives to conduct customer service | |
| CN105630512A (en) | Method and system for implementing mobile device data tracking through software development toolkit | |
| CN107908637B (en) | Entity updating method and system based on knowledge base | |
| CN107645410A (en) | A kind of virtual machine management system and method based on OpenStack cloud platforms | |
| CN108133024A (en) | Towards the geographical spatial data service integration method of the dynamic configuration of mobile client | |
| WO2019076001A1 (en) | Information updating method and device | |
| CN104915902A (en) | Cloud platform based implementation method for take-out order online delivery | |
| CN107451301B (en) | Processing method, device, equipment and storage medium for real-time delivery bill mail | |
| CN107451227A (en) | Paging query method and device | |
| CN109360106A (en) | Portrait construction method, system, medium and computer system | |
| US9256641B1 (en) | Dynamic optimization of data aggregation | |
| CN107729394A (en) | Data Mart management system and its application method based on Hadoop clusters | |
| CN107016128A (en) | A kind of data processing method and device | |
| CN109446167A (en) | A kind of storage of daily record data, extracting method and device | |
| CN106130757B (en) | Information acquisition method and device | |
| CN114168624B (en) | Data analysis method, computing device and storage medium | |
| CN110347654A (en) | A kind of method and apparatus of online cluster features | |
| CN109246035A (en) | A kind of method and device of data transfer management | |
| CN112783924B (en) | Dirty data identification method, device and system | |
| CN104035856A (en) | Virtual desktop monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |