CN1703041A - Method for parallel processing big quantities of data in a communication server - Google Patents
Method for parallel processing big quantities of data in a communication server Download PDFInfo
- Publication number
- CN1703041A CN1703041A CN 200510018872 CN200510018872A CN1703041A CN 1703041 A CN1703041 A CN 1703041A CN 200510018872 CN200510018872 CN 200510018872 CN 200510018872 A CN200510018872 A CN 200510018872A CN 1703041 A CN1703041 A CN 1703041A
- Authority
- CN
- China
- Prior art keywords
- data
- buffer memory
- communication server
- network interface
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
This invention relates to mobile object positioning system large data merging and process method, which comprises the following steps: the communication server receiving one frame of data from outer net card and writing into the line data buffer for line processing and sending it into monitor center; receiving one frame of down line data from inner net card and writing it into data buffer and sending it to the terminal through line process. The invention is used in double rapid receiving, processing, transferring and the three-layer structure of multiple terminals; one communication server and several monitor centers.
Description
Technical field
The present invention relates to the concurrent processing method of communication server big data quantity in the moving target navigation system.
Background technology
The moving target global position system generally is made up of three parts at present: terminal equipment, the communication server, Surveillance center.Terminal is carried on the wireless data service, and for example CDMA 2000 1X of the GPRS of Yi Donging, UNICOM send data on the assigned ip address of the communication server.The communication server is positioned at the centre of three-decker, on the one hand the data in real time from different terminals is forwarded to Surveillance center under the terminal, on the other hand the order that Surveillance center issues is forwarded to terminal in real time.Surveillance center is responsible for demonstration, control and management etc.
The general handling process of the communication server:
Upstream data: receive the data of self terminal, inquiry disk database to obtain this terminal related information, carry out protocol conversion, are forwarded to Surveillance center at fixed address.
Downlink data: receive order, the current communication mode of inquiry Be Controlled terminal, be forwarded to terminal from Surveillance center at fixed address.
Present moving target global position system applying unit, commercial vehicle company for example, the number of terminals that requires each communication server to handle is more, by 6000 terminals, each terminal sent a locator data in per 6 seconds, promptly require the communication server to have the ability of handling 1000 data average each second, this does not also comprise the data that Surveillance center issues.
If according to general handling process, receive the database on visit this machine disk after the data or the remote computer, be not reach 1000/second concurrent data processing performance.
Summary of the invention
The object of the present invention is to provide a kind of method that is used for moving target navigation system communication server concurrent processing big data quantity, realize that bidirectional high speed ground receives, handles, transmits the function of data.
The technical scheme that the present invention is adopted for its technical problem of solution comprises:
A kind of method that is used for moving target navigation system communication server concurrent processing big data quantity, outside the communication server adopts, interior two network interface cards configuration, outer network interface card is used for data that receiving terminal reports and sends data to terminal, interior network interface card is used to receive data that Surveillance center issues and sends data to Surveillance center, and data processing is divided into upstream data processing module and downlink data processing module, upstream data processing module processing terminal reported data, the downlink data processing module is handled Surveillance center's transmitting order to lower levels, link together by buffer memory between two network interface cards, the concrete implementation step of this method is: the communication server receives a frame upstream data from outer network interface card, write the upstream data buffer memory, through thread process, send to Surveillance center by interior network interface card; Receive a frame downlink data from the Intranet clamping, write the downlink data buffer memory, generate suitable number through described upstream data buffer memory according to the concurrent data volume of reality, downlink data amount buffer memory only generates one, and each buffer memory is handled with a thread.The IP network communication technology based on the TCP/IP technology.Two network interface card configurations on the communication server, have been adopted, one of them has disposed public network IP address, be called " outer network interface card ", be used for data that receiving terminal reports and send data to terminal, another has disposed internal lan IP address, be called " interior network interface card ", be used to receive data that Surveillance center issues and send data to Surveillance center.One frame upstream data receives from " outer network interface card ", writes the upstream data dedicated cache, through handling, sends to Surveillance center by " interior network interface card "; One frame downlink data is received by " interior network interface card ", writes the downlink data dedicated cache, through handling, sends to terminal by " outer network interface card ".The multithreading caching process.Connect with buffer memory between " outer network interface card " and " interior network interface card ", in view of upstream data is very many, deposit upstream data so opened up a plurality of buffer memorys, each buffer memory is handled with a thread.The downlink data amount is little, only opens up a buffer memory and is used to deposit downlink data.
The object based programming technology.Encapsulated buffer memory and processing threads in class, a class object is just represented a buffer memory and processing threads, can generate the object of suitable number, promptly a plurality of buffer memorys and processing threads according to the concurrent data volume of reality.Described thread process: exactly buffer memory and processing threads are encapsulated in the same class, class object of every generation, just generate a buffer memory and corresponding processing threads, between a plurality of threads without mutual exclusion, deal with the buffer memory of oneself separately, the process of thread process data comprises from buffer memory reading of data, inquires about relevant parameter, protocol conversion, transmission in memory database, in addition after receiving data, write each buffer memory in turn, data are evenly distributed in a plurality of buffer memorys.
During communication server program start, the reading of data library information uses passive renewal or active update method to obtain the data that change in the database in internal memory then, and when deal with data, information needed is all inquired about from internal memory.Passive renewal refers to that the communication server is provided with timer, and regularly the reading of data library information is recorded in the internal memory; When initiatively more new index changes according to the data in the storehouse, the information that writes down in Surveillance center's notifying communication server update internal memory.
In internal memory during the record data library information, used back ten of the SIM/UIM number that carries on the terminal to convert the unsigned long data to, be that key assignments is set up hash table with this integer then.The memory database technology.Speed is slow during owing to visit disk database especially remote access, in the technical program relevant information is stored in the internal memory, regularly upgrades, and directly inquires about in internal memory when needing.
HASH hash index technology.Be used with memory database, further improved inquiry velocity.
The present invention is used for the two-way data that receive apace, handle, transmit, and is particularly suitable for a plurality of terminals, communication server, the such three-decker of a plurality of Surveillance center.In realization, the reading of data library information is recorded in the internal memory, replaces Query Database with audit memory, and, in internal memory, adopted the HASH hashing technique during recorded information, greatly improved inquiry velocity.On the basis of original communication server, do not increase any hardware device, only, just can promote handling property greatly, simplified system configuration, saved system cost by the renewal of software implementation method.
Description of drawings
Fig. 1 is the flow process that the communication server is handled upstream data.
Embodiment
The flow process of communication server processing upstream data has been described as shown in Figure 1.
The communication server is divided into 2 big modules, comprises the module of reception, processing, the module of transmitting terminal data and reception, processing, forwarding Surveillance center data.
The communication server after startup, just opened up n buffer memory and with buffer memory n processing threads one to one, receive after the data of terminal to report, for speed up processing, write n buffer memory in turn, data just are dispersed in n the buffer memory like this, rather than concentrate in the buffer memory, the data in the own buffer memory of each thread independent process, from memory database, search information needed, be forwarded to the Surveillance center under this terminal.
As shown in table 1, the communication server has defined relevant data structure when program realizes, after the communication server starts, reading database is transferred to all relevant informations in the internal memory, later inquiry is all carried out in internal memory, used the thought of " sacrifice internal memory and exchange speed for ", this way is fully desirable under the cheap day by day reality of hardware develop rapidly nowadays, price.If what if the data in the database have taken place to change? in the specific implementation, when considering some function items that has only by the operation supervise and control center software, just can cause the variation of data in the database, therefore the communication server has adopted two kinds of ways: one, the timely notifying communication server of Surveillance center after data in database change, the relative recording in the passive updating memory of the communication server; Its two, the communication server is reading database regularly, initiatively the record in the updating memory.If the employing second method, after database may occurring and changing, the situation that the communication server can not make an immediate response, but for fear of the frequent access database, defining a rational access time also is desirable at interval.
The structure that table 1 adopts for communication server record terminal related information.
| Sequence number | Content | Explanation |
| ??1 | Terminal SIM/UIM number | Get back ten and transfer unsigned long to |
| ??2 | Whether arrearage | |
| ??3 | User ID | |
| ??4 | The address of Surveillance center | Use the array record, because the data of a terminal may mail to a plurality of Surveillance center |
Terminal with the SIM/UIM number as unique identification, therefore, in data structure shown in the table 1, get terminal SIM/UIM number back 10, convert the unsigned long data to, set up the HASH hash table with this integer as key assignments then.Have the SIM/UIM number in the data of terminal to report, therefore the communication server takes out number after receiving data, converts the unsigned long data to back 10 to, by this integer index HASH hash table, can find the out of Memory of this terminal fast then.
Claims (6)
1, a kind of method that is used for moving target navigation system communication server concurrent processing big data quantity, outside the communication server adopts, interior two network interface cards configuration, outer network interface card is used for data that receiving terminal reports and sends data to terminal, interior network interface card is used to receive data that Surveillance center issues and sends data to Surveillance center, and data processing is divided into upstream data processing module and downlink data processing module, upstream data processing module processing terminal reported data, the downlink data processing module is handled Surveillance center's transmitting order to lower levels, link together by buffer memory between two network interface cards, the concrete implementation step of this method is: the communication server receives a frame upstream data from outer network interface card, write the upstream data buffer memory, through thread process, send to Surveillance center by interior network interface card; Receive a frame downlink data from the Intranet clamping, write the downlink data buffer memory,, send to terminal by outer network interface card through thread process.
2, method according to claim 1 is characterized in that: described upstream data buffer memory generates suitable number according to the concurrent data volume of reality, and downlink data amount buffer memory only generates one, and each buffer memory is handled with a thread.
3, method according to claim 2, it is characterized in that: described thread process: exactly buffer memory and processing threads are encapsulated in the same class, class object of every generation, just generate a buffer memory and corresponding processing threads, between a plurality of threads without mutual exclusion, deal with the buffer memory of oneself separately, the process of thread process data comprises from buffer memory reading of data, inquires about relevant parameter, protocol conversion, transmission in memory database, in addition after receiving data, write each buffer memory in turn, data are evenly distributed in a plurality of buffer memorys.
4, method according to claim 1, it is characterized in that: during communication server program start, the reading of data library information uses passive renewal or active update method to obtain the data that change in the database in internal memory then, when deal with data, information needed is all inquired about from internal memory.
5, method according to claim 4 is characterized in that: in internal memory during the record data library information, having used back ten of the SIM/UIM number that carries on the terminal to convert the unsigned long data to, is that key assignments is set up hash table with this integer then.
6, method according to claim 4 is characterized in that: passive renewal refers to that the communication server is provided with timer, and regularly the reading of data library information is recorded in the internal memory; When initiatively more new index changes according to the data in the storehouse, the information that writes down in Surveillance center's notifying communication server update internal memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005100188721A CN100477658C (en) | 2005-06-08 | 2005-06-08 | Method for parallel processing big quantities of data in a communication server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005100188721A CN100477658C (en) | 2005-06-08 | 2005-06-08 | Method for parallel processing big quantities of data in a communication server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1703041A true CN1703041A (en) | 2005-11-30 |
| CN100477658C CN100477658C (en) | 2009-04-08 |
Family
ID=35632556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2005100188721A Active CN100477658C (en) | 2005-06-08 | 2005-06-08 | Method for parallel processing big quantities of data in a communication server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100477658C (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100387015C (en) * | 2006-02-20 | 2008-05-07 | 华为技术有限公司 | Business scheduling system and method in wireless net-like network |
| CN101452422B (en) * | 2007-11-29 | 2011-03-30 | 联芯科技有限公司 | Chip data read-write method, corresponding apparatus and system |
| CN101090401B (en) * | 2007-05-25 | 2011-05-18 | 金蝶软件（中国）有限公司 | Data buffer store method and system at duster environment |
| CN102546140A (en) * | 2012-01-10 | 2012-07-04 | 苏州阔地网络科技有限公司 | Duplex communication method and communication server |
| CN102609476A (en) * | 2012-01-19 | 2012-07-25 | 北京神州数码思特奇信息技术股份有限公司 | Method for managing data of handheld business hall |
| CN103198108A (en) * | 2013-03-27 | 2013-07-10 | 新浪网技术(中国)有限公司 | Index data updating method, retrieval server and index data updating system |
| CN103530239A (en) * | 2013-10-18 | 2014-01-22 | 深圳市新格林耐特通信技术有限公司 | Method for improving performance of communication server by using memory pool |
| CN104536407A (en) * | 2014-12-17 | 2015-04-22 | 中国南方电网有限责任公司调峰调频发电公司 | Method for issuing concurrency control instruction to IEC61850 intelligent device |
| CN105868123A (en) * | 2016-04-07 | 2016-08-17 | 武汉数字派特科技有限公司 | Concurrent data caching structure and method |
| CN106599071A (en) * | 2016-11-18 | 2017-04-26 | 青岛海信移动通信技术股份有限公司 | Short message display method and device |
| CN110493769A (en) * | 2019-07-05 | 2019-11-22 | 南京航空航天大学 | Cell-phone number security scheme based on lightweight Hash mapping |
| CN110674093A (en) * | 2019-08-28 | 2020-01-10 | 金蝶汽车网络科技有限公司 | File data processing method and device, computer equipment and storage medium |
| CN112416069A (en) * | 2020-10-19 | 2021-02-26 | 温州大学 | Real-time data analysis and statistical reporting method |
| CN115801809A (en) * | 2022-11-03 | 2023-03-14 | 广州辰创科技发展有限公司 | Cross-network data synchronization control system, method and medium |
| CN115914418A (en) * | 2023-03-09 | 2023-04-04 | 北京全路通信信号研究设计院集团有限公司 | Railway interface gateway equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1198143A (en) * | 1997-09-17 | 1999-04-09 | Toshiba Corp | Atm repeater |
| CN1165846C (en) * | 2000-12-07 | 2004-09-08 | 华为技术有限公司 | Method of raising message processing capacity of short message dispatching center |
| CN1277217C (en) * | 2002-12-31 | 2006-09-27 | 中兴通讯股份有限公司 | Mobile customer high performance HLR/AUC mobile customer data bank implementing method |
-
2005
- 2005-06-08 CN CNB2005100188721A patent/CN100477658C/en active Active
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100387015C (en) * | 2006-02-20 | 2008-05-07 | 华为技术有限公司 | Business scheduling system and method in wireless net-like network |
| CN101090401B (en) * | 2007-05-25 | 2011-05-18 | 金蝶软件（中国）有限公司 | Data buffer store method and system at duster environment |
| CN101452422B (en) * | 2007-11-29 | 2011-03-30 | 联芯科技有限公司 | Chip data read-write method, corresponding apparatus and system |
| CN102546140A (en) * | 2012-01-10 | 2012-07-04 | 苏州阔地网络科技有限公司 | Duplex communication method and communication server |
| WO2013104208A1 (en) * | 2012-01-10 | 2013-07-18 | Hu Jiaming | Method and communication server for duplex communication |
| CN102609476A (en) * | 2012-01-19 | 2012-07-25 | 北京神州数码思特奇信息技术股份有限公司 | Method for managing data of handheld business hall |
| CN103198108A (en) * | 2013-03-27 | 2013-07-10 | 新浪网技术(中国)有限公司 | Index data updating method, retrieval server and index data updating system |
| CN103198108B (en) * | 2013-03-27 | 2016-08-10 | 新浪网技术(中国)有限公司 | A kind of index data update method, retrieval server and system |
| CN103530239A (en) * | 2013-10-18 | 2014-01-22 | 深圳市新格林耐特通信技术有限公司 | Method for improving performance of communication server by using memory pool |
| CN104536407A (en) * | 2014-12-17 | 2015-04-22 | 中国南方电网有限责任公司调峰调频发电公司 | Method for issuing concurrency control instruction to IEC61850 intelligent device |
| CN105868123A (en) * | 2016-04-07 | 2016-08-17 | 武汉数字派特科技有限公司 | Concurrent data caching structure and method |
| CN105868123B (en) * | 2016-04-07 | 2018-10-09 | 武汉数字派特科技有限公司 | A kind of buffer storage and method of concurrent data |
| CN106599071A (en) * | 2016-11-18 | 2017-04-26 | 青岛海信移动通信技术股份有限公司 | Short message display method and device |
| CN110493769A (en) * | 2019-07-05 | 2019-11-22 | 南京航空航天大学 | Cell-phone number security scheme based on lightweight Hash mapping |
| CN110674093A (en) * | 2019-08-28 | 2020-01-10 | 金蝶汽车网络科技有限公司 | File data processing method and device, computer equipment and storage medium |
| CN112416069A (en) * | 2020-10-19 | 2021-02-26 | 温州大学 | Real-time data analysis and statistical reporting method |
| CN115801809A (en) * | 2022-11-03 | 2023-03-14 | 广州辰创科技发展有限公司 | Cross-network data synchronization control system, method and medium |
| CN115801809B (en) * | 2022-11-03 | 2023-11-03 | 广州辰创科技发展有限公司 | Cross-network data synchronous control system, method and medium |
| CN115914418A (en) * | 2023-03-09 | 2023-04-04 | 北京全路通信信号研究设计院集团有限公司 | Railway interface gateway equipment |
| CN115914418B (en) * | 2023-03-09 | 2023-06-30 | 北京全路通信信号研究设计院集团有限公司 | Railway interface gateway equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100477658C (en) | 2009-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100477658C (en) | Method for parallel processing big quantities of data in a communication server | |
| CN202058147U (en) | Distribution type real-time database management system | |
| CN102375837B (en) | Data acquiring system and method | |
| CN102932846B (en) | Data management system for distributed heterogeneous sensing network | |
| CN105138615A (en) | Method and system for building big data distributed log | |
| CN102761454A (en) | Method and system for monitoring internet of things | |
| CN101170452A (en) | Content distribution network service provision node system for enhancing management capability and its affiliated network | |
| CN204598011U (en) | A kind of OPC data acquisition and monitoring intelligent terminal based on TD-LTE | |
| CN102724307A (en) | Information fusion engine and information fusion method for Internet of Things | |
| CN102752322A (en) | Method for storing and querying distributed data in internet of things | |
| CN105183299A (en) | Human-computer interface service processing system and method | |
| CN101535954A (en) | Information processing system and related method | |
| CN101887388A (en) | Data backup system and method based on memory database | |
| CN106161644A (en) | Distributed system for data processing and data processing method thereof | |
| CN102916844A (en) | Mass data fusion and real-time monitoring system | |
| CN110855724B (en) | Cloud operation and maintenance management method and system of intelligent camera and application of cloud operation and maintenance management method and system in parking charging | |
| CN100498780C (en) | Data inquire system and method | |
| CN103970530A (en) | Data acquisition and implementation method based on Intouch configuration technology of OPC interface | |
| CN102737061B (en) | Distributed ticket query management system and method | |
| CN101556587B (en) | Operation agent device for heterogeneous database | |
| CN102098170B (en) | Data acquisition optimization method and system | |
| CN104202389A (en) | Monitoring method for storage space and running state in cloud environment and cloud storage system | |
| CN109726360A (en) | A kind of crops monitoring data processing method and system | |
| CN104270432A (en) | Real-time data service system and data interaction method based on drilling industry | |
| CN101383733B (en) | Method and device for monitoring network state when calling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 430205 Hubei city of Wuhan province Jiangxia Hidden Dragon Island Tan lake two Road No. 1 Patentee after: CITIC Mobile Communication Technology Co., Ltd Address before: 430074 Wuhan, Hongshan Province District Road, Department of mail, No. 88 hospital Patentee before: Wuhan Hongxin Telecommunication Technologies Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 430205 No.1 tanhu 2nd Road, Canglong Island, Jiangxia District, Wuhan City, Hubei Province Patentee after: CITIC Mobile Communication Technology Co.,Ltd. Address before: 430205 No.1 tanhu 2nd Road, Canglong Island, Jiangxia District, Wuhan City, Hubei Province Patentee before: CITIC Mobile Communication Technology Co., Ltd |