The method of communication server concurrent processing big data quantity
Technical field
The present invention relates to the concurrent processing method of communication server big data quantity in the moving target navigation system.
Background technology
The moving target global position system generally is made up of three parts at present: terminal equipment, the communication server, Surveillance center.Terminal is carried on the wireless data service, and for example CDMA 2000 1X of the GPRS of Yi Donging, UNICOM send data on the assigned ip address of the communication server.The communication server is positioned at the centre of three-decker, on the one hand the data in real time from different terminals is forwarded to Surveillance center under the terminal, on the other hand the order that Surveillance center issues is forwarded to terminal in real time.Surveillance center is responsible for demonstration, control and management etc.
The general handling process of the communication server:
Upstream data: receive the data of self terminal, inquiry disk database to obtain this terminal related information, carry out protocol conversion, are forwarded to Surveillance center at fixed address.
Downlink data: receive order, the current communication mode of inquiry Be Controlled terminal, be forwarded to terminal from Surveillance center at fixed address.
Present moving target global position system applying unit, commercial vehicle company for example, the number of terminals that requires each communication server to handle is more, by 6000 terminals, each terminal sent a locator data in per 6 seconds, promptly require the communication server to have the ability of handling 1000 data average each second, this does not also comprise the data that Surveillance center issues.
If according to general handling process, receive the database on visit this machine disk after the data or the remote computer, be not reach 1000/second concurrent data processing performance.
Summary of the invention
The object of the present invention is to provide a kind of method that is used for moving target navigation system communication server concurrent processing big data quantity, realize that bidirectional high speed ground receives, handles, transmits the function of data.
The technical scheme that the present invention is adopted for its technical problem of solution comprises:
A kind of method that is used for moving target navigation system communication server concurrent processing big data quantity, outside the communication server adopts, interior two network interface cards configuration, outer network interface card is used for data that receiving terminal reports and sends data to terminal, interior network interface card is used to receive data that Surveillance center issues and sends data to Surveillance center, and data processing is divided into upstream data processing module and downlink data processing module, upstream data processing module processing terminal reported data, the downlink data processing module is handled Surveillance center's transmitting order to lower levels, link together by buffer memory between two network interface cards, the concrete implementation step of this method is: the communication server receives a frame upstream data from outer network interface card, write the upstream data buffer memory, through thread process, send to Surveillance center by interior network interface card; Receive a frame downlink data from the Intranet clamping, write the downlink data buffer memory, generate suitable number through described upstream data buffer memory according to the concurrent data volume of reality, downlink data amount buffer memory only generates one, and each buffer memory is handled with a thread.The IP network communication technology based on the TCP/IP technology.Two network interface card configurations on the communication server, have been adopted, one of them has disposed public network IP address, be called " outer network interface card ", be used for data that receiving terminal reports and send data to terminal, another has disposed internal lan IP address, be called " interior network interface card ", be used to receive data that Surveillance center issues and send data to Surveillance center.One frame upstream data receives from " outer network interface card ", writes the upstream data dedicated cache, through handling, sends to Surveillance center by " interior network interface card "; One frame downlink data is received by " interior network interface card ", writes the downlink data dedicated cache, through handling, sends to terminal by " outer network interface card ".The multithreading caching process.Connect with buffer memory between " outer network interface card " and " interior network interface card ", in view of upstream data is very many, deposit upstream data so opened up a plurality of buffer memorys, each buffer memory is handled with a thread.The downlink data amount is little, only opens up a buffer memory and is used to deposit downlink data.
The object based programming technology.Encapsulated buffer memory and processing threads in class, a class object is just represented a buffer memory and processing threads, can generate the object of suitable number, promptly a plurality of buffer memorys and processing threads according to the concurrent data volume of reality.Described thread process: exactly buffer memory and processing threads are encapsulated in the same class, class object of every generation, just generate a buffer memory and corresponding processing threads, between a plurality of threads without mutual exclusion, deal with the buffer memory of oneself separately, the process of thread process data comprises from buffer memory reading of data, inquires about relevant parameter, protocol conversion, transmission in memory database, in addition after receiving data, write each buffer memory in turn, data are evenly distributed in a plurality of buffer memorys.
During communication server program start, the reading of data library information uses passive renewal or active update method to obtain the data that change in the database in internal memory then, and when deal with data, information needed is all inquired about from internal memory.Passive renewal refers to that the communication server is provided with timer, and regularly the reading of data library information is recorded in the internal memory; When initiatively more new index changes according to the data in the storehouse, the information that writes down in Surveillance center's notifying communication server update internal memory.
In internal memory during the record data library information, used back ten of the SIM/UIM number that carries on the terminal to convert the unsigned long data to, be that key assignments is set up hash table with this integer then.The memory database technology.Speed is slow during owing to visit disk database especially remote access, in the technical program relevant information is stored in the internal memory, regularly upgrades, and directly inquires about in internal memory when needing.
HASH hash index technology.Be used with memory database, further improved inquiry velocity.
The present invention is used for the two-way data that receive apace, handle, transmit, and is particularly suitable for a plurality of terminals, communication server, the such three-decker of a plurality of Surveillance center.In realization, the reading of data library information is recorded in the internal memory, replaces Query Database with audit memory, and, in internal memory, adopted the HASH hashing technique during recorded information, greatly improved inquiry velocity.On the basis of original communication server, do not increase any hardware device, only, just can promote handling property greatly, simplified system configuration, saved system cost by the renewal of software implementation method.
Description of drawings
Fig. 1 is the flow process that the communication server is handled upstream data.
Embodiment
The flow process of communication server processing upstream data has been described as shown in Figure 1.
The communication server is divided into 2 big modules, comprises the module of reception, processing, the module of transmitting terminal data and reception, processing, forwarding Surveillance center data.
The communication server after startup, just opened up n buffer memory and with buffer memory n processing threads one to one, receive after the data of terminal to report, for speed up processing, write n buffer memory in turn, data just are dispersed in n the buffer memory like this, rather than concentrate in the buffer memory, the data in the own buffer memory of each thread independent process, from memory database, search information needed, be forwarded to the Surveillance center under this terminal.
As shown in table 1, the communication server has defined relevant data structure when program realizes, after the communication server starts, reading database is transferred to all relevant informations in the internal memory, later inquiry is all carried out in internal memory, used the thought of " sacrifice internal memory and exchange speed for ", this way is fully desirable under the cheap day by day reality of hardware develop rapidly nowadays, price.If what if the data in the database have taken place to change? in the specific implementation, when considering some function items that has only by the operation supervise and control center software, just can cause the variation of data in the database, therefore the communication server has adopted two kinds of ways: one, the timely notifying communication server of Surveillance center after data in database change, the relative recording in the passive updating memory of the communication server; Its two, the communication server is reading database regularly, initiatively the record in the updating memory.If the employing second method, after database may occurring and changing, the situation that the communication server can not make an immediate response, but for fear of the frequent access database, defining a rational access time also is desirable at interval.
The structure that table 1 adopts for communication server record terminal related information.
Sequence number |
Content |
Explanation |
??1 |
Terminal SIM/UIM number |
Get back ten and transfer unsigned long to |
??2 |
Whether arrearage |
|
??3 |
User ID |
|
??4 |
The address of Surveillance center |
Use the array record, because the data of a terminal may mail to a plurality of Surveillance center |
Terminal with the SIM/UIM number as unique identification, therefore, in data structure shown in the table 1, get terminal SIM/UIM number back 10, convert the unsigned long data to, set up the HASH hash table with this integer as key assignments then.Have the SIM/UIM number in the data of terminal to report, therefore the communication server takes out number after receiving data, converts the unsigned long data to back 10 to, by this integer index HASH hash table, can find the out of Memory of this terminal fast then.