CN102447628B - Data packet transmission processing method and system - Google Patents
Data packet transmission processing method and system Download PDFInfo
- Publication number
- CN102447628B CN102447628B CN201110433104.8A CN201110433104A CN102447628B CN 102447628 B CN102447628 B CN 102447628B CN 201110433104 A CN201110433104 A CN 201110433104A CN 102447628 B CN102447628 B CN 102447628B
- Authority
- CN
- China
- Prior art keywords
- thread
- service
- type
- priority level
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data packet transmission processing method and system. The method comprises steps as follows: a Socket thread receives to-be-processed data packets; a preprocessing thread merges incomplete data packets read from a buffer zone and the to-be-processed data packets received by the Socket thread, and scans the merged data packets through a regular expression to judge whether the merged data packets are matched with a special data structure; if the merged data packets are matched with the special data structure, matched complete data packets are put into a receiving queue; and an executive device schedules the complete data packets from the receiving queue and calls for corresponding threads for processing according to the service types of the complete data packets, wherein each service type has a priority level, and different service types are correspondingly processed by different threads.
Description
Technical field
The invention belongs to field of data transmission, relate in particular to a kind of data packet transmission processing method and system.
Background technology
At present, the application program based on TCP Internet Transmission there will be sticky bag phenomenon, and some data that transmit leg sends stick into a bag in the time that recipient receives.For this sticky bag phenomenon, the communication server must by coherent bag separately, then carry out business logic processing.All will carry out subpackage processing because the communication server often receives a packet, therefore, the efficiency of subpackage processing becomes one of key factor of the whole communication server efficiency of restriction.If subpackage efficiency is slow, not only greatly reduce the efficiency of communication server processing messages, also can cause overstocking the packet of not processing in a large number in internal memory, cause communication server collapse.In addition, the processing of the service logic after packet parsing also affects the efficiency of the communication server to a great extent.
Summary of the invention
The invention provides a kind of data packet transmission processing method and system, to address the above problem.
The invention provides a kind of data packet transmission processing method.Said method comprises the following steps: Socket thread receives pending packet; Preliminary treatment thread splices the pending packet that the deficiency of data bag that reads from buffering area and Socket thread receive, by regular expression Overlap-scanning packet with judge splicing packet whether mate specific data structure, if splice data packet matched specific data structure, the complete data packet of coupling is put into receiving queue; Final controlling element is dispatched complete data packet from receiving queue, and calls corresponding thread according to the type of service of complete data packet and process, and wherein, type of service has priority level, and different types of service is by different thread alignment processings.
The present invention also provides a kind of data packet transmission treatment system, comprises receiving system, pretreatment unit and final controlling element.Receiving system connects pretreatment unit, and pretreatment unit connects final controlling element.Wherein, receiving system is used for receiving pending packet.The pending packet that pretreatment unit receives for splicing the deficiency of data bag that reads from buffering area and receiving system, by regular expression Overlap-scanning packet with judge splicing packet whether mate specific data structure, if splice data packet matched specific data structure, the complete data packet of coupling is put into receiving queue.Final controlling element is used for dispatching complete data packet from receiving queue, and calls corresponding thread according to the type of service of complete data packet and process.Wherein, type of service has priority level, and different types of service is by different thread alignment processings.
Compared to prior art, according to data packet transmission processing method provided by the invention and system, preliminary treatment thread splices the pending packet that the deficiency of data bag that reads from buffering area and Socket thread receive, by regular expression Overlap-scanning packet to judge whether splicing packet mates specific data structure, and the complete data packet that coupling is obtained is put into receiving queue.So, improve subpackage efficiency, prevented that in internal memory, overstocking mass data bag causes communication server collapse.In addition, final controlling element is dispatched complete data packet from receiving queue, and calls corresponding thread according to the type of service of complete data packet and process.So, improved the treatment effeciency of business.
Brief description of the drawings
Accompanying drawing described herein is used to provide a further understanding of the present invention, forms the application's a part, and schematic description and description of the present invention is used for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Figure 1 shows that the flow chart of the data packet transmission processing method that preferred embodiment according to the present invention provides;
Figure 2 shows that the schematic diagram of the data packet transmission processing unit that preferred embodiment according to the present invention provides.
Embodiment
Hereinafter also describe the present invention in detail with reference to accompanying drawing in conjunction with the embodiments.It should be noted that, in the situation that not conflicting, the feature in embodiment and embodiment in the application can combine mutually.
Figure 1 shows that the flow chart of the data packet transmission processing method that preferred embodiment according to the present invention provides.As shown in Figure 1, the data packet transmission processing method that preferred embodiment of the present invention provides comprises step 101～103.
In step 101, Socket thread receives pending packet.Particularly, Socket thread receives after the pending packet of user's transmission, take out preliminary treatment thread and carry out subsequent step, and Socket thread returns to the reception task that continues to carry out next pending packet immediately from thread pool.So, improved the receiving efficiency of Socket thread.
In step 102, preliminary treatment thread splices the pending packet that the deficiency of data bag that reads from buffering area and Socket thread receive, scan described splicing packet to judge whether described splicing packet mates specific data structure by regular expression, if the data packet matched specific data structure of described splicing, puts into receiving queue by the complete data packet of coupling.
In the present embodiment, complete packet has specific data structure, and above-mentioned specific data structure comprises data head, data content and data tail.For example, specific data structure starts with " <root ", end up with "/> ", centre is any random length character, as: <root type=' text ' from=' 1 ' to=' 2 ' say=' hello '/>.But the present invention is not limited thereto.In practical application, anyly start and the data structure that ends up all may be defined as specific data structure with corresponding fixed character.
Particularly, preliminary treatment thread is postponed to rush and in district, is taken out deficiency of data bag, then by the pending packet splicing that deficiency of data bag and Socket thread receive, obtains splicing packet.Then,, by regular expression macroscopic view Overlap-scanning packet, judge whether splicing packet comprises the specific data structure that meets " <root.../> ".If splicing packet does not have above-mentioned specific data structure, splicing packet is put into buffering area; If splicing packet has above-mentioned specific data structure, use again regular expression microcosmic Overlap-scanning packet, the packet that is about to meet " <root.../> " specific data structure is put into receiving queue, the packet that does not meet " <root.../> " specific data structure is put into buffering area, wait for data splicing next time.
For example, the pending packet that Socket thread receives is for the first time: <root from=' sam ' to=' sam ' type=' text ' say=' hello '/><root from=' sam '.The pending packet receiving is for the second time: to=' sam ' type=' text ' say=' hello '/><root.The pending packet receiving is for the third time: from=' sam ' to=' sam ' type=' text ' say=' hello '/><root from=' sam ' to=' sam ' type=' text ' say=' hello '/>.
In the present embodiment, Socket thread receives after pending packet for the first time, and preliminary treatment thread can from buffering area, take out deficiency of data bag and described pending packet splices.In this, because buffering area does not initially have packet, therefore the splicing packet that, preliminary treatment thread obtains: <root from=' sam ' to=' sam ' type=' text ' say=' hello '/><rootfrom=' sam '.Then, the above-mentioned splicing packet of regular expression macroscopic view scanning can find to exist the packet " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " that meets " <root.../> " data structure, then scan above-mentioned splicing packet with regular expression microcosmic, the complete data packet " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " obtaining is put into receiving queue, simultaneously by " <root from=' sam " " put into buffering area.
Then, receive for the second time after pending packet in Socket thread, preliminary treatment thread can take out deficiency of data bag " <root from=' sam " from buffering area ", will after the pending packet splicing of deficiency of data bag and this reception, obtain splicing packet: <root from=' sam ' to=' sam ' type=' text ' say=' hello '/><root.Then, the above-mentioned splicing packet of regular expression macroscopic view scanning, find to exist the packet " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " that meets " <root.../> " data structure, then scan above-mentioned splicing packet with regular expression microcosmic, the complete data packet " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " obtaining is put into receiving queue, " <root " put into buffering area simultaneously.
Then, receive for the third time after pending packet in Socket thread, preliminary treatment thread is postponed to rush in district and is taken out deficiency of data bag " <root ", will after deficiency of data bag and this pending packet splicing receiving, obtain splicing packet: <rootfrom=' sam ' to=' sam ' type=' text ' say=' hello '/><root from=' sam ' to=' sam ' type=' text ' say=' hello '/>.Then, the above-mentioned splicing packet of regular expression macroscopic view scanning, find to exist the packet " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/><root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " that meets " <root.../> " data structure, then scan above-mentioned splicing packet with regular expression microcosmic, the complete data packet " <rootfrom=' sam ' to=' sam ' type=' text ' say=' hello '/> " " <root from=' sam ' to=' sam ' type=' text ' say=' hello '/> " obtaining is put into receiving queue, empty the deficiency of data bag in buffering area simultaneously.
In step 103, final controlling element is dispatched complete data packet from receiving queue, and calls corresponding thread according to the type of service of complete data packet and process, and wherein, type of service has priority level, and different types of service is by different thread alignment processings.In addition,, in the time that final controlling element detects that the thread of the type of service that processing priority level is higher does not have the free time, final controlling element calls the higher type of service of thread process priority level corresponding to type of service that priority level is lower.And, in this, final controlling element timing detects the actual thread borrowing time of the type of service that priority level is higher, if what actual thread borrowing time was equal to or greater than thread corresponding to type of service that priority level is lower can borrowing time, final controlling element calls idle thread corresponding to type of service that other priority levels are lower and processes the higher type of service of priority level.
Particularly, the table of comparisons that final controlling element storage service type, thread and thread can borrowing times.Described table of comparisons example is as shown in the table.
Wherein, the priority level of above-mentioned four kinds of types of service from height to low order is: login business, publish business, text message transmission business, group message transport service.In addition, the thread a～c in thread pool is preset as the business of logining of processing, and thread d～g is preset as the business of publishing of processing, and thread h～k is preset as the text message transmission business of processing, and thread l～n is preset as and processes group message transport service.Wherein, owing to logining, the priority level of business is the highest, therefore in the time that business is carried out, need first guarantee that it processes, and the thread that therefore set handling is logined business is for using.And, thread borrowing time that can be corresponding different according to the difference that can use number of threads.But the present invention is not limited in any way this.In practical application, can borrowing time can the setting according to actual needs of the thread that different types of service is corresponding and thread.
In the present embodiment, particularly, in the time that final controlling element detects that processing the thread of logining business does not have the free time (, thread a～c is all in running order), final controlling element can call idle thread processing corresponding to type of service that priority level is lower according to priority orders and login business.In this, determine the idle thread of using from low paramount order according to the priority level of other types of service.Because the priority level type of service lower than the business of logining comprises the business of publishing, text message transmission business, group message transport service, therefore the thread of, logining business is used order: group message transport service, text message transmission business, publish business.In other words, first final controlling element can call idle thread processing corresponding to group message transport service and login business, in the case of still cannot meeting the processing demands of the business of logining, then calls successively text message transmission business or idle thread corresponding to the business of publishing.But the present invention is not limited thereto.In practical application, can set according to actual needs thread and use order.
For example, if when thread corresponding to the business of logining do not have the free time, now, if desired process newly while logining business, first final controlling element can call idle thread corresponding to group message transport service.If corresponding thread m, the n of group message transport service is idle, final controlling element separately invokes thread m or thread n process the business of logining, or business is logined in invokes thread m, n processing simultaneously.Known according to the above-mentioned table of comparisons, when the independent invokes thread m of final controlling element or thread n process while logining business, thread m or thread n's can borrowing time be 80s, and in the time that business is logined in final controlling element invokes thread m, n processing simultaneously, thread m, n's can borrowing time be 70s.
Then, login business as example taking final controlling element invokes thread m, n processing simultaneously.Detect if final controlling element carries out timing in the time that beginning thread is used, and be 10s interval time.So, when the actual thread borrowing time of logining business (for example, 70s) equal thread m, n can borrowing time (for example, 70s) time, final controlling element can be regained thread m, n and call idle thread corresponding to type of service that other priority levels are lower.Now, if text message transmission business correspondence exists idle thread, final controlling element can call idle thread processing corresponding to text message transmission business and login business.Use so, as required idle thread corresponding to type of service that priority level is lower until the business of logining does not need thread to use.
In another preferred embodiment, in the time that final controlling element calls the higher type of service of thread process priority level corresponding to type of service that priority level is lower, final controlling element starts timing, and timing be thread corresponding to type of service that priority level is lower can borrowing time, in the time arriving timing, if the higher type of service of priority level is still by thread process corresponding to the lower type of service of priority level, final controlling element calls idle thread corresponding to type of service that other priority levels are lower and processes the type of service that described priority level is higher.
According to the above-mentioned table of comparisons for example, login business as example taking final controlling element invokes thread m, n processing simultaneously.In the time of final controlling element invokes thread m, n, start timing, and timing be thread m, n can borrowing time, for example, 70s.Therefore,, in the time arriving timing, if the business of logining is still processed by thread m, n, final controlling element can be regained thread m, n and call idle thread corresponding to type of service that other priority levels are lower.Now, if text message transmission business correspondence exists idle thread, final controlling element can call idle thread processing corresponding to text message transmission business and login business.Use so, as required idle thread corresponding to type of service that priority level is lower until the business of logining does not need thread to use.
Figure 2 shows that the schematic diagram of the data packet transmission treatment system that preferred embodiment provides according to the present invention.As shown in Figure 2, the data packet transmission treatment system that preferred embodiment of the present invention provides comprises receiving system 20, pretreatment unit 21 and final controlling element 22.Receiving system 21 connects pretreatment unit 21, and pretreatment unit 21 connects final controlling element 22.Wherein, receiving system 20 is for receiving pending packet.The pending packet that pretreatment unit 21 receives for splicing the deficiency of data bag that reads from buffering area and receiving system, and scan described splicing packet to judge whether described splicing packet mates specific data structure by regular expression, if the data packet matched specific data structure of described splicing, puts into receiving queue by the complete data packet of coupling.Final controlling element 22 is for dispatching described complete data packet from described receiving queue, and call corresponding thread according to the type of service of described complete data packet and process, wherein, described type of service has priority level, and different types of service is by different thread alignment processings.About the specific operation process of described system with described in said method, therefore repeat no more in this.
In sum, the data packet transmission processing method that preferred embodiment provides according to the present invention and system, preliminary treatment thread splices the pending packet that the deficiency of data bag that reads from buffering area and Socket thread receive, by regular expression Overlap-scanning packet to judge whether splicing packet mates specific data structure, and the complete data packet that coupling is obtained is put into receiving queue.So, improve subpackage efficiency, prevented that in internal memory, overstocking mass data bag causes communication server collapse.In addition, final controlling element is dispatched complete data packet from receiving queue, and calls corresponding thread according to the type of service of complete data packet and process.So, improved the treatment effeciency of business.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.
Claims (1)
1. a data packet transmission processing method, is characterized in that, comprises the following steps:
Socket thread receives pending packet;
Preliminary treatment thread splices the pending packet that the deficiency of data bag that reads from buffering area and described Socket thread receive, by regular expression Overlap-scanning packet to judge whether described splicing packet mates specific data structure, if the data packet matched specific data structure of described splicing, puts into receiving queue by the complete data packet of coupling;
Final controlling element is dispatched described complete data packet from described receiving queue, and call corresponding thread according to the type of service of described complete data packet and process, wherein, described type of service has priority level, and different types of service is by different thread alignment processings;
In the time that described final controlling element detects that the thread of processing the higher type of service of priority level does not have the free time, described final controlling element calls the higher type of service of priority level described in thread process corresponding to type of service that priority level is lower;
Described final controlling element timing detects the actual thread borrowing time of the type of service that described priority level is higher, if what described actual thread borrowing time was equal to or greater than thread corresponding to type of service that described priority level is lower can borrowing time, described final controlling element call idle thread corresponding to type of service that other priority levels are lower process type of service that described priority level is higher or
In the time of type of service that described in described final controlling element calls thread process corresponding to type of service that priority level is lower, priority level is higher, described final controlling element starts timing, and timing be thread corresponding to type of service that described priority level is lower can borrowing time, in the time arriving described timing, if the higher type of service of described priority level is still by thread process corresponding to the lower type of service of described priority level, described final controlling element calls idle thread corresponding to type of service that other priority levels are lower and processes the type of service that described priority level is higher.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110433104.8A CN102447628B (en) | 2011-12-22 | 2011-12-22 | Data packet transmission processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110433104.8A CN102447628B (en) | 2011-12-22 | 2011-12-22 | Data packet transmission processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102447628A CN102447628A (en) | 2012-05-09 |
| CN102447628B true CN102447628B (en) | 2014-07-02 |
Family
ID=46009729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110433104.8A Active CN102447628B (en) | 2011-12-22 | 2011-12-22 | Data packet transmission processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102447628B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572290A (en) * | 2013-10-11 | 2015-04-29 | 中兴通讯股份有限公司 | Method and device for controlling message processing threads |
| CN105991223A (en) * | 2015-02-13 | 2016-10-05 | Tcl集团股份有限公司 | Method and system for preventing packet splicing in TCP network data transmission |
| CN105071976B (en) * | 2015-09-08 | 2019-05-03 | 安一恒通（北京）科技有限公司 | Data transmission method and device |
| CN108156192A (en) * | 2016-12-02 | 2018-06-12 | 联芯科技有限公司 | Android RIL message handling systems and method |
| CN107273190A (en) * | 2017-05-19 | 2017-10-20 | 武汉票据交易中心有限公司 | A kind of batch scheduled service processing method and system |
| CN107995061A (en) * | 2017-11-30 | 2018-05-04 | 北京卓讯科信技术有限公司 | Collection and playback system and method when more specification 10Gbps network signals are long |
| CN108259243A (en) * | 2018-01-12 | 2018-07-06 | 深圳市卓讯信息技术有限公司 | Data processing method, terminal and computer storage media based on micro services Technical Architecture |
| CN110768910A (en) * | 2018-07-25 | 2020-02-07 | 视联动力信息技术股份有限公司 | Data transmission method and device |
| CN109451035A (en) * | 2018-12-04 | 2019-03-08 | 宁波耘瑞智能科技有限公司 | A kind of method, apparatus and system of separate type collection group agent and service |
| CN114125081B (en) * | 2021-10-27 | 2023-09-22 | 桂林长海发展有限责任公司 | Method and device for processing received data and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1402492A (en) * | 2002-09-29 | 2003-03-12 | 清华大学 | Method for implementing stream medium transmission based on real time transmission protocol and transmission control protocol |
| CN101959159A (en) * | 2009-07-17 | 2011-01-26 | 浙江省公众信息产业有限公司 | System and method for performing cross-platform on-site disposal by using mobile terminal |
-
2011
- 2011-12-22 CN CN201110433104.8A patent/CN102447628B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1402492A (en) * | 2002-09-29 | 2003-03-12 | 清华大学 | Method for implementing stream medium transmission based on real time transmission protocol and transmission control protocol |
| CN101959159A (en) * | 2009-07-17 | 2011-01-26 | 浙江省公众信息产业有限公司 | System and method for performing cross-platform on-site disposal by using mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102447628A (en) | 2012-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102447628B (en) | Data packet transmission processing method and system | |
| WO2021057500A1 (en) | Message sending management method and device | |
| CN104133724B (en) | Concurrent tasks dispatching method and device | |
| US20120254204A1 (en) | Techniques to manage file conversions | |
| CN102447778A (en) | Message display method and device | |
| WO2021190087A1 (en) | Task execution method, device and system, and server | |
| CN104932932A (en) | Asynchronous business processing method, device and system | |
| CN101242360B (en) | A network address conversion method and system based on priority queue | |
| CN102891809B (en) | Multi-core network device message presses interface order-preserving method and system | |
| CN103491162B (en) | Information sharing method based on mobile Internet and system | |
| CN102014282A (en) | Distributed video transcoding scheduling method and system | |
| CN102299843A (en) | Network data processing method based on graphic processing unit (GPU) and buffer area, and system thereof | |
| CN103873523A (en) | Client cluster access method and device | |
| CN101136870A (en) | Symbol based message transferring method and system | |
| CN101442498A (en) | Method for self-timing transmission of e-mail | |
| Hwang et al. | Modification of mosquitto broker for delivery of urgent MQTT message | |
| CN107577541A (en) | A kind of application and a kind of call method of component | |
| CN105208004A (en) | Data input method based on OBD equipment | |
| CN101789105A (en) | Packet-level dynamic mail attachment virus detection method | |
| JP2007529051A (en) | Method and system for maintaining font size on different platforms | |
| CN112817745A (en) | Task processing method and device | |
| CN117271137A (en) | Multithreading data slicing parallel method | |
| CN101072172A (en) | Method and device for realizing network data high-efficiency transfer | |
| CN101419618B (en) | In-text information displaying method and apparatus | |
| CN107911484B (en) | Message processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Suzhou City, Jiangsu province 215121 Fengting Avenue Suzhou Industrial Park No. 666 Weiting Intelligent Industrial Park Building 8 Patentee after: Kuo Di education and science company limited Address before: Suzhou City, Jiangsu province 215121 Fengting Avenue Suzhou Industrial Park No. 666 Weiting Intelligent Industrial Park Building 8 Patentee before: Suzhou Kuodi Network Technology Co., Ltd. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20201125 Address after: 226100 Guangzhou road Jiangsu city Nantong Province Haimen Economic and Technological Development Zone No. 999 Patentee after: Nantong Construction Engineering Co.,Ltd. Address before: Suzhou City, Jiangsu province 215121 Fengting Avenue Suzhou Industrial Park No. 666 Weiting Intelligent Industrial Park Building 8 Patentee before: CODYY EDUCATION TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220301 Address after: 226000 building 10, Jiangong Lake Science and innovation center, No. 588, Hong Kong Road, Haimen Economic and Technological Development Zone, Nantong City, Jiangsu Province Patentee after: Nantong haisai future digital technology Co.,Ltd. Address before: 226100 Guangzhou Road 999 Haimen Economic and Technological Development Zone, Nantong City, Jiangsu Province Patentee before: Nantong Construction Engineering Co.,Ltd. |
|
| TR01 | Transfer of patent right |