CN105654259A - Mass agent freight rate search compression method - Google Patents
Mass agent freight rate search compression method Download PDFInfo
- Publication number
- CN105654259A CN105654259A CN201510994413.0A CN201510994413A CN105654259A CN 105654259 A CN105654259 A CN 105654259A CN 201510994413 A CN201510994413 A CN 201510994413A CN 105654259 A CN105654259 A CN 105654259A
- Authority
- CN
- China
- Prior art keywords
- freight rate
- proxy
- group
- binding
- agreement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006835 compression Effects 0.000 title claims abstract description 21
- 238000007906 compression Methods 0.000 title claims abstract description 21
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000012795 verification Methods 0.000 claims description 11
- 241001101988 Proxys Species 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 8
- 239000003795 chemical substances by application Substances 0.000 description 5
- 230000003203 everyday effect Effects 0.000 description 4
- 230000002596 correlated effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
- G06Q10/0834—Choice of carriers
- G06Q10/08345—Pricing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
Abstract
The invention provides a mass agent freight rate search compression method comprising the steps that configuration files are loaded; binding relation data are read; a group and agreed freight rate binding relation object set and a group and prescribed freight rate binding relation object set are constructed; agent numbers are compared; and the agent numbers suitable for the freight rate inquiring result are determined. According to the method, similar agents can be compressed based on the freight rate agreement so that efficiency of freight rate searching can be enhanced and occupation of system resources can be reduced.
Description
Technical field
The present invention relates to proxy's freight rate search field, it is specifically related to a kind of big batch proxy's freight rate search compression method.
Background technology
In traditional domestic freight rate search system, some being had to proxy's platform of a large amount of proxy, if needing to find out minimum freight rate for all proxys, traditional method is that each agency is called domestic freight rate search engine per capita. This kind of traditional method extremely wastes time and energy, and inquiry efficiency is lower.
In addition, although each proxy has the freight rate data of its uniqueness, but the freight rate of major part is all common, and therefore, this kind of traditional method also can take valuable system resource and go to perform identical freight rate calculating.
Therefore, urgent needs are a kind of can be responsible for the method processed for big batch proxy's freight rate search specially, and to reduce the freight rate search response time, and freight rate is searched for taking of system resource by minimizing.
Summary of the invention
In view of this, the present invention provides a kind of big batch proxy's freight rate search compression method, it is possible to similar proxy is compressed based on freight rate agreement, thus improves the efficiency of freight rate search, reduces taking of system resource.
The present invention provides a kind of big batch proxy's freight rate search compression method, comprising:
Step 1: loading configuration file, comprises the two character code lists of airline complete or collected works, each proxy's list;
Step 2: read binding relationship data, comprise group and agent binding relation table, group and agreement freight rate binding relationship table; Wherein, described group and agent binding relation table contain the mapping relation between group and its agency comprised; Described group and agreement freight rate binding relationship table contain the mapping relation between group and its agreement freight rate;
Step 3: based on the binding relationship data read, builds group and agreement freight rate binding relationship object set; Wherein, described group and agreement freight rate binding relationship object set contain each mapping relation acted on behalf of between its agreement freight rate in group;
Step 4: based on group and agreement freight rate binding relationship object set, sorts according to the quantity of institute's binding protocol to the proxy number in set;
Step 5: judge whether the binding protocol of the proxy number that the quantity of institute's binding protocol is maximum equals the intersection of the binding protocol of part residue proxy number, if, then proxys number maximum for the quantity of described institute binding protocol is deleted in described set, and the maximum proxy number of the quantity recording described institute binding protocol remains the relation of inclusion of proxy number with described part, then step 4 is returned, until there is not any proxy number in described set;Otherwise proxys number maximum for the quantity of described institute binding protocol deleted in described set, and be stored in the first database, then return step 4, until in described set, there is not any proxy number;
Step 6: the proxy number in the first database is carried out freight rate inquiry, and based on described relation of inclusion, it is determined that the proxy number that freight rate Query Result is suitable for.
Described step 4 also comprises, and before the proxy number in set being sorted according to the quantity of institute's binding protocol, the proxy number in set is filtered based on verification condition, retains the proxy number meeting verification condition.
Described verification condition comprises origin, point of destination and Query Dates.
Present method can adopt multiple parallel thread to carry out processed compressed.
In sum, the present invention proposes a kind of big batch proxy's freight rate search compression method, it is possible to similar proxy is compressed based on freight rate agreement, thus improves the efficiency of freight rate search, reduces taking of system resource. In addition, the present invention also by adopting parallel thread to carry out processed compressed, further increases the efficiency of freight rate search, decreases the time of response of freight rate search.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or scheme of the prior art, it is briefly described to the accompanying drawing used required in embodiment below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skills, under the prerequisite not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 gives the schematic flow sheet of big batch proxy's freight rate search compression method that one embodiment of the invention provides.
Fig. 2 gives group and the schematic diagram of agent binding relation table;
Fig. 3 to agreement freight rate the schematic diagram of main integral part;
Fig. 4 gives group and the schematic diagram of agreement freight rate binding relationship object set;
Fig. 5 meets, after giving verification, the data items schematic diagram required.
Embodiment
For making the object of the embodiment of the present invention, technical scheme and advantage clearly, the technical scheme in the embodiment of the present invention is carried out clear, complete description below, it is clear that described embodiment is the present invention's part embodiment, instead of whole embodiments. Based on the embodiment in the present invention, other embodiments all that those of ordinary skill in the art obtain under the prerequisite not making creative work, all belong to the scope of protection of the invention.
As shown in Figure 1, giving the schematic flow sheet of big batch proxy's freight rate search compression method that one embodiment of the invention provides, big batch proxy's freight rate search compression method of the present embodiment comprises:
Step one, the dependence resource being correlated with by the loading of self-defined Classloader, and load required configuration file simultaneously. Mainly comprise: the main configuration file that the two character code lists of airline complete or collected works, proxy's list file of different proxy, program are run.
Wherein defining the time point of timing task execution every day in main configuration file, different proxy needs to compress the important parameter such as total number of days of data, the number of days of data buffer memory.
After the configuration information that step 2, loading are correlated with, from database, read relevant binding relationship data. Mainly comprise: group and agent binding relation table, group and agreement freight rate binding relationship. Fig. 1 gives group and the schematic diagram of agent binding relation table.Fig. 2 to agreement freight rate the schematic diagram of main integral part.
After step 3, the configuration information in early stage and binding relationship data encasement are complete, the data issued with group's form are arranged, arranges as satisfactory data structure. Main structure comprises such as lower part: group and agreement freight rate binding relationship object set. Fig. 3 gives group and the schematic diagram of agreement freight rate binding relationship object set.
Step 4, execution compression operation: all freight rates issued with group's form are carried out School Affairs filtration, retains the data satisfied condition, carry out proxy's compression. Verification condition mainly comprises: the verification of origin and point of destination, the verification of Query Dates. Fig. 4 meets, after giving verification, the data items schematic diagram required.
Proxy number in group and agreement freight rate binding relationship object set is sorted according to the quantity of institute's binding protocol; Judge whether the binding protocol of the proxy number that the quantity of institute's binding protocol is maximum equals the intersection of the binding protocol of part residue proxy number, if, then proxys number maximum for the quantity of described institute binding protocol is deleted in described set, and the maximum proxy number of the quantity recording described institute binding protocol remains the relation of inclusion of proxy number with described part; Otherwise proxys number maximum for the quantity of described institute binding protocol deleted in described set, and be stored in the first database, until in described set, there is not any proxy number; Proxy number in first database is carried out freight rate inquiry, and based on described relation of inclusion, it is determined that the proxy number that freight rate Query Result is suitable for. Illustrate: table one is the mapping relation of proxy number in certain set with its binding protocol.
Table one
| Proxy number | Binding protocol |
| PEK099 | REF_1, REF_2, REF_3, REF_4, FBR_1, FBR_2, FBR_3, FBR_4 |
| SHA888 | REF_1, FBR_1 |
| CAN777 | REF_2, FBR_2 |
| CTU555 | REF_3, FBR_3 |
| CKQ222 | REF_1, REF_2, FBR_1, FBR_2 |
| PEK123 | REF_2, REF_3, FBR_2, FBR_3 |
| SHA444 | REF_1, REF_4, FBR_1, FBR_4 |
| HRB012 | REF_5, FBR_5 |
Proxy number in this set is compressed, obtains following result:
SHA888/PEK099/CKQ222
CAN777/PEK099/CKQ222/PEK123
CTU555/PEK099/PEK123
SHA444/PEK099
HRB012
This result shows: the un-compressed proxy number being stored in the first database is respectively: SHA888, CAN777, CTU555, SHA444, HRB012; And the binding protocol of PEK099 and CKQ222 includes the binding protocol of SHA888; The binding protocol of PEK099, CKQ222, PEK123 includes the binding protocol of CAN777; The binding protocol of PEK099, PEK123 includes the binding protocol of CTU555; The binding protocol of PEK099 includes the binding protocol of SHA444. Such that it is able to think that the freight rate of SHA888 is applicable to PEK099, CKQ222; The freight rate of CAN777 is applicable to PEK099, CKQ222, PEK123; The freight rate of CTU555 is applicable to PEK099, PEK123; The freight rate of SHA444 is applicable to PEK099.
Owing to the quantity of the freight rate information of Mei Jia airline every day is 1,000,000 ranks, therefore to be compressed the data of magnanimity like this, only go processing efficiency very low by single thread. In order to improve the efficiency of program, in compressed logic certain applications, GroovyGpars concurrent framework improves the efficiency of program. Illustrate, all freight rate data can be re-started grouping according to inquiry number of days, when adopting 5 concurrent compression threads, the data that can give tacit consent to every 5 days are as a task executing units, to each task executing units, enable 5 compression threads to be responsible for respectively the data of every day being compressed, and the result after compression is pushed to buffer memory server and preserves.
After step 5, the freight rate data having compressed configuration file given number of days, the proxy's result after all compressions has all been carried out buffer memory.
Now, timing task can enter monitoring state, again performs compression operation at the 2nd day fixed time point, and every day moves in circles.
Last it is noted that above embodiment is only in order to illustrate the technical scheme of the present invention, it is not intended to limit; Although with reference to previous embodiment to invention has been detailed description, it will be understood by those within the art that: the technical scheme described in foregoing embodiments still can be modified by it, or wherein part technology feature is carried out equivalent replacement, and these amendments or replacement, do not make the spirit and scope of the essence disengaging various embodiments of the present invention technical scheme of appropriate technical solution.
Claims (4)
1. big batch proxy's freight rate search compression method, it is characterised in that, described method comprises:
Step 1: loading configuration file, comprises the two character code lists of airline complete or collected works, each proxy's list;
Step 2: read binding relationship data, comprise group and agent binding relation table, group and agreement freight rate binding relationship table; Wherein, described group and agent binding relation table contain the mapping relation between group and its agency comprised; Described group and agreement freight rate binding relationship table contain the mapping relation between group and its agreement freight rate;
Step 3: based on the binding relationship data read, builds group and agreement freight rate binding relationship object set; Wherein, described group and agreement freight rate binding relationship object set contain each mapping relation acted on behalf of between its agreement freight rate in group;
Step 4: based on group and agreement freight rate binding relationship object set, sorts according to the quantity of institute's binding protocol to the proxy number in set;
Step 5: judge whether the binding protocol of the proxy number that the quantity of institute's binding protocol is maximum equals the intersection of the binding protocol of part residue proxy number, if, then proxys number maximum for the quantity of described institute binding protocol is deleted in described set, and the maximum proxy number of the quantity recording described institute binding protocol remains the relation of inclusion of proxy number with described part, then step 4 is returned, until there is not any proxy number in described set; Otherwise proxys number maximum for the quantity of described institute binding protocol deleted in described set, and be stored in the first database, then return step 4, until in described set, there is not any proxy number;
Step 6: the proxy number in the first database is carried out freight rate inquiry, and based on described relation of inclusion, it is determined that the proxy number that freight rate Query Result is suitable for.
2. the big batch proxy's freight rate search compression method of one according to claim 1, it is characterized in that, described step 4 also comprises, before the proxy number in set is sorted according to the quantity of institute's binding protocol, based on verification condition, the proxy number in set is filtered, retain the proxy number meeting verification condition.
3. the big batch proxy's freight rate search compression method of one according to claim 2, it is characterised in that, described verification condition comprises origin, point of destination and Query Dates.
4. the big batch proxy's freight rate search compression method of one according to claim 3, it is characterised in that, adopt multiple parallel thread to carry out processed compressed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510994413.0A CN105654259B (en) | 2015-12-25 | 2015-12-25 | Large-batch agent freight rate search compression method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510994413.0A CN105654259B (en) | 2015-12-25 | 2015-12-25 | Large-batch agent freight rate search compression method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105654259A true CN105654259A (en) | 2016-06-08 |
| CN105654259B CN105654259B (en) | 2021-07-06 |
Family
ID=56476954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510994413.0A Active CN105654259B (en) | 2015-12-25 | 2015-12-25 | Large-batch agent freight rate search compression method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105654259B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103399944A (en) * | 2013-08-14 | 2013-11-20 | 曙光信息产业(北京)有限公司 | Implementation method and implementation device for data duplication elimination query |
| CN104050269A (en) * | 2014-06-23 | 2014-09-17 | 上海帝联信息科技股份有限公司 | Log compression method and device and log decompression method and device |
| CN104462527A (en) * | 2014-12-22 | 2015-03-25 | 龙信数据（北京）有限公司 | Data deduplication method and device |
-
2015
- 2015-12-25 CN CN201510994413.0A patent/CN105654259B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103399944A (en) * | 2013-08-14 | 2013-11-20 | 曙光信息产业(北京)有限公司 | Implementation method and implementation device for data duplication elimination query |
| CN104050269A (en) * | 2014-06-23 | 2014-09-17 | 上海帝联信息科技股份有限公司 | Log compression method and device and log decompression method and device |
| CN104462527A (en) * | 2014-12-22 | 2015-03-25 | 龙信数据（北京）有限公司 | Data deduplication method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105654259B (en) | 2021-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104899199A (en) | Data processing method and system for data warehouse | |
| CN102129425B (en) | The access method of big object set table and device in data warehouse | |
| CN104598376A (en) | Data driving layered automation test system and method | |
| CN111813805A (en) | Data processing method and device | |
| CN106909554B (en) | Method and device for loading database text table data | |
| US20140059000A1 (en) | Computer system and parallel distributed processing method | |
| CN106354817B (en) | Log processing method and device | |
| CN103902544A (en) | Data processing method and system | |
| CN110058940B (en) | Data processing method and device in multi-thread environment | |
| CN105550225A (en) | Index construction method and query method and apparatus | |
| CN105354256A (en) | Data pagination query method and apparatus | |
| CN106557307B (en) | Service data processing method and system | |
| CN103970853A (en) | Method and device for optimizing search engine | |
| CN111400288A (en) | Data quality inspection method and system | |
| CN102915344B (en) | SQL (structured query language) statement processing method and device | |
| CN105630934A (en) | Data statistic method and system | |
| CN111460019A (en) | Data conversion method and middleware of heterogeneous data source | |
| CN107798111B (en) | Method for exporting data in large batch in distributed environment | |
| CN103116641B (en) | Obtain method and the collator of the statistics of sequence | |
| CN109885642A (en) | Classification storage method and device towards full-text search | |
| CN102207935A (en) | Method and system for establishing index | |
| CN103064908A (en) | Method for rapidly removing repeated list through a memory | |
| CN106407442A (en) | Massive text data processing method and apparatus | |
| CN106649385B (en) | Data reordering method and device based on HBase database | |
| CN112527836B (en) | Big data query method based on T-BOX platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |