CN102156744A - Method for eliminating repetition of memory dialog list - Google Patents
Method for eliminating repetition of memory dialog list Download PDFInfo
- Publication number
- CN102156744A CN102156744A CN 201110097351 CN201110097351A CN102156744A CN 102156744 A CN102156744 A CN 102156744A CN 201110097351 CN201110097351 CN 201110097351 CN 201110097351 A CN201110097351 A CN 201110097351A CN 102156744 A CN102156744 A CN 102156744A
- Authority
- CN
- China
- Prior art keywords
- internal memory
- ticket
- concordance list
- list
- writing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for eliminating repetition of memory dialog list, comprising the following steps of: step 1, reading a dialog list document into a memory; step 2, reading a dialog list record from the dialog list document; step 3, finding the index list which is corresponding to the dialog list record from the memory according to the key information of the dialog list record; step 4, piecing the field content of the dialog list record to form a character string, working out the MD5 value as the index of the dialog list record; step 5, interpolating the MD5 value in the index list; if the interpolating operation is successful, writing the dialog list record into a normal dialog list document; otherwise, writing the dialog list record into a repeated list document; and step 6, repeating the second step to the fifth step until traversing all dialog list records of the dialog list document. The method provided by the invention eliminates the repetition by using the index list and the MD5 value, realizes the operation for implementing relatively quick repetition eliminating treatment in a relatively small memory space, and the repetition eliminating processing performance of the dialog list is not relative to the data volume of the dialog list.
Description
Technical field
The present invention relates to a kind of double recipe formula of picking of ticket writing, particularly in the inner heavy method of internal memory level row that solves magnanimity level file of telecommunication system.
Background technology
Along with telecom operators constantly develop, professional expansion is more and more wider, and various new business emerge in an endless stream; More and more cheap along with rate, the user constantly increases, also cumulative year after year of user's call volume simultaneously, detailed single amount of user is increasing.Original more and more data ticket of magnanimity of heavy pin that picks, pressure is increasing, and system upgrade is imperative.Following several mode database mode, internal memory mode and file system mode are adopted in current most telecom operation commercial city, and existing these methods realize picks heavy product, advance to be more or less in defectiveness aspect performance and the IO in mass data processing.
Database mode: database picks heavy common practices and builds a series of tables exactly, uses unique index or major key to carry out the judgement of uniqueness.This tactful biggest advantage is developed exactly than being easier to, and has done because database is given in a lot of work; Shortcoming is very obvious, wasting space, inefficiency.Because table data and index data will take the double space in the database, even if the table data have only critical field, also to waste one times space, because write the double data, so efficient also will be hanged down one times.
Traditional file systems mode: in the ticket preprocessing process, set up heavy single comparison document of checking every day.To every standard ticket through generating after the pre-service, unpack field (character string), open corresponding heavy single file of checking according to condition, utilize HASH algorithm and adaptation function to search fast heavy single the inspection in the file, if the match is successful, it is heavy single that this ticket belongs to, and then carries out heavy accordingly single the processing.If coupling is unsuccessful, this work is single not to be heavy single, and above-mentioned three character rates are added corresponding heavy single the inspection in the file in this ticket, are used for heavy single inspection the to follow-up ticket.For the time of the matched and searched that reduces the HASH algorithm, heavyly singly check that file also should be the smaller the better.It is relevant with the size of performance and file that this method will be searched in full text spare, the little performance of ticket amount also may, measure the IO operation big can strengthen the performance meeting influential.
Pure internal memory mode: will weigh and single check that file is placed in the internal memory and move, this method function that dependency database did not provide, processing speed is fast, but heavy single inspection quantity of documents is more, except will taking a part of hard-disc storage resource, also need system that bigger memory source is arranged, at the processing mass tickets host memory is had relatively and be strict with.
Summary of the invention
Technical matters to be solved by this invention provides a kind of new removing call ticket repeat method, realizes using less relatively memory headroom to carry out ticket faster and picks heavily processing, and make that the data volume of heavy handling property of picking of ticket and ticket is irrelevant.
The technical scheme that the present invention solves the problems of the technologies described above is as follows: a kind of internal memory removing call ticket repeat method comprises the steps:
Step 1: CDR file is read in internal memory;
Step 2: from described CDR file, read a ticket writing;
Step 3:, find in the internal memory and the corresponding concordance list of described ticket writing according to the key message in the described ticket writing;
Step 4: the field contents in the described ticket writing is combined into a character string, and asks the index of MD5 value as this ticket writing;
Step 5: described MD5 value is inserted in the described concordance list,, then described ticket writing is write in the normal CDR file,, then described ticket writing is write heavy monofile if insert failure if insert successfully;
Step 6: repeating step 2 is to step 5, all ticket writings in having traveled through described CDR file.
The invention has the beneficial effects as follows: utilize concordance list and adopt the MD5 value to pick heavily in internal memory, than traditional database mode and file system mode, speed of the present invention is faster; Than traditional pure internal memory mode, because the present invention has adopted the mode of MD5 value, it is heavy to need not that each field contents in the ticket writing is compared row, has saved the time; Adopt the mode of directly the MD5 value being inserted in the concordance list to pick heavily, faster than inquiry speed relatively; Method of the present invention only need be operated with the corresponding concordance list of described ticket writing in internal memory, and concordance list that therefore will be not all is written into internal memory, and also the mode than traditional is lower to the demand of internal memory; The present invention has realized using less relatively memory headroom to carry out ticket faster and has picked heavily processing, and makes that the data volume of heavy handling property of picking of ticket and ticket is irrelevant.
On the basis of technique scheme, the present invention can also do following improvement.
Further, described concordance list adopts the T-tree index.
Adopt the beneficial effect of above-mentioned further scheme to be, the traversal searching speed is fast, and this T-tree index also is topmost a kind of indexed mode in the memory database.
Further, the key message number of being section and/or the date in the described ticket writing.
Adopt the beneficial effect of above-mentioned further scheme to be, employing section and/or date are as key message, can classify preferably to ticket, such as all tickets of certain number section all records of certain hour at certain day are placed in the table, this number every day section just has 24 tables, corresponding concordance list also has 24, will be convenient to like this to record management, search and pick heavily and wait.
Further, a char (32) type-word section is only arranged in the described concordance list, record all the non-heavy unirecord MD5 values in the CDR file.
Adopt the beneficial effect of above-mentioned further scheme to be, make that taking up room of concordance list is little, inquiry velocity is fast, and loading and unloading speed is fast, flexible operation.
Further, in the step 3,, then described concordance list is loaded in the internal memory from its storer of preserving if do not exist and the corresponding concordance list of described ticket writing in the internal memory; If all do not exist and the corresponding concordance list of described ticket writing in described internal memory and the storer, then newly-built concordance list conduct and the corresponding concordance list of described ticket writing in internal memory.
Adopt the beneficial effect of above-mentioned further scheme to be, load concordance list as required, realized better control, needn't load whole concordance lists simultaneously and pick heavily, avoid the waste in unnecessary memory space EMS memory occupation.
Further, described method is provided with about ticket writing and handles bar number or the limit value of ticket writing time, when the ticket writing of handling reaches described limit value, the concordance list that is of little use in the internal memory is unloaded, and be kept in the storer of appointment.
Adopt the beneficial effect of above-mentioned further scheme to be, after having handled a collection of ticket writing, corresponding meeting is loaded into the concordance list of some in the internal memory, carry out regular unloading according to preset condition, to not re-use or use concordance list seldom to unload, only keep the concordance list (concordance list commonly used) of dsc data, the concordance list after the unloading reloads in the internal memory when to be needed, continuation that internal memory uses and the dirigibility of using have been guaranteed.
Description of drawings
Fig. 1 is the way to manage schematic block diagram of concordance list in the internal memory removing call ticket repeat method of the present invention;
Fig. 2 is a kind of embodiment process flow diagram of internal memory removing call ticket repeat method of the present invention;
Fig. 3 is the system diagram that flow process shown in Figure 2 adopts.
Embodiment
Below in conjunction with accompanying drawing principle of the present invention and feature are described, institute gives an actual example and only is used to explain the present invention, is not to be used to limit scope of the present invention.
Internal memory removing call ticket repeat method of the present invention comprises the steps:
Step 1: CDR file is read in internal memory;
Step 2: from described CDR file, read a ticket writing;
Step 3:, find in the internal memory and the corresponding concordance list of described ticket writing according to the key message in the described ticket writing;
Step 4: the field contents in the described ticket writing is combined into a character string, and asks the index of MD5 value as this ticket writing;
Step 5: described MD5 value is inserted in the described concordance list,, then described ticket writing is write in the normal CDR file,, then described ticket writing is write heavy monofile if insert failure if insert successfully;
Step 6: repeating step 2 is to step 5, all ticket writings in having traveled through described CDR file.
Concordance list wherein adopts the T-tree index, and a char (32) type-word section is only arranged in the concordance list, records all the non-heavy unirecord MD5 values in the CDR file.The mode that key message in the ticket writing can the number of employing section, the date combines, with different number sections not the ticket that produces of same date, time classify, so that search, also can adopt other ticket content to classify as required certainly as key message.Internal memory removing call ticket repeat method of the present invention is provided with about ticket writing and handles bar number or the limit value of ticket writing time, when the ticket writing of handling reaches described limit value, the concordance list that is of little use in the internal memory is unloaded, and be kept in the storer of appointment.
In step 3,, then described concordance list is loaded in the internal memory from its storer of preserving if do not exist and the corresponding concordance list of described ticket writing in the internal memory; If all do not exist and the corresponding concordance list of described ticket writing in described internal memory and the storer, then newly-built concordance list conduct and the corresponding concordance list of described ticket writing in internal memory.
According to as mentioned above, adopt mode for the management of concordance list among the present invention as Fig. 1.Among Fig. 1, loading zone is set in internal memory, on other storeies, as hard disk etc., unload zone is set, obsolete concordance list is positioned in the unload zone, when needs use, the concordance list that use is loaded in the loading zone in the internal memory uses.Because concordance list adopts the T-tree index, char (a 32) field of preserving the MD5 value is wherein only arranged, and classify according to number section and date, therefore each concordance list occupation space is all very little, be convenient to loading and unloading like this to concordance list, only will be loaded in the loading zone of internal memory with the corresponding concordance list of current ticket writing, other obsolete all having carried out in internal memory unload and are kept in the unload zone of storer, thereby guaranteed less take of concordance list, saved memory source internal memory.
As shown in Figure 2, for using an idiographic flow of the inventive method, Fig. 3 is the system diagram that flow process shown in Figure 2 adopts.This flow process comprises:
1, program loads from CDR file and database and picks reconfiguration information.
2, confirm whether the CDR file of not handling last time (handling the CDR file of half) is arranged after finishing when 1, promptly the breakpoint file is obtained the filename of processing and the line number of processing if having from the breakpoint file, begins to handle from the processing line number of record; If there is not the breakpoint file, directly walk downward, to follow-up flow process.
3, obtain CDR file from program entry, obtain file by configuration information and pick heavy type:
(1) appends index (only in concordance list, index information, do not generate outlet)
(2) do not pick heavily (directly moving on to program exit, not recording indexes in concordance list) from program entry
(3) normally pick heavy file, get each the bar ticket writing in the CDR file, by configuration information generate the index table name as: find concordance list (if table does not exist then from disk table is loaded into the internal memory) by number section and date, if the situation of Out of Memory when inserting, occurs, program can be used lru algorithm, concordance list useless is unloaded to disk, when unloading, writes the breakpoint file simultaneously.
4, ticket writing is all fields are combined into a long character string, carry out the calculating of MD5 value, and be inserted in the internal memory concordance list, if success then represent that this ticket writing is not heavy single, if failure and newspaper repeat, representing that then this record is heavy single, is not that heavy single ticket outputs to outlet, weighs and singly outputs to heavy monofile.
5, write the processing daily record after handling a file, write down the situation of the processing of these all records of file,, then write the detailed error message of running log newspaper if having unusually.
6, every one batch, i.e. the time of appointment, then the daily record under the temp directory is outputed to formal catalogue, simultaneously with concordance list unloading useless in the internal memory.
Said process, in index management, create several concordance lists (may be 1 to 2 thousand sheets) by number section, date (it is configurable specifically how dividing), adopt the T-Tree index, every concordance list has only a field, be char (32), the MD5 character string of the index value of ticket is left a record in the table.Read picking heavy index field in the ticket according to the heavy set of fields of picking of joining in the configuration information, be combined into a character string after, ask MD5 value, insert corresponding internal memory concordance list then, if success then be normal ticket, if the major key constraint then is to weigh list.In the flow process shown in Figure 2, also keep to append index function and skip not pick and weigh function, and support the function of a plurality of catalogues of routine processes, breakpoint restore funcitons when the support processing is unusual, index field customization index, the customization of index condition, also support multiple form ticket to pick heavily (ascii scale-of-two split), adopt quick internal memory replacement algorithm to unload some internal memory dynamic index tables, keep the internal memory dynamic index table of dsc data.
About internal memory control problem
(1) for fear of the excessive internal memory of use, need the dynamic quantity of showing in the internal memory that controls, general the data load with the same day gets final product to internal memory, the table of ticket correspondence then is loaded into internal memory with corresponding tables not in internal memory if discovery is newly arrived; If the table quantity in the discovery internal memory has reached maximal value, then unload the less table of some utilization rate.
(2) since in the actual production ticket be that the time is continuous, so load table, the frequent odds of unload table are little, can not produce big influence to efficient.
(3) memory database need cooperate transformation, increases by two functions of " load table " and " unload table ".
Unload table: similar drop, difference is the full dose file of not deleting under the corresponding catalogue;
Load table: on the basis of creating table, the full dose file load under the corresponding catalogue is arrived internal memory, and rebuild the T-Tree index.
For example: with Sichuan No. 20,091,011 one day ticket of a main frame is example: have section 1240 numbers, wherein 1038 numbers sections have record, and single section dominant record number is 281519, the smallest record number is 1, ticket adds up to 100,000,000, estimates to take to be stored as 5.5G, and the expectation committed memory is 8G.
About efficiency
(1) memory database is to pick the insert function of bringing up again for a special use, gets around SQL and resolves, and directly inserts internal memory, and in addition, affairs adopt does not write logging mode, and (whole file) inserts efficient (contain and generate the MD5 sign indicating number) and be about 30000 to 50000/second.
(2) need to optimize memory table " loading and unloading " switchover policy, it is little to guarantee to switch probability.
About the data security problem
Because when inserting, affairs are in does not write logging mode, in case memory database is unusual or main frame is unusual, after restarting, data will be lost, and therefore need security guarantee:
(1) pick file of heavy every processing after, export formal ticket, submit affairs to, the inlet ticket moves into the A catalogue;
(2) pick heavyly, carry out the exp order, internal storage data is landed fully to disk, and the CDR file of A catalogue on the disk is moved into the B catalogue every one batch (perhaps handling a batch file);
(3) pick heavy opening after, if find the A catalogue file is arranged, then the A catalogue file is inserted in the memory management again, but does not export any file, handle the back and move into the B catalogue, purpose is the main record that indexes;
(4) when a file processing when intact, because Out of Memory, when taking place, the concordance list unloading generates the breakpoint file, the line number that the record breakpoint is handled, catalogue, effectively write down number, wrong unirecord number, heavy unirecord number, handle this file when restarting, leap to breakpoint and write down the row processing, former wrong list and heavy list all keep.
(5) a plurality of processes all may have access to same concordance list, and mutually exclusive operation has been done in memory management for this reason, and data access safety compares human nature, can observe the situation of concordance list.
The internal memory ticket that the present invention is based on md5, memory shared, dynamic T-tree concordance list and quick internal memory replacement algorithm picks heavily, reach the win-win result of data storage usage space and treatment effeciency, adopt internal memory dynamic table algorithm to pick heavily, and unload some internal memory dynamic index tables by quick internal memory replacement algorithm, keep dsc data internal memory dynamic table.This method adopts md5, memory shared, dynamic T-tree concordance list and quick internal memory replacement algorithm, has high performance processing power, has realized that handling property and data volume are irrelevant.Pick the storage organization of heavy information in internal memory, possess and share and feature such as concurrent, time window.
Pick the embodiment of heavy index: use the probability in the database, can and hour create memory table by preceding 7 dates that add the air time of phone number, with the table space subregion that hour carries out similar database of air time, reach the data payload balance simultaneously.As table name be: T_VC_1355227_2010072001 represent the cell-phone number section be all tickets of 1355227 at 2010-7-20, all tickets are put in this table in the time of 1, simultaneously this table are placed on table space and are on 01 the file system table space.If have 1000 numbers sections these 1000 numbers sections will be divided into 24 parts like this, divide the file system of 24 settings equally.Picking heavy process is:
(1) reads a CDR file;
(2) from this CDR file, read a ticket writing;
(3) navigate to concordance list in the memory management according to the phone number of this record and time;
(4) if this concordance list does not exist, then load required concordance list from unload zone automatically, newly-built concordance list if unload zone does not have concordance list yet;
(5) MD5 is intact index inserts, and is not heavy single if successfully represent this ticket, if unsuccessful, represents that then this ticket is heavy single, writes heavy monofile with this ticket;
(6) when the higher limit that reaches setting counted in the record of handling, as: 100 ten thousand records, program is carried out unloading operation to concordance list according to optimal algorithm; Or the time of arrival appointment, will carry out unloading operation automatically, keep dsc data in internal memory.
Below for adopting the profile information of the inventive method, the presents configuration be the process operation time essential information used, adopt the ini file form.
[SYS_INFO]
The # Log Directory
LOGDIR=/tpt/mmdb/work/zhuoch/wbin/data/log
[CONFIG]
IDXCTRL?=?/tpt/mmdb/work/zhuoch/wbin/cfg/IndexFields.cfg
The ####{ option }, when handling the fedx ticket, use by needs.Be adding-D__USE_FEDX__ among the makefile
The #FEDX configuration file
FEDXCTRL?=?${FEDX_CONFIG_PATH}/config.xml
[CONTROL]
The #{ option } the program run sign, acquiescence weed_dup
CTRLFLAG?=?weed_dup
The #{ option } program start and stop control table, acquiescence sys_proc
CNTTAB =?sys_proc
The #{ option } pick and heavily show prefix.Acquiescence t_dup
TABPRE =?t_bill1
The #{ option } there is maximum number simultaneously in table in the memory database, acquiescence 1000
MAXTAB?=?300
The #{ option } when reaching the table maximum number, each unload table number, acquiescence 200
DELTAB?=?50
The #{ option } daily record batch (second), gave tacit consent to 900 seconds
LOGBATCH?=?900
The #{ option } catalogue handle number of files next time. and acquiescence 100, handling a Directory Value all the time is 0
FILECOUNT?=?0
[DATABASE]
# memory database encrypt file
LOGIN =?${DCI_HOME}/cfg/login.db
# memory database Instance Name
SERVER =?imdb
#DIR begins can only be since 01, comes the front if repeat to give tacit consent to get
[DIR01]
The # file layout that enters the mouth
INDIR =?/tpt/mmdb/work/zhuoch/wbin/data/in/A*.chk
# export document form
OUTDIR =?/tpt/mmdb/work/zhuoch/wbin/data/out/P%s
# exports temp directory
OUTTMP =/tpt/mmdb/work/zhuoch/wbin/data/outtmp
The # backup directory
BAKDIR =?/tpt/mmdb/work/zhuoch/wbin/data/bak
# backs up temp directory
BAKTMP =?/tpt/mmdb/work/zhuoch/wbin/data/baktmp
The fixed elongated cut-point of ##
CHANGEDATE?=?20090701
The #{ option }, inlet does not pick heavily, directly writes the outlet file layout, acquiescence * .nodup
IGNOREFILE?=?y. .nodup
The #{ option }, inlet does not pick heavily, and does not write outlet, only appends the index file form, acquiescence * .addidx
ADDIDXFILE?=?y. .addidx
The #{ option }, inlet does not pick heavily, and does not write outlet, only deletes index index file form in the table, acquiescence * .delidx
DELIDXFILE?=?y. .delidx
The #{ option }, rs chacter (only effective when handling separator record).Space ^ ~ replacement, tab replaces with ^^, the acquiescence space
SPLIT?=
#split separator form ascii f format fedx FEDX form
What branch was preceding among the #RECTYPE contrasts CHANGEDATE constantly with current system, than the big new that uses of CHANGEDATE, little of old (principle: preceding new back is old).
[VC/VC2] expression uses file-level to pick heavily below the #, and promptly a file is a kind of form, if there is not the expression record level of [...] to pick heavily.In the manipulative indexing configuration, judge according to a certain field condition
RECTYPE?= new:ascii[A];old:split
Below be the index condition file, the available respectively two kinds of different type files of index file represent can, both get one in practice
(1) xml pattern
(2) cfg pattern
Attention: wherein, must have libxml to resolve the storehouse simultaneously if use xml pattern should add grand-D__USE_XML__ in makefile.[CONFIG] middle IDXCTRL changes corresponding xml file in the configuration file
Below be xml schema file and cfg schema file
The XML file:
＜DUP〉＜!--pick and reshuffle--〉
＜REC_CFG〉＜!--one pick reshuffle--〉
＜REC_FIELD〉[A]＜/REC_FIELD〉＜!--file-level is picked and is reshuffled--〉
＜FILE_HEAD〉y, a＜/FILE_HEAD〉＜!--configuration of file reputation--〉
＜CON_FIELD〉[substr (svcName, 1,2)=00] | [svcName=01]＜/CON_FIELD〉＜!--picking heavy record condition configuration--〉
＜MDB_FIELD〉＜!--picking heavy memory database configuration--〉
＜TABLE〉substr (msisdn, 1,7), substr (start_datetime, 1,8)＜/TABLE〉＜!--picking heavily table configuration--〉
＜TABLESPACE〉substr (start_datetime, 7,2)＜/TABLESPACE〉＜!--picking heavy table space configuration--〉
</MDB_FIELD>
＜IND_FIELD〉svcName, msisdn, other_party, start_datetime＜/IND_FIELD〉＜!--picking heavy index field configuration--〉
＜KEY_FIELD〉＜!--picking heavy field configuration--〉
＜FIELD seq='0'fieldName='svcName'startPos='0'length='2'type='0' desc=' subsystem code '/
＜FIELD seq='1'fieldName='msisdn'startPos='40'length='15'type='0 ' desc=' subsystem code '/
＜FIELD seq='2'fieldName='start_datetime'startPos='55'length='14 ' type='0'desc=' subsystem code '/
＜FIELD seq='3'fieldName='other_party'startPos='117'length='24't ype='0'desc=' subsystem code '/
</KEY_FIELD>
</REC_CFG>
＜REC_CFG〉＜!--another picking reshuffled--〉
…
</REC_CFG>
</DUP>
Cfg file:
Second parameter opens the beginning position since 1 among all substr of ##
##[KEY_FIELD] in all fields open the beginning position since 0, separate to begin first field among the split also since 0
## distributes rationally to improve to pick and reshuffles inquiry velocity
Have under the same case among ##1, a plurality of REC_FIELD, picking that the CON_FIELD condition is complicated more reshuffled, and please fits over the back as far as possible
# record level is picked when weighing by following configuration; If picking heavily, file-level changes [REC_FIELD] [VC] into
[REC_FIELD]?[A]
#{ option }
#[FILE_HEAD] y, a
#{ option }
#[CON_FIELD]?[?substr(billing_type,1,2)?=?00?]?|?[?billing_type?=?01?]
# according to field spell table name TABPRE}_{$1}_{$2}, table space must be the parts of field in the table name, substr (since 1), perhaps table space is unworthy of, and gets the default table space
[MDB_FIELD]?TABLE:substr(msisdn,1,7),substr(start_datetime,1,8) TABLESPACE:substr(start_datetime,7,2)
The # field format
[IND_FIELD]?svcName,msisdn,?other_party,?start_datetime
[KEY_FIELD]
The # field name, field location in the separator form (since 0), field starting position (since 0), field length, field type (0:ascii 1:int 2:float), remarks
So # notices that the field starting position is useless among the fedx. get center section content in the field value, after taking-up value is earlier come, substr again
0 svcName 0 2 0?aaaa
1 msisdn 40 15?0?aaaa
2 start_datetime 55 14?0?aaaa
3 other_party 117 24?0?aaaa
The above only is preferred embodiment of the present invention, and is in order to restriction the present invention, within the spirit and principles in the present invention not all, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (6)
1. internal memory removing call ticket repeat method comprises:
Step 1: CDR file is read in internal memory;
Step 2: from described CDR file, read a ticket writing;
Step 3:, find in the internal memory and the corresponding concordance list of described ticket writing according to the key message in the described ticket writing;
Step 4: the field contents in the described ticket writing is combined into a character string, and asks the index of MD5 value as this ticket writing;
Step 5: described MD5 value is inserted in the described concordance list,, then described ticket writing is write in the normal CDR file,, then described ticket writing is write heavy monofile if insert failure if insert successfully;
Step 6: repeating step 2 is to step 5, all ticket writings in having traveled through described CDR file.
2. internal memory removing call ticket repeat method according to claim 1 is characterized in that: described concordance list adopts the T-tree index.
3. internal memory removing call ticket repeat method according to claim 1 is characterized in that: the key message in the described ticket writing is number section and/or date.
4. internal memory removing call ticket repeat method according to claim 1 is characterized in that: a char (32) type-word section is only arranged in the described concordance list, record all the non-heavy unirecord MD5 values in the CDR file.
5. internal memory removing call ticket repeat method according to claim 1 is characterized in that: in the step 3, if do not exist and the corresponding concordance list of described ticket writing in the internal memory, then described concordance list is loaded in the internal memory from its storer of preserving; If all do not exist and the corresponding concordance list of described ticket writing in described internal memory and the storer, then newly-built concordance list conduct and the corresponding concordance list of described ticket writing in internal memory.
6. according to each described internal memory removing call ticket repeat method of claim 1 to 5, it is characterized in that: described method is provided with about ticket writing and handles bar number or the limit value of ticket writing time, when the ticket writing of handling reaches described limit value, the concordance list that is of little use in the internal memory is unloaded, and be kept in the storer of appointment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110097351 CN102156744A (en) | 2011-04-18 | 2011-04-18 | Method for eliminating repetition of memory dialog list |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110097351 CN102156744A (en) | 2011-04-18 | 2011-04-18 | Method for eliminating repetition of memory dialog list |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102156744A true CN102156744A (en) | 2011-08-17 |
Family
ID=44438243
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110097351 Pending CN102156744A (en) | 2011-04-18 | 2011-04-18 | Method for eliminating repetition of memory dialog list |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102156744A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591855A (en) * | 2012-01-13 | 2012-07-18 | 广州从兴电子开发有限公司 | Data identification method and data identification system |
| CN103020131A (en) * | 2012-11-20 | 2013-04-03 | 北京思特奇信息技术股份有限公司 | Method and system for realizing storage and query of communication detailed ticket |
| CN103491518A (en) * | 2013-09-09 | 2014-01-01 | 北京思特奇信息技术股份有限公司 | Method for safely and effectively avoiding malice arrearage of mobile subscriber |
| CN105930396A (en) * | 2016-04-15 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Database based duplicate removal method and system |
| CN106599326A (en) * | 2017-01-23 | 2017-04-26 | 北京思特奇信息技术股份有限公司 | Duplication eliminating method and system for recorded data under cloud architecture |
| CN109582674A (en) * | 2018-11-28 | 2019-04-05 | 亚信科技（南京）有限公司 | A kind of date storage method and system |
| CN109976896A (en) * | 2019-04-09 | 2019-07-05 | 中国联合网络通信集团有限公司 | Business re-scheduling treating method and apparatus |
| CN112069510A (en) * | 2020-07-24 | 2020-12-11 | 北京思特奇信息技术股份有限公司 | Data encryption and de-duplication method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1838124A (en) * | 2006-02-20 | 2006-09-27 | 南京联创科技股份有限公司 | Method for rapidly positioning grid + T tree index in mass data memory database |
| CN101442731A (en) * | 2008-12-12 | 2009-05-27 | 中国移动通信集团安徽有限公司 | Method and apparatus for removing call ticket repeat |
-
2011
- 2011-04-18 CN CN 201110097351 patent/CN102156744A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1838124A (en) * | 2006-02-20 | 2006-09-27 | 南京联创科技股份有限公司 | Method for rapidly positioning grid + T tree index in mass data memory database |
| CN101442731A (en) * | 2008-12-12 | 2009-05-27 | 中国移动通信集团安徽有限公司 | Method and apparatus for removing call ticket repeat |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591855A (en) * | 2012-01-13 | 2012-07-18 | 广州从兴电子开发有限公司 | Data identification method and data identification system |
| CN103020131A (en) * | 2012-11-20 | 2013-04-03 | 北京思特奇信息技术股份有限公司 | Method and system for realizing storage and query of communication detailed ticket |
| CN103020131B (en) * | 2012-11-20 | 2016-08-03 | 北京思特奇信息技术股份有限公司 | A kind of the most single storing and the method and system of inquiry that realize communicating |
| CN103491518A (en) * | 2013-09-09 | 2014-01-01 | 北京思特奇信息技术股份有限公司 | Method for safely and effectively avoiding malice arrearage of mobile subscriber |
| CN103491518B (en) * | 2013-09-09 | 2016-06-15 | 北京思特奇信息技术股份有限公司 | A kind of method of safely and effectively avoiding malice arrearage of mobile subscriber |
| CN105930396B (en) * | 2016-04-15 | 2019-04-09 | 北京思特奇信息技术股份有限公司 | A kind of repetition removing method and system based on database |
| CN105930396A (en) * | 2016-04-15 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Database based duplicate removal method and system |
| CN106599326A (en) * | 2017-01-23 | 2017-04-26 | 北京思特奇信息技术股份有限公司 | Duplication eliminating method and system for recorded data under cloud architecture |
| CN106599326B (en) * | 2017-01-23 | 2020-02-04 | 北京思特奇信息技术股份有限公司 | Recorded data duplication eliminating processing method and system under cloud architecture |
| CN109582674A (en) * | 2018-11-28 | 2019-04-05 | 亚信科技（南京）有限公司 | A kind of date storage method and system |
| CN109582674B (en) * | 2018-11-28 | 2023-12-22 | 亚信科技（南京）有限公司 | Data storage method and system |
| CN109976896A (en) * | 2019-04-09 | 2019-07-05 | 中国联合网络通信集团有限公司 | Business re-scheduling treating method and apparatus |
| CN109976896B (en) * | 2019-04-09 | 2021-06-29 | 中国联合网络通信集团有限公司 | Service re-ranking processing method and device |
| CN112069510A (en) * | 2020-07-24 | 2020-12-11 | 北京思特奇信息技术股份有限公司 | Data encryption and de-duplication method |
| CN112069510B (en) * | 2020-07-24 | 2024-01-30 | 北京思特奇信息技术股份有限公司 | Data encryption and duplication elimination method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102156744A (en) | Method for eliminating repetition of memory dialog list | |
| CN101136783B (en) | Backup, recovery method and apparatus for network management system to configure data | |
| CN102117306B (en) | Method and system for monitoring ETL (extract-transform-load) data processing process | |
| CN1848849B (en) | Method and device for replicating modifications of a directory | |
| CN107391306A (en) | A kind of isomeric data library backup file access pattern method | |
| CN100590620C (en) | System and method for moving records between partitions | |
| JP2003520363A (en) | Data maintenance method in a partially replicated database system network | |
| CN105808653B (en) | A kind of data processing method and device based on user tag system | |
| CN108388604A (en) | User right data administrator, method and computer readable storage medium | |
| US20110167041A1 (en) | Method and device for maintaining a changelog in data synchronization | |
| CN113420026B (en) | Database table structure changing method, device, equipment and storage medium | |
| CN105787058B (en) | A kind of user tag system and the data delivery system based on user tag system | |
| CN101533407A (en) | Method for detecting exceptional data in ETL flow | |
| KR20070039067A (en) | Information service system, information server, portable terminal, information service control program and portable terminal control program | |
| CN102279885A (en) | Method and device for operating data by memory database | |
| CN107491558B (en) | Metadata updating method and device | |
| US8756246B2 (en) | Method and system for caching lexical mappings for RDF data | |
| US20070061092A1 (en) | Generational global name table | |
| CN106802928B (en) | Power grid historical data management method and system | |
| CN107992354A (en) | For reducing the method and device of memory load | |
| CN103778176B (en) | For the system of data long term archival, method and computer usable medium in MES system | |
| CN106844497A (en) | The check device and method of a kind of database code | |
| CN112506869A (en) | File processing method, device and system | |
| CN116701355A (en) | Data view processing method, device, computer equipment and readable storage medium | |
| CN115952146A (en) | File management system applied to key information supervision of direct-current control protection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent for invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 100085 Haidian District, Zhongguancun, South Street, No. 6,, building information, floor, No. 16 Applicant after: SI-TECH Information Technology Ltd. Address before: 100085, Beijing, Haidian District on the nine Street 9 digital science and Technology Plaza, two floor Applicant before: Beijing Digital China SI-TECH Information Technology Co., Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: BEIJING DIGITAL CHINA SI-TECH INFORMATION TECHNOLOGY LTD. TO: BEIJING SI-TECH INFORMATION TECHNOLOGY LTD. |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110817 |