CN109254618A - Time conversion parallel computing method in big data environment - Google Patents
Time conversion parallel computing method in big data environment Download PDFInfo
- Publication number
- CN109254618A CN109254618A CN201810958924.0A CN201810958924A CN109254618A CN 109254618 A CN109254618 A CN 109254618A CN 201810958924 A CN201810958924 A CN 201810958924A CN 109254618 A CN109254618 A CN 109254618A
- Authority
- CN
- China
- Prior art keywords
- time
- seconds
- month
- january
- converted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 27
- 238000004364 calculation method Methods 0.000 title claims abstract description 9
- 230000006870 function Effects 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 6
- 230000003068 static effect Effects 0.000 claims description 3
- 238000000151 deposition Methods 0.000 claims 1
- 238000006467 substitution reaction Methods 0.000 description 2
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/04—Generating or distributing clock signals or signals derived directly therefrom
- G06F1/14—Time supervision arrangements, e.g. real time clock
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
Abstract
The invention provides a time conversion parallel computing method in a big data environment, belonging to the technical field of time conversion, and the method comprises the steps of providing a quick look-up table with the distance between 1 # of each month and 1 # 0/1/1900 year and the time of 0/min/s in a memory in advance, quickly comparing the time to be converted with the quick look-up table to find the time of the month in which the time to be converted is located, and then adding the difference value between the time to be converted and the time of 0/min/0 s in the time of 1 # 0/h/0/h in the month to quickly calculate the time to be converted and the time of 0/min/0 s in the time of 1/0/h/1900 year. The time conversion of the invention can be calculated in parallel, and the concurrency performance can reach hundreds times of the mktime function according to the difference of the CPU core number and the concurrency degree.
Description
Technical field
The present invention relates to time converting parallel calculation methods in time switch technology more particularly to a kind of big data environment.
Background technique
In computer information system, the time almost occurs everywhere, as user log in time, user allow log in when
Between, user operation time, certain data library record insertion time, the founding of People's Republic of China's time, someone go out
Raw time, the time that someone works etc..The expression of time can use text mode such as the " morning 11 on October 1st, 2015
When 12 divide 9 seconds " or " 2015.10.1 11:12:9 " or " 20151001111209 " etc., these times, representation people compared
It is understandable, but be exactly character string in computer disposal, need to occupy the memory space of more than ten to two ten bytes, in big data
Such expression perhaps has several hundred million or even many trillion item in environment, can occupy the memory space of more preciousness, and such expression
Computer is not easy to calculate and influence performance.A mktime function is provided in computer (SuSE) Linux OS, it can be incited somebody to action
The 0 integer value for dividing 0 second number of seconds, the integer when time character string of a byte representation more than ten is converted to distance 1 day 0 January in 1900
Value is 4 bytes or 8 bytes, saves many memory spaces, storage performance is fast and is easy to time calculating.In practical applications
Mktime function generally can meet demand, but data volume much need parallel carry out time conversion big data occasion,
Performance cannot be not only improved when mktime function concurrent invocation, and performance is lower instead, and occupies more CPU, can not meet
High-performance needs.
Summary of the invention
In order to solve the above technical problems, the invention proposes time converting parallel calculating sides in a kind of big data environment
Method solves the problems, such as that time conversion function mktime concurrency performance is low in big data environment, passes through the function pot life of substitution
Conversion performance improves hundreds of times.
The technical scheme is that
Time converting parallel calculation method in a kind of big data environment, it is monthly No. 1 annual by the way that one is provided previously in memory
0 zoom table for dividing 0 second number of seconds, the time for needing to convert and the zoom table quick comparison when distance 1 day 0 January in 1900, finding out needs
The number of seconds in month where the time to be converted, 0 divides the difference of 0 second number of seconds just when then adding this month of time gap No. 10 of conversion
0 divides 0 second number of seconds when can quickly calculate time gap 1 day 0 January in 1900 of conversion.Committed memory of the present invention is few, can be parallel
It calculates, according to the difference of CPU core number and concurrency, concurrency performance can achieve the hundreds of times of mktime function.
There is provided the alternative functions mktime1, mktime1 of a mktime parameter and return value and mktime it is completely simultaneous
Hold, facilitates application call.When mktime1 function is called for the first time, mktime1 function initializes one one in memory
The 0 static zoom table for dividing 0 second number of seconds when annual monthly No. 1 distance 1 day 0 January in 1900 in section time range, it is annual in zoom table
Monthly the 0 mktime function for dividing 0 second number of seconds that can provide by system obtains when No. 1 distance 1 day 0 January in 1900；After
When mktime1 function call, it is only necessary to by the time of conversion and the zoom table quick comparison, find out the time place for needing to convert
The number of seconds in month, 0 divides the difference of 0 second number of seconds that can quickly calculate conversion when then adding this month of time gap No. 10 of conversion
Time gap 1 day 0 January in 1900 when 0 divide 0 second number of seconds, do not recall mktime function.
Zoom table in memory is an array, plays on December 1st, 2199, January monthly 1 from January 1st, 1900
The storage unit of 8 bytes is occupied, the memory space for only needing 28800 bytes is added up to, it is interior in modern computer system
Deposit that occupied space is almost negligible to be disregarded.
When mktime1 function specifically calculates, the month of time to be converted is first judged whether within the scope of zoom table, if do not existed
In range, then former mktime function is called to calculate；The storage unit in month where quickly navigating to if in range, takes out the second
Then number can calculate time to be converted and 1 day 0 January in 1900 plus the difference of time to be converted and place month number of seconds
When 0 divide 0 second number of seconds.The difference of time to be converted and place month number of seconds calculate can with the complete number of days on the date multiplied by 24 again
Multiplied by 3600, in addition conversion day hourage is obtained multiplied by 3600 in addition conversion day the number of minutes multiplied by 60, finally adds conversion day number of seconds
?.
The beneficial effects of the invention are as follows
The mktime versus time concurrency performance for solving the problems, such as that system itself provides is low, and new alternative functions are according to CPU core
Number is different with concurrency, and concurrency performance can achieve the hundreds of times of mktime function, effectively big data is supported to calculate.
Detailed description of the invention
Fig. 1 is time converting parallel calculation method schematic diagram in a kind of big data environment.
Specific embodiment
More detailed elaboration is carried out to the contents of the present invention below:
Time converting parallel calculation method in a kind of big data environment of the invention, is mainly to provide the substitution letter of a mktime
Number mktime1, the parameter and return value of mktime1 is completely compatible with mktime, facilitates application call.In mktime1 letter
When number is called for the first time, mktime1 function initializes annual monthly No. 1 distance 1900 within the scope of an a period of time in memory
The 0 static zoom table for dividing 0 second number of seconds when on January 1,00 divides 0 when annual monthly No. 1 distance 1 day 0 January in 1900 in zoom table
Second number of seconds can pass through the mktime function that system provides and obtain；When later mktime1 function call, it is only necessary to by the time of conversion
With the zoom table quick comparison, the number of seconds in month where finding out the time for needing to convert then should plus the time gap of conversion
00 divides 0 second second when dividing the difference of 0 second number of seconds that can quickly calculate time gap 1 day 0 January in 1900 of conversion when month No. 10
Number, does not recall mktime function.
As shown, the zoom table in memory is an array, on December 1st, 2199 is played from January 1st, 1900, often
The January 1 of the moon occupies the storage unit of 8 bytes, adds up to the memory space for only needing 28800 bytes, in the calculating in modern times
In machine system, memory footprint is almost negligible to be disregarded.
When mktime1 function specifically calculates, the month of time to be converted is first judged whether within the scope of zoom table, if do not existed
In range, then former mktime function is called to calculate；The storage unit in month where quickly navigating to if in range, takes out the second
Then number can calculate time to be converted and 1 day 0 January in 1900 plus the difference of time to be converted and place month number of seconds
When 0 divide 0 second number of seconds.The difference of time to be converted and place month number of seconds calculate can with the complete number of days on the date multiplied by 24 again
Multiplied by 3600, in addition conversion day hourage is obtained multiplied by 3600 in addition conversion day the number of minutes multiplied by 60, finally adds conversion day number of seconds
?.For when 12 days 15 October in 2015 20 divide 10 seconds, difference calculation formula is (12-1) * 24*3600+15*3600+20*
60+10= 1005610。
Claims (4)
1. time converting parallel calculation method in a kind of big data environment, which is characterized in that
0 zoom table for dividing 0 second number of seconds when annual monthly No. 1 distance 1 day 0 January in 1900 being provided previously in memory, need to
The time to be converted compares with the zoom table, finds out the number of seconds in month where needing time for converting, then plus conversion when
Between this month of distance No. 10 when 0 time gap for dividing the difference of 0 second number of seconds to calculate conversion 1 day 0 January in 1900 when 0 divide 0 second second
Number.
2. the method according to claim 1, wherein
There is provided the alternative functions mktime1, mktime1 of a mktime parameter and return value and mktime it is completely compatible；?
When mktime1 function is called for the first time, mktime1 function initializes within the scope of an a period of time annual monthly 1 in memory
The 0 static zoom table for dividing 0 second number of seconds when number distance 1 day 0 January in 1900, annual monthly No. 1 distance January 1 in 1900 in zoom table
The 0 mktime function for dividing 0 second number of seconds that can provide by system obtains when number 0；When later mktime1 function call, it is only necessary to will
The time of conversion and the zoom table quick comparison, the number of seconds in month where finding out the time for needing to convert, then plus conversion
0 when dividing the difference of 0 second number of seconds that can quickly calculate time gap 1 day 0 January in 1900 of conversion at this month of time gap No. 10
0 point of 0 second number of seconds, does not recall mktime function.
3. it is method according to claim 1 or 2, it is characterized in that: the zoom table in memory was an array, from January 1 in 1900
Number on December 1st, 2199 is played, the storage unit for occupying 8 bytes on January 1 monthly is total to need depositing for 28800 bytes
Store up space.
4. method as claimed in claim 3, it is characterized in that: first judging the time to be converted when mktime1 function specifically calculates
Month whether within the scope of zoom table, if not in range, then called former mktime function to calculate；It is quickly fixed if in range
The storage unit in place month is arrived in position, takes out number of seconds, then can count plus time to be converted and the difference of place month number of seconds
It calculates the time to be converted and 0 divides 0 second number of seconds when 1 day 0 January in 1900；The difference meter of time to be converted and place month number of seconds
Calculate can with the complete number of days on the date multiplied by 24 multiplied by 3600, plus day hourage is converted multiplied by 3600, in addition converting bu
Clock number is finally obtained plus conversion day number of seconds multiplied by 60.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810958924.0A CN109254618A (en) | 2018-08-22 | 2018-08-22 | Time conversion parallel computing method in big data environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810958924.0A CN109254618A (en) | 2018-08-22 | 2018-08-22 | Time conversion parallel computing method in big data environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109254618A true CN109254618A (en) | 2019-01-22 |
Family
ID=65049645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810958924.0A Pending CN109254618A (en) | 2018-08-22 | 2018-08-22 | Time conversion parallel computing method in big data environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109254618A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6643664B1 (en) * | 1999-09-17 | 2003-11-04 | International Business Machines Corporation | Data conversions solving time limitations of object-oriented software programs |
| CN101795020A (en) * | 2010-02-08 | 2010-08-04 | 四方华能电网控制系统有限公司 | Automatic time-tick synchronization system of intelligent electronic equipment of transformer substation |
| CN101800643A (en) * | 2009-12-29 | 2010-08-11 | 北京飞天诚信科技有限公司 | Method for processing time information and dynamic token |
| CN102281114A (en) * | 2010-06-10 | 2011-12-14 | 郑州威科姆科技股份有限公司 | Service time leap second processing method |
| CN103399484A (en) * | 2013-07-23 | 2013-11-20 | 深圳市元征科技股份有限公司 | Local clock calibrating method and vehicle-mounted equipment |
| CN104092509A (en) * | 2014-06-13 | 2014-10-08 | 华电众信（北京）技术有限公司 | Time data processing method, device and system |
-
2018
- 2018-08-22 CN CN201810958924.0A patent/CN109254618A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6643664B1 (en) * | 1999-09-17 | 2003-11-04 | International Business Machines Corporation | Data conversions solving time limitations of object-oriented software programs |
| CN101800643A (en) * | 2009-12-29 | 2010-08-11 | 北京飞天诚信科技有限公司 | Method for processing time information and dynamic token |
| CN101795020A (en) * | 2010-02-08 | 2010-08-04 | 四方华能电网控制系统有限公司 | Automatic time-tick synchronization system of intelligent electronic equipment of transformer substation |
| CN102281114A (en) * | 2010-06-10 | 2011-12-14 | 郑州威科姆科技股份有限公司 | Service time leap second processing method |
| CN103399484A (en) * | 2013-07-23 | 2013-11-20 | 深圳市元征科技股份有限公司 | Local clock calibrating method and vehicle-mounted equipment |
| CN104092509A (en) * | 2014-06-13 | 2014-10-08 | 华电众信（北京）技术有限公司 | Time data processing method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8825617B2 (en) | Limiting deduplication based on predetermined criteria | |
| CN106202548B (en) | Date storage method, lookup method and device | |
| JP2017123151A (en) | Reference block aggregating into reference set for deduplication in memory management | |
| CN107577420B (en) | File processing method and device and server | |
| US20120203742A1 (en) | Remote data protection in a networked storage computing environment | |
| US10769114B2 (en) | Database syncing | |
| CN110134668B (en) | Data migration method, device and equipment applied to blockchain | |
| CN109542907A (en) | Database caches construction method, device, computer equipment and storage medium | |
| US20130117274A1 (en) | Electronic device with address book and address book management method | |
| EP3543843B1 (en) | System and method of rolling upgrades of data traits | |
| US10656860B2 (en) | Tape drive library integrated memory deduplication | |
| US20190079914A1 (en) | Reduced memory processing of data files | |
| US20190129992A1 (en) | Data stream processing method and system for processing transactions in a data stream | |
| CN111639132B (en) | Log synchronization method and equipment | |
| US11175993B2 (en) | Managing data storage system | |
| CN115599807A (en) | Data access method, device, application server and storage medium | |
| US8700676B2 (en) | System and method for appending metadata to objects | |
| EP3343395B1 (en) | Data storage method and apparatus for mobile terminal | |
| CN109254618A (en) | Time conversion parallel computing method in big data environment | |
| CN110795920B (en) | Document generation method and device | |
| CN104866431B (en) | A kind of data processing method and device | |
| US20190384825A1 (en) | Method and device for data protection and computer readable storage medium | |
| CN104407946A (en) | Method for conveniently backing up mail to HDFS | |
| CN112632266B (en) | Data writing method and device, computer equipment and readable storage medium | |
| CN113836157A (en) | Method and device for acquiring incremental data of database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |