CN105373623A - Method for quickly generating big data in custom table - Google Patents

Method for quickly generating big data in custom table Download PDF

Info

Publication number
CN105373623A
CN105373623A CN201510901705.5A CN201510901705A CN105373623A CN 105373623 A CN105373623 A CN 105373623A CN 201510901705 A CN201510901705 A CN 201510901705A CN 105373623 A CN105373623 A CN 105373623A
Authority
CN
China
Prior art keywords
memory
data
user
defined data
userinfo
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN201510901705.5A
Other languages
Chinese (zh)
Inventor
张来卿
徐震宇
季统凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
G Cloud Technology Co Ltd
Original Assignee
G Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by G Cloud Technology Co Ltd filed Critical G Cloud Technology Co Ltd
Priority to CN201510901705.5A priority Critical patent/CN105373623A/en
Publication of CN105373623A publication Critical patent/CN105373623A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462Approximate or statistical queries

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of databases, in particular to a method for quickly generating big data in a custom table. The method provided by the invention comprises the following steps: firstly, generating data in a memory table by a function and a memory process according to the characteristic of a high mysql memory table insertion speed; and then inserting the data into the custom table from the memory table, so as to quickly generate the big data in the custom table. According to the method, the problem of quickly generating the big data in the custom table is solved; and the method can be applied to data processing of tables.

Description

A kind of method generating large data fast in user-defined data table
Technical field
The present invention relates to database technical field, particularly a kind of method generating large data fast in user-defined data table.
Background technology
MySQL is a Relational DBMS, and memory table uses Hash hash index to store the data in internal memory, therefore has the speed be exceedingly fast; If the data processing of user-defined format form can be used for, will greatly promote the efficiency of data processing.
Summary of the invention
The technical matters that the present invention solves is to provide a kind of method generating large data fast in user-defined data table, solves the problem generating large data fast in user-defined data table.
The technical scheme that the present invention solves the problems of the technologies described above is:
The feature that described method is fast according to mysql memory table insertion speed, first utilizes function and storing process to generate data in memory table, and then data is inserted user-defined data table from memory table, thus in user-defined data table, generates large data fast.
Described method comprises following step:
Step one, establishment user-defined data table userInfo;
Step 2, create the memory table userInfo_memory identical with user-defined data table;
Step 3, realize in memory table, generate large data function rand_struserid and storing process add_userInfo_memory, and require that userid is 18, be made up of at random letter and number;
Step 4, adjustment System internal memory, arranging memory parameters max_heap_table_size is 5000M;
Step 5, Calling Stored Procedure add_userInfo_memory, the large data required for raw in memory table;
Step 6, employing SQL statement are inserted into data in user-defined data table from memory table.
Beneficial effect of the present invention: the feature fast according to mysql memory table insertion speed, first function and storing process is utilized to generate data in memory table, and then from memory table, data are inserted user-defined data table, thus reach the object generating large data fast in user-defined data table.
Accompanying drawing explanation
Below in conjunction with accompanying drawing, the present invention is further described:
Fig. 1 is method flow diagram of the present invention.
Embodiment
As shown in Figure 1, the present invention adopts following steps, the feature that the present invention is fast according to mysql memory table insertion speed, first function and storing process is utilized to generate data in memory table, and then from memory table, data are inserted user-defined data table, thus reach the object generating large data fast in user-defined data table.
Step one, establishment user-defined data table userInfo; As:
createtable`userinfo`(
`id`int(4)NOTNULLauto_increment,
`userid`varchar(20)NOTNULL,
`username`varchar(20)NOTNULL,
`createtime`datetimeNOTNULL,
PRIMARYKEY(`id`)
)ENGINE=InnoDBDEFAULTCHARSET=utf8;
Step 2, create the memory table identical with user-defined data table;
createtableuserInfo_memory(
`id`INT(11)NOTNULLAUTO_INCREMENT,
`userid`varchar(20)NOTNULL,
`username`varchar(20)NOTNULL,
`createtime`datetimeNOTNULL,
PRIMARYKEY(`id`)
)ENGINE=MEMORYDEFAULTCHARSET=utf8;
Step 3, realize in memory table, generate large data function and storing process, and require that userid is 18, be made up of at random letter and number.
Step 4, adjustment System internal memory, arrange memory parameters max_heap_table_size
The size of internal memory is used to be specified by the max_heap_table_size in My.cnf, setting max_heap_table_size=5000M,
Step 5, Calling Stored Procedure, the large data required for raw in memory table;
calladd_userInfo_memory(999999)
Step 6, employing SQL statement are inserted into data in user-defined data table from memory table.
insertintouserInfoselect*fromuserInfo_memory。

Claims (2)

1. one kind generates the method for large data fast in user-defined data table, it is characterized in that: the feature that described method is fast according to mysql memory table insertion speed, first function and storing process is utilized to generate data in memory table, and then data inserted user-defined data table from memory table, thus in user-defined data table, generate large data fast.
2. a kind of method generating large data fast in user-defined data table according to claim 1, is characterized in that: described method comprises following step:
Step one, establishment user-defined data table userInfo;
Step 2, create the memory table userInfo_memory identical with user-defined data table;
Step 3, realize in memory table, generate large data function rand_struserid and storing process add_userInfo_memory, and require that userid is 18, be made up of at random letter and number;
Step 4, adjustment System internal memory, arranging memory parameters max_heap_table_size is 5000M;
Step 5, Calling Stored Procedure add_userInfo_memory, the large data required for raw in memory table;
Step 6, employing SQL statement are inserted into data in user-defined data table from memory table.
CN201510901705.5A 2015-12-08 2015-12-08 Method for quickly generating big data in custom table Withdrawn CN105373623A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510901705.5A CN105373623A (en) 2015-12-08 2015-12-08 Method for quickly generating big data in custom table

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510901705.5A CN105373623A (en) 2015-12-08 2015-12-08 Method for quickly generating big data in custom table

Publications (1)

Publication Number Publication Date
CN105373623A true CN105373623A (en) 2016-03-02

Family

ID=55375821

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510901705.5A Withdrawn CN105373623A (en) 2015-12-08 2015-12-08 Method for quickly generating big data in custom table

Country Status (1)

Country Link
CN (1) CN105373623A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102937955A (en) * 2011-11-29 2013-02-20 Ut斯达康通讯有限公司 Main memory database achieving method based on My structured query language (SQL) double storage engines
US9152691B2 (en) * 2013-10-06 2015-10-06 Yahoo! Inc. System and method for performing set operations with defined sketch accuracy distribution

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102937955A (en) * 2011-11-29 2013-02-20 Ut斯达康通讯有限公司 Main memory database achieving method based on My structured query language (SQL) double storage engines
US9152691B2 (en) * 2013-10-06 2015-10-06 Yahoo! Inc. System and method for performing set operations with defined sketch accuracy distribution

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王献美等: "可扩展分布式关系型云数据库方案", 《华中科技大学学报(自然科学版)》 *

Similar Documents

Publication Publication Date Title
EP3285178A1 (en) Data query method in crossing-partition database, and crossing-partition query device
US9378233B2 (en) For all entries processing
US9817858B2 (en) Generating hash values
WO2013001535A3 (en) System, method and data structure for fast loading, storing and access to huge data sets in real time
CN103699620A (en) Method and system for achieving database operation by utilizing object relational mapping (ORM) frame in object orientation
MX371406B (en) File service using a shared file access-rest interface.
JP2017518561A5 (en)
US20160147820A1 (en) Variable Sized Database Dictionary Block Encoding
EP3779715A1 (en) Method and apparatus for deleting duplicate data
EP2784699A1 (en) Computer-implemented method for storing unlimited amount of data as a mind map in relational database systems
JP2013531844A5 (en)
WO2011130706A3 (en) Methods and systems for performing cross store joins in a multi-tenant store
US10261950B2 (en) Table as query language parameter
RU2005129003A (en) LONG-TERM STORAGE OF TYPES AND INSTANCES OF .NET DATA
CN102323947A (en) Generation method of pre-join table on ring-shaped schema database
CN103729453A (en) HBase table conjunctive query optimization method
CN105740410A (en) Data statistics method based on Hbase secondary index
GB2547361A (en) System generator module for electronic document and electronic file
CN105938479A (en) Structural transfer method of relational tables and non-relational tables
WO2017079175A3 (en) Data transformation during recycling
CN105373623A (en) Method for quickly generating big data in custom table
CN105786946A (en) Virtualized data integration and query system and implementation method thereof
WO2016169322A1 (en) Query method and device for database, and computer storage medium
WO2023116428A1 (en) Cross-database data migration comparison method and apparatus, and device and storage medium
WO2014193470A8 (en) Method of managing relational data in a single matrix representation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20160302