CN106886568B8 - One kind divides table method, apparatus and electronic equipment - Google Patents
One kind divides table method, apparatus and electronic equipment Download PDFInfo
- Publication number
- CN106886568B8 CN106886568B8 CN201710023995.7A CN201710023995A CN106886568B8 CN 106886568 B8 CN106886568 B8 CN 106886568B8 CN 201710023995 A CN201710023995 A CN 201710023995A CN 106886568 B8 CN106886568 B8 CN 106886568B8
- Authority
- CN
- China
- Prior art keywords
- sub
- open source
- class
- data
- source framework
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 230000006870 function Effects 0.000 claims description 30
- 238000012217 deletion Methods 0.000 claims description 10
- 230000037430 deletion Effects 0.000 claims description 10
- 238000001028 reflection method Methods 0.000 claims description 10
- 238000012986 modification Methods 0.000 claims description 9
- 230000004048 modification Effects 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 229940004975 interceptor Drugs 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
一种分表方法、装置及电子设备A meter distribution method, device and electronic equipment
技术领域technical field
[0001]本发明涉及数据处理技术领域,尤其涉及数据库分表处理技术。[0001] The present invention relates to the technical field of data processing, in particular to database sub-table processing technology.
背景技术Background technique
[0002] Hibernate是一个基于Java的开源的持久化中间件,对JDBC (Java Data Base Connectivity Java 数据库连接)做了轻量的封装。其采用ORM (Object Relation Mapping, 对象关系映射)机制,实现Java对象和关系数据库之间的映射,把sql语句传给数据库,并且 把数据库返回的结果封装成对象。内部封装了 JDBC访问数据库的操作,向上层应用提供了 面向对象的数据库访问API。由于其可以以对象的形式操作数据,更换数据库时只要修改配 值文件,而不用关心数据库种类,因而提高了开发效率。Hibernate is an open-source persistence middleware based on Java, and JDBC (Java Data Base Connectivity Java database connection) has been done lightweight encapsulation. It adopts the ORM (Object Relation Mapping) mechanism to realize the mapping between Java objects and relational databases, transmit SQL statements to the database, and encapsulate the results returned by the database into objects. Internally encapsulates the operation of JDBC to access the database, and provides an object-oriented database access API to upper-level applications. Because it can operate data in the form of objects, it only needs to modify the allocation file when replacing the database, and it does not need to care about the type of database, thus improving the development efficiency.
[0003] 分表是将一张大表切分为同一个数据库的多张表,抵抗因数据量过大而影响性能 的一种方法。发明人在实现本发明的过程中发现,Hi bernat e-Shards开源框架是基于 Hibernate的强大的分库框架,但是不支持分表。Hibernate NamingStrategy是Hibernate 提供的表名替换策略,其可以进行简单的分表操作,但是不能根据实例参数动态决定表名, 一般只能用于按时间进行分表的场景。Hibernate Query Interceptor是Hibernate提供的 查询拦截器,用户可以在生成sql语句时进行拦截,将相应的表名动态替换成具体的分表, 需要解析sql语句,实现复杂,性能和维护性都较差,而且也不可以根据实例参数动态决定 表名,一般只能用户按时间进行分表。Table sub-table is a method of cutting a large table into multiple tables of the same database, and resisting performance impact due to excessive data volume. The inventor found in the process of implementing the present invention that the Hibernate e-Shards open source framework is a powerful sub-database framework based on Hibernate, but does not support sub-tables. Hibernate NamingStrategy is a table name replacement strategy provided by Hibernate. It can perform simple table splitting operations, but it cannot dynamically determine the table name based on instance parameters. Generally, it can only be used in scenarios where tables are split by time. Hibernate Query Interceptor is a query interceptor provided by Hibernate. Users can intercept SQL statements when they are generated, and dynamically replace the corresponding table names with specific sub-tables. SQL statements need to be parsed, which is complicated to implement and poor in performance and maintainability. Moreover, the table name cannot be dynamically determined according to the instance parameters. Generally, the user can only divide the table by time.
[0004] 因此,需要一种基于Hibernate的更加简单有效的分表处理方法。Therefore, need a kind of more simple and effective sub-table processing method based on Hibernate.
发明内容Contents of the invention
[0005] 有鉴于此,本发明实施例提供了一种分表方法、装置及电子设备,至少部分的解决 现有技术中存在的问题。Summary of the invention In view of this, the embodiment of the present invention provides a kind of table sub-method, device and electronic equipment, at least partially solve the problems existing in the prior art.
[0006] 第一方面,本发明实施例提供了 一种分表方法,包括:First aspect, the embodiment of the present invention provides a kind of sub-table method, comprising:
[0007] 动态生成多个子表模型类;A plurality of sub-table model classes are dynamically generated;
[0008] 在开源框架的配置中心注册所述子表模型类;Register described sub-table model class in the configuration center of open source framework;
[0009] 选择所述子表模型类中需要进行数据操作的子表模型类;Select the sub-table model class that needs to carry out data operation in described sub-table model class;
[0010] 利用所述开源框架提供的数据操作功能,对所述需要进行数据操作的子表模型类 的实例执行相应的数据操作。Utilize the data operation function that described open-source framework provides, carry out corresponding data operation to the instance of the sub-table model class that needs to carry out data operation.
[0011] 根据本发明实施例的一种具体实现方式,在所述动态生成多个子表模型类之前, 所述方法还包括:According to a kind of specific implementation of the embodiment of the present invention, before described dynamically generating a plurality of sub-table model classes, described method also includes:
[0012] 在所述开源框架中定义单表模型类。[0012] Define the single-table model class in the open-source framework.
[0013] 根据本发明实施例的一种具体实现方式,所述动态生成多个子表模型类,包括:According to a kind of concrete implementation of the embodiment of the present invention, described dynamic generation multiple sub-table model classes, comprise:
[0014] 获取所述开源框架关联的数据库的分表需求;Obtain the sub-table requirements of the database associated with the open source framework;
[0015] 基于所述分表需求,利用第三方开源类库动态生成多个子表模型类。[0015] Based on the sub-table requirements, a third-party open source class library is used to dynamically generate a plurality of sub-table model classes.
[0016] 根据本发明实施例的一种具体实现方式,所述在开源框架的配置中心注册所述子 表模型类,包括:According to a kind of specific implementation mode of the embodiment of the present invention, described in the configuration center of open source framework, described sub-table model class is registered, comprising:
[0017] 生成所述子表模型类对应的类文件;Generate the class file corresponding to the sub-table model class;
[0018] 将所述类文件保存在所述开源框架的预设扫描位置。[0018] The class file is stored in the preset scanning position of the open source framework.
[0019] 根据本发明实施例的一种具体实现方式,所述在开源框架的配置中心注册所述子 表模型类,包括:According to a kind of specific implementation mode of the embodiment of the present invention, described in the configuration center of open source framework, described sub-table model class is registered, comprising:
[0020] 获取所述开源框架的会话接口文件;Obtain the session interface file of described open source framework;
[0021] 基于所述会话接口文件,判断所述子表模型类是否采用动态方式进行加载;Based on the session interface file, judge whether the sub-table model class is loaded in a dynamic manner;
[0022]当所述子表模型类采用动态方式进行加载时,更新所述开源框架的配置文件,并 重新创建所述开源框架的会话接口文件。When the sub-table model class is loaded in a dynamic manner, update the configuration file of the open source framework, and recreate the session interface file of the open source framework.
[0023] 根据本发明实施例的一种具体实现方式,所述选择所述子表模型类中需要进行数 据操作的子表模型类,包括:According to a kind of specific implementation mode of the embodiment of the present invention, the sub-table model class that needs to carry out data operation in the described sub-table model class of described selection, comprises:
[0024] 基于所述开源框架关联的数据库的分表需求定义散列方法;Based on the sub-table requirement definition hash method of the database associated with the open source framework;
[0025] 获取所述开源框架关联的数据库的预设字段;Obtain the preset field of the database associated with the open source framework;
[0026] 基于所述散列方法对所述预设字段进行数据运算;Carry out data operation to described preset field based on described hash method;
[0027] 根据所述数据运算的结果确定需要进行数据操作的子表模型类。[0027] According to the result of the data operation, the sub-table model class that needs to be operated on is determined.
[0028]根据本发明实施例的一种具体实现方式,所述利用所述开源框架提供的数据操作 功能,对所述需要进行数据操作的子表模型类的实例执行相应的数据操作,包括:According to a kind of specific implementation mode of the embodiment of the present invention, described utilize the data operation function that described open-source framework provides, carry out corresponding data operation to the instance of the subtable model class described needing to carry out data operation, comprising:
[0029] 使用所述开源框架提供的数据增加、数据删除、数据修改及数据查询功能,显式指 定所述需要进行数据操作的子表模型类进行相关数据操作。Use the data increase, data deletion, data modification and data query function provided by the open source framework, explicitly specify the sub-table model classes that need to perform data operations to perform relevant data operations.
[0030] 根据本发明实施例的一种具体实现方式,所述利用所述开源框架提供的数据操作 功能,对所述需要进行数据操作的子表模型类的实例执行相应的数据操作,还包括:According to a kind of concrete implementation mode of the embodiment of the present invention, described utilize the data operation function that described open source framework provides, carry out corresponding data operation to the instance of the sub-table model class that needs to carry out data operation, also include :
[0031] 使用反射方法构建所述需要进行数据操作的子表模型类的模型实例;Use reflection method to build the model instance of the sub-table model class that needs to carry out data operation;
[0032] 使用不包含重复元素的类集对所述模型实例进行数据填充。[0032] The model instance is filled with data using a class set that does not contain repeated elements.
[0033] 第二方面,本发明实施例还提供了 一种分表装置,包括:Second aspect, the embodiment of the present invention also provides a kind of sub-table device, comprising:
[0034] 生成模块,用于动态生成多个子表模型类;Generating module, for dynamically generating a plurality of sub-table model classes;
[0035] 注册模块,用于在开源框架的配置中心注册所述子表模型类;Registration module, for registering described sub-table model class in the configuration center of open source framework;
[0036] 选择模块,用于选择所述子表模型类中需要进行数据操作的子表模型类;Selection module, for selecting the sub-table model class that needs to carry out data operation in described sub-table model class;
[0037] 执行模块,用于利用所述开源框架提供的数据操作功能,对所述需要进行数据操 作的子表模型类的实例执行相应的数据操作。Executing module is used for utilizing the data operation function that described open source framework provides, to the described needing to carry out data operation The example of subtable model class carries out corresponding data operation.
[0038] 根据本发明实施例的一种具体实现方式,所述装置还包括:According to a kind of specific implementation of the embodiment of the present invention, described device also comprises:
[0039] 定义模块,用于在所述开源框架中定义单表模型类。Define module, be used for defining single table model class in described open source framework.
[0040] 根据本发明实施例的一种具体实现方式,所述生成模块,还用于:According to a kind of concrete implementation of the embodiment of the present invention, described generating module is also used for:
[0041] 获取所述开源框架关联的数据库的分表需求;Obtain the sub-table requirements of the database associated with the open source framework;
[0042] 基于所述分表需求,利用第三方开源类库动态生成多个子表模型类。[0042] Based on the sub-table requirements, a third-party open source class library is used to dynamically generate a plurality of sub-table model classes.
[0043] 根据本发明实施例的一种具体实现方式,所述注册模块,还用于:According to a kind of specific implementation mode of the embodiment of the present invention, described registration module is also used for:
[0044] 生成所述子表模型类对应的类文件;Generate the class file corresponding to the sub-table model class;
[0045] 将所述类文件保存在所述开源框架的预设扫描位置。[0045] The class file is stored in the preset scanning position of the open source framework.
[0046] 根据本发明实施例的一种具体实现方式,所述注册模块,还用于:According to a kind of concrete implementation of the embodiment of the present invention, described registration module is also used for:
[0047] 获取所述开源框架的会话接口文件;Obtain the session interface file of described open source framework;
[0048]基于所述会话接口文件,判断所述子表模型类是否采用动态方式进行加载;Based on the session interface file, judge whether the sub-table model class is loaded in a dynamic manner;
[0049]当所述子表模型类采用动态方式进行加载时,更新所述开源框架的配置文件,并 重新创建所述开源框架的会话接口文件。When the sub-table model class is loaded in a dynamic manner, update the configuration file of the open source framework, and recreate the session interface file of the open source framework.
[0050] 根据本发明实施例的一种具体实现方式,所述选择模块,包括:According to a kind of specific implementation of the embodiment of the present invention, described selection module comprises:
[0051] 定义模块,用于基于所述开源框架关联的数据库的分表需求定义散列方法;Definition module, for the sub-table requirement definition hash method based on the database associated with the open source framework;
[0052] 获取模块,用于获取所述开源框架关联的数据库的预设字段;Obtaining module, for obtaining the preset field of the database associated with described open source framework;
[0053]运算模块,用于基于所述散列方法对所述预设字段进行数据运算;Operation module, for carrying out data operation to described preset field based on described hash method;
[0054]确定模块,用于根据所述数据运算的结果确定需要进行数据操作的子表模型类。 [0055]根据本发明实施例的一种具体实现方式,所述执行模块,包括:[0054] Determining module, for determining the sub-table model class that needs to be operated on according to the result of the data operation. [0055] According to a specific implementation of the embodiments of the present invention, the execution module includes:
[0056]指定模块,用于使用所述开源框架提供的数据增加、数据删除、数据修改及数据查 询功能,显式指定所述需要进行数据操作的子表模型类进行相关数据操作。Appointed module, for using the data increase that described open source framework provides, data deletion, data modification and data inquiry function, explicitly designate the sub-table model class that needs to carry out data operation to carry out relevant data operation.
[0057]根据本发明实施例的一种具体实现方式,所述执行模块,还包括:[0057] According to a specific implementation of the embodiments of the present invention, the execution module also includes:
[0058]构建模块,用于使用反射方法构建所述需要进行数据操作的子表模型类的模型实 例;Building block, for using reflection method to construct the model instance of the sub-table model class that needs to carry out data operation;
[0059]填充模块,用于使用不包含重复元素的类集对所述模型实例进行数据填充。[0059] A filling module, configured to fill the model instance with data using a class set that does not contain repeated elements.
[0060] 第三方面,本发明实施例还提供了一种电子设备,该电子设备包括:In a third aspect, the embodiment of the present invention also provides a kind of electronic equipment, and this electronic equipment comprises:
[0061] 至少一个处理器;以及,at least one processor; and,
[0062] 与该至少一个处理器通信连接的存储器;其中,A memory connected in communication with the at least one processor; wherein,
[0063]该存储器存储有可被该至少一个处理器执行的指令,该指令被该至少一个处理器 执行,以使该至少一个处理器能够执行前述任第一方面或第一方面的任一实现方式中的分 表方法。[0063] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform any of the aforementioned first aspects or any implementation of the first aspect The sub-table method in the method.
[0064]第四方面,本发明实施例还提供了一种非暂态计算机可读存储介质,该非暂态计 算机可读存储介质存储计算机指令,该计算机指令用于使该计算机执行前述第一方面或第 一方面的任一实现方式中的分表方法。[0064] In a fourth aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the aforementioned first A table-splitting method in any implementation manner of the aspect or the first aspect.
[0065]第五方面,本发明实施例还提供了一种计算机程序产品,该计算机程序产品包括 存储在非暂态计算机可读存储介质上的计算程序,该计算机程序包括程序指令,当该程序 指令被计算机执行时,使该计算机执行前述第一方面或第一方面的任一实现方式中的分表 方法。In the fifth aspect, the embodiment of the present invention also provides a computer program product, the computer program product includes a computing program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program When the instructions are executed by a computer, the computer is made to execute the table splitting method in the aforementioned first aspect or any implementation manner of the first aspect.
[0066]本发明实施例提供的分表方法、装置、电子设备、非暂态计算机可读存储介质及计 算机程序,通过动态生成多个子表模型类并结合Hibernate的数据操作功能,进而能够在不 明显影响性能和维护性的前提下实现了基于Hibernate框架来实现分表0RM的功能。由于分 表过程中能够根据实例参数动态决定表名,用户仅需制定根据实例参数动态决定表名的规 则即可,故达到了实现方式简单、可理解性强并大幅减少了学习代价和维护成本的技术效 果,总体上提高了分表的效率。The sub-table method, device, electronic equipment, non-transitory computer-readable storage medium and computer program that the embodiment of the present invention provides, by dynamically generating a plurality of sub-table model classes and combining the data operation function of Hibernate, and then being able to Under the premise of obviously affecting the performance and maintainability, the function of realizing sub-table ORM based on the Hibernate framework is realized. Since the table name can be dynamically determined according to the instance parameters during the table splitting process, the user only needs to formulate the rules for dynamically determining the table name according to the instance parameters, so the realization method is simple, the comprehension is strong, and the learning cost and maintenance cost are greatly reduced. The technical effect improves the efficiency of sub-metering on the whole.
附图说明Description of drawings
[0067]为了更清楚地说明本发明实施例的技术方案,下面将对实施例中所需要使用的附 图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域 普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。In order to illustrate the technical scheme of the embodiment of the present invention more clearly, the accompanying drawing that needs to use in the embodiment will be briefly introduced below, obviously, the accompanying drawing in the following description is only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained based on these drawings without creative work.
[0068]图1为本发明实施例提供的一种分表方法的流程示意图;Fig. 1 is the schematic flow chart of a kind of sub-table method that the embodiment of the present invention provides;
[0069]图2为本发明实施例提供的另一种分表方法的流程示意图;Fig. 2 is the schematic flow chart of another kind of sub-table method that the embodiment of the present invention provides;
[0070]图3为本发明实施例提供的一种动态加载子表模型类到Hibernate配置中心的流 程不意图;Fig. 3 is that a kind of dynamic loading child table model class that the embodiment of the present invention provides is to the flow diagram of Hibernate configuration center;
[0071]图4为本发明实施例提供的一种基于散列的子表模型类选择流程示意图;Fig. 4 is a kind of hash-based sub-table model class selection flow schematic diagram that the embodiment of the present invention provides;
[0072]图5为本发明实施例提供的一种对子表模型类的实例执行相应的数据操作的流程 示意图;Fig. 5 is a kind of schematic flow chart that carries out corresponding data operation to the instance of subtable model class that the embodiment of the present invention provides;
[0073]图6为本发明实施例提供的一种分表装置的示意性框图;Fig. 6 is a schematic block diagram of a kind of sub-table device provided by the embodiment of the present invention;
[0074]图7为本发明实施例提供的另一种分表装置的示意性框图;Fig. 7 is a schematic block diagram of another kind of sub-table device provided by the embodiment of the present invention;
[0075]图8为本发明实施例提供的一种选择模块的示意性框图;Fig. 8 is a schematic block diagram of a selection module provided by an embodiment of the present invention;
[0076]图9为本发明实施例提供的一种执行模块的示意性框图;Fig. 9 is a schematic block diagram of an execution module provided by an embodiment of the present invention;
[0077]图10为本发明实施例提供的一种电子设备的结构示意图。[0077] FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
具体实施方式detailed description
[0078]下面结合附图对本发明实施例进行详细描述。[0078] The embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.
[0079]应当明确,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基 于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其 它实施例,都属于本发明保护的范围。[0079] It should be clear that the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0080]图1为本发明实施例提供的一种分表方法,参见图1,该方法包括以下步骤:Fig. 1 is a kind of sub-table method that the embodiment of the present invention provides, referring to Fig. 1, this method comprises the following steps:
[0081] S101,动态生成多个子表模型类。[0081] S101, dynamically generate a plurality of sub-table model classes.
[0082]当数据库中的数据容量发展到一定程度之后,为了提高数据库的响应速度,需要 将整个数据库的单个大表拆分成多个小表。在进行拆分时,可以采用多种预设规则来进行, 比如:After the data capacity in the database develops to a certain extent, in order to improve the response speed of the database, it is necessary to split the single large table of the entire database into multiple small tables. When splitting, a variety of preset rules can be used, such as:
[0083] (1)按自然时间来分表(1) sub-table by natural time
[0084]如一个业务的统计数据量达到一定的规模(如200万条左右),那么就可以考虑用 一年的数据来作为一个表存储,例如,表名为app,那么2010年的数据就是app_2010,app— 2011,. . •。当然也可以米用其他时间方式,如按照月来进彳丁来分,如app_2〇l〇_〇l,app_ 2010—02,…。As the statistical data volume of a business reaches certain scale (as about 2,000,000), then just can consider using the data of one year to store as a table, for example, the name of table is app, then the data in 2010 is app_2010, app— 2011, . . •. Of course, other time methods can also be used, such as dividing according to the month, such as app_2010_01, app_2010-02, ....
[0085] (2)按数字类型hash分表(2) sub-table by digital type hash
[0086]举例而言,如果要存储用户的信息,而业务数据的量很大,用单表是不能满足存储 需求,那么就可以用用户的编号来进行hash,常见的是用取余操作,如果要分30张表来存储 用户的信息,那么用户编号为1的用户1%30 = 1,那么就存在user_01表里,如用户的编号为 500,那么500%30 = 20,将此用户的信息存储在user_2〇的表里。For example, if the information of the user is to be stored, and the amount of business data is very large, the storage requirement cannot be met with a single table, then the user's numbering can be used for hashing, and it is common to use the remainder operation, If you want to store user information in 30 tables, then user 1%30 = 1 with user number 1, then it will be stored in user_01 table, if the user number is 500, then 500%30 = 20, the user's The information is stored in the user_20 table.
[0087] ⑶按md5值来分表(3) sub-table by md5 value
[0088]假设要存储用户上传的文件,如果上传量大的话,也会带来系统的瓶颈问题,比 如,在一个数据库表格下如果超过20000个文件的话,文件的浏览效率会降低。此时可以用 文件的用户名来md5或者用文件的md5校验值来做,可以用!^5的前3位来做hash,这样最多 我们就可以得到16~3 = 40%个表,每次在存储文件的时候,就可以用文件名的md5值的前3 位来确定这个文件该存哪张表。Assuming that the file uploaded by the user is to be stored, if the upload volume is large, the bottleneck problem of the system will also be brought. For example, if there are more than 20,000 files in a database table, the browsing efficiency of the file will be reduced. At this time, you can use the user name of the file to md5 or use the md5 check value of the file to do it. You can use the first 3 digits of !^5 to do the hash, so that we can get 16~3 = 40% of the tables at most, each When storing a file for the first time, you can use the first 3 digits of the md5 value of the file name to determine which table the file should be stored in.
[0089]上述预设规则生成的每一个表均对应一个子表模型类。具体的,可以用ByteBuddy 动态生成多个子表模型类,当然也可以使用cglib或javaassist等其他第三方开源类库,来 动态生成多个子表模型类。这三个类库都是用了 asm这个开源类库,可以动态操作java子节 码来构造动态Java类。[0089] Each table generated by the above preset rules corresponds to a sub-table model class. Specifically, ByteBuddy can be used to dynamically generate multiple sub-table model classes, and of course other third-party open source libraries such as cglib or javaassist can also be used to dynamically generate multiple sub-table model classes. These three class libraries all use the open source class library asm, which can dynamically manipulate java subsection codes to construct dynamic Java classes.
[0090] S102,在开源框架的配置中心注册所述子表模型类。[0090] S102, register the sub-table model class in the configuration center of the open source framework.
[0091]注册到hibernate配置中心可以采用静态方式加载,也可以采用动态方式加载。 [0092] Hibernate通常是在web容器启动的时候根据相关配置信息,扫描Class文件,然后 注册到SessionFactory中,可以操作Java子节码来构造动态Java类,具体为通过编写Java 的汇编指令序列化生成一个类文件,然后使用classloader来加载这个类文件。[0091] Registering to the hibernate configuration center can be loaded in a static mode or in a dynamic mode. Hibernate usually scans the Class file according to the relevant configuration information when the web container starts, and then registers in the SessionFactory, and can operate the Java subsection code to construct the dynamic Java class, which is specifically generated by the assembly instruction serialization of writing Java A class file, and then use classloader to load this class file.
[0093] 对于在容器启动已经完成后,程序正常运行期间产生的类,由于Hibernate不能识 别这个新创建的类,此时不能直接使用Hibernate的接口。为此可以采用动态注册的方式来 进行。考虑到安全性问题,不再使用原来已经存在的SessionFactory,而是获取 SessionFactory,判断子表模型类是否动态加载到JVM中,如果是,则更新当前的 Configuration,并重新创建一个 SessionFactory。After the container start has been completed, the class produced during the normal operation of the program, because Hibernate cannot recognize this newly created class, the interface of Hibernate cannot be directly used this moment. This can be done by means of dynamic registration. Considering security issues, instead of using the existing SessionFactory, obtain the SessionFactory to determine whether the subtable model class is dynamically loaded into the JVM. If so, update the current Configuration and recreate a SessionFactory.
[0094] S103,选择所述子表模型类中需要进行数据操作的子表模型类。[0094] S103. Select the sub-table model class that needs to be operated on in the sub-table model class.
[0095]定义散列方法,根据特定的散列参数选择相应的子表模型类,该散列参数可以是 模型主键Id,也可以是模型的其他字段,还可以是时间参数等。Define hash method, select corresponding sub-table model class according to specific hash parameter, this hash parameter can be Model primary key Id, also can be other fields of model, can also be time parameter etc.
[0096] 举例而言,比如对于记录用户的所有购买记录的数据库,如果将所有记录放入到 一张表userbuy表中,势必会非常巨大。所以可以通过对用户id进行分表处理,将不同用户 的购买记录放入不同的分表中。具体做法如下:For example, such as for the database of all purchase records of record user, if all records are put into a table userbuy table, certainly will be very huge. Therefore, the purchase records of different users can be put into different sub-tables by sub-table processing of user id. The specific method is as follows:
[0097] 预估业务量,将最初的userbuy表,扩展为100张相同表结构的userbuy_index表, 原表结构如下:Estimate business volume, with initial userbuy table, be expanded into the userbuy_index table of 100 identical table structures, former table structure is as follows:
[0098] 表一 userbuy表数据结构Table one userbuy table data structure
[0099] _ 字段 |类型 | Null |健值 |默认 id bigint (20) unsigned NO PRI NULL uid int (10)unsigned NO 0 goods varchar (128) NO num int (10)unsigned NO 0 ctime int (10)unsigned NO 0_field | type | Null | healthy value | default id bigint (20) unsigned NO PRI NULL uid int (10) unsigned NO 0 goods varchar (128) NO num int (10) unsigned NO 0 ctime int (10) unsigned NO 0
[0l00]所有的分表采用完全相同的表结构,当需要新增一条分表记录时,过程如下:首先 对用户id进行hash计算,得到hash值,该hash值必然是0-99,目的是将对应的用户id数据, 写入到同一张分表中,此时,基于该hash值便可以选取子表模型类。[0100] All sub-tables adopt exactly the same table structure. When a new sub-table record needs to be added, the process is as follows: first, the user id is hash calculated to obtain a hash value. This hash value must be 0-99, and the purpose is to Write the corresponding user id data into the same sub-table. At this time, the sub-table model class can be selected based on the hash value.
[0101] S104,利用所述开源框架提供的数据操作功能,对所述需要进行数据操作的子表 模型类的实例执行相应的数据操作。S104, utilize the data operation function that described open-source framework provides, carry out corresponding data operation to the instance of the sub-table model class that needs to carry out data operation.
[0102]使用原生的Hibernate数据增加、数据删除、数据修改、及数据查询等数据操作功 能,显式指定具体子表模型类进行相关数据操作。通过显示指定分表,使用者可以明确指定 是要向哪个分表发起相关数据操作,使用者可以清晰理解数据的物理数据结构。Use original Hibernate data to increase, data deletion, data modification, and data query and other data manipulation functions, explicitly designate specific sub-table model classes to carry out relevant data manipulations. By displaying the specified sub-table, the user can clearly specify which sub-table to initiate related data operations, and the user can clearly understand the physical data structure of the data.
[0103]具体而言,通过构造子表模型类的实例,并对该子表模型类的实例进行数据操作 来来实现数据操作功能。构造模型类的实例时,先根据散列参数选择相应的子表模型类,再 使用反射方法Class.newlnstance 〇构造对应子类的模型实例,最后再使用set方法填充 对象。Specifically, by constructing the instance of the sub-table model class, and performing data manipulation on the instance of the sub-table model class to realize the data operation function. When constructing an instance of a model class, first select the corresponding subtable model class according to the hash parameters, then use the reflection method Class.newlnstance 〇 to construct the model instance of the corresponding subclass, and finally use the set method to fill the object.
[0104]在实现分表的过程中,除了上述实施例中的步骤S101〜S104之外,可选的,参见图 2,该分表方法还可以包括如下步骤:In the process of realizing sub-table, except steps S101~S104 in the above-mentioned embodiment, optional, referring to Fig. 2, this sub-table method can also comprise the following steps:
[0105] S201,在所述开源框架中定义单表模型类。[0105] S201, define a single table model class in the open source framework.
[0106]具体的,可以在Hibernate的配置文件hibernate .cfg. xml中添加如下代码:Concrete, can add following code in the configuration file hibernate.cfg.xml of Hibernate:
[0107] <properties><properties>
[0108] 〈property name = ’’hibernate. hbm2ddl • auto〃value = 〃create〃/><property name=''hibernate.hbm2ddl • auto"value="create"/>
[0109] </properties〉[0109] </properties>
[0110]通过在开源框架中定义单表模型类,能够在步骤Sl〇l中动态生成的多个子表模型 类时,复用步骤S2〇l中定义的单表模型类代码,在简化代码编写的同时,更便于对代码进行 维护。By defining the single-table model class in the open source framework, when a plurality of sub-table model classes dynamically generated in step S101, the single-table model class code defined in step S201 can be reused, writing in simplified code At the same time, it is easier to maintain the code.
[0111] 本实施例中的分表方法,实现简单,使用简单,有很强的可理解性,大幅减少了学 习代价和维护成本,性能接近原生的0RM单表操作。The sub-table method in the present embodiment is simple to implement, easy to use, has strong comprehensibility, greatly reduces the learning cost and maintenance cost, and the performance is close to the original ORM single-table operation.
[0112]可选的,本发明实施例还提供了 一种动态加载子表模型类到Hibernate配置中心 的方法,参见图3,该方法包括如下步骤:Optionally, the embodiment of the present invention also provides a kind of method of dynamically loading subtable model class to Hibernate configuration center, referring to Fig. 3, the method comprises the steps:
[0113] S301,获取所述开源框架的会话接口文件。[0113] S301. Obtain the session interface file of the open source framework.
[0m] 对于Hibernate来讲,需要获取SessionFactory接口,具体的,可以通过 Configuation 实例创建 SessionFactory,如下所示:[0m] For Hibernate, you need to obtain the SessionFactory interface. Specifically, you can create a SessionFactory through a Configuation instance, as shown below:
[0115] Configuration config=new Configuration () .configure ();[0115] Configuration config=new Configuration (). configure ();
[0116] SessionFactory SessionFactory二config.buildSessionFactory 〇 ;[0116] SessionFactory SessionFactory two config.buildSessionFactory 〇;
[0117] Configuration实例config会根据当前的配置信息,构造SessionFactory实例并 返回。Configuration instance config can construct SessionFactory instance and return according to current configuration information.
[0118] S302,基于所述会话接口文件,判断所述子表模型类是否采用动态方式进行加载。[0118] S302. Based on the session interface file, judge whether the sub-table model class is loaded in a dynamic manner.
[0119]具体而言,可以通过 SessionFactory 的getAllClassMetadata 方法得到一个 Map, Map的key是实体类的名称,value是ClassMetadata对象,通过判断map .keySet ()中是否含 有所述子表模型类的方式来判断所述子表模型类是否采用动态方式进行加载。比如:[0119] Specifically, a Map can be obtained through the getAllClassMetadata method of the SessionFactory, the key of the Map is the name of the entity class, and the value is the ClassMetadata object, by judging whether the map.keySet () contains the mode of the sub-table model class It is judged whether the sub-table model class is loaded in a dynamic manner. for example:
[0120] M^<String,CI_Metadata>maft.sesSionFacto_ge^^lassM_a_ta0; Set<String> set = map.keySet(); if(!set.contams(entityClass.getName〇)){ }[0120] M^<String, CI_Metadata>maft.sesSionFacto_ge^^lassM_a_ta0; Set<String> set = map.keySet(); if(!set.contams(entityClass.getName〇)){ }
[0121] S303,当所述子表模型类采用动态方式进行加载时,更新所述开源框架的配置文 件,并重新创建所述开源框架的会话接口文件。S303, when the sub-table model class is loaded in a dynamic manner, update the configuration file of the open source framework, and recreate the session interface file of the open source framework.
[0122]具体的,可以通过设置函数configuration.addAnnotatedClass ()更新ffibernate 的配置文件,以及通过设置函数sessionFactoryList.add ()重新创建Hibernate的 SessionFactory〇Concrete, can update the configuration file of Wibernate by setting function configuration.addAnnotatedClass (), and recreate the SessionFactory of Hibernate by setting function sessionFactoryList.add ().
[0123]通过上述方式,能够通过动态方式加载子表模型类至Hibernate的配置中心。[0123] Through the above method, the sub-table model class can be dynamically loaded to the configuration center of Hibernate.
[0124]参见图4,本发明实施例还提供了一种基于散列的子表模型类选择方法,包括如下 步骤:Referring to Fig. 4, the embodiment of the present invention also provides a kind of subtable model class selection method based on hash, comprises the following steps:
[0125] S401,基于所述开源框架关联的数据库的分表需求定义散列方法。[0125] S401, define a hash method based on the sub-table requirements of the database associated with the open source framework.
[0126]根据具体的分表需求,定义散列方法,以表一所示的数据库表单结构为例,要把数 据库的表分成100个子表,此时,可以选择使用Crc32方式进行hash计算。According to concrete sub-table requirement, define hashing method, take the database form structure shown in Table 1 as example, the table of database will be divided into 100 sub-tables, at this moment, can choose to use Crc32 mode to carry out hash calculation.
[0127] S402,获取所述开源框架关联的数据库的预设字段。[0127] S402. Acquire the preset fields of the database associated with the open source framework.
[0128]具体而言,可以采取表一中的任一字段进行hash计算,比如可以选择id作为预设 字段。Specifically, any field in Table 1 can be adopted to carry out hash calculation, for example, id can be selected as a preset field.
[0129] S4〇3,基于所述散列方法对所述预设字段进行数据运算。[0129] S403. Perform data operations on the preset field based on the hash method.
[0130] crc32计算结果为一个10进制数,在不同bit位的操作系统下,得到的值可能会不 同,所以对其进行一个统一的处理,然后对这个自然数进行取余计算,取余的除数就是分表 数,比如这里为100。The crc32 calculation result is a decimal number, under the operating system of different bit positions, the value obtained may be different, so it is carried out a unified process, then this natural number is carried out to get remainder calculation, take remainder The divisor is the sub-table number, for example, 100 here.
[0131] S404,根据所述数据运算的结果确定需要进彳丁数据操作的子表模型类。[0131] S404, determine the sub-table model class that needs to be operated on according to the result of the data operation.
[0132]通过计算数据库的预设字段,这样就得到了 一个在〇-99以内的表后缀数index,通 过将index数值与子表模型类进行关联,即可确定需要进行数据操作的子表模型类。By calculating the preset field of the database, a table suffix number index within 0-99 has been obtained, and by associating the index value with the sub-table model class, the sub-table model that needs to be operated on can be determined kind.
[0133] 通过上述实施例中的方法,能够将单个表较为均匀分配到各个子表中。[0133] Through the method in the above-mentioned embodiment, a single table can be more evenly distributed to each sub-table.
[0134]参见图5,本发明实施例还提供了一种对子表模型类的实例执行相应的数据操作 的方法,包括如下步骤:Referring to Fig. 5, the embodiment of the present invention also provides a kind of method that carries out corresponding data operation to the instance of subtable model class, comprises the steps:
[0135] S501,使用所述开源框架提供的数据增加、数据删除、数据修改及数据查询功能, 显式指定所述需要进行数据操作的子表模型类进行相关数据操作。[0135] S501, use the data addition, data deletion, data modification and data query functions provided by the open source framework to explicitly designate the sub-table model classes that require data operations to perform related data operations.
[0136]使用原生的Hibernate数据增加、数据删除、数据修改、及数据查询等数据操作功 能,显式指定具体子表模型类进行相关数据操作。通过显示指定分表,使用者可以明确指定 是要向哪个分表发起相关数据操作,使用者可以清晰理解数据的物理数据结构。Use native Hibernate data to increase, data deletion, data modification, and data query and other data manipulation functions, explicitly designate specific sub-table model classes to carry out relevant data manipulations. By displaying the specified sub-table, the user can clearly specify which sub-table to initiate related data operations, and the user can clearly understand the physical data structure of the data.
[0137] 以子表模型类的删除操作为例,可以采用如下代码进行:Taking the deletion operation of the sub-table model class as an example, the following code can be adopted to carry out:
[0138] tx = session.beginTransaction ();[0138] tx=session.beginTransaction();
[0139] user= (UserEntity) session.get (UserEntity.shardFor (”2〃),〃2");[0139] user=(UserEntity) session.get(UserEntity.shardFor("2"), "2");
[0140] System.out.println (user.getName());[0140] System.out.println(user.getName());
[0141] session.delete (user);[0141] session.delete(user);
[0142] tx.commit ();[0142] tx.commit();
[0143] S502,使用反射方法构建所述需要进行数据操作的子表模型类的模型实例。[0143] S502, using a reflection method to construct a model instance of the sub-table model class that requires data manipulation.
[0144]具体的,先根据散列参数选择相应的子表模型类,再使用反射方法 Class • newlnstance ()构造对应子类的模型实例。[0144] Specifically, first select the corresponding sub-table model class according to the hash parameter, and then use the reflection method Class • newlnstance () to construct the model instance of the corresponding sub-class.
[0145] S503,使用不包含重复元素的类集对所述模型实例进行数据填充。[0145] S503. Fill the model instance with data using a cluster that does not contain repeated elements.
[0146]具体的,可以采用使用set方法对所述模型实例进行数据填充,比如:Concrete, can adopt and use set method to carry out data filling to described model instance, such as:
[0147] UserEntityuser= (UserEntity) session• get (UserEntity • shardFor (〃2〃)," 2");[0147] UserEntityuser=(UserEntity) session·get(UserEntity·shardFor("2")," 2");
[0148] user•setName (〃user—modified")[0148] user setName ("user_modified")
[0149] 通过本实施例中的方法,能够通过显式方式指定所述需要进行数据操作的子表模 型类进行相关数据操作。By the method in this embodiment, can specify the sub-table model class that needs to carry out data operation by explicit mode and carry out relevant data operation.
[0150]图6为本发明实施例提供的一种分表装置,参见图6,该装置包括:Fig. 6 is a kind of sub-table device provided by the embodiment of the present invention, referring to Fig. 6, the device includes:
[0151]生成模块601,用于动态生成多个子表模型类。[0151] The generation module 601 is used to dynamically generate multiple sub-table model classes.
[0152]当数据库中的数据容量发展到一定程度之后,为了提高数据库的响应速度,可以 将整个数据库的单个大表拆分成多个小表。在进行拆分时,可以采用多种预设规则来进行, 比如:After the data capacity in the database develops to a certain extent, in order to improve the response speed of the database, the single large table of the entire database can be split into multiple small tables. When splitting, a variety of preset rules can be used, such as:
[0153] (1)按自然时间来分表(1) sub-table by natural time
[0154]如一个业务的统计数据量达到一定的规模(如2〇0万条左右),那么就可以考虑用 一年的数据来作为一个表存储,例如,表名为app,那么2〇10年的数据就是app_2010,app_ 2011,• • •。当然也可以采用其他时间方式,如按照月来进行来分,如app_2〇l〇_〇l,app_ 2010—02,___〇[0154] If the amount of statistical data of a business reaches a certain scale (such as about 2,000,000), then one year's data can be considered to be stored as a table, for example, the name of the table is app, then 2010 The data of the year is app_2010, app_ 2011, • • •. Of course, other time methods can also be used, such as dividing according to the month, such as app_2〇1〇_〇l, app_2010-02, ___〇
[0155] (2)按数字类型hash分表(2) sub-table by digital type hash
[0156]举例而言,如果要存储用户的信息,而业务数据的量很大,用单表是不能满足存储 需求,那么就可以用用户的编号来进行hash,常见的是用取余操作,如果要分30张表来存储 用户的信息,那么用户编号为1的用户1%30 = 1,那么就存在user_01表里,如用户的编号为 500,那么500 % 30 = 20,将此用户的信息存储在user_20的表里。For example, if the user's information is to be stored, and the amount of business data is very large, the storage requirement cannot be met with a single table, then the user's numbering can be used for hashing, and it is common to use the remainder operation, If you want to store user information in 30 tables, then user 1%30 = 1 with user number 1, then it will be stored in the user_01 table. If the user number is 500, then 500% 30 = 20, the user's The information is stored in the table user_20.
[0157] (3)按md5值来分表(3) divide table by md5 value
[0158] 假设要存储用户上传的文件,如果上传量大的话,也会带来系统的瓶颈问题,比 如,在一个数据库表格下如果超过20000个文件的话,文件的浏览效率会降低。此时可以用 文件的用户名来md5或者用文件的md5校验值来做,可以用md5的前3位来做hash,这样最多 我们就可以得到16~3 = 4〇96个表,每次在存储文件的时候,就可以用文件名的md5值的前3 位来确定这个文件该存哪张表。Assuming that the files uploaded by the user are to be stored, if the upload volume is large, it will also bring about a bottleneck problem in the system. For example, if there are more than 20,000 files in a database table, the browsing efficiency of the files will be reduced. At this time, you can use the user name of the file to md5 or use the md5 check value of the file to do it. You can use the first 3 digits of md5 to do the hash, so that we can get 16~3 = 4096 tables at most, each time When storing a file, you can use the first 3 digits of the md5 value of the file name to determine which table the file should be stored in.
[0159]上述预设规则生成的每一个表均对应一个子表模型类。具体的,可以用ByteBuddy 动态生成多个子表模型类,当然也可以使用cglib和javaassist第三方开源类库,来动态生 成多个子表模型类。这三个类库都是用了 asm这个开源类库,它可以动态操作java子节码来 构造动态Java类[0159] Each table generated by the above preset rules corresponds to a sub-table model class. Specifically, you can use ByteBuddy to dynamically generate multiple sub-table model classes, and of course you can also use cglib and javaassist third-party open source class libraries to dynamically generate multiple sub-table model classes. These three class libraries all use the open source class library asm, which can dynamically manipulate java subsection codes to construct dynamic Java classes
[0160]注册模块602,用于在开源框架的配置中心注册所述子表模型类。[0160] The registration module 602 is configured to register the sub-table model class in the configuration center of the open source framework.
[0161]注册到hibernate配置中心可以采用静态方式加载,也可以采用动态方式加载。 [0162] Hibernate通常是在web容器启动的时候根据相关配置信息,扫描Class文件,然后 注册到SessionFactory中,可以操作Java子节码来构造动态Java类,具体为通过编写java 的汇编指令序列化生成一个类文件,然后使用class loader来加载这个类文件。[0161] Registering to the hibernate configuration center can be loaded in a static mode or in a dynamic mode. Hibernate usually scans the Class file according to the relevant configuration information when the web container starts, and then registers in the SessionFactory, and can operate the Java subsection code to construct the dynamic Java class, which is specifically generated by serializing the assembly instructions of writing java A class file, and then use the class loader to load this class file.
[0163]对于在容器启动已经完成后,程序正常运行期间产生的类,由于h i bernate不能识 别这个新创建的类,此时不能直接使用hibernate的接口。此时可以采用动态注册的方式来 进行。考虑到安全性问题,此时不再使用原来的SessionFactory 了,而是获取 SessionFactory,判断子表模型类是否动态加载到JVM中,如果是,则更新当前的 Configuration,并重亲jf 创建一个 SessionFactory。After the container startup has been completed, the class produced during the normal operation of the program, because hibernate cannot recognize this newly created class, the interface of hibernate cannot be directly used this moment. At this time, dynamic registration can be used. Considering security issues, the original SessionFactory is no longer used at this time, but the SessionFactory is obtained to determine whether the sub-table model class is dynamically loaded into the JVM. If so, update the current Configuration and recreate a SessionFactory with jf.
[0164]选择模块㈤3,用于选择所述子表模型类中需要进行数据操作的子表模型类。[0164] The selection module (5) is used to select the sub-table model classes that need to perform data operations in the sub-table model classes.
[0165] 定义散列方法,根据特定的散列参数选择相应的子表模型类,该散列参数可以是 模型主键Id,也可以是模型的其他字段,还可以是时间参数等。Define hash method, select corresponding sub-table model class according to specific hash parameter, this hash parameter can be model primary key Id, also can be other fields of model, can also be time parameter etc.
[0166] 举例而言,比如对于记录用户的所有购买记录的数据库,如果将所有记录放入到 一张表userbuy表中,势必会非常巨大。所以可以通过对用户id进行分表处理,将不同用户 的购买记录放入不同的分表中。具体做法如下:For example, such as for the database of all purchase records of record user, if all records are put into a table userbuy table, certainly will be very huge. Therefore, the purchase records of different users can be put into different sub-tables by sub-table processing of user id. The specific method is as follows:
[0167] 预估业务量,将最初的userbuy表,扩展为100张相同表结构的userbuy_index表, 原表结构如表一所不:Estimate business volume, with initial userbuy table, be extended to the userbuy_index table of 100 identical table structures, former table structure is as shown in Table 1:
[0168] 所有的分表采用完全相同的表结构,当需要新增一条分表记录时,过程如下:首先 对用户id进行hash计算,得到hash值,该hash值必然是0-99,目的是将对应的用户id数据, 写入到同一张分表中,此时,基于该hash值便可以选取子表模型类。All sub-tables adopt identical table structure, when needing to add a sub-table record, process is as follows: first user id is carried out hash calculation, obtains hash value, and this hash value must be 0-99, purpose is Write the corresponding user id data into the same sub-table. At this time, the sub-table model class can be selected based on the hash value.
[0169] 执行模块604,用于利用所述开源框架提供的数据操作功能,对所述需要进行数据 操作的子表模型类的实例执行相应的数据操作。Execution module 604 is used for utilizing the data operation function provided by the open source framework to perform corresponding data operations on the instance of the sub-table model class that requires data operations.
[0170] 使用原生的Hibernate数据增加、数据删除、数据修改、及数据查询等数据操作功 能,显式指定具体子表模型类进行相关数据操作。通过显示指定分表,使用者可以明确指定 是要向哪个分表发起相关数据操作,使用者可以清晰理解数据的物理数据结构。Use original Hibernate data increase, data deletion, data modification, and data query and other data manipulation functions to explicitly specify specific sub-table model classes to perform related data manipulations. By displaying the specified sub-table, the user can clearly specify which sub-table to initiate related data operations, and the user can clearly understand the physical data structure of the data.
[0171] 具体而言,通过构造子表模型类的实例,并对该子表模型类的实例进行数据操作 来来实现数据操作功能。构造模型类的实例时,先根据散列参数选择相应的子表模型类,再 使用反射方法Class.newlnstance ()构造对应子类的模型实例,最后再使用set方法填充 对象。之所以使用反射方法,因为相应的子表模型类是动态定义的。Specifically, by constructing the instance of the sub-table model class, and performing data manipulation on the instance of the sub-table model class to realize the data operation function. When constructing an instance of a model class, first select the corresponding subtable model class according to the hash parameter, then use the reflection method Class.newlnstance () to construct a model instance of the corresponding subclass, and finally use the set method to fill the object. The reflection method is used because the corresponding child table model classes are defined dynamically.
[0172] 上述分表装置60,除了包括上述模块之外,可选的,参见图7,分表装置60还可以包 括:Above-mentioned sub-table device 60, except comprising above-mentioned module, optional, referring to Fig. 7, sub-table device 60 can also comprise:
[0173] 定义模块701,用于在所述开源框架中定义单表模型类。[0173] Definition module 701 is used to define the single form model class in the open source framework.
[0174] 具体的,可以在Hibernate的配置文件hibernate • cfg • xml中添加如下代码:Concrete, can add following code in the configuration file hibernate·cfg·xml of Hibernate:
[0175] <properties><properties>
[0176] 〈property name = "hibernate • hbm2ddl • auto"value = "create〃/><property name="hibernate • hbm2ddl • auto" value="create"/>
[0177] </properties>[0177] </properties>
[0178] 通过在开源框架中定义单表模型类,能够在步骤S101中动态生成的多个子表模型 类时,复用步骤S2〇l中定义的单表模型类代码,在简化代码编写的同时,更便于对代码进行 维护。By defining the single-table model class in the open source framework, when a plurality of sub-table model classes dynamically generated in step S101, the single-table model class code defined in step S201 can be reused, while simplifying code writing , making it easier to maintain the code.
[0179]本实施例中的分表装置,实现简单,使用简单,有很强的可理解性,大幅减少了学 习代价和维护成本,性能接近原生的0RM单表操作。The sub-table device in this embodiment is simple to implement, easy to use, has strong comprehensibility, greatly reduces learning costs and maintenance costs, and has performance close to the original ORM single-table operation.
[0180] 可选的,参见图8,选择模块603,可以包括:Optionally, referring to Fig. 8, selection module 603 may include:
[0181]定义模块8〇1,用于基于所述开源框架关联的数据库的分表需求定义散列方法; [0182]获取模块802,用于获取所述开源框架关联的数据库的预设字段;[0181] The definition module 801 is used to define a hash method based on the sub-table requirements of the database associated with the open source framework; [0182] The acquisition module 802 is used to obtain the preset fields of the database associated with the open source framework;
[0183]运算模块803,用于基于所述散列方法对所述预设字段进行数据运算;Operation module 803, for carrying out data operation to described preset field based on described hash method;
[0184]确定模块804,用于根据所述数据运算的结果确定需要进行数据操作的子表模型 类。Determining module 804, is used for determining the subtable model class that needs to carry out data operation according to the result of described data operation.
[0185] 可选的,参见图9,执行模块604,可以包括:Optionally, referring to Fig. 9, execution module 604 may include:
[0186]指定模块901,用于使用所述开源框架提供的数据增加、数据删除、数据修改及数 据查询功能,显式指定所述需要进行数据操作的子表模型类进行相关数据操作;Appointment module 901, for using the data increase that described open source framework provides, data deletion, data modification and data query function, explicitly designate the sub-table model class that needs to carry out data operation to carry out relevant data operation;
[0187]构建模块902,用于使用反射方法构建所述需要进行数据操作的子表模型类的模 型实例;Construction module 902, for using reflection method to construct the model instance of the sub-table model class that needs to carry out data operation;
[0188]填充模块903,用于使用不包含重复元素的类集对所述模型实例进行数据填充。 [0189]图8-9实施例中各模块的功能与其对应的方法实施例中的内容相对应,在此不再 赘述。[0188] The filling module 903 is configured to fill the model instance with data using a class set that does not contain repeated elements. The function of each module in the embodiment of Fig. 8-9 is corresponding to the content in its corresponding method embodiment, repeats no more here.
[0190]图10示出了本发明实施例提供的电子设备100的结构示意图,电子设备1〇〇包括至 少一个处理器1001 (例如CPU),至少一个输入输出接口 1〇〇4,存储器1002,和至少一个通信 总线1003,用于实现这些部件之间的连接通信。至少一个处理器1〇01用于执行存储器10〇2 中存储的计算机指令,以使所述至少一个处理器1001能够执行前述任一分表方法的实施 例。存储器1〇〇2为非暂态存储器(non-transitory memory),其可以包含易失性存储器,例 如高速随机存取存储器(RAM:Random Access Memory),也可以包括非易失性存储器(nonvolatile memory) ,例如至少一个磁盘存储器。通过至少一个输入输出接口 i〇〇4(可以是有 线或者无线通信接口)实现与至少一个其他设备或单元之间的通信连接。[0190] FIG. 10 shows a schematic structural diagram of an electronic device 100 provided by an embodiment of the present invention. The electronic device 100 includes at least one processor 1001 (such as a CPU), at least one input and output interface 1004, and a memory 1002. And at least one communication bus 1003, used to realize connection and communication between these components. At least one processor 1001 is configured to execute the computer instructions stored in the memory 1002, so that the at least one processor 1001 can execute any of the aforementioned embodiments of the table splitting method. The memory 1002 is a non-transitory memory (non-transitory memory), which may include a volatile memory, such as a high-speed random access memory (RAM: Random Access Memory), and may also include a nonvolatile memory (nonvolatile memory ), such as at least one disk storage. The communication connection with at least one other device or unit is realized through at least one input and output interface 1004 (which may be a wired or wireless communication interface).
[0191] 在一些实施方式中,存储器1002存储了程序10021,处理器1001执行程序10021,用 于执行前述任一分表方法实施例中的内容。In some implementations, the memory 1002 stores the program 10021, and the processor 1001 executes the program 10021, which is used to execute the content in any of the aforementioned embodiments of the sub-table method.
[0192] 该电子设备可以以多种形式存在,包括但不限于:[0192] The electronic device may exist in a variety of forms, including but not limited to:
[0193] (1)移动通信设备:这类设备的特点是具备移动通信功能,并且以提供话音、数据 通信为主要目标。这类终端包括:智能手机(例如iPhone)、多媒体手机、功能性手机,以及低 端手机等。(1) Mobile communication equipment: the feature of this type of equipment is to possess mobile communication function, and to provide voice and data communication as the main goal. Such terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.
[0194] (2)超移动个人计算机设备:这类设备属于个人计算机的范畴,有计算和处理功 能,一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等,例如iPad。(2) Ultra-mobile personal computer equipment: this type of equipment belongs to the category of personal computers, has calculation and processing functions, and generally also possesses the characteristics of mobile Internet access. Such terminals include: PDA, MID and UMPC equipment, such as iPad.
[0195] (3)便携式娱乐设备:这类设备可以显示和播放多媒体内容。该类设备包括:音频、 视频播放器(例如iPod),掌上游戏机,电子书,以及智能玩具和便携式车载导航设备。(3) Portable entertainment equipment: This type of equipment can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.
[0196] (4)特定服务器:提供计算服务的设备,服务器的构成包括处理器、硬盘、内存、系 统总线等,服务器和通用的计算机架构类似,但是由于需要提供高可靠的服务,因此在处理 能力、稳定性、可靠性、安全性、可扩展性、可管理性等方面要求较高。(4) Specific server: the equipment that provides computing service, the composition of server comprises processor, hard disk, internal memory, system bus etc., server and general computer architecture are similar, but because need provide highly reliable service, therefore in processing Higher requirements in terms of capability, stability, reliability, security, scalability, and manageability.
[0197] (5)其他具有数据交互功能的电子设备。(5) other electronic equipment with data interaction function.
[0198]需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实 体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存 在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖 非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要 素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备 所固有的要素。在没有更多限制的情况下,由语句“包括一个......”限定的要素,并不排除 在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。[0198] It should be noted that, in this article, relational terms such as first and second etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.
[0199]本说明书中的各个实施例均采用相关的方式描述,各个实施例之间相同相似的部 分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。Each embodiment in this description is described in a relevant manner, and the same and similar parts between the various embodiments can be referred to each other, and what each embodiment focuses on is the difference with other embodiments.
[0200] 尤其,对于装置实施例而言,由于其基本相似于方法实施例,所以描述的比较简 单,相关之处参见方法实施例的部分说明即可。Especially, for the device embodiment, because it is basically similar to the method embodiment, so the description is relatively simple, and relevant parts can refer to the part description of the method embodiment.
[0201]在流程图中表示或在此以其他方式描述的逻辑和/或步骤,例如,可以被认为是用 于实现逻辑功能的可执行指令的定序列表,可以具体实现在任何计算机可读介质中,以供 指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执 行系统、装置或设备取指令并执行指令的系统)使用,或结合这些指令执行系统、装置或设 备而使用。就本说明书而言,〃计算机可读介质〃可以是任何可以包含、存储、通信、传播或传 输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装 置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下:具有一个或多个布线的电 连接部(电子装置),便携式计算机盘盒(磁装置),随机存取存储器(RAM),只读存储器 (ROM),可擦除可编辑只读存储器(EPROM或闪速存储器),光纤装置,以及便携式光盘只读存 储器(CDR0M)。另外,计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的 介质,因为可以例如通过对纸或其他介质进行光学扫描,接着进行编辑、解译或必要时以其 他合适方式进行处理来以电子方式获得所述程序,然后将其存储在计算机存储器中。[0201] The logic and/or steps represented in a flowchart or otherwise described herein, for example, may be considered as a sequenced listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use in, or in connection with, an instruction execution system, apparatus, or device (such as a computer-based system, system including a processor, or other system capable of fetching and executing instructions from an instruction execution system, apparatus, or device) system, device or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disk Read Only Memory (CDR0M). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and stored in computer memory.
[0202]应当理解,本发明的各部分可以用硬件、软件、固件或它们的组合来实现。[0202] It should be understood that various parts of the present invention may be implemented by hardware, software, firmware or a combination thereof.
[0203]在上述实施方式中,多个步骤或方法可以用存储在存储器中且由合适的指令执行 系统执行的软件或固件来实现。例如,如果用硬件来实现,和在另一实施方式中一样,可用 本领域公知的下列技术中的任一项或他们的组合来实现:具有用于对数据信号实现逻辑功 能的逻辑门电路的离散逻辑电路,具有合适的组合逻辑门电路的专用集成电路,可编程门 阵列(PGA),现场可编程门阵列(FPGA)等。[0203] In the above-described embodiments, various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
[0204]以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何 熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到的变化或替换,都应 涵盖在本发明的保护范围之内。因此,本发明的保护范围应以权利要求的保护范围为准。The above is only the specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, and any skilled person familiar with the technical field can easily think of changes within the technical scope disclosed in the present invention Or replacement, all should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims (15)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710023995.7A CN106886568B8 (en) | 2017-01-12 | 2017-01-12 | One kind divides table method, apparatus and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710023995.7A CN106886568B8 (en) | 2017-01-12 | 2017-01-12 | One kind divides table method, apparatus and electronic equipment |
Publications (3)
Publication Number | Publication Date |
---|---|
CN106886568A CN106886568A (en) | 2017-06-23 |
CN106886568B CN106886568B (en) | 2018-03-30 |
CN106886568B8 true CN106886568B8 (en) | 2018-05-01 |
Family
ID=59176381
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710023995.7A Active CN106886568B8 (en) | 2017-01-12 | 2017-01-12 | One kind divides table method, apparatus and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106886568B8 (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107423387A (en) * | 2017-07-19 | 2017-12-01 | 北京北信源软件股份有限公司 | The method and apparatus of table is divided in a kind of point storehouse based on middleware |
CN108108434A (en) * | 2017-12-19 | 2018-06-01 | 福建中金在线信息科技有限公司 | A kind of method and device for managing database |
CN108334576A (en) * | 2018-01-24 | 2018-07-27 | 深圳市金政软件技术有限公司 | A kind of data exhibiting system and its method for exhibiting data |
CN111026985A (en) * | 2019-12-02 | 2020-04-17 | 北京齐尔布莱特科技有限公司 | Short link generation method, device and server |
CN111177148B (en) * | 2019-12-06 | 2023-03-24 | 南京南瑞水利水电科技有限公司 | Method for automatically building and dividing tables of hydropower database |
CN111199386A (en) * | 2019-12-27 | 2020-05-26 | 天阳宏业科技股份有限公司 | Workflow engine and implementation method thereof |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101697152A (en) * | 2009-10-23 | 2010-04-21 | 金蝶软件(中国)有限公司 | Database storage system and method and device for splitting data thereof |
CN102682044A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Method for splitting tables for PHP MYSQL database |
CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
CN103678609A (en) * | 2013-12-16 | 2014-03-26 | 中国科学院计算机网络信息中心 | Large data inquiring method based on distribution relation-object mapping processing |
CN105550349A (en) * | 2015-12-28 | 2016-05-04 | 北京邮电大学 | Persistent layer device and database operation method |
-
2017
- 2017-01-12 CN CN201710023995.7A patent/CN106886568B8/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101697152A (en) * | 2009-10-23 | 2010-04-21 | 金蝶软件(中国)有限公司 | Database storage system and method and device for splitting data thereof |
CN102682044A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Method for splitting tables for PHP MYSQL database |
CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
CN103678609A (en) * | 2013-12-16 | 2014-03-26 | 中国科学院计算机网络信息中心 | Large data inquiring method based on distribution relation-object mapping processing |
CN105550349A (en) * | 2015-12-28 | 2016-05-04 | 北京邮电大学 | Persistent layer device and database operation method |
Non-Patent Citations (1)
Title |
---|
数据库动态生成表在Hibernate中映射问题的研究与解决;姚振宇;《计算机与现代化》;20091031(第10期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN106886568B (en) | 2018-03-30 |
CN106886568A (en) | 2017-06-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106886568B8 (en) | One kind divides table method, apparatus and electronic equipment | |
US7599948B2 (en) | Object relational mapping layer | |
US10180992B2 (en) | Atomic updating of graph database index structures | |
JP5235665B2 (en) | Method, apparatus and computer program product for automatically obtaining custom interface elements by querying a remote repository upon UI theme change | |
US9311373B2 (en) | Taxonomy driven site navigation | |
WO2019153554A1 (en) | Data relationship display method and device, computer equipment and storage medium | |
US20200065313A1 (en) | Extensible content object metadata | |
US11100173B2 (en) | Autolayout of visualizations based on graph data | |
US20130055291A1 (en) | Describing native application programming interfaces of an operating system with metadata | |
US9740713B1 (en) | Dynamic modeling of data in relational databases | |
JP2010519624A (en) | REST for entities | |
CN104881490A (en) | WEB form data access method and system | |
US20170154083A1 (en) | Integrated framework for secured data provisioning and management | |
CN114115844A (en) | Page generation method and device, computer equipment and storage medium | |
US20130325534A1 (en) | Terminal management system, management server, and method therefor | |
US8413109B2 (en) | Systems and methods for metamodel transformation | |
US10289620B1 (en) | Reporting and data governance management | |
WO2024113740A1 (en) | Data query method, and system, device and computer-readable storage medium | |
US8650534B2 (en) | Metaobject enhancement objects | |
US10019519B2 (en) | Methods and systems for utilizing global entities in software applications | |
TWI629602B (en) | Method for automatically generating website interactive program | |
US10318524B2 (en) | Reporting and data governance management | |
CN113076086B (en) | Metadata management system and method for modeling model object using the same | |
CN111984257B (en) | Solid modeling customized extension method and device | |
US8910183B2 (en) | Access to context information in a heterogeneous application environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CB03 | Change of inventor or designer information |
Inventor after: Qian Wenpin Inventor after: Liu Weiping Inventor after: Wang Liang Inventor after: Yu Fei Inventor after: Chen Chao Inventor before: Qian Wenpin Inventor before: Yu Fei Inventor before: Chen Chao |
|
CB03 | Change of inventor or designer information | ||
CI03 | Correction of invention patent |
Correction item: Inventor Correct: Qian Wenpin|Liu Weiping|Wang Liang|Yu Fei|Chen Chao False: Qian Wenpin|Yu Fei|Chen Chao Number: 13-02 Page: The title page Volume: 34 Correction item: Inventor Correct: Qian Wenpin|Liu Weiping|Wang Liang|Yu Fei|Chen Chao False: Qian Wenpin|Yu Fei|Chen Chao Number: 13-02 Volume: 34 |
|
CI03 | Correction of invention patent |