CN113590609A - Database partitioning method and device, storage medium and electronic equipment - Google Patents

Database partitioning method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN113590609A
CN113590609A CN202110695181.4A CN202110695181A CN113590609A CN 113590609 A CN113590609 A CN 113590609A CN 202110695181 A CN202110695181 A CN 202110695181A CN 113590609 A CN113590609 A CN 113590609A
Authority
CN
China
Prior art keywords
rule
library
database
label
feature data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110695181.4A
Other languages
Chinese (zh)
Inventor
樊晓威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kuangshi Technology Co Ltd
Original Assignee
Beijing Kuangshi Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kuangshi Technology Co Ltd filed Critical Beijing Kuangshi Technology Co Ltd
Priority to CN202110695181.4A priority Critical patent/CN113590609A/en
Publication of CN113590609A publication Critical patent/CN113590609A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to the technical field of data processing, and provides a data base dividing method and device, a storage medium and electronic equipment. The data sub-base is applied to the middle layer system, and the method comprises the following steps: receiving image feature data from an upstream system; judging whether the image characteristic data is matched with a sub-library rule established on the middle layer system; and if the image characteristic data is matched with a created library dividing rule, issuing the image characteristic data to a downstream base corresponding to the library dividing rule. According to the method, the image feature data issued by the upstream system are divided into different downstream base databases by using the base dividing rule established on the intermediate layer system, so that the flexibility of feature data comparison in the downstream system is obviously improved, the overlarge data amount in a single base database can be avoided, and the efficiency of feature data comparison is improved. In addition, the method is executed by the middle layer system, the original logics of the upstream system and the downstream system are not affected, and the large-scale upgrading and transformation of the original system are avoided.

Description

Database partitioning method and device, storage medium and electronic equipment
Technical Field
The invention relates to the technical field of data processing, in particular to a data base dividing method and device, a storage medium and electronic equipment.
Background
At present, application scenarios for comparison based on image feature data are increasing, for example, in a task of identifying a target ID, a camera is used to collect a target image, and extract feature data of the target image, similarity calculation is performed on the feature data and feature data stored in a base library, if the similarity exceeds a certain threshold, it is indicated that the feature data is successfully matched, and at this time, target ID information corresponding to the feature data matched in the base library can be further obtained, and so on.
In these application scenarios, for the consideration of factors such as safety, a typical architecture mode is adopted to divide an upstream system and a downstream system, the upstream system is responsible for providing image feature data and issuing the image feature data to the downstream system, and the downstream system uses the feature data to perform data comparison and obtain a result, and then pulls other dimension information corresponding to the feature data from the upstream system. For example, the upstream system issues image feature data to the downstream system in batch, the downstream system constructs a base by using the feature data, similarity calculation is performed on the feature data of the target image and the feature data stored in the base, and if the similarity exceeds a certain threshold, target ID information corresponding to the feature data matched in the base is further pulled from the upstream system. The upstream system does not issue the full amount of target ID information to the downstream system, and only when the feature data in the downstream system is successfully matched, the upstream system is allowed to pull the information of other dimensions of the target, so that the leakage of the target information is effectively avoided.
In the prior art, an upstream system usually only designates one base when issuing image feature data, so that a downstream system can only construct one base according to the feature data, and applications in the downstream system use the base to compare the feature data, which is poor in flexibility. Moreover, the feature data issued by the upstream system may reach the level of millions or even tens of millions, which results in a huge amount of data in the base library and low efficiency of comparing the feature data.
Disclosure of Invention
An embodiment of the present invention provides a database partitioning method and apparatus, a storage medium, and an electronic device, so as to solve the above technical problem.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, an embodiment of the present application provides a database partitioning method, which is applied to a middle layer system, and the method includes: receiving image feature data from an upstream system; judging whether the image characteristic data is matched with a sub-library rule established on the intermediate layer system; and if the image characteristic data is matched with a created library dividing rule, issuing the image characteristic data to a downstream base corresponding to the library dividing rule.
According to the method, the intermediate layer system is additionally arranged between the upstream system and the downstream system, so that the decoupling between the upstream system and the downstream system is realized. The database partitioning rule can be established on the middle layer system, and the image characteristic data issued by the upstream system is partitioned into different downstream bases by utilizing the established database partitioning rule, wherein the database partitioning rule and the downstream bases can be set as required, so that the flexibility of characteristic data comparison in the downstream system is obviously improved. Moreover, by means of the characteristic data sub-database, the problem of overlarge data volume in a single base database can be avoided, and the efficiency of characteristic data comparison is improved. Finally, the method is executed by the middle-layer system, and the original logics of the upstream system and the downstream system are not affected, so that the function of the original system is enhanced, and meanwhile, the large-scale upgrading and reconstruction of the original system are avoided.
In an implementation manner of the first aspect, issuing the image feature data to a downstream base corresponding to a sublibrary rule if the image feature data matches the created sublibrary rule includes: if the image characteristic data is matched with a created database partitioning rule, searching a downstream database corresponding to a middle-layer database specified in the database partitioning rule from a database mapping relation maintained on the middle-layer system; and issuing the image characteristic data to the searched downstream base.
In some implementations, the sub-library rule may be directly bound to the downstream base library, but in the above implementations, the sub-library rule is bound to the middle-layer base library, and the middle-layer system maintains the mapping relationship between the middle-layer base library and the downstream base library, so that the mapping relationship between the sub-library rule and the downstream base library is more flexible in configuration. Note that the intermediate-level chassis does not need to store the actual image feature data therein, since it merely acts as an intermediary between the binning rules and the downstream chassis.
In an implementation manner of the first aspect, the image feature data received from the upstream system is provided with tag information, where the tag information includes a tag, or a tag and a value thereof; the judging whether the image feature data is matched with the established sublibrary rule on the intermediate layer system includes: and judging whether the image characteristic data is matched with the established database partitioning rule on the intermediate layer system or not according to the label information carried by the image characteristic data.
The upstream system may add some label information to the image feature data, and these labels may be regarded as attributes of the image feature data, so that when the image feature data is sorted, corresponding sorting rules may be set according to the label information carried by the image feature data and the image feature data is sorted.
In an implementation manner of the first aspect, the determining, according to the tag information carried by the image feature data, whether the image feature data matches with a sub-library rule created on the middle layer system includes: determining all possible sub-library rules according to the labels carried by the image characteristic data and the values of the labels; each possible sub-library rule corresponds to a combination, and the combination is a combination of a label and a value thereof; judging whether a database partitioning rule exists in the created database partitioning rules on the intermediate layer system, wherein the label and the value of the database partitioning rule are consistent with the label and the value of a possible database partitioning rule; if the image characteristic data exists, the image characteristic data is determined to be matched with the database partitioning rule, otherwise, the image characteristic data is determined not to be matched with the database partitioning rule established on the middle layer system.
In an implementation manner of the first aspect, the determining, according to the tag information carried by the image feature data, whether the image feature data matches with a sub-library rule created on the middle layer system includes: for each database partitioning rule established on the intermediate layer system, judging whether the label and the value thereof specified in the database partitioning rule are contained in the label and the value thereof carried by the image characteristic data; if the image feature data is contained in the database, determining that the image feature data is matched with the database partitioning rule, and otherwise, determining that the image feature data is not matched with the database partitioning rule.
There are many different implementations of matching image feature data with the sub-library rules, two of which are given above.
In an implementation manner of the first aspect, if the image feature data matches a created library partitioning rule, issuing the image feature data to a downstream base corresponding to the library partitioning rule, where the issuing includes: if the image characteristic data is matched with and only matched with a created library partitioning rule, the image characteristic data is issued to a downstream base corresponding to the library partitioning rule; the method further comprises the following steps: and if the image characteristic data are matched with the plurality of established database partitioning rules, outputting error information of repeated matching.
In the implementation manner, each library partitioning rule is required to be mutually exclusive, and the same image feature data is prevented from being repeatedly stored by different downstream base libraries, so that if one image feature data is successfully matched with a plurality of created library partitioning rules, the existence of the non-mutually exclusive library partitioning rules is indicated, at this moment, error information needs to be output, and a user can timely modify the improperly set library partitioning rules.
In one implementation form of the first aspect, the method further comprises: if the image characteristic data is not matched with the established library dividing rule on the middle layer system, establishing a new library dividing rule which can be matched with the image characteristic data; instructing the downstream system to create a downstream base corresponding to the new base partitioning rule; and issuing the image characteristic data to the newly created downstream base.
In the implementation manner, if there is image feature data that cannot be processed by the existing binning rule, the intermediate layer system may automatically create a binning rule that can be matched with the image feature data, and instruct the downstream system to create a corresponding downstream base (of course, the intermediate layer system may also create an intermediate layer base), so as to implement binning of the image feature data.
In an implementation manner of the first aspect, the image feature data received from the upstream system is provided with tag information, where the tag information includes a tag, or a tag and a value thereof; if the image feature data is not matched with the created library splitting rule on the middle layer system, creating a new library splitting rule which can be matched with the image feature data, wherein the new library splitting rule comprises the following steps: and if the image characteristic data is not matched with the established library dividing rule on the middle layer system, establishing a new library dividing rule which can be matched with the image characteristic data according to the label information of the image characteristic data.
If the image characteristic data is provided with the labels, when a database partitioning rule aiming at the image characteristic data is created, the database partitioning rule can be created according to the labels so as to realize the accurate database partitioning of the image characteristic data.
In an implementation manner of the first aspect, before the determining whether the image feature data matches with a sub-library rule already created on the middle-layer system, the method further includes: loading the created library dividing rules on the middle layer system from a database into a memory; the method further comprises the following steps: and polling the database partitioning rules stored in the database, and loading the updated database partitioning rules from the database to a memory when a polling result shows that the database partitioning rules stored in the database are updated.
Before rule matching, the sub-library rules are loaded into the memory from the database (generally on a hard disk), which is beneficial to improving the access speed of the rules in the matching process, thereby improving the rule matching efficiency. In addition, since the database partitioning rule allows updating, but in order to maintain the consistency of the database partitioning process, the database partitioning rule loaded in the memory is generally not directly updated, but the database partitioning rule in the database is updated, which may cause inconsistency between the database partitioning rule in the database and the memory, so that in some implementation manners, the change of the database partitioning rule in the database may be discovered in a manner of polling the database, and the rule in the memory is updated correspondingly.
In one implementation form of the first aspect, the method further comprises: before the image characteristic data is judged to be matched with the established library dividing rule on the intermediate layer system, establishing a new library dividing rule; and instructing the downstream system to create a downstream base corresponding to the new library partitioning rule.
Before the image feature data is subjected to library partitioning by using the library partitioning rule, the library partitioning rule and a corresponding downstream base should be created.
In an implementation manner of the first aspect, the creating a new sublibrary rule includes: creating a new banking rule in which a downstream base is specified; the instructing the downstream system to create a downstream base corresponding to the new base partitioning rule includes: instructing the downstream system to create a downstream base specified in the new banking rule.
The downstream base created in the above implementation is directly bound to the sub-base rule.
In an implementation manner of the first aspect, the creating a new sublibrary rule includes: creating a new library partitioning rule in which a middle-layer base is specified; the instructing the downstream system to create a downstream base corresponding to the new base partitioning rule includes: and instructing a downstream system to create a downstream base corresponding to the intermediate-layer base, and storing a mapping relation between the intermediate-layer base and the downstream base on the intermediate-layer system.
The downstream base created in the implementation mode is not directly bound with the base dividing rule but is associated with the middle-layer base, and the base dividing rule is bound with the middle-layer base, so that the mapping relation between the base dividing rule and the downstream base is more flexible in configuration.
In one implementation manner of the first aspect, the image feature data received from the upstream system is provided with tag information, and the tag information includes a tag and a value thereof; the creating of the new sublibrary rule comprises the following steps: and creating a new library rule according to the combination of the specified unique label.
In the implementation manner, the sub-library rule is created according to the combination of the only one tag, so that the created sub-library rule is mutually exclusive, and the condition that the image feature data is simultaneously matched with a plurality of sub-library rules is not generated subsequently.
In a second aspect, an embodiment of the present application provides a database partitioning apparatus configured in a middle-layer system, where the apparatus includes: a data receiving module for receiving image feature data from an upstream system; the rule matching module is used for judging whether the image characteristic data is matched with the established sub-library rule on the middle layer system; and the data sending module is used for issuing the image characteristic data to a downstream base corresponding to the database partitioning rule when the image characteristic data is matched with the established database partitioning rule.
In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the method provided by the first aspect or any one of the possible implementation manners of the first aspect.
In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory in which computer program instructions are stored, and a processor, where the computer program instructions are read and executed by the processor to perform the method provided by the first aspect or any one of the possible implementation manners of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 illustrates a system architecture provided by an embodiment of the present application;
FIG. 2 shows a possible flow of a database partitioning method provided by an embodiment of the present application;
FIG. 3 illustrates a matching process of image feature data and a banking rule provided by an embodiment of the present application;
FIG. 4 illustrates a database selection-by-database interface provided by an embodiment of the present application;
FIG. 5 illustrates a base creation interface provided by an embodiment of the present application;
FIG. 6 shows a possible structure of a database partitioning apparatus provided in an embodiment of the present application;
fig. 7 shows a possible structure of an electronic device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Fig. 1 shows a system architecture provided in an embodiment of the present application, where the system architecture includes an upstream system, an intermediate layer system, and a downstream system, image feature data is sent to the intermediate layer system by the upstream system, then sent to the downstream system by the intermediate layer system, and then stored in a base (referred to as a downstream base) of the downstream system, and an application in the downstream system may execute a feature data comparison task based on the downstream base. The base library is an image feature database, and in a commonly used scene, feature comparison is performed on image feature data Q of a query image and image feature data in the base library, and the image feature data in the base library matched with Q is used as a comparison result. In the architecture, the upstream system and the downstream system may be existing systems, and the intermediate layer system is a system added to support the database partitioning method provided by the embodiment of the present application.
The above simple description of the functional layers of the upstream system, the intermediate layer system, and the downstream system does not limit how each system corresponds to a physical device when actually deployed. For example, if the downstream server performance is high, both the downstream system and the middle tier system may be deployed on the downstream server. In addition, the upstream system and the downstream system in actual deployment may have certain business implications, for example, the upstream system is a system used by a main department, and the downstream system is a system used by a basic department.
Further details regarding the other aspects shown in fig. 1 will be described later. Fig. 2 shows a possible flow of a database partitioning method provided in an embodiment of the present application, where the method may be performed by an intermediate layer system in fig. 1, and the intermediate layer system may be deployed on the electronic device shown in fig. 7, and a specific structure of the electronic device is described later with reference to fig. 7. Referring to fig. 1, the method includes:
step S110: the middle tier system receives image characteristic data from the upstream system.
The image feature data is a common situation in which features extracted from an image are extracted, and one piece of image feature data is extracted from one image.
The upstream system may take a number of strategies in issuing image feature data: for example, if there is no data in the downstream base, the upstream system may issue the data in full volume, for example, several millions or even tens of millions of image feature data at a time. Of course, if there is already a large amount of data in the downstream base, only a small amount of data needs to be added, and the upstream system may also adopt an incremental issuing manner, for example, issuing several or tens of pieces of image feature data at one time.
In an upstream system, some label information may be added to the image feature data according to business requirements, and the label information may be regarded as attributes of the image feature data, so that in a subsequent step, when the image feature data is sorted, corresponding sorting rules may be set according to the label information carried by the image feature data and the image feature data is sorted, so that the image feature data with some same attributes are divided into the same base.
In some implementations, the tag information may include only tags, e.g., a piece of image feature data with a "source" tag and a "target type" tag; in other implementation manners, the tag information may include tags and values thereof, and a certain piece of image feature data has a "source" tag and a "target type" tag, where the "source" tag takes the value of "source 1" and the "target type" tag takes the value of "type 1".
Further, when the upstream system issues the image feature data, each piece of image feature data may have independent tag information, or a batch of image feature data may have common tag information. For example, 1000 pieces of feature data issued by an upstream system all have a "source" tag and a "target type" tag, and the tag information only needs to be stored in 1 part of the issued data, and does not need to be stored in 1000 parts.
In the database partitioning method provided in the embodiment of the present application, each piece of image feature data issued by an upstream system may be partitioned into databases separately, and the database partitioning processes of each piece of image feature data are similar. Of course, in some implementations, a batch of image feature data may be binned at a time. For example, following the above example, 1000 pieces of image feature data sent by the upstream system all have a "source" tag and a "target type" tag, and the 1000 pieces of image feature data may be sorted at one time according to the tag information. As will be described in detail later with respect to how the tag information is used for binning, for simplicity, a single image feature data is generally used as an example when describing the data binning.
Step S120: the intermediate layer system judges whether the image characteristic data is matched with the established library dividing rule on the intermediate layer system.
Step S130: and when the image characteristic data is matched with a created library dividing rule, the intermediate layer system issues the image characteristic data to a downstream base corresponding to the library dividing rule.
Step S120 and step S130 are put together for description. Before step S120 is executed, several rules for the database are already created and stored on the middle layer system, and the creation of the rules for the database will be described in detail later. The created library dividing rules can be stored in a database (generally located on a hard disk), and are loaded into a memory from the database when the image characteristic data needs to be divided, and the library dividing rules are always maintained in the memory in the rule matching process (namely step S120), so that the query from the database is avoided during each use, and the access speed of the rules is improved, and the rule matching efficiency is improved.
In some implementations, the middle layer system allows the user to update the library rules during the use process, where the updating includes creating new library rules, modifying created library rules, deleting created library rules, and the like, and the middle layer system may provide a corresponding interface for the user to do so. However, in order to maintain the consistency of the banking process, it is generally not allowed to directly update the banking rules that are already loaded into the memory (because these banking rules may be in use), but only update the banking rules in the database, which may cause the banking rules in the database to be inconsistent with the banking rules loaded in the memory. In order to solve the problem, the intermediate layer system may periodically query (i.e., poll) the library splitting rules stored in the database, and when the query result indicates that the library splitting rules stored in the database are updated, load the updated library splitting rules from the database into the memory, and replace or supplement the original library splitting rules in the memory (or delete the original library splitting rules in the memory). It will be appreciated that the periodic setting of the query database should be on the principle of not affecting the consistency of the banking process.
The general form of the banking rule may be:
(1) condition ← → downstream base
In step S120, it is substantially determined whether the image feature data matches the binning rule, if the image feature data matches the condition specified in the binning rule, the image feature data matches the binning rule, otherwise the image feature data does not match the binning rule. The conditions may include, and are described later.
In the form (1), the bank partitioning rule specifies a downstream bank to which the image feature data is to be partitioned, in addition to a condition for matching, so that, once the image feature data matches a certain bank partitioning rule, the image feature data can be transmitted to the corresponding downstream bank for storage according to the information of the downstream bank specified in the bank partitioning rule. The information of the downstream base library may include information such as an IP address and a port number thereof. It will be appreciated that the downstream base specified in the banking rule has already been created in the downstream system before step S120 is performed.
Some implementation manners only allow one downstream base to be specified in the base partitioning rule, and other implementation manners allow a plurality of downstream bases to be specified in the base partitioning rule, and for the latter, if a certain image feature data is successfully matched with the base partitioning rule, the image feature data can be respectively issued to the plurality of downstream bases specified in the successfully matched base partitioning rule.
In fig. 1, a thick dotted line shows a flow of the intermediate layer system for binning image feature data using a binning rule having the form (1).
(2) Condition ← → middle tier base
The meaning of the conditions in form (2) is similar to that in form (1), and will not be repeated. In addition to the conditions, the middle-layer base into which the image feature data is divided is specified in the binning rule under the form (2). The intermediate level repository is created by the intermediate level system and is not a repository for actually storing image feature data (otherwise, storing image feature data by both the intermediate level repository and the downstream repository would result in duplicate storage), but merely as an intermediary between the banking rules and the downstream repository. The intermediate layer system maintains the mapping relation between the intermediate layer base and the downstream base, once the image characteristic data matches a certain base dividing rule, the mapping relation is searched by using the information (such as the identification of the base) of the intermediate layer base specified in the base dividing rule, the corresponding downstream base can be obtained, and the image characteristic data can be sent to the searched downstream base for storage.
The middle-layer base can be implemented as a data object in the middle-layer system, or a table in a database, etc. Although image feature data is not stored in the intermediate layer system, it is not excluded to store other information therein.
Some implementation modes only allow the middle-layer base to establish a mapping relation with one downstream base, other implementation modes allow the middle-layer base to establish a mapping relation with a plurality of downstream bases, and for the latter, if matching of certain image feature data and the database partitioning rule is successful, the image feature data can be respectively issued to the downstream bases associated with the middle-layer base specified in the rule.
In fig. 1, a thick solid line shows a flow of the intermediate layer system for binning image feature data using a binning rule having the form (2).
Simply analyzing the difference between the database partitioning rule and the database partitioning rule in the form (1) and the form (2), wherein when the form (1) is adopted, the database partitioning rule is directly bound with a downstream base, and the implementation form of the database partitioning rule is simple and direct; when the form (2) is adopted, the library dividing rule is bound with the middle-layer base library, and the middle-layer system maintains the mapping relation between the middle-layer base library and the downstream base library, so that the mapping relation between the library dividing rule and the downstream base library is more flexible in configuration.
The following continues with the conditions in the banking rules:
in some implementations, the condition may be a period of time that the middle tier system receives image characteristic data, such as:
library partitioning rule 01: time period 1 ← → bottom library 1
Library splitting rule 02: time period 2 ← → bottom library 2
The base library 1 and the base library 2 may refer to a downstream base library (corresponding to the format (1)) or an intermediate base library (corresponding to the format (2)), and are not specifically distinguished herein. For a certain piece of image characteristic data, if the time t when the intermediate layer system receives the data falls into the time period 1, matching the upper database partitioning rule 01, if the time t when the intermediate layer system receives the data falls into the time period 2, matching the upper database partitioning rule 02, otherwise, matching the data with no database partitioning rule.
In some implementations, the condition may be a range of amounts of image feature data that the middle tier system has received, such as:
library splitting rule 11: number range 1 ← → base 1
Library partitioning rule 12: number range 2 ← → base library 2
For a certain piece of image feature data, the data is the x-th piece of image feature data received by the middle layer system, if x falls into the quantity range 1, the data is matched with the upper database partitioning rule 11, if x falls into the quantity range 2, the data is matched with the upper database partitioning rule 12, and otherwise, the data is not matched with any database partitioning rule.
In some implementations, if the image feature data carries tag information, it can also be determined whether the image feature data matches a sub-library rule that has been created on the middle-layer system according to the tag information carried by the image feature data. The following is described according to the content of the tag information:
(1) the label information comprises labels and values thereof
In this case, each created library rule specifies a combination of a label and its value, and it should be noted that "label and its value" should be regarded as a whole, for example:
library splitting rule 21: label a ═ 1, label C ═ 2 ← → base library 1
Library partitioning rule 22: label a ═ 1, label C ═ 3 ← → base library 2
Library splitting rule 23: label a ═ 2, label C ═ 3 ← → base library 3
In this case, the combination of "tag a ═ 1" and "tag C ═ 2" is specified in the banking rule 21, the combination of "tag a ═ 1" and "tag C ═ 3" is specified in the banking rule 22, and the combination of "tag a ═ 2" and "tag C ═ 3" is specified in the banking rule 23.
It is not assumed that tag information carried by a certain piece of image feature data is:
tag a is 1, tag B is 2, and tag C is 3
At this time, whether the image feature data matches one of the created library rules can be judged at least in the following two ways:
in a first mode
Firstly, determining all possible sub-library rules according to labels carried by image characteristic data and values thereof, wherein each possible sub-library rule corresponds to a combination, and the combination is a combination of the labels and the values thereof. For example, for three tags "tag a is equal to 1", "tag B is equal to 2", and "tag C is equal to 3" and their values, a total of seven combinations can be generated, corresponding to seven possible binning rules, namely:
possible banking rule a 1: label a ═ 1
Possible banking rule b 1: label B2
Possible banking rule c 1: label C ═ 3
Possible banking rule d 1: label a is 1 and label B is 2
Possible banking rule e 1: label B is 2 and label C is 3
Possible banking rule f 1: label a is 1 and label C is 3
Possible banking rule g 1: tag a is 1, tag B is 2, and tag C is 3
These seven possible binning rules are shown in the left-hand oval dashed box of fig. 3. Note that there is no need to specify any base library in these possible banking rules.
Then, whether the created library splitting rules on the middle-layer system have the following library splitting rules is judged: the label and the value thereof are consistent with a possible library division rule; if the image feature data exists, the image feature data is determined to be matched with the database partitioning rule, otherwise, the image feature data is determined not to be matched with the database partitioning rule established on the middle layer system.
With continued reference to FIG. 3, the created banking rules are the above banking rule 21, the banking rule 22 and the banking rule 23, as shown in the right-hand oval dashed box in the figure.
The possible library splitting rule f1 and the library splitting rule 22 both include labels a and C, the value of the label a is 1, and the value of the label C is 3, which indicates that the image feature data matches the library splitting rule 22 according to the above explanation. It is also not difficult to see from fig. 3 that the two dashed oval boxes present an intersection region where the rule is just the possible rule f1 or the banking rule 22.
Mode two
Aiming at each sub-library rule established on the intermediate layer system, judging whether the label and the value thereof appointed in the rule are contained in the label and the value thereof carried by the image characteristic data; if the image feature data is contained in the database, determining that the image feature data is matched with the database partitioning rule, otherwise, determining that the image feature data is not matched with the database partitioning rule.
For example, as for the above library partitioning rule 21, it can be determined whether the image feature data includes the label a and whether the value of the label a is 1, and whether the image feature data includes the label C and whether the value of the label C is 3, where a result of the second determination is "no", so that the image feature data does not match the library partitioning rule 21. For the banking rule 22 and the banking rule 23, similar judgment is made and it is determined that the image feature data matches the banking rule 22.
(2) Including only tags in the tag information
In this case, each created sublibrary rule specifies a combination of labels, for example:
library splitting rule 31: label a, label B ← → base library 1
Library partitioning rule 32: label B, label C ← → base library 2
Library splitting rule 33: label C, label D ← → base 3
Wherein, the combination of label a and label B is specified in the library-splitting rule 31, the combination of label B and label C is specified in the library-splitting rule 32, and the combination of label C and label D is specified in the library-splitting rule 33.
It is not assumed that tag information carried by a certain piece of image feature data is:
label A, label C, label D
At this time, whether the image feature data matches one of the created library rules can be judged at least in the following two ways:
in a first mode
Firstly, all possible sub-library rules are determined according to labels carried by image feature data, wherein each possible sub-library rule corresponds to a combination of labels.
For example, for three tags, tag a, tag C, and tag D, seven combinations can be generated, i.e. seven possible banking rules are generated, i.e.:
possible banking rule a 2: label A
Possible banking rule b 2: label C
Possible banking rule c 2: label D
Possible banking rule d 2: label A, label C
Possible banking rule e 2: label C, Label D
Possible banking rule f 2: label A, label D
Possible banking rule g 2: label A, label C, label D
Then, whether the created library splitting rules on the middle-layer system have the following library splitting rules is judged: the label of the database is consistent with a possible database dividing rule; if the image feature data exists, the image feature data is determined to be matched with the database partitioning rule, otherwise, the image feature data is determined not to be matched with the database partitioning rule established on the middle layer system.
Where both the possible banking rule e2 and the banking rule 33 contain labels C and D, this indicates that the image feature data matches the banking rule 33, in accordance with the above explanations.
Mode two
Aiming at each sub-library rule established on the intermediate layer system, judging whether a label specified in the rule is contained in a label carried by the image characteristic data; if the image feature data is contained in the database, determining that the image feature data is matched with the database partitioning rule, otherwise, determining that the image feature data is not matched with the database partitioning rule.
For example, with respect to the above binning rule 31, it is determined whether or not the image feature data includes the label a and the label B, and the determination result is "no", so that the image feature data does not match the binning rule 31. For the banking rule 32 and the banking rule 33, similar judgment is made and it is determined that the image feature data matches the banking rule 33.
For the above situation (1), if the image feature data sent by the upstream system has a tag and a value thereof, and some image feature data only have a tag, a library partitioning rule that only includes a tag but not a value thereof may also be set, for example:
library splitting rule 41: label X ← → base 4
Meaning that if an image feature carries a label X but does not contain a value, it is divided into the base 4.
For the above case (2), if there is a label in the image feature data sent by the upstream system, or there is no label in the image feature data, a sub-library rule without a label may also be set, for example:
library partitioning rule 51: no label ← → base 5
Meaning that if the image features do not have labels, they are partitioned into the base library 5.
In some implementations, a piece of image feature data is allowed to simultaneously match multiple created binning rules, for example, for the following created binning rules:
library splitting rule 61: label a ═ 1, label C ═ 2 ← → base library 1
Library partitioning rule 62: label a ═ 1, label C ═ 3 ← → base library 2
Library partitioning rule 63: label B ═ 2, label C ═ 3 ← → base library 3
If the label information carried by a certain piece of image characteristic data is:
tag a is 1, tag B is 2, and tag C is 3
According to the rule matching method set forth above, the image feature data can be simultaneously matched with the library dividing rule 62 and the library dividing rule 63, and after the matching is successful, the image feature data is respectively issued to the downstream base corresponding to the library dividing rule 62 and the downstream base corresponding to the library dividing rule 63 by the intermediate layer system.
In other implementations, one piece of image feature data is not allowed to be simultaneously matched with multiple created binning rules, in order to avoid that the same piece of image feature data is repeatedly stored by different downstream base banks. That is, if the image feature data is matched with and only matched with one established sub-library rule, the image feature data is issued to a downstream base corresponding to the sub-library rule; if the image characteristic data are matched with the created library dividing rules, error information of repeated matching is output, and a user can timely modify the improperly set library dividing rules.
To avoid duplicate matches, the conditions in the banking rules may be set mutually exclusive, e.g., the above banking rules are modified (the banking rules 63 therein are altered):
library splitting rule 61: label a ═ 1, label C ═ 2 ← → base library 1
Library partitioning rule 62: label a ═ 1, label C ═ 3 ← → base library 2
Library partitioning rule 63: label a ═ 2, label C ═ 3 ← → base library 3
The condition parts of the three database rules all comprise a label A and a label C, but the values of the label A and the label C are not completely the same, so that one image characteristic data can only be matched with at most one of the three database rules if the image characteristic data is provided with the label A and the value thereof, and the label C and the value thereof.
For example, if a certain piece of image feature data carries tag information:
tag a is 1, tag B is 2, and tag C is 3
Only the upper library rule 62 can be matched.
In the process of rule matching, besides the case that the matching of the image feature data and the sub-library rule is successful, there may be a case that the image feature data does not match all created images. Aiming at the situation, the intermediate layer system has different processing strategies, one is direct error reporting, the other is that a new library partitioning rule which can be matched with the image characteristic data is automatically established aiming at the image characteristic data, then the downstream system is instructed to establish a downstream base corresponding to the new library partitioning rule, and finally the image characteristic data is issued to the newly established downstream base, so that the image characteristic data which fails to be matched is also included in the downstream base to participate in the subsequent characteristic data comparison task. It can be understood that, if the middle-layer system is implemented by using the middle-layer base, when creating a new library splitting rule, it is also necessary to create the middle-layer base and store a mapping relationship between the newly created middle-layer base and the newly created downstream base. How to build the rules of the sub-base is further described in more detail below.
Taking the case that the image feature data carries the label information, and the label information includes the label and its value as an example, how to create a new library splitting rule for a certain image feature data according to the label information carried by the image feature data if the image feature data cannot be matched with the created library splitting rule will be described below.
Assume that the created banking rules are:
library splitting rule 71: label a ═ 1, label C ═ 2 ← → base library 1
Library splitting rule 72: label a ═ 1, label C ═ 3 ← → base library 2
Library splitting rule 73: label a ═ 2, label C ═ 3 ← → base library 3
If the label information of a certain piece of image characteristic data is:
tag a is 1, tag B is 2, and tag C is 1
Obviously, the image feature data cannot match any created library rule, but the library rules have the same library basis, namely, the label combination on which the library depends, here, the combination of the label a and the label C. Moreover, the sub-library basis is also included in the label information carried in the image feature data, and only in the created sub-library rule and the image feature data, the value of the label included in the sub-library basis is not completely the same, and at this time, the existing sub-library basis can be directly maintained to be unchanged to create a sub-library rule, and the value of the label in the sub-library rule is completely the same as the value of the label in the image feature data, for example:
banking rule 74: label a ═ 1, label C ═ 1 ← → base library 4
As another example, assume that the created banking rules are:
library splitting rule 81: label a ═ 1, label C ═ 2 ← → base library 1
Library splitting rule 82: label a ═ 1, label C ═ 3 ← → base library 2
Library partitioning rule 83: label a ═ 2, label C ═ 3 ← → base library 3
If the label information of a certain piece of image characteristic data is:
tag B is 2, tag C is 1, and tag D is 2
Obviously, the image feature data cannot match any created library rule, and the library basis used in the created library rule is also not included in the tag information carried in the image feature data, and at this time, a library rule may be created according to some default manner. For example, a sublibrary rule is created by using the combination of all tags in the image feature data as a sublibrary basis, as follows:
library splitting rule 84: label B ═ 2, label C ═ 1, label D ═ 2 ← → base library 4
The following is a description of how to create the banking rules on the middle tier system before using the banking rules in step S120:
in some implementations, the middle layer system may provide a visual interface (e.g., a web page, a client interface, etc.) for a user to create the sublibrary rule, and in other implementations, the user may also create the sublibrary rule by inputting an instruction, writing a configuration file, etc., which is mainly described below with reference to fig. 4 and 5 by taking the visual manner as an example. In these examples, it is assumed that the image feature data has tag information, and the tag information includes a tag and a value thereof, so that a combination of the tag and the value thereof needs to be specified in the rule when creating the sublibrary rule.
Fig. 4 shows a library basis selection interface provided in this embodiment of the present application, in which a user may select tags constituting a library basis (according to the foregoing, the library basis is a combination of tags), and fig. 4 shows a total of 3 selectable tags, which are respectively "source", "target type", and "attention level", and if a certain tag is desired to be included in the library basis, only a radio box in front of the tag needs to be checked, for example, 3 tags are all selected as the library basis in fig. 4. After the database classification basis is selected, the user can click the lower 'save' button and the lower 'next' button to enter the interface in fig. 5.
When a new sublibrary rule is created on the middle-layer system, the downstream system is also instructed to create a downstream base corresponding to the new sublibrary rule. As mentioned in the description of fig. 1, the library partitioning rule may be directly bound to the downstream base library, and for this implementation, when creating a new library partitioning rule, the downstream base library may be directly specified in the rule, and then the downstream system is instructed to create the downstream base library corresponding to the new library partitioning rule by issuing a command or a notification message, etc. In addition, the library partitioning rule may also be indirectly bound through the middle-layer base and the downstream base, and for this implementation, when creating a new library partitioning rule, it is necessary to first specify the middle-layer base in the rule, then instruct the downstream system to create the downstream base corresponding to the middle-layer base by issuing a command or a notification message, and the like, and store the mapping relationship between the middle-layer base and the downstream base on the middle-layer system. It should be noted that if the base libraries are already created in the downstream system before the middle layer system is added, it is not excluded to directly specify these existing base libraries in the sublibrary rule without creating new downstream base libraries.
Fig. 5 illustrates a base creation interface provided in an embodiment of the present application, which is also a creation interface of a sublibrary rule. In the interface, the first three rows "source", "target type", "attention level", i.e. the label selected in fig. 4 as the basis of the sub-library, may select a value for the label by means of a pull-down list, for example, the value of "source" in fig. 5 is set as "source 1", "the value of target type" is set as "type 1", and "attention level" is set as "third level". The last three rows "library name", "library type", "library level" are used to set the attribute of the base library to be created, for example, the value of "library name" is set to "test 01", "the value of library type" is set to "test library", and "library level" is set to "level two" in fig. 5. It is not assumed here that the intermediate-level base is created and the intermediate-level base and the downstream base are in a one-to-one correspondence relationship, and therefore it is also equivalent to creating the downstream base and setting the attribute of the downstream base. After the contents of the upper six rows are set, the user can click a 'create' button on the lower part, and at the moment, the library division rule, the bottom libraries (the middle-layer bottom library and the downstream bottom library) and the mapping relation among the bottom libraries are created and stored.
Taking fig. 5 as an example, the created library splitting rule is:
the label "source" is "source 1",
the label "target type ═ type 1",
label "level of attention" - "three-level"
←→
Bottom gallery
The attribute "library name" is "test 01",
the attribute "library type" is "test library",
attribute "library level" - "second level" }
It is understood that fig. 4 and 5 are only examples, and the interface used in creating the sublibrary rule may take other design forms.
Further, in some implementations, when creating a new sublibrary rule, only one unique combination of tags is allowed to be designated as a sublibrary basis, and a plurality of different sublibrary bases are not allowed to be designated. The reason is that if a unique library dividing basis is designated, when each library dividing rule is created on the basis of the library dividing basis, each created library dividing rule can have mutual exclusivity only by selecting a value which is not identical to other library dividing rules for a label contained in the library dividing basis, and the problem of repeated matching of the rules is avoided. For example, if "source" and "target type" are selected as the bases of the sub-libraries in fig. 4, the following rules are created with mutual exclusivity:
library splitting rule 91:
label "source" ═ source 1 ", label" target type "=" type 1 "← → base library 1
Library partitioning rule 92:
label "source" ═ source 2 ", label" target type "=" type 1 "← → base library 2
Library splitting rule 93:
label "source" ═ source 1 ", label" target type "=" type 2 "← → base library 3
Library rules 94:
label "source" ═ source 2 ", label" target type "=" type 2 "← → base library 4
To sum up, the data partitioning method provided in the embodiment of the present application realizes decoupling between the upstream system and the downstream system by adding the intermediate layer system between the upstream system and the downstream system. The database partitioning rule can be established on the middle layer system, and the image characteristic data issued by the upstream system is partitioned into different downstream bases by utilizing the established database partitioning rule, wherein the database partitioning rule and the downstream bases can be set as required, so that the flexibility of characteristic data comparison in the downstream system is obviously improved. Moreover, by means of the characteristic data sub-database, the problem of overlarge data volume in a single base database can be avoided, and the efficiency of characteristic data comparison is improved. Finally, the method is executed by the middle-layer system, and the original logics of the upstream system and the downstream system are not affected, so that the function of the original system is enhanced, and meanwhile, the large-scale upgrading and reconstruction of the original system are avoided.
It should be noted that the various implementations of the newly created binning rule given above may be performed not only before the binning is used in step S120, but also when it is determined in step S120 that the image feature data does not match the already created binning rule, or when the binning rule is updated (equivalent to creating a new rule instead of an old rule).
Fig. 6 shows a possible structure of the data banking apparatus 200 provided in the embodiment of the present application. Referring to fig. 6, the data banking apparatus 200 includes:
a data receiving module 210 for receiving image characteristic data from an upstream system;
a rule matching module 220, configured to determine whether the image feature data matches a sub-library rule created on the middle-layer system;
and the data sending module 230 is configured to, when the image feature data matches a created library partitioning rule, issue the image feature data to a downstream base corresponding to the library partitioning rule.
In an implementation manner of the data banking apparatus 200, when the image feature data matches a created banking rule, the data sending module 230 issues the image feature data to a downstream base corresponding to the banking rule, where the method includes: if the image characteristic data is matched with a created database partitioning rule, searching a downstream database corresponding to a middle-layer database specified in the database partitioning rule from a database mapping relation maintained on the middle-layer system; and issuing the image characteristic data to the searched downstream base.
In one implementation of the database partitioning apparatus 200, the image feature data received from the upstream system is provided with tag information, where the tag information includes a tag, or a tag and a value thereof; the rule matching module 220 determines whether the image feature data matches with a sub-library rule created on the middle layer system, including: and judging whether the image characteristic data is matched with the established database partitioning rule on the intermediate layer system or not according to the label information carried by the image characteristic data.
In an implementation manner of the data library apparatus 200, the tag information includes a tag and a value thereof, and the determining, by the rule matching module 220, whether the image feature data matches with the library partitioning rule created on the middle layer system according to the tag information carried by the image feature data includes: determining all possible sub-library rules according to the labels carried by the image characteristic data and the values of the labels; each possible sub-library rule corresponds to a combination, and the combination is a combination of a label and a value thereof; judging whether a database partitioning rule exists in the created database partitioning rules on the intermediate layer system, wherein the label and the value of the database partitioning rule are consistent with the label and the value of a possible database partitioning rule; if the image characteristic data exists, the image characteristic data is determined to be matched with the database partitioning rule, otherwise, the image characteristic data is determined not to be matched with the database partitioning rule established on the middle layer system.
In an implementation manner of the data library apparatus 200, the tag information includes a tag and a value thereof, and the determining, by the rule matching module 220, whether the image feature data matches with the library partitioning rule created on the middle layer system according to the tag information carried by the image feature data includes: for each database partitioning rule established on the intermediate layer system, judging whether the label and the value thereof specified in the database partitioning rule are contained in the label and the value thereof carried by the image characteristic data; if the image feature data is contained in the database, determining that the image feature data is matched with the database partitioning rule, and otherwise, determining that the image feature data is not matched with the database partitioning rule.
In an implementation manner of the data banking apparatus 200, when the image feature data matches a created banking rule, the data sending module 230 issues the image feature data to a downstream base corresponding to the banking rule, where the method includes: if the image characteristic data is matched with and only matched with a created library partitioning rule, the image characteristic data is issued to a downstream base corresponding to the library partitioning rule; the device further comprises: and the error prompt module is used for outputting error information of repeated matching when the image characteristic data is matched with a plurality of established database partitioning rules.
In one implementation manner of the database partitioning apparatus 200, the apparatus further includes: the rule creating module is used for creating a new database partitioning rule which can be matched with the image characteristic data when the image characteristic data is not matched with the database partitioning rule created on the intermediate layer system; the base creation module is used for indicating a downstream system to create a downstream base corresponding to the new base dividing rule; the data sending module 230 is further configured to: and issuing the image characteristic data to the newly created downstream base.
In one implementation of the database partitioning apparatus 200, the image feature data received from the upstream system is provided with tag information, where the tag information includes a tag, or a tag and a value thereof; when the image characteristic data is not matched with the created library splitting rule on the intermediate layer system, the rule creating module creates a new library splitting rule which can be matched with the image characteristic data, and the rule creating module comprises: and if the image characteristic data is not matched with the established library dividing rule on the middle layer system, establishing a new library dividing rule which can be matched with the image characteristic data according to the label information of the image characteristic data.
In one implementation manner of the database partitioning apparatus 200, the apparatus further includes: a rule loading module, configured to load the created sublibrary rule on the middle-layer system from a database into a memory before the rule matching module 220 determines whether the image feature data matches the created sublibrary rule on the middle-layer system; and the database management system is used for polling the database partitioning rules stored in the database, and loading the updated database partitioning rules from the database to the memory when the polling result shows that the database partitioning rules stored in the database are updated.
In one implementation manner of the database partitioning apparatus 200, the apparatus further includes: a rule creating module, configured to create a new banking rule before the rule matching module 220 determines whether the image feature data matches a banking rule already created on the middle-layer system; and the bottom base creating module is used for indicating the downstream system to create a downstream bottom base corresponding to the new library dividing rule.
In one implementation manner of the data banking device 200, the rule creating module creates a new banking rule, including: creating a new banking rule in which a downstream base is specified; the base library creating module instructs a downstream system to create a downstream base library corresponding to the new library partitioning rule, and comprises the following steps: instructing the downstream system to create a downstream base specified in the new banking rule.
In one implementation manner of the data banking device 200, the rule creating module creates a new banking rule, including: creating a new library partitioning rule in which a middle-layer base is specified; the base library creating module instructs a downstream system to create a downstream base library corresponding to the new library partitioning rule, and comprises the following steps: and instructing a downstream system to create a downstream base corresponding to the intermediate-layer base, and storing a mapping relation between the intermediate-layer base and the downstream base on the intermediate-layer system.
In one implementation of the database partitioning apparatus 200, the image feature data received from the upstream system is provided with tag information, where the tag information includes a tag and a value thereof; the rule creating module creates a new sublibrary rule, comprising: and creating a new library rule according to the combination of the specified unique label.
The implementation principle and the resulting technical effect of the database partitioning device 200 provided in the embodiment of the present application have been introduced in the foregoing method embodiments, and for the sake of brief description, no mention in the apparatus embodiment may be made to refer to the corresponding contents in the method embodiments.
Fig. 7 shows a possible structure of an electronic device 300 provided in an embodiment of the present application. Referring to fig. 7, the electronic device 300 includes: a processor 310, a memory 320, and a communication interface 530, which are interconnected and in communication with each other via a communication bus 340 and/or other form of connection mechanism (not shown).
The processor 310 includes one or more (only one is shown), which may be an integrated circuit chip having signal processing capability. The Processor 310 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Micro Control Unit (MCU), a Network Processor (NP), or other conventional processors; the Processor may also be a dedicated Processor, including a Graphics Processing Unit (GPU), a Neural-Network Processing Unit (NPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, and a discrete hardware component. Also, when there are multiple processors 510, some of them may be general-purpose processors and others may be special-purpose processors.
The Memory 320 includes one or more (Only one is shown in the figure), which may be, but not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an electrically Erasable Programmable Read-Only Memory (EEPROM), and the like. The processor 310, as well as possibly other components, may access, read, and/or write data to the memory 320.
One or more computer program instructions may be stored in the memory 320, and may be read and executed by the processor 310 to implement the database partitioning method provided by the embodiments of the present application.
Communication interface 330 includes one or more (only one shown) that may be used to communicate directly or indirectly with other devices for the purpose of data interaction. Communication interface 330 may include an interface to communicate wired and/or wireless.
It will be appreciated that the configuration shown in fig. 7 is merely illustrative and that electronic device 300 may include more or fewer components than shown in fig. 7 or have a different configuration than shown in fig. 7. The components shown in fig. 7 may be implemented in hardware, software, or a combination thereof. The electronic device 300 may be a physical device, such as a PC, a laptop, a tablet, a mobile phone, a server, an embedded device, etc., or may be a virtual device, such as a virtual machine, a virtualized container, etc. The electronic device 300 is not limited to a single device, and may be a combination of a plurality of devices or a cluster including a large number of devices.
The embodiment of the present application further provides a computer-readable storage medium, where computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are read and executed by a processor of a computer, the database partitioning method provided in the embodiment of the present application is executed. The computer-readable storage medium may be implemented as, for example, memory 320 in electronic device 300 in fig. 7.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (13)

1. A data banking method is applied to a middle layer system, and comprises the following steps:
receiving image feature data from an upstream system;
judging whether the image characteristic data is matched with a sub-library rule established on the intermediate layer system;
and if the image characteristic data is matched with a created library dividing rule, issuing the image characteristic data to a downstream base corresponding to the library dividing rule.
2. The database partitioning method according to claim 1, wherein the issuing the image feature data to a downstream base corresponding to a created partitioning rule if the image feature data matches the partitioning rule includes:
if the image characteristic data is matched with a created database partitioning rule, searching a downstream database corresponding to a middle-layer database specified in the database partitioning rule from a database mapping relation maintained on the middle-layer system;
and issuing the image characteristic data to the searched downstream base.
3. The database partitioning method according to claim 1 or 2, wherein the image feature data received from the upstream system is provided with label information, and the label information includes a label, or a label and a value thereof;
the judging whether the image feature data is matched with the established sublibrary rule on the intermediate layer system includes:
and judging whether the image characteristic data is matched with the established database partitioning rule on the intermediate layer system or not according to the label information carried by the image characteristic data.
4. The data banking method according to claim 3, wherein the tag information includes a tag and a value thereof, and the determining whether the image feature data matches a banking rule created on the middle layer system according to the tag information carried by the image feature data includes:
determining all possible sub-library rules according to the labels carried by the image characteristic data and the values of the labels; each possible sub-library rule corresponds to a combination, and the combination is a combination of a label and a value thereof;
judging whether a database partitioning rule exists in the created database partitioning rules on the intermediate layer system, wherein the label and the value of the database partitioning rule are consistent with the label and the value of a possible database partitioning rule;
if the image characteristic data exists, determining that the image characteristic data is matched with the database partitioning rule, otherwise determining that the image characteristic data is not matched with the database partitioning rule established on the middle layer system;
or,
for each database partitioning rule established on the intermediate layer system, judging whether the label and the value thereof specified in the database partitioning rule are contained in the label and the value thereof carried by the image characteristic data;
if the image feature data is contained in the database, determining that the image feature data is matched with the database partitioning rule, and otherwise, determining that the image feature data is not matched with the database partitioning rule.
5. The database partitioning method according to any one of claims 1 to 4, wherein said method further comprises:
if the image characteristic data is not matched with the established library dividing rule on the middle layer system, establishing a new library dividing rule which can be matched with the image characteristic data;
instructing the downstream system to create a downstream base corresponding to the new base partitioning rule;
and issuing the image characteristic data to the newly created downstream base.
6. The database partitioning method according to claim 5, wherein the image feature data received from the upstream system is provided with label information, and the label information includes a label, or a label and a value thereof;
if the image feature data is not matched with the created library splitting rule on the middle layer system, creating a new library splitting rule which can be matched with the image feature data, wherein the new library splitting rule comprises the following steps:
and if the image characteristic data is not matched with the established library dividing rule on the middle layer system, establishing a new library dividing rule which can be matched with the image characteristic data according to the label information of the image characteristic data.
7. The data banking method of any one of claims 1-6, wherein prior to the determining whether the image feature data matches a banking rule already created on the intermediate layer system, the method further comprises:
loading the created library dividing rules on the middle layer system from a database into a memory;
the method further comprises the following steps:
and polling the database partitioning rules stored in the database, and loading the updated database partitioning rules from the database to a memory when a polling result shows that the database partitioning rules stored in the database are updated.
8. The database partitioning method according to any one of claims 1 to 7, wherein said method further comprises:
before the image characteristic data is judged to be matched with the established library dividing rule on the intermediate layer system, establishing a new library dividing rule;
and instructing the downstream system to create a downstream base corresponding to the new library partitioning rule.
9. The data banking method of claim 8, wherein the creating of the new banking rule comprises:
creating a new banking rule in which a downstream base is specified;
the instructing the downstream system to create a downstream base corresponding to the new base partitioning rule includes:
instructing the downstream system to create a downstream base specified in the new base partitioning rule;
or,
and instructing a downstream system to create a downstream base corresponding to the intermediate-layer base, and storing a mapping relation between the intermediate-layer base and the downstream base on the intermediate-layer system.
10. The database partitioning method according to claim 8 or 9, wherein the image feature data received from the upstream system is provided with label information, and the label information includes a label and a value thereof;
the creating of the new sublibrary rule comprises the following steps:
and creating a new library rule according to the combination of the specified unique label.
11. A database partitioning apparatus, configured in a middle layer system, the apparatus comprising:
a data receiving module for receiving image feature data from an upstream system;
the rule matching module is used for judging whether the image characteristic data is matched with the established sub-library rule on the middle layer system;
and the data sending module is used for issuing the image characteristic data to a downstream base corresponding to the database partitioning rule when the image characteristic data is matched with the established database partitioning rule.
12. A computer-readable storage medium having stored thereon computer program instructions which, when read and executed by a processor, perform the method of any one of claims 1-10.
13. An electronic device comprising a memory and a processor, the memory having stored therein computer program instructions that, when read and executed by the processor, perform the method of any of claims 1-10.
CN202110695181.4A 2021-06-22 2021-06-22 Database partitioning method and device, storage medium and electronic equipment Pending CN113590609A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110695181.4A CN113590609A (en) 2021-06-22 2021-06-22 Database partitioning method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110695181.4A CN113590609A (en) 2021-06-22 2021-06-22 Database partitioning method and device, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN113590609A true CN113590609A (en) 2021-11-02

Family

ID=78244312

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110695181.4A Pending CN113590609A (en) 2021-06-22 2021-06-22 Database partitioning method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN113590609A (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030194689A1 (en) * 2002-04-12 2003-10-16 Mitsubishi Denki Kabushiki Kaisha Structured document type determination system and structured document type determination method
CN104750848A (en) * 2015-04-10 2015-07-01 腾讯科技(北京)有限公司 Image file treating method, server and image display device
JP2016071412A (en) * 2014-09-26 2016-05-09 キヤノン株式会社 Image classification apparatus, image classification system, image classification method, and program
WO2017201970A1 (en) * 2016-05-21 2017-11-30 乐视控股(北京)有限公司 Branch base database system and routing method therefor
CN109165639A (en) * 2018-10-15 2019-01-08 广州广电运通金融电子股份有限公司 A kind of finger vein identification method, device and equipment
CN109284675A (en) * 2018-08-13 2019-01-29 阿里巴巴集团控股有限公司 A kind of recognition methods of user, device and equipment
CN110022397A (en) * 2018-01-10 2019-07-16 广东欧珀移动通信有限公司 Image processing method, device, storage medium and electronic equipment
CN111126224A (en) * 2019-12-17 2020-05-08 成都通甲优博科技有限责任公司 Vehicle detection method and classification recognition model training method
CN111737270A (en) * 2019-11-12 2020-10-02 北京京东尚科信息技术有限公司 Data processing method and system, computer system and computer readable medium
CN111950364A (en) * 2020-07-07 2020-11-17 北京思特奇信息技术股份有限公司 System and method for identifying face of tens of millions of base libraries in different libraries
CN112395293A (en) * 2020-11-27 2021-02-23 浙江诺诺网络科技有限公司 Warehouse and table dividing method, warehouse and table dividing device, warehouse and table dividing equipment and storage medium
CN112784087A (en) * 2021-01-29 2021-05-11 平安科技(深圳)有限公司 Image retrieval method, image retrieval device, computer equipment and storage medium

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030194689A1 (en) * 2002-04-12 2003-10-16 Mitsubishi Denki Kabushiki Kaisha Structured document type determination system and structured document type determination method
JP2016071412A (en) * 2014-09-26 2016-05-09 キヤノン株式会社 Image classification apparatus, image classification system, image classification method, and program
CN104750848A (en) * 2015-04-10 2015-07-01 腾讯科技(北京)有限公司 Image file treating method, server and image display device
WO2017201970A1 (en) * 2016-05-21 2017-11-30 乐视控股(北京)有限公司 Branch base database system and routing method therefor
CN110022397A (en) * 2018-01-10 2019-07-16 广东欧珀移动通信有限公司 Image processing method, device, storage medium and electronic equipment
CN109284675A (en) * 2018-08-13 2019-01-29 阿里巴巴集团控股有限公司 A kind of recognition methods of user, device and equipment
CN109165639A (en) * 2018-10-15 2019-01-08 广州广电运通金融电子股份有限公司 A kind of finger vein identification method, device and equipment
CN111737270A (en) * 2019-11-12 2020-10-02 北京京东尚科信息技术有限公司 Data processing method and system, computer system and computer readable medium
CN111126224A (en) * 2019-12-17 2020-05-08 成都通甲优博科技有限责任公司 Vehicle detection method and classification recognition model training method
CN111950364A (en) * 2020-07-07 2020-11-17 北京思特奇信息技术股份有限公司 System and method for identifying face of tens of millions of base libraries in different libraries
CN112395293A (en) * 2020-11-27 2021-02-23 浙江诺诺网络科技有限公司 Warehouse and table dividing method, warehouse and table dividing device, warehouse and table dividing equipment and storage medium
CN112784087A (en) * 2021-01-29 2021-05-11 平安科技(深圳)有限公司 Image retrieval method, image retrieval device, computer equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
江梓豪;黄刘生;徐宏力;: "基于数据分片与分布式架构的室内定位中间件", 电子技术, no. 07, 25 July 2017 (2017-07-25) *
谭鹏;朱艳辉;杨芸桦;: "分布式数据访问层中间件的研究与实现", 电脑知识与技术, no. 24, 25 August 2018 (2018-08-25) *
陆伟;罗梦奇;丁恒;李信;: "深度学习图像标注与用户标注比较研究", 数据分析与知识发现, no. 05, 25 May 2018 (2018-05-25) *

Similar Documents

Publication Publication Date Title
KR101355273B1 (en) A computing system, a method for controlling thereof, and a computer-readable recording medium having a computer program for controlling thereof
CN112860777B (en) Data processing method, device and equipment
CN110134681B (en) Data storage and query method and device, computer equipment and storage medium
CN114327374A (en) Business process generation method and device and computer equipment
CN115168162B (en) Multi-gray-scale issuing method and device based on ingess controller in container environment and storage medium
CN110221829A (en) Information processing method and its system, computer system and computer-readable medium
CA3142770A1 (en) Component linkage configuration method, device, computer equipment and storage medium
CN109726295A (en) Brand knowledge map display methods, device, figure server and storage medium
CN114185895A (en) Data import and export method and device, electronic equipment and storage medium
CN116383223A (en) Asset data processing method, related device and storage medium
KR102153259B1 (en) Data domain recommendation method and method for constructing integrated data repository management system using recommended domain
CN112631716A (en) Database container scheduling method and device, electronic equipment and storage medium
CN112035471A (en) Transaction processing method and computer equipment
US11048756B2 (en) Inserting datasets into database systems utilizing hierarchical value lists
KR20100083778A (en) Acquisition and expansion of storage area network interoperation relationships
CN111858366A (en) Test case generation method, device, equipment and storage medium
CN113590609A (en) Database partitioning method and device, storage medium and electronic equipment
CN112162731B (en) Data expansion method, device, storage medium and electronic device
CN111339245A (en) Data storage method, device, storage medium and equipment
CN114020986B (en) Content retrieval system
CN112929398B (en) Data sharing method, device, server and storage medium
CN117270909A (en) Project optimization method, apparatus, computer device, storage medium, and product
CN114201538B (en) Data processing method and device, intelligent equipment and computer storage medium
WO2022004455A1 (en) Computer system and application programming interface device for achieving cooperation between objects with categorized input/output using object groups in which placeable object categories are defined
CN118409956A (en) Performance test method, apparatus, computer device, storage medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination