CN115357607A - Mass data query method and device based on personal health big data platform - Google Patents

Mass data query method and device based on personal health big data platform Download PDF

Info

Publication number
CN115357607A
CN115357607A CN202210998876.4A CN202210998876A CN115357607A CN 115357607 A CN115357607 A CN 115357607A CN 202210998876 A CN202210998876 A CN 202210998876A CN 115357607 A CN115357607 A CN 115357607A
Authority
CN
China
Prior art keywords
data
type
query
dimensional data
database
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210998876.4A
Other languages
Chinese (zh)
Inventor
钱军波
汤人杰
陈俊杰
王清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Mobile Information System Integration Co ltd
Original Assignee
Zhejiang Mobile Information System Integration Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Mobile Information System Integration Co ltd filed Critical Zhejiang Mobile Information System Integration Co ltd
Priority to CN202210998876.4A priority Critical patent/CN115357607A/en
Publication of CN115357607A publication Critical patent/CN115357607A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2219Large Object storage; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of big data storage and query application, in particular to a mass data query method and a mass data query device based on a personal health big data platform, which comprise the following steps: receiving a query request and query information sent by a user from an application end, wherein the query information comprises A-type dimension data; sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data; inquiring corresponding B-type dimension data in a Redis memory real-time database based on the index value; inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data; generating result data based on A-type dimensional data, B-type dimensional data and C-type dimensional data corresponding to the query information; and sending the result data to an application end which sends the query request and the query information.

Description

Mass data query method and device based on personal health big data platform
Technical Field
The invention relates to the technical field of big data storage and query application, in particular to a mass data query method and a mass data query device based on a personal health big data platform.
Background
The sources and kinds of personal health data are various, including clinical data of medical institutions, physical examination data of community health services, health monitoring data collected by wearable devices, dynamic personal health data of user apps and the like. Conventional personal health systems manage and analyze personal health data only through a simple data list. This does not fully exploit the capabilities of different types of health data in health analysis and prognosis. Millions of individual users can generate billions of data, and an individual health big data platform needs to support fast query analysis of massive real-time streaming data. Meanwhile, as user data continuously expands for a long time, continuously accumulated mass historical data needs a new method for supporting a query mode different from real-time stream data.
The conventional real-time stream data storage adopts an internal memory or a cache database, and has the constraints of volatility and limited space. The vast amount of total historical data presents a greater challenge to storage space and distributed query analysis. The storage capacity, query efficiency, index length and the like of a common relational database or a non-relational database cannot be limited, and mass data information of a larger data quantity hierarchy cannot be stored and queried more efficiently.
Disclosure of Invention
Aiming at the defects in the prior art, one of the purposes of the invention is to provide a mass data query method based on a personal health big data platform, which can provide different query methods for real-time data and historical data according to various database characteristics, realize the storage of mass health data and quickly query relevant data based on an application request.
The technical purpose of the invention is realized by the following technical scheme:
a mass data query method and device based on a personal health big data platform comprise the following steps:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimensional data, and the A-type dimensional data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-class dimensional data and the C-class dimensional data, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database.
Further, when the index table in the elastic search index database is updated in real time, the updating of the elastic search index database is triggered by monitoring whether the Key value of the hash mapping table in the Redis database changes, and when the real-time data is synchronously written into the Redis database, the updated index value is written into the elastic search index database in real time.
Furthermore, when a receiving user sends a query request and query information from the application terminal, the synchronous response verifies the request information sent by the requesting user, and responses and feedbacks are carried out according to the determination information of the request.
Further, the method for converting the class B dimension model part into the class C dimension model comprises the following steps: and storing the operation command for modifying the data into the ASAP file, re-executing the command in the ASAP file after Redis restarting, and performing persistent data storage on the real-time data by adopting an AOF (automated optical resource Format) method of a Redis database.
Further, index values in the HBase distributed database are changed periodically.
Further, when sending the result data to the application terminal that sends the query request and the query information, the method further includes:
and forming the result data fed back to the application end into a periodic curve feedback value application end.
Aiming at the defects in the prior art, the invention provides a mass data query device based on a personal health big data platform, which can provide different query methods for real-time data and historical data according to various database characteristics, realize the storage of mass health data and quickly query relevant data based on application requests.
The technical purpose of the invention is realized by the following technical scheme:
mass data inquiry unit based on personal health big data platform includes: the system comprises a data acquisition unit, a business server side, a server side and an application side;
the server side comprises:
the user request receiving/feedback module is used for receiving the query request and the query information sent by the application terminal and sending result data to the application terminal which sends the query request and the query information;
the keyword index value query module is used for sending a request to an elastic search index database of the platform service server side, querying the index value and determining the index value corresponding to the A-type dimensional data;
the real-time stream data query analysis module is used for querying corresponding B-type dimensional data in a Redis memory real-time database based on the index value;
the historical data query analysis module is used for querying corresponding C-type dimensional data in the HBase distributed database based on the index value;
and the result data generation module generates result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information.
Aiming at the defects in the prior art, the invention also aims to provide electronic equipment which can provide different query methods for real-time data and historical data according to various database characteristics, realize storage of massive health data and quickly query related data based on application requests.
The technical purpose of the invention is realized by the following technical scheme:
an electronic device comprises a RAM memory, a ROM memory, a communication interface, a processor and a bus;
wherein, the processor, the communication interface, the RAM memory and the ROM memory are connected through a bus,
the RAM memory is used to store computer programs to support the processor in performing the following operations:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-class dimensional data and the C-class dimensional data, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database;
the ROM memory is used to store data and the processor is configured to execute programs stored in the RAM memory.
In view of the defects in the prior art, a fourth object of the present invention is to provide a machine-readable medium, which can provide different query methods for real-time data and historical data according to various database characteristics, so as to store massive health data and quickly query relevant data based on an application request.
The technical purpose of the invention is realized by the following technical scheme:
a machine-readable medium having stored thereon machine-executable instructions that, when invoked and executed by a processor, cause the processor to:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-type dimensional data and the C-type dimensional data, establishing an index data table aiming at the extracted key index values and the A-type dimensional data, and storing the index data table in an elastic search index database.
In conclusion, the invention has the following beneficial effects:
the method comprises the steps that a data dimension grading mode is adopted to store and manage data in a grading mode, an application end sends A-class dimension data related to a personal health user according to application of the data, according to the data, a key index value is quickly inquired from an elastic search index database, and then corresponding B-class dimension data is inquired in a Redis memory database according to the index value so as to support quick response of a real-time inquiry request; and inquiring C-type dimension data in the HBase distributed database according to the index value so as to support more complete full-data inquiry and detailed analysis processing.
Drawings
FIG. 1 is a schematic diagram illustrating data dimension classification in a health big data platform according to an embodiment of the present invention;
fig. 2 is a flow chart of steps of a mass data query method based on a health big data platform according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an application of mass data query based on a health big data platform according to an embodiment of the present invention;
fig. 4 is an exemplary diagram of an association relationship between three types of dimensional information of three databases and an index according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a hardware architecture of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
For a more clear description of the objects, solutions and features of the embodiments of the present invention, the solutions of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described herein are not all embodiments of the present invention, and the detailed steps of the implementation process of the technical solution are described only by one exemplary embodiment. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, the technical scheme aiming at mass data storage and query of a healthy big data platform comprises the following steps:
1) Redis is a high-performance key-value database, and the real-time stream data is more suitably stored by using memory databases such as Redis and the like. First, it supports storing more value types and richer operations, and these operations are all atomic. Secondly, the data are all cached in the memory, and the efficiency is improved. The implementation of master-slave synchronization then helps scalability of read operations and data redundancy. However, the problem with memory databases is that a large increase in data in a short time will result in memory overflow. And the single thread makes a single server unable to fully utilize the CPU of the multi-core server.
2) HBase is a distributed database which is based on columns, high in reliability, high in performance, scalable, and capable of reading and writing in real time. When a non-relational database such as HBase is used for mass data storage, the high-availability PB-level data support and high-performance instant write quantity can be solved. However, the inherent limitations of a single RowKey make it impossible to efficiently support multi-condition queries and unsuitable for a wide range of scan queries. The multi-dimensional query requirement of mass data under the personal health platform is difficult to support through the limited index.
3) Relational databases such as MySQL and the like store data by using a relational table under the support of a relational data model. Single point, replication, cluster architectures may be employed to adapt to applications of different scales. Meanwhile, multidimensional query can be carried out on the personal health data through a compound index method. However, in the mass storage process of MySQL, the memory usage amount of each connection will be greatly increased. Making it more difficult to develop storage processes with complex business logic. When the query application for mass data is oriented, the limited storage process and the low query efficiency are influenced.
In order to better understand the specific method mentioned in this embodiment, a mass data query method based on a personal health big data platform disclosed in the embodiment of the present invention is further described in detail. The personal health information is divided into multi-dimensional attributes and multi-class dimensional data (A, B, C three classes of dimensions) in the embodiment of the invention. As shown in fig. 1, the a-type dimension data mainly includes personal basic health information such as basic data and general data; the B-type dimension data mainly comprises real-time acquisition monitoring information such as life data and behavior data; the C-type dimension data mainly comprises historical medical disease information such as periodic physical examination data and medical clinical data.
In summary, the embodiment of the invention provides a mass data query method and device based on a personal health big data platform. Firstly, providing different query methods for real-time data and historical data by adopting various database models to realize query of massive health data; secondly, aiming at multi-dimensional health data of billions level in practical application, quick response is realized.
As shown in fig. 2, the mass data query method based on the personal health big data platform includes:
step S01: the server receives a query request and query information sent from an application terminal, where the query information includes a type a dimension data, and the type a dimension data includes basic data and personal basic health information of conventional data, where the type a dimension data in this embodiment includes one or more of the following: community number, time spent in bed, age, etc.
Step S02: according to the A-type dimensional data obtained by the application end, the server end sends a request to the healthy big data platform service server, information query is carried out in an elastic search index database of the platform service server end, and an index value corresponding to the A-type dimensional data is determined.
Step S03: inquiring corresponding B-type dimension data in a Redis memory real-time database based on the index value, wherein the B-type dimension data comprises real-time acquisition monitoring information of life data and behavior data;
the B-type dimension data is stored in a Redis database in an organization mode of establishing Key-Value data by real-time data, the Redis adopts a primary Key for query, the query efficiency is O (1) and is not influenced by the number of Key Value pairs, in addition, the health data storage scheme utilizes a hash data structure of the Redis, when one primary Key is queried, all contained information can be obtained, and the real-time query operation is further simplified.
Step S04: inquiring corresponding C-type dimension data in an HBase distributed database based on the index value, wherein the C-type dimension data comprises health full history data;
the C-type dimension data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data.
Step S05: the server side generates result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
the result data comprises the acquired data information and a corresponding analysis result.
Step S06: and the server side sends the result data to the application side which sends the query request and the query information.
In the actual application process, a user sends a query request to a server through an application end of the big health data platform, wherein the query request corresponds to health information in the A-type dimensional data. The server side starts searching in the elastic search index database aiming at the information, and searches the corresponding memory database index value and the distributed database index value. And searching the real-time stream data and the historical data according to the index values respectively. Under the index value, query analysis aiming at the B-type dimension data is carried out by querying from a Redis database; query analysis for the C-type dimension data is performed through the HBase distributed database. Therefore, multi-dimensional storage of mass data can be realized for diversified healthy big data, and millisecond-level response can be met for data with different characteristics. By using the query method of the embodiment of the invention, the dimension attribute conditions of the real-time monitoring data meeting the specific conditions can be quickly found according to the A-type dimension data, and the corresponding instant service is simply statistically analyzed and responded; the health history data can be quickly searched, and more comprehensive service is further provided through an intelligent or complex analysis method.
As shown in fig. 3, in the health big data platform, there are data collected in a real-time manner, such as health data of wearable devices such as self-bracelet and heart rate belt, sleep and diet data collected from App terminal, data collected from nursing monitoring instrument, and data collected from physical sign index detecting instrument. There are also medical record data and physical examination data transferred or loaded in an off-line batch processing mode. When the number of participated people reaches a certain scale, the whole storage capacity can reach mass data of billions of orders of magnitude. The data acquisition work is the basis of the platform and is the premise of storage and query analysis. Aiming at data of different types and different sources, database writing is realized through data monitoring, real-time data receiving cluster, flow calculation, protocol analysis, cleaning, transferring, loading and the like of batch processing data. As an embodiment supporting diversified query analysis, the health big data platform can provide various application-side services for different users, such as services of early warning, prediction, statistics and the like through real-time analysis and big data analysis.
The application end is taken as an example to explain the application of the application end for assisting the healthcare and nursing of the senile diseases. The nursing process of the senile diseases is mostly influenced by seasonal and regional characteristics, the change of cardiopulmonary sign indexes, blood pressure and the like of the old at different time periods and the good or bad diet and sleep conditions are generally required to be observed, and the early prevention is carried out on seasonal diseases and symptoms, particularly cardiovascular diseases. The embodiment utilizes wearable equipment, monitoring instruments and the like to collect real-time data such as sign data and life behavior data, and the community is used as a unit to support nursing staff to give health care suggestions in time through query analysis. And more accurate analysis and suggestion can be performed by combining the real-time data of the target old people and the historical data of the target old people such as physical examination, medical history and the like.
In step S01, before sending the query request and the data of the query information from the application to the server, the first processing operation is not to directly connect to the service server for data search, which includes:
and step S01.1, performing necessary verification on the request information sent by the requester user through synchronous response, and performing response and feedback according to the determination information of the request.
Similarly, when the server and the application reach the request confirmation, the query request information needs to be sent to the service server first before the step S02 starts querying the index database, and the connection confirmation is attempted.
Accordingly, the method in an embodiment further comprises:
and step S02.1, if the server establishes connection with the service server through the determined request information, further sending data and waiting for the service server to return a result to the server through specific service inquiry and analysis. The server side feeds back result information to the application side.
In an embodiment of practical application, the a-dimensional data input by the healthcare application for geriatric diseases at the application end is listed as: inquiring target old people with community number (CommID, data type: character type) of A1, bed-ridden time (TiB, data type: numerical type, unit: hour) of more than half a year and Age (Age, data type: numerical type) of more than 70, and performing health condition inquiry analysis and nursing prediction. The service end inquires the B-type dimension real-time data of the old people meeting the conditions from the business server to perform inquiry statistics, and further inquires C-type dimension disease medical record, physical examination and other full-health historical data of specific targets to perform more targeted nursing prediction on individuals.
A specific processing procedure example: firstly, the A-type dimension data of "CommID = A1", "TiB >4320" and "Age >70" are respectively inquired from the elastic search database, and the target index value meeting the condition is inquired. Secondly, real-time monitoring data of the current index value target, such as Blood Pressure (BP), blood oxygen value (SaO 2) and the like, are inquired from the Redis database in real time according to the index value, and a primary statistical analysis result is carried out and returned to the server side. And then, further inquiring a distributed database HBase according to the index value, inquiring the corresponding full-health data of the target old man, carrying out off-line big data analysis and prediction, and sequentially returning the result to the server.
The real-time data of the B-type dimensional data continuously changes along with the monitoring process, the detection process and the acquisition of equipment. These changes should not all be purged with the stream data. Partial data should be converted into full-health data and written into the HBase database.
After step S03, in order to further retain valuable content in the real-time data, it is necessary to store the valuable content in the disk, and to maintain consistency and correctness of the data. The present embodiment is based on the characteristics of the Redis database, and further includes:
step S03.1: and (3) adopting an AOF (application Only File) method of a Redis database to store persistent data of the real-time data.
First, all the operation commands for modifying data are saved in the ASAP file. And secondly, after the Redis is restarted, executing the commands in the ASAP file again. The method only loses 1 second of data under the condition of power failure or write-once error, and the stored data has good reliability.
In a specific embodiment, after the method queries the full-health data, the queried full-health data is analyzed, and in step S0.6, the analysis result and the queried comprehensive full-health data of the target object are returned. The business server of the health big data platform provides query results and analysis results to the server side in an interface mode. And then, the server feeds back the analysis result and the inquired full-health data to an application page through service combination packaging, and the analysis result and the inquired full-health data are presented to the user in a key index tracking monitoring mode, a problem index periodic curve changing mode and the like.
In an embodiment of practical application, the establishment and planning of the database are the premise of mass data query of a health big data platform. Therefore, the method of the present invention further comprises, before step S01:
step S00.1, establishing a data table for the historical data obtained by the data acquisition end, namely C-type dimension data is stored in the HBase distributed database in the data table organizing mode. As shown in the example of the database table in fig. 4, the collected historical data is stored in the HBase database and organized according to the C-type dimension data. The social security number is a main key, and an index is established to uniquely identify each row of target patient individual data. The attributes in this embodiment are only illustrated as community, hospital, confirmed disease and hospital stay, but not limited to the current attribute classification.
Step S00.2, a Key-Value database is established for the real-time data obtained by the data acquisition end, that is, the B-type dimensional data is stored in the Redis database in an organization mode of the Key-Value data. As shown in the example of the database table in fig. 4, the collected real-time data is stored in the Redis database, and is organized according to the B-type dimension data, and the data types are processed in the manner of a hash mapping table. The social security number and the mobile phone number of the user name are keys (keys), the device ID, the step number, the blood oxygen Value and the like are fields (fields) of a hash table, and the specific Value (Value) is used for identifying real-time data of the individual user name or the social security number. In this embodiment, only some domains are listed, but not limited to these.
And step S00.3, extracting key index values from the Redis database and the HBase database based on the C-type dimensional data and the B-type dimensional data. In the example database of fig. 4, the index value is the user's mobile phone number or social security number. In the embodiment, for the A-type dimension data, community codes, user mobile phone numbers, social security number relationships, bed-bound time, user mobile phone numbers, social security number relationships and age, user mobile phone numbers and social security number relationships are extracted.
Step S00.4, an index data table can be established for the relation between the key index value and the A-type dimensional data extraction, and the index data table is stored in an elastic search index database. As shown in the example of fig. 4, three tables are created according to the class a dimension data and the index value of the present embodiment. The data organization corresponding to the rest of the A-type dimension data is similar.
The specific step of S00.1 may further include step S00.1.1. And based on the B-type dimension data, acquiring a corresponding index value in real time aiming at the real-time data in the Redis database. In order to ensure the consistency and accuracy of the application-side query, the index table in the elastic search index database needs to be updated in real time.
And acquiring a corresponding index value from a hash mapping table of the Redis database according to the updating triggering condition and the B-type dimension data. The triggering condition comprises a condition that a Key value changes in the hash mapping table is monitored. And synchronously writing the real-time data into a Redis database in real time through a preset updating rule, and writing the index relation updating data into an elastic search index database in real time.
Similarly, the change of the historical data will also affect the consistency of the index database, and in order to maintain the consistency of the index database, the method further comprises the following steps:
step S00.1.2, performing periodic maintenance on distributed data in the HBase database based on the C-type dimension data. Historical data in the distributed database do not need to meet high timeliness, and the index table is updated without monitoring data changes in real time. When only the main key information or the index value in the table is considered to be changed, the elastic search index database is periodically updated by extracting the changed index value.
In an exemplary application of the embodiment, the process of step S00.1.1 is illustrated by specific data. As shown in fig. 4, tiB in the elastic search index database records the relationship between the attribute of time in bed and the key index value, but the index relationship needs to be updated in real time as the time in bed changes. If the mobile phone number of the user is '13849578999', and the social security number is '28395038', the time-in-bed data acquired through the wearable device is 8769 hours. With time change, the data collected in real time changes to 8770 hours, and update data will be triggered according to step S03.1.1. '13849578999' and '28395038' will be added to the user's mobile phone number and social security number attribute with TiB 8770 in the indexing relationship. And simultaneously, deleting the corresponding user mobile phone number and social security number with TiB of 8769.
The process of step S00.1.2 is illustrated by the specific data. As shown in FIG. 4, commID in the elastic search index database records the relationship between the community attribute and the key index value. When the community of the user changes or some new user data enters the HBase database, the index relationship needs to be changed accordingly. If the mobile phone number of the user is '13849578999', the community with the social security number of '28395038' is 'A1'. When the community of the user is changed, if the community is changed to 'B3', the index database is updated according to the step S00.1.2 during regular maintenance. '13849578999' and '28395038' are added to the user's mobile phone number and social security number attribute with CommID B3 in the index relationship. And simultaneously, deleting the corresponding user mobile phone number and social security number in the data with CommID A1. This update operation also occurs when new data is added to the class C dimensional data.
In an embodiment of an exemplary use case of the present invention, as shown in fig. 4, for a basic situation of a target elderly person, such as a demand of a physical sign index, an exercise situation, a hospitalization situation, etc., with a query community number A1, and a bedridden time of more than half a year and an age of more than 70, which is proposed by an application, a server sends a request to an elastic search index database. Inquiring user information (13849578999, 28395038) and (13245328976, 21134910) of CommI = A1 in a CommID-user mobile phone number-social security number relation; inquiring user information (13849578999, 28395038) of TiB >4320 in the relationship of TiB-user mobile phone number-social security number; the user information of Age >70 is inquired in the relationship of Age-user mobile phone number-social security number, and all user index values of ages 82 and 73 are included in the user information. The index value satisfying three conditions at the same time by comprehensive consideration is (13849578999, 28395038). Finally, querying from the Redis database based on the two index values, it can be known that the current step number of the user meeting the condition is 7865, the ID of the device for collection is 001, and the time spent in bed is 8769 hours. The user can also be inquired that the real-time physical examination result of the QA786 institution is the blood oxygen value 96 and the hypertension 123. Meanwhile, based on two index values, the HBase database is inquired, the community where the user meets the conditions is A1, the hospital stays is YDEY, the diagnosed disease is degenerative, and the hospitalization time is 2020.10.11. On the basis of inquiring the target data, according to the functions of analysis, prediction and the like provided by the service server, the inquiry result can be further processed, and is pushed to the server side for packaging and fed back to the application side user.
It can be understood that the real-time data and the full-health data of the big health data platform include many data, such as dietary calories, sleep duration, other physical signs, family medical history, hospitalization medicine, and the like. The embodiments of the present invention are merely illustrative of several data for convenience of description, and are not limited thereto.
The mass data inquiry device based on the personal health big data platform comprises the following components in one implementation mode: the system comprises a data acquisition device, a service server side, a server side and an application side.
Wherein, the server side includes:
the user request receiving/feedback module is used for receiving the query request and the query information sent by the application terminal and sending result data to the application terminal which sends the query request and the query information;
the keyword index value query module is used for sending a request to an elastic search index database of the platform service server side, querying the index value and determining the index value corresponding to the A-type dimensional data;
the real-time stream data query analysis module is used for querying the corresponding B-type dimensional data in the Redis memory real-time database based on the index value;
the historical data query analysis module is used for querying corresponding C-type dimensional data in the HBase distributed database based on the index value;
and the result data generation module generates result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information.
In the following, detailed description is given of an electronic device according to an embodiment of the present application, and in a hardware level, in this embodiment, the electronic device includes a wearable device, a smart phone, a personal computer, a notebook computer, a monitoring device, a detection instrument, a server, and other computer devices with analysis and processing capabilities.
As an exemplary embodiment, see fig. 5, an electronic device comprises a communication interface, a processor, a RAM memory, a ROM memory, and a bus.
The processor, the communication interface and the two memories are connected through a bus; the RAM memory is used to store computer programs that support the processor to perform the mass data querying method, the ROM memory is used to store data, and the processor is configured to execute the programs stored in the memory.
A machine-readable storage medium as referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (random Access Memory), volatile Memory, and nonvolatile Memory. Non-volatile memory includes, among other things, flash memory, a storage drive (e.g., a hard disk drive), any type of storage disk (e.g., an optical disk, dvd, etc.), or similar storage media, or a combination thereof.
The RAM memory is used to store computer programs to support the processor in performing the following operations:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by the real-time data;
inquiring corresponding C-type dimensional data in the HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by using the history data;
generating result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on B-type dimensional data and C-type dimensional data, establishing an index data table aiming at the extracted key index values and A-type dimensional data, and storing the index data table in an elastic search index database;
the ROM memory is used to store data and the processor is configured to execute programs stored in the RAM memory.
The mass data query method disclosed in the embodiment of fig. 1 of the present application may be applied to a processor, or may be implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, etc. as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and combines hardware thereof to complete the steps of the method.
Of course, besides the software implementation, the electronic device of the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.
Embodiments of the present application also provide a machine-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the method of the embodiment shown in fig. 1, and are specifically configured to perform the following method:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by the real-time data;
inquiring corresponding C-type dimensional data in the HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by using the history data;
generating result data based on A-type dimensional data, B-type dimensional data and C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from the B-class dimensional data and the C-class dimensional data in a Redis database and an HBase distributed database, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present embodiment is only for explaining the present invention, and it is not limited to the present invention, and those skilled in the art can make modifications of the present embodiment without inventive contribution as needed after reading the present specification, but all of them are protected by patent law within the scope of the claims of the present invention.

Claims (9)

1. A mass data query method based on a personal health big data platform is characterized by comprising the following steps:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on A-type dimensional data, B-type dimensional data and C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-class dimensional data and the C-class dimensional data, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database.
2. The mass data query method based on the personal health big data platform according to claim 1, characterized in that when updating the index table in the elastic search index database in real time, it triggers the updating of the elastic search index database by monitoring whether the Key value of the hash mapping table in the Redis database changes, and when real-time data is synchronously written into the Redis database, the updated index value is written into the elastic search index database in real time.
3. The mass data query method based on the personal health big data platform according to claim 1, characterized in that when receiving a query request and query information sent by a user from an application terminal, a synchronous response verifies the request information sent by a requesting user, and responses and feedbacks are performed according to the determination information of the request.
4. The mass data query method based on the personal health big data platform according to claim 1, wherein the method for converting the class B dimensional model part into the class C dimensional model comprises the following steps: and storing the operation command for modifying the data into the ASAP file, re-executing the command in the ASAP file after Redis restarting, and performing persistent data storage on the real-time data by adopting an AOF (automated optical resource Format) method of a Redis database.
5. The mass data query method based on the personal health big data platform according to claim 4, wherein the index value in the HBase distributed database is changed periodically.
6. The mass data query method based on the personal health big data platform according to claim 1, wherein when sending the result data to the application terminal sending the query request and the query information, the method further comprises:
and forming the result data fed back to the application end into a periodic curve feedback value application end.
7. A mass data inquiry device based on a personal health big data platform is characterized by comprising: the system comprises a data acquisition unit, a business server side, a server side and an application side;
the server side comprises:
the user request receiving/feedback module is used for receiving the query request and the query information sent by the application terminal and sending result data to the application terminal which sends the query request and the query information;
the keyword index value query module is used for sending a request to an elastic search index database of the platform service server side, querying the index value and determining the index value corresponding to the A-type dimensional data;
the real-time stream data query analysis module is used for querying the corresponding B-type dimensional data in the Redis memory real-time database based on the index value;
the historical data query analysis module is used for querying corresponding C-type dimensional data in the HBase distributed database based on the index value;
and the result data generation module generates result data based on the A-type dimension data, the B-type dimension data and the C-type dimension data corresponding to the query information.
8. An electronic device, comprising a RAM memory, a ROM memory, a communication interface, a processor, a bus;
wherein, the processor, the communication interface, the RAM memory and the ROM memory are connected through a bus,
the RAM memory is used to store computer programs to support the processor in performing the following operations:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on the A-type dimensional data, the B-type dimensional data and the C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-class dimensional data and the C-class dimensional data, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database;
the ROM memory is used to store data and the processor is configured to execute programs stored in the RAM memory.
9. A machine-readable medium having stored thereon machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform operations comprising:
receiving a query request and query information sent by a user from an application terminal, wherein the query information comprises A-type dimension data, and the A-type dimension data comprises basic data and personal basic health information of conventional data;
sending a request to an elastic search index database of a platform service server side, inquiring an index value, and determining an index value corresponding to the A-type dimensional data;
inquiring corresponding B-type dimensional data in a Redis memory real-time database based on the index Value, wherein the B-type dimensional data comprises real-time acquisition monitoring information of life data and behavior data, and the B-type dimensional data is stored in the Redis database in an organization mode of establishing Key-Value data by real-time data;
inquiring corresponding C-type dimensional data in an HBase distributed database based on the index value, wherein the C-type dimensional data comprises health full history data, and the C-type dimensional data is stored in the HBase distributed database in an organization mode of establishing a data table by historical data;
generating result data based on A-type dimensional data, B-type dimensional data and C-type dimensional data corresponding to the query information;
sending result data to an application end which sends a query request and query information;
wherein,
extracting key index values from a Redis database and an HBase distributed database based on the B-class dimensional data and the C-class dimensional data, establishing an index data table aiming at the extracted key index values and the A-class dimensional data, and storing the index data table in an elastic search index database.
CN202210998876.4A 2022-08-19 2022-08-19 Mass data query method and device based on personal health big data platform Pending CN115357607A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210998876.4A CN115357607A (en) 2022-08-19 2022-08-19 Mass data query method and device based on personal health big data platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210998876.4A CN115357607A (en) 2022-08-19 2022-08-19 Mass data query method and device based on personal health big data platform

Publications (1)

Publication Number Publication Date
CN115357607A true CN115357607A (en) 2022-11-18

Family

ID=84002847

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210998876.4A Pending CN115357607A (en) 2022-08-19 2022-08-19 Mass data query method and device based on personal health big data platform

Country Status (1)

Country Link
CN (1) CN115357607A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116108020A (en) * 2023-04-10 2023-05-12 科技日报社 Data analysis and storage method and device for media information base containing complex information source

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116108020A (en) * 2023-04-10 2023-05-12 科技日报社 Data analysis and storage method and device for media information base containing complex information source
CN116108020B (en) * 2023-04-10 2023-06-06 科技日报社 Data analysis and storage method and device for media information base containing complex information source

Similar Documents

Publication Publication Date Title
CN110109887B (en) Data retrieval method, electronic device, and computer storage medium
Eswari et al. Predictive methodology for diabetic data analysis in big data
JP6956107B2 (en) Indistinguishable Healthcare Database Hospital Matching Without Clear Matching Information
US20160203050A1 (en) Data replication in a database management system
US9026550B2 (en) Temporal pattern matching in large collections of log messages
GB2541231A (en) Hybrid data storage system and method and program for storing hybrid data
Lin et al. Temporal event tracing on big healthcare data analytics
WO2013090413A1 (en) Storing structured and unstructured clinical information for information retrieval
CN106933859B (en) Medical data migration method and device
CN111046036A (en) Data synchronization method, device, system and storage medium
CN109087688B (en) Patient information acquisition method, apparatus, computer device and storage medium
US20180046779A1 (en) Caching technology for clinical data sources
CN115357607A (en) Mass data query method and device based on personal health big data platform
Sauvayre Types of errors hiding in Google Scholar Data
CN113658712A (en) Doctor-patient matching method, device, equipment and storage medium
JP2015018462A (en) Medical chart system and medical chart search method
CN115516432A (en) Method and system for identifying, managing and monitoring data dependencies
Bukhari et al. Hadoop based demography big data management system
JP7213890B2 (en) Accelerated large-scale similarity computation
Zhang et al. A distributed storage model for healthcare big data designed on HBase
Godinho et al. Assessing the relational database model for optimization of content discovery services in medical imaging repositories
Xie et al. Cool, a COhort OnLine analytical processing system
C. Flint et al. Automated extraction of structured data from text notes in the electronic medical record
Gamal et al. Integrated Document-Based Electronic Health Records Persistence Framework
Cerbulescu et al. Proposed Architecture to manage critical states predictions in IoT applications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination