CN117196694A - Medicine market data analysis method and system based on big data - Google Patents
Medicine market data analysis method and system based on big data Download PDFInfo
- Publication number
- CN117196694A CN117196694A CN202311444060.8A CN202311444060A CN117196694A CN 117196694 A CN117196694 A CN 117196694A CN 202311444060 A CN202311444060 A CN 202311444060A CN 117196694 A CN117196694 A CN 117196694A
- Authority
- CN
- China
- Prior art keywords
- data
- market
- module
- analysis
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003814 drug Substances 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000007405 data analysis Methods 0.000 title claims abstract description 65
- 229940079593 drug Drugs 0.000 title claims description 71
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 87
- 238000004458 analytical method Methods 0.000 claims abstract description 58
- 238000012544 monitoring process Methods 0.000 claims abstract description 45
- 238000011160 research Methods 0.000 claims abstract description 41
- 238000002648 combination therapy Methods 0.000 claims abstract description 37
- 238000004140 cleaning Methods 0.000 claims abstract description 22
- 230000003993 interaction Effects 0.000 claims abstract description 16
- 238000003012 network analysis Methods 0.000 claims abstract description 13
- 238000013480 data collection Methods 0.000 claims abstract description 11
- 230000000694 effects Effects 0.000 claims abstract description 8
- 238000005457 optimization Methods 0.000 claims description 65
- 238000005516 engineering process Methods 0.000 claims description 48
- 206010013710 Drug interaction Diseases 0.000 claims description 43
- 239000000890 drug combination Substances 0.000 claims description 35
- 238000001914 filtration Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 20
- 230000008859 change Effects 0.000 claims description 18
- 238000013136 deep learning model Methods 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 17
- 238000011049 filling Methods 0.000 claims description 13
- 238000005065 mining Methods 0.000 claims description 12
- 238000000513 principal component analysis Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 10
- 238000013079 data visualisation Methods 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 238000012300 Sequence Analysis Methods 0.000 claims description 8
- 238000005206 flow analysis Methods 0.000 claims description 8
- 238000013439 planning Methods 0.000 claims description 8
- 238000012800 visualization Methods 0.000 claims description 7
- 238000000342 Monte Carlo simulation Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 5
- 238000012731 temporal analysis Methods 0.000 claims description 5
- 238000002560 therapeutic procedure Methods 0.000 claims description 5
- 238000000700 time series analysis Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000009193 crawling Effects 0.000 claims description 4
- 230000007774 longterm Effects 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000010223 real-time analysis Methods 0.000 claims description 3
- 238000011425 standardization method Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 2
- 238000013481 data capture Methods 0.000 claims 1
- 238000011282 treatment Methods 0.000 abstract description 8
- 206010067484 Adverse reaction Diseases 0.000 abstract description 3
- 230000006838 adverse reaction Effects 0.000 abstract description 3
- 238000009412 basement excavation Methods 0.000 abstract 1
- 238000013135 deep learning Methods 0.000 abstract 1
- 230000008676 import Effects 0.000 description 16
- 230000001932 seasonal effect Effects 0.000 description 5
- 230000004044 response Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000002483 medication Methods 0.000 description 3
- 241000238631 Hexapoda Species 0.000 description 2
- 208000025174 PANDAS Diseases 0.000 description 2
- 208000021155 Paediatric autoimmune neuropsychiatric disorders associated with streptococcal infection Diseases 0.000 description 2
- 240000004718 Panda Species 0.000 description 2
- 235000016496 Panda oleosa Nutrition 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000007621 cluster analysis Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000009509 drug development Methods 0.000 description 2
- 238000009472 formulation Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010224 classification analysis Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 235000014347 soups Nutrition 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of data analysis methods, in particular to a medicine market data analysis method and system based on big data, comprising the following steps: based on data sources of medical institutions, medical research centers and patient self-reporting, data collection and cleaning are performed by adopting a data grabbing and cleaning algorithm, and a purified market original data set is generated. According to the invention, through deep excavation of medical institutions, research centers and patient data, comprehensive and real data analysis is ensured, deep time sequence and Bayesian deep learning are used for improving analysis depth of market history data, accurately capturing market dynamics, supporting medicine circulation and inventory management, realizing market real-time monitoring and deep medicine interaction research through real-time data flow and multi-level network analysis, improving medicine safety and effectiveness, optimizing medicine combination therapy, introducing personalized medicine recommendation, ensuring that patients receive more accurate treatment, improving curative effect, and reducing adverse reaction and medical cost.
Description
Technical Field
The invention relates to the technical field of data analysis methods, in particular to a medicine market data analysis method and system based on big data.
Background
The technical field of data analysis methods is mainly focused on processing, analyzing and interpreting a large amount of data by using special algorithms and statistical techniques to mine patterns, relationships or trends in the data. In the present day, with the popularity of the internet, the internet of things and mobile devices, the speed and scale of data generation are rapidly increasing, which has led to an increasing need for efficient data analysis tools and techniques. Data analysis may be applied in various fields, such as finance, medicine, retail, advertising, etc., to provide valuable insight and decision support.
The medicine market data analysis method based on big data is a method which is specially used for analyzing medicine market data by combining big data technology. It processes large-scale data sets, which may be derived from sales records, patient feedback, drug development results, market research, etc., to refine valuable information about the drug market through specific data processing, cleaning, mining techniques. The method aims to abstract the core trends, demands, problems or opportunities of the drug market from massive amounts of data to help pharmaceutical companies, medical institutions, policy makers and other related parties make more intelligent and informed decisions.
In the existing medicine market data analysis method, the data sources are single and updated with lag, and a quick response mechanism for real-time market change is lacked, so that a market analysis report is often disjointed from the current market condition, and decision delay and resource waste are caused. The existing method depends on the traditional statistical method, lacks enough depth and breadth, and is difficult to capture complex market dynamics and patient demands, so that inaccuracy of medicine supply and demand prediction is caused. In the aspects of drug interaction and personalized drug recommendation, the existing method cannot fully utilize multi-source data and advanced algorithms, lacks accuracy, cannot provide the most effective drug combination for patients, and increases treatment risks and cost.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides a medicine market data analysis method and system based on big data.
In order to achieve the above purpose, the present invention adopts the following technical scheme: a medicine market data analysis method based on big data comprises the following steps:
s1: based on data sources of medical institutions, medical research centers and patient self-reporting, performing data collection and cleaning by adopting a data grabbing and cleaning algorithm to generate a purified market original data set;
S2: based on the purified market original data set, performing market history data feature analysis by adopting a deep time sequence analysis method to generate a market history data feature report;
s3: based on the market history data feature report, performing market trend prediction by adopting a Bayesian deep learning model and combining probability programming, and generating a market trend prediction report;
s4: based on the market trend prediction report, real-time data flow analysis technology and Internet of things data integration are adopted to execute real-time market monitoring and inventory optimization, and a real-time market monitoring and regional inventory optimization strategy is generated;
s5: based on the real-time market monitoring and regional inventory optimization strategy, a multi-level network analysis method is adopted to execute the research of a drug interaction network, and a drug combination therapy scheme report is generated;
s6: based on the drug combination therapy scheme report, a collaborative filtering algorithm of a mixed model is adopted to execute personalized drug recommendation, and a personalized drug recommendation scheme is generated.
As a further scheme of the invention, based on data sources of medical institutions, medical research centers and patient self-reporting, a data grabbing and cleaning algorithm is adopted to perform data collection and cleaning, and the steps of generating a purified market original data set are specifically as follows:
S101: based on the web crawler technology, data grabbing is carried out on data sources of medical institutions, medical research centers and patient self-reporting, and an original medical data set is generated;
s102: based on the original medical data set, preprocessing data by adopting a K-nearest neighbor filling algorithm, and generating a preprocessed data set;
s103: generating a standardized data set by adopting a minimum maximum standardized data conversion method based on the preprocessed data set;
s104: and optimizing data through a principal component analysis algorithm based on the standardized data set, and generating a purified market original data set.
As a further scheme of the invention, based on the purified market original data set, a deep time sequence analysis method is adopted to execute market history data feature analysis, and the step of generating a market history data feature report specifically comprises the following steps:
s201: analyzing the long-term trend of the purified market original data set by adopting a time sequence analysis method to generate a time sequence decomposition result;
s202: based on the time sequence decomposition result, adopting a Fourier transform algorithm to perform data feature mining to generate a historical data feature set;
S203: based on the historical data feature set, adopting a matplotlib library of Python to perform visualization processing to generate a data feature visualization report;
s204: and based on the data characteristic visual report, using a report writing tool to sort data to form a market history data characteristic report.
As a further scheme of the invention, based on the market history data feature report, a Bayesian deep learning model is adopted to combine with probability programming, and the step of executing market trend prediction and generating a market trend prediction report comprises the following steps:
s301: training by using the characteristic report of the market history data by adopting a Bayesian deep learning model, finding a probability relation between data, and generating a pre-trained Bayesian model;
s302: based on the pre-trained Bayesian model, adopting a PyMC3 probability programming method to simulate and infer future market change, and generating market change probability distribution;
s303: based on the market change probability distribution, predicting market trend and change by adopting a Monte Carlo method, and generating a market trend prediction result;
s304: and integrating analysis data and the prediction result by using a report generating tool according to the market trend prediction result to obtain a market trend prediction report.
As a further scheme of the invention, based on the market trend prediction report, the real-time data flow analysis technology and the data integration of the Internet of things are adopted to execute real-time market monitoring and inventory optimization, and the steps for generating the real-time market monitoring and regional inventory optimization strategy are specifically as follows:
s401: based on the input real-time market data, adopting an Apache Kafka data stream processing framework to collect and process data streams and generate real-time market data streams;
s402: based on the real-time market data stream, adopting an Apache Flink stream data analysis technology to perform real-time data analysis and processing to generate a real-time market analysis result;
s403: based on the real-time market analysis result, integrating and analyzing inventory data by adopting an Internet of things equipment collection technology and a data fusion algorithm to generate integrated inventory data;
s404: based on the integrated inventory data, adopting a dynamic planning algorithm and an inventory optimization model to optimize and adjust the inventory quantity, and generating a real-time inventory optimization strategy;
s405: based on the real-time inventory optimization strategy and the real-time market analysis result, a regional strategy making model is adopted to carry out comprehensive analysis on regional inventory and market strategies, and real-time market monitoring and regional inventory optimization strategies are generated.
As a further scheme of the present invention, based on the real-time market monitoring and regional inventory optimization strategy, a multi-level network analysis method is adopted to perform research on a drug interaction network, and the steps of generating a drug combination therapy scheme report are specifically as follows:
s501: based on the real-time market monitoring and regional inventory optimization strategy, extracting and analyzing the drug interaction data by adopting a network mining technology to generate a primary drug interaction network;
s502: based on the preliminary drug interaction network, adopting a graph theory analysis method and a core node identification algorithm to identify and analyze core nodes and correlations thereof, and generating a core drug interaction network;
s503: based on the core drug interaction network, classifying and clustering the drugs by adopting a modularity analysis technology and a clustering algorithm to generate a drug clustering module;
s504: based on the drug clustering module, a multilevel network analysis method and a therapy optimization algorithm are adopted to conduct research and optimization of drug combination, and a drug combination therapy scheme report is generated.
As a further aspect of the present invention, based on the report of the drug combination therapy scheme, a collaborative filtering algorithm of a hybrid model is adopted to execute personalized drug recommendation, and the steps of generating the personalized drug recommendation scheme specifically include:
S601: based on the drug combination therapy scheme report, adopting a user historical data analysis technology and a preliminary screening algorithm to analyze and screen user historical medication data, and generating a user drug use candidate set;
s602: based on the user medicine use candidate set, adopting a collaborative filtering technology and a user preference modeling algorithm to analyze and model user preference, and generating a user preference model;
s603: based on the user preference model and the drug combination therapy scheme report, adopting a mixed model collaborative filtering algorithm to calculate and analyze personalized drug recommendation, and generating a preliminary personalized drug recommendation list;
s604: based on the preliminary personalized medicine recommendation list, a user feedback analysis technology and a model fine adjustment algorithm are adopted to verify the recommendation effect and optimize and adjust the model, and a personalized medicine recommendation scheme is generated.
The system comprises a data acquisition module, a data preprocessing module, a data analysis module, a market prediction module, a real-time monitoring module, a drug interaction research module and a personalized recommendation module.
As a further scheme of the invention, the data acquisition module is used for capturing data of medical institutions, medical research centers and self-reporting of patients based on a web crawler technology to generate an original medical data set;
the data preprocessing module is used for filling missing values by adopting a K-nearest neighbor filling algorithm based on an original medical data set, converting the data into a unified standard space by a minimum and maximum standardization method and generating a standardized data set;
the data analysis module adopts principal component analysis to perform dimension reduction and feature extraction based on a standardized data set, and then performs data visualization processing through a matplotlib library to generate a market history data feature report;
the market prediction module adopts a Bayesian deep learning model and a PyMC3 probability programming method based on the market history data characteristic report to reveal the probability relation between data and simulate and infer future market change so as to generate a market trend prediction report;
the real-time monitoring module is used for carrying out real-time analysis on data by adopting an Apache Kafka data stream processing frame and an Apache Flink stream data analysis technology based on real-time market data and market trend prediction reports, and generating a real-time market monitoring and regional inventory optimization strategy through a dynamic planning algorithm and an inventory optimization model;
The drug interaction research module analyzes interactions and correlations among drugs based on real-time market monitoring and regional inventory optimization strategies by adopting a network mining technology and a graph theory analysis method to generate a drug combination therapy scheme report;
the personalized recommendation module generates a personalized medicine recommendation scheme for a user by adopting a collaborative filtering technology and a user preference modeling algorithm based on historical data of the user and a medicine combination therapy scheme report.
As a further aspect of the present invention, the data acquisition module includes a network crawling insect sub-module;
the data preprocessing module comprises a data cleaning sub-module and a data standardization sub-module;
the data analysis module comprises a data optimization sub-module, a data visualization sub-module and a first report generation sub-module;
the market prediction module comprises a model training sub-module, a prediction simulation sub-module and a second report generation sub-module;
the real-time monitoring module comprises a data collection sub-module, a first data analysis sub-module and an inventory optimization sub-module;
the medicine interaction research module comprises a network research sub-module, a second data analysis sub-module and a scheme making sub-module;
The personalized recommendation module comprises a user analysis sub-module, a recommendation calculation sub-module and a result optimization sub-module.
Compared with the prior art, the invention has the advantages and positive effects that:
according to the invention, the comprehensiveness and the authenticity of data analysis are ensured by deep mining of data of medical institutions, medical research centers and self-reporting of patients. By adopting the deep time sequence analysis and the Bayesian deep learning model, the analysis depth of the market history data is improved, the market dynamic change is more accurately captured and predicted, and powerful support is provided for medicine circulation and inventory management. Real-time monitoring of the market and deep research on drug interaction are realized through a real-time data flow analysis and a multi-level network analysis method, so that the safety and effectiveness of the drug are improved, and the formulation of a drug combination therapy is optimized. The introduction of the personalized medicine recommendation scheme ensures that the patient receives a more accurate treatment scheme, improves the treatment effect and reduces adverse reaction and medical cost.
Drawings
FIG. 1 is a schematic workflow diagram of the present invention;
FIG. 2 is a S1 refinement flowchart of the present invention;
FIG. 3 is a S2 refinement flowchart of the present invention;
FIG. 4 is a S3 refinement flowchart of the present invention;
FIG. 5 is a S4 refinement flowchart of the present invention;
FIG. 6 is a S5 refinement flowchart of the present invention;
FIG. 7 is a S6 refinement flowchart of the present invention;
FIG. 8 is a system flow diagram of the present invention;
FIG. 9 is a schematic diagram of a system framework of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the description of the present invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present invention and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention. Furthermore, in the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Embodiment one:
referring to fig. 1, the present invention provides a technical solution: a medicine market data analysis method based on big data comprises the following steps:
s1: based on data sources of medical institutions, medical research centers and patient self-reporting, performing data collection and cleaning by adopting a data grabbing and cleaning algorithm to generate a purified market original data set;
s2: based on the purified market original data set, performing market history data feature analysis by adopting a deep time sequence analysis method to generate a market history data feature report;
s3: based on the market history data feature report, a Bayesian deep learning model is adopted to combine with probability programming, market trend prediction is executed, and a market trend prediction report is generated;
s4: based on the market trend prediction report, real-time data flow analysis technology and Internet of things data integration are adopted, real-time market monitoring and inventory optimization are executed, and a real-time market monitoring and regional inventory optimization strategy is generated;
s5: based on real-time market monitoring and regional inventory optimization strategies, a multi-level network analysis method is adopted to execute research on a drug interaction network, and a drug combination therapy scheme report is generated;
S6: based on the report of the drug combination therapy scheme, a collaborative filtering algorithm of a mixed model is adopted to execute personalized drug recommendation, and a personalized drug recommendation scheme is generated.
First, a large amount of raw data is extracted from a number of authoritative data sources, including medical institutions, medical research centers, and patient self-reported data. The range of such data coverage is wide, which helps reflect the general view of the pharmaceutical market. The data grabbing and cleaning algorithm is adopted to ensure the purity and quality of the data, so that the subsequent data analysis is more accurate.
Secondly, through a deep time series analysis method, the method can be used for insights into hidden features and modes in market history data. This is difficult to achieve by conventional methods. Such deep analysis not only helps understand past market trends, but also provides a solid basis for predicting the future.
The application of the Bayesian deep learning model in combination with probability programming in the step S3 makes market trend prediction more scientific and prospective. Compared with the traditional statistical prediction method, the method can handle complex nonlinear relations and potential uncertainties, and provides a more comprehensive and detailed prediction report for a decision maker.
The integration of real-time data flow analysis technology and Internet of things data brings real-time insight to the medical market. All of this can be captured and analyzed in real time, whether market dynamics, consumer behavior, or inventory changes. This is critical to handling market incidents, optimizing inventory, and developing strategies.
The drug interaction is researched by a multi-level network analysis method, so that the effect of a single drug can be obtained, and the interaction and potential benefits of multiple drugs in combination can be understood. This is of great value in advancing drug development, optimizing therapy selection, and reducing the risk of adverse reactions between drugs.
Finally, personalized medicine recommendations embody a trend of the medical industry towards precise medical treatment. The collaborative filtering algorithm combined with the mixed model not only recommends according to the medical history and physique of the patient, but also provides the most suitable drug scheme for each patient based on a large amount of data analysis.
Referring to fig. 2, based on the data sources of the medical institution, the medical research center and the patient self-report, the data capturing and cleaning algorithm is adopted to perform data collection and cleaning, and the steps of generating the purified market original data set are specifically as follows:
S101: based on the web crawler technology, data grabbing is carried out on data sources of medical institutions, medical research centers and patient self-reporting, and an original medical data set is generated;
s102: preprocessing data by adopting a K-nearest neighbor filling algorithm based on an original medical data set to generate a preprocessed data set;
s103: based on the preprocessed data set, generating a standardized data set by adopting a minimum and maximum standardized data conversion method;
s104: based on the standardized dataset, the data is optimized through a principal component analysis algorithm, and a purified market original dataset is generated.
In S101, data grabbing is performed
The method comprises the following steps: data is extracted using web crawler technology, such as the Scrapy framework of Python or the beautifu so library, to access websites of medical institutions, medical research centers, and patient self-reports.
Code example:
import requests
from bs4 import BeautifulSoup
url = "http://example.com/medical_data"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# Extract relevant data from the webpage
in S102, data preprocessing is performed
The method comprises the following steps: and filling missing values according to the similarity of the data by using a K-neighbor filling algorithm, so as to ensure the integrity of the data set.
Algorithm description: the K-nearest neighbor filling algorithm will find the K data points closest to the missing values and then fill the missing values based on the values of these neighbors.
Code example:
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
data_filled = imputer.fit_transform(data)
in S103, data normalization is performed
The method comprises the following steps: the data is scaled to between 0 and 1 using a minimum maximum normalized data conversion method.
Algorithm description: the minimum maximum normalization scales by subtracting the minimum value and dividing by the range.
Code example:
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
data_normalized = scaler.fit_transform(data)
in S104, data optimization is performed
The method comprises the following steps: using principal component analysis algorithm (PCA), dimension reduction and data optimization.
Algorithm description: PCA projects the raw data into a new coordinate system through linear transformation to reduce the dimensionality and preserve most of the data variance.
Code example:
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
data_optimized = pca.fit_transform(data)
referring to fig. 3, based on the purified market raw data set, a deep time series analysis method is adopted to perform market history data feature analysis, and the step of generating a market history data feature report specifically includes:
s201: analyzing the long-term trend of the purified market original data set by adopting a time sequence analysis method to generate a time sequence decomposition result;
s202: based on a time sequence decomposition result, adopting a Fourier transform algorithm to perform data feature mining to generate a historical data feature set;
s203: based on the historical data feature set, adopting a matplotlib library of Python to perform visualization processing to generate a data feature visualization report;
S204: based on the data characteristic visualization report, the data is collated by using a report writing tool to form a market history data characteristic report.
In S201, performing a time series analysis this step will analyze the long-term trend of the dataset using the time series analysis function of the library of Python. statsmodels
import statsmodels.api as sm
# decomposition of time series data
result = sm.tsa.seasonal_decompose(data, model='multiplicative')
Trend, # extraction, seasonal and noise component
trend = result.trend
seasonal = result.seasonal
residual = result.resid
In S202, performing a fourier transform uses the fourier transform to mine the periodic features of the data.
import numpy as np
# Fourier transform
frequencies = np.fft.fft(trend.dropna())
# extraction principal frequency
main_frequencies = frequencies[:len(frequencies)//2]
In S203, the analysis result is visualized using the library by performing data visualization. matplotlib
import matplotlib.pyplot as plt
# plotting time series decomposition results
plt.figure(figsize=(12, 8))
plt.subplot(4, 1, 1)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(4, 1, 2)
plt.plot(seasonal, label='Seasonal')
plt.legend(loc='best')
plt.subplot(4, 1, 3)
plt.plot(residual, label='Residual')
plt.legend(loc='best')
plt.subplot(4, 1, 4)
plt.plot(np.abs(main_frequencies), label='Frequencies from FFT')
plt.legend(loc='best')
plt.tight_layout()
plt.show()
In S204, the execution generates a report, which may be collated and formed using tools like Jupyter Notebook, including codes, analysis results and related specifications.
Referring to fig. 4, based on the market history data feature report, the steps of performing market trend prediction by using a bayesian deep learning model in combination with probability programming to generate a market trend prediction report are specifically as follows:
s301: training by using a Bayesian deep learning model and utilizing a market history data characteristic report to find a probability relation between data, and generating a pre-trained Bayesian model;
S302: based on a pre-trained Bayesian model, adopting a PyMC3 probability programming method to simulate and infer future market change, and generating market change probability distribution;
s303: based on the market change probability distribution, predicting market trend and change by adopting a Monte Carlo method, and generating a market trend prediction result;
s304: and integrating the analysis data and the prediction result by using a report generating tool according to the market trend prediction result to obtain a market trend prediction report.
In S301, bayes deep learning model training is performed
At this step, a bayesian deep learning model will be trained using PyMC 3.
import pymc3 as pm
# creation of PyMC3 model
with pm.Model() as bayesian_model:
# definition model structure and parameters
# e.g. neural network layer, weights, biases, etc
Model training using historical data feature report
observed_data = historical_data_features
Bayesian inference is performed #)
trace = pm.sample(1000, tune=1000)
In S302, pyMC3 probability programming is performed
PyMC3 was used to simulate and infer future market changes.
Simulation of future market changes #)
with bayesian_model:
Input data defining future market changes
future_data = ...
# extrapolate
posterior = pm.sample_posterior_predictive(trace, samples=1000)
In S303, a Monte Carlo method is performed to predict market trends
Market trends and changes were predicted using the monte carlo method.
# market trend prediction based on Monte Carlo method
predictions = np.mean(posterior['output'], axis=0)
In S304, generation report is executed
A market trend forecast report is generated using a report generating tool to integrate the analysis data and the forecast results.
Referring to fig. 5, based on the market trend prediction report, the real-time data flow analysis technology and the internet of things data integration are adopted to execute real-time market monitoring and inventory optimization, and the steps for generating the real-time market monitoring and regional inventory optimization strategy are specifically as follows:
s401: based on the input real-time market data, adopting an Apache Kafka data stream processing framework to collect and process data streams and generate real-time market data streams;
s402: based on the real-time market data flow, adopting an Apache Flink flow data analysis technology to conduct real-time data analysis and processing, and generating a real-time market analysis result;
s403: based on real-time market analysis results, integrating and analyzing inventory data by adopting an Internet of things equipment collection technology and a data fusion algorithm to generate integrated inventory data;
s404: based on the integrated inventory data, adopting a dynamic planning algorithm and an inventory optimization model to optimize and adjust the inventory quantity, and generating a real-time inventory optimization strategy;
s405: based on the real-time inventory optimization strategy and the real-time market analysis result, a regional strategy making model is adopted to carry out comprehensive analysis on the regional inventory and the market strategy, and the real-time market monitoring and the regional inventory optimization strategy are generated.
In S401, data producer example, performing data flow collection and processing-Apache KafkaKafka:
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='your_kafka_broker')
producer.send('market_data_topic', value=real_time_data)
data consumer example of Kafka:
from kafka import KafkaConsumer
consumer = KafkaConsumer('market_data_topic', bootstrap_servers='your_kafka_broker')
for message in consumer:
process_real_time_data(message.value)
in S402, a real-time data analysis example of real-time data analysis-Apache FlinkFlink is performed:
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<RealTimeData> dataStream = env.addSource(new KafkaSource<>(...));
DataStream<AnalysisResult> resultStream = dataStream
keyBy(...)
timeWindow(...)
apply(new RealTimeDataAnalysisFunction());
resultStream.addSink(new KafkaSink<>(...));
env.execute("Real-time Market Analysis");
in S403, an inventory data integration-internet of things device and data fusion algorithm is executed
The example code varies with application and device type, requiring writing a data acquisition script for the internet of things device, and then integrating the data using a data fusion algorithm.
In S404, an inventory optimization-dynamic planning algorithm and an inventory model are executed
Inventory optimization typically involves inventory models, such as EOQ model (economic order quantity model) and dynamic planning algorithms. The following is an example Python code segment:
def calculate_optimal_order_quantity(demand, holding_cost, ordering_cost):
optimal_order_quantity = math.sqrt((2demand/>ordering_cost) / holding_cost)
return optimal_order_quantity
in S405, market monitoring and strategy synthesis-regional strategy formulation model is performed
The policy-making model will vary depending on the actual needs, including analyzing market trends, inventory status, and other factors, and making optimal policies using decision trees, linear programming, or other algorithms. The following is a simple example:
if market_trend == "Up" and inventory_level < threshold:
implement_strategy_A()
elif market_trend == "Down" and inventory_level > threshold:
implement_strategy_B()
else:
implement_default_strategy()
referring to fig. 6, based on real-time market monitoring and regional inventory optimization strategies, a multi-level network analysis method is adopted to perform research on a drug interaction network, and the steps for generating a report of a drug combination therapy scheme are specifically as follows:
S501: based on real-time market monitoring and regional inventory optimization strategies, extracting and analyzing drug interaction data by adopting a network mining technology to generate a primary drug interaction network;
s502: based on the preliminary drug interaction network, adopting a graph theory analysis method and a core node identification algorithm to identify and analyze core nodes and interrelation thereof, and generating a core drug interaction network;
s503: based on a core drug interaction network, classifying and clustering the drugs by adopting a modularity analysis technology and a clustering algorithm to generate a drug clustering module;
s504: based on the drug clustering module, a multilevel network analysis method and a therapy optimization algorithm are adopted to conduct research and optimization of drug combination, and a drug combination therapy scheme report is generated.
In S501, data extraction and generation of a preliminary drug interaction network are performed
Real-time market monitoring data and regional inventory data are collected, including sales data, inventory, price information, etc. for the medications. The data is preprocessed and cleaned using network mining techniques to generate data that can be used to construct a drug interaction network. Constructing a drug interaction network, wherein drugs are represented as nodes in the network, interaction relationships between drugs are represented as edges, and the interaction relationships can be implemented by using a common graph theory library (such as network x), and the following are example code segments:
import networkx as nx
# create an empty directed graph
drug_interactions_network = nx.DiGraph()
# add medicine node
for drug in drugs:
drug_interactions_network.add_node(drug)
Add interaction relationship #
for interaction in drug_interactions:
drug1, drug2 = interaction
drug_interactions_network.add_edge(drug1, drug2)
In S502, core node identification and core drug interaction network are performed
Preliminary drug interaction networks, such as centrality, tight centrality, etc., are analyzed using graph theory analysis methods to identify core nodes. Core node identification algorithms, such as PageRank, betweenness centrality, etc., are applied to find the most important nodes in the network. Based on the core nodes, a core drug interaction network is constructed, which comprises the core nodes and the sub-networks interacted with the core nodes.
# identifying core node
core_nodes = nx.degree_centrality(drug_interactions_network)
# find the most important node
top_core_nodes = sorted(core_nodes, key=core_nodes.get, reverse=True)[:N]
Construction of core drug interaction network
core_drug_interactions_network = drug_interactions_network.subgraph(top_core_nodes)
In S503, drug classification and cluster analysis are performed
The core drug interaction network is partitioned into different modules or communities using a modular analysis technique, such as a community detection algorithm. Further classification and analysis of the drugs is performed using clustering algorithms (e.g., K-means clustering or hierarchical clustering).
# Community detection
communities = nx.algorithms.community.greedy_modularity_communities(core_drug_interactions_network)
# Cluster analysis
from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters=K)
cluster_assignments = kmeans.fit_predict(drug_features)
In S504, drug combination therapy regimen report generation is performed
And integrating the core drug interaction network, the drug classification and the clustering information by using a multi-level network analysis method to construct a multi-level drug interaction network. Based on a multi-level network, a therapy optimization algorithm, such as a genetic algorithm or simulated annealing algorithm, is applied to find the optimal drug combination therapy. A medication combination therapy regimen report is generated including information on suggested medication combinations, dosages, treatment regimens, and the like.
Referring to fig. 7, based on a report of a drug combination therapy scheme, a collaborative filtering algorithm of a hybrid model is adopted to execute personalized drug recommendation, and the steps of generating the personalized drug recommendation scheme are specifically as follows:
s601: based on the report of the drug combination therapy scheme, adopting a user historical data analysis technology and a preliminary screening algorithm to analyze and screen user historical medication data, and generating a user drug use candidate set;
s602: based on a user medicine use candidate set, adopting a collaborative filtering technology and a user preference modeling algorithm to analyze and model user preference, and generating a user preference model;
s603: based on a user preference model and a drug combination therapy scheme report, adopting a mixed model collaborative filtering algorithm to calculate and analyze personalized drug recommendation, and generating a preliminary personalized drug recommendation list;
s604: based on the initial personalized medicine recommendation list, a user feedback analysis technology and a model fine adjustment algorithm are adopted to verify the recommendation effect and optimize and adjust the model, and a personalized medicine recommendation scheme is generated.
In S601, analysis and screening of user history medication data is performed
User historical medication data is collected, including medication name, dosage, duration of treatment, and the like. And (5) performing data cleaning and preprocessing, and processing missing values, abnormal values and the like. The user medication data is primarily screened, and indexes such as frequency, duration and the like can be considered to generate a user medication use candidate set.
# assume data processing Using pandas
import pandas as pd
# reading user historical medication data
user_medication_data = pd.read_csv("user_medication_data.csv")
Data washing and screening #
cleaned_data = preprocess_data(user_medication_data)
Initial screening of user drug use candidate set
user_candidate_medications = select_candidate_medications(cleaned_data)
In S602, analysis and modeling of user preferences is performed
A user drug preference model is established using collaborative filtering techniques, such as user-based collaborative filtering. The user historical medication data is modeled using a user preference modeling algorithm, such as a matrix factorization algorithm (e.g., ALS or SGD).
from surprise import Dataset, Reader, SVD
from surprise.model_selection import train_test_split
Creation of Surprise dataset
reader = Reader(rating_scale=(0, 5))
data = Dataset.load_from_df(cleaned_data[['user_id', 'medication_id', 'rating']], reader)
# division training set and test set
trainset, testset = train_test_split(data, test_size=0.2)
Establishing user drug preference model using SVD algorithm
model = SVD()
model.fit(trainset)
In S603, a calculation and analysis of personalized medicine recommendations is performed
And constructing a mixed model collaborative filtering algorithm by combining the user preference model and the drug combination therapy scheme report, and considering the specificity of the drug combination.
And generating a preliminary personalized medicine recommendation list for the user by using a mixed model collaborative filtering algorithm.
Mixed model collaborative filtering based on user preference model and drug combination therapy regimen report
def hybrid_recommendation(user_id, medications_in_combination):
user_preferences = model.predict(user_id, medications_in_combination)
Calculation of other hybrid models.
return personalized_recommendations
In S604, user feedback analysis and model fine tuning are performed
And collecting feedback data of the user on the primary personalized medicine recommendation list. And evaluating the recommendation effect by using a user feedback analysis technology, such as indexes of accuracy, coverage range and the like. And a model fine tuning algorithm, such as gradient descent or Bayesian optimization, is used for adjusting the model, so that the accuracy of personalized medicine recommendation is improved.
# collect user feedback data
user_feedback = collect_user_feedback()
# analyze user feedback and make model fine-tuning
model_tuned = tune_model(model, user_feedback)
Referring to fig. 8, a big data-based pharmaceutical market data analysis system is used for executing the big data-based pharmaceutical market data analysis method, and the system includes a data acquisition module, a data preprocessing module, a data analysis module, a market prediction module, a real-time monitoring module, a pharmaceutical interaction research module and a personalized recommendation module.
The data acquisition module is used for capturing data of a medical institution, a medical research center and a patient self-report based on a web crawler technology and generating an original medical data set;
the data preprocessing module is used for filling missing values by adopting a K-nearest neighbor filling algorithm based on an original medical data set, converting the data into a unified standard space by a minimum and maximum standardization method, and generating a standardized data set;
the data analysis module adopts principal component analysis to perform dimension reduction and feature extraction based on a standardized data set, and then performs data visualization processing through a matplotlib library to generate a market history data feature report;
the market prediction module adopts a Bayesian deep learning model and a PyMC3 probability programming method based on the market history data characteristic report to reveal the probability relation between the data and simulate and infer future market change so as to generate a market trend prediction report;
The real-time monitoring module is used for carrying out real-time analysis on data by adopting an Apache Kafka data stream processing frame and an Apache Flink stream data analysis technology based on real-time market data and market trend prediction reports, and generating a real-time market monitoring and regional inventory optimization strategy through a dynamic planning algorithm and an inventory optimization model;
the drug interaction research module analyzes interactions and correlations among drugs based on real-time market monitoring and regional inventory optimization strategies by adopting a network mining technology and a graph theory analysis method to generate a drug combination therapy scheme report;
the personalized recommendation module generates a personalized medicine recommendation scheme for the user by adopting a collaborative filtering technology and a user preference modeling algorithm based on the historical data of the user and the medicine combination therapy scheme report.
Firstly, the system can comprehensively and quickly collect various related data through the data acquisition module. This extensive data acquisition provides a solid foundation for comprehensive market analysis, ensuring the comprehensiveness and depth of the analysis results.
Secondly, the data preprocessing module ensures the height of data quality, so that the subsequent data analysis is more accurate. The K-neighbor filling algorithm processes the missing value, so that the data integrity can be effectively improved, and the data standardization enables the data with different sources and different scales to be fused, so that a unified and standardized data set is formed.
The data analysis module adopts principal component analysis and data visualization technology, so that the data dimension is reduced, the data is easier to understand, the information hidden behind the data is presented in an intuitive mode, and the assistance decision maker rapidly acquires key information.
The market prediction module adopts a Bayesian deep learning model and a PyMC3 probability programming method, and the Bayesian deep learning model and the PyMC3 probability programming method are proved to be very powerful tools in the prediction field, can accurately predict future market trends, and provide decision support for enterprises.
The real-time data processing and stream data analysis technology of the real-time monitoring module means that enterprises can respond to market changes rapidly, adjust strategies in real time, effectively reduce inventory cost and improve market response speed.
The introduction of the medicine interaction research module provides valuable information for medical institutions and patients, and is beneficial to popularization of safer and more effective medicine combination therapies, so that the treatment effect and the life quality of the patients are improved.
Finally, the personalized recommendation module can provide targeted and personalized medicine recommendation, so that the treatment satisfaction degree of the patient is remarkably improved, and meanwhile, guidance is provided for medicine production and sales.
Referring to fig. 9, the data acquisition module includes a network crawling insect sub-module;
the data preprocessing module comprises a data cleaning sub-module and a data standardization sub-module;
the data analysis module comprises a data optimization sub-module, a data visualization sub-module and a first report generation sub-module;
the market prediction module comprises a model training sub-module, a prediction simulation sub-module and a second report generation sub-module;
the real-time monitoring module comprises a data collection sub-module, a first data analysis sub-module and an inventory optimization sub-module;
the medicine interaction research module comprises a network research sub-module, a second data analysis sub-module and a scheme making sub-module;
the personalized recommendation module comprises a user analysis sub-module, a recommendation calculation sub-module and a result optimization sub-module.
And a data acquisition module: the network crawling sub-module can effectively acquire key medical data from the internet, including market data, inventory information and drug interaction data, which provides the basis data for subsequent analysis.
And a data preprocessing module: the data cleaning submodule and the data standardization submodule can ensure the quality and consistency of data. The cleaning data is helpful to remove noise and abnormal values, so that the data is more reliable, and the standardization can unify the data format, thereby facilitating the subsequent analysis.
And a data analysis module: the data optimization sub-module helps to extract useful information and patterns and helps to formulate more efficient strategies. The data visualization sub-module provides a tool to graphically present data, making it easier for a user to understand and interpret the results. The first report generating sub-module then assists in presenting the analysis results to the decision maker.
Market prediction module: the model training sub-module uses the historical market data to construct a predictive model that provides a reference for future market trends. The prediction simulation sub-module can help simulate different market situations and provide decision support for decisions. The second report generating sub-module is capable of generating detailed market forecast reports supporting strategic decisions.
And the real-time monitoring module is used for: the data collection sub-module periodically acquires the latest market and inventory information, and keeps the freshness of the data. The first data analysis submodule can quickly find market change trend, and the inventory optimization submodule helps to improve inventory efficiency and reduce cost.
Drug interaction research module: the network research submodule analyzes the drug interaction network and provides a basis for drug combination therapy. The second data analysis sub-module further explores drug interactions in depth, supporting drug combination strategies for the protocol-making sub-module.
And a personalized recommendation module: the user analysis sub-module uses the personal historical medication data to build a user preference model. The recommendation calculation sub-module combines the user's preferences and the medication combination therapies to provide personalized medication recommendations for the user. The results optimization sub-module continually refines the model to provide more accurate recommendations.
The present invention is not limited to the above embodiments, and any equivalent embodiments which can be changed or modified by the technical disclosure described above can be applied to other fields, but any simple modification, equivalent changes and modification made to the above embodiments according to the technical matter of the present invention will still fall within the scope of the technical disclosure.
Claims (10)
1. The medicine market data analysis method based on big data is characterized by comprising the following steps:
based on data sources of medical institutions, medical research centers and patient self-reporting, performing data collection and cleaning by adopting a data grabbing and cleaning algorithm to generate a purified market original data set;
based on the purified market original data set, performing market history data feature analysis by adopting a deep time sequence analysis method to generate a market history data feature report;
Based on the market history data feature report, performing market trend prediction by adopting a Bayesian deep learning model and combining probability programming, and generating a market trend prediction report;
based on the market trend prediction report, real-time data flow analysis technology and Internet of things data integration are adopted to execute real-time market monitoring and inventory optimization, and a real-time market monitoring and regional inventory optimization strategy is generated;
based on the real-time market monitoring and regional inventory optimization strategy, a multi-level network analysis method is adopted to execute the research of a drug interaction network, and a drug combination therapy scheme report is generated;
based on the drug combination therapy scheme report, a collaborative filtering algorithm of a mixed model is adopted to execute personalized drug recommendation, and a personalized drug recommendation scheme is generated.
2. The big data based pharmaceutical market data analysis method according to claim 1, wherein the step of performing data collection and cleaning by using a data capture and cleaning algorithm based on data sources of medical institutions, medical research centers and patient self-reporting, and generating a purified market raw data set is specifically as follows:
based on the web crawler technology, data grabbing is carried out on data sources of medical institutions, medical research centers and patient self-reporting, and an original medical data set is generated;
Based on the original medical data set, preprocessing data by adopting a K-nearest neighbor filling algorithm, and generating a preprocessed data set;
generating a standardized data set by adopting a minimum maximum standardized data conversion method based on the preprocessed data set;
and optimizing data through a principal component analysis algorithm based on the standardized data set, and generating a purified market original data set.
3. The big data based pharmaceutical market data analysis method according to claim 1, wherein the step of performing market history data feature analysis based on the purified market raw dataset by using a deep time series analysis method, and generating a market history data feature report specifically comprises:
analyzing the long-term trend of the purified market original data set by adopting a time sequence analysis method to generate a time sequence decomposition result;
based on the time sequence decomposition result, adopting a Fourier transform algorithm to perform data feature mining to generate a historical data feature set;
based on the historical data feature set, adopting a matplotlib library of Python to perform visualization processing to generate a data feature visualization report;
And based on the data characteristic visual report, using a report writing tool to sort data to form a market history data characteristic report.
4. The big data based medicine market data analysis method according to claim 1, wherein based on the market history data feature report, a bayesian deep learning model is adopted in combination with probability programming to perform market trend prediction, and the step of generating a market trend prediction report specifically comprises:
training by using the characteristic report of the market history data by adopting a Bayesian deep learning model, finding a probability relation between data, and generating a pre-trained Bayesian model;
based on the pre-trained Bayesian model, adopting a PyMC3 probability programming method to simulate and infer future market change, and generating market change probability distribution;
based on the market change probability distribution, predicting market trend and change by adopting a Monte Carlo method, and generating a market trend prediction result;
and integrating analysis data and the prediction result by using a report generating tool according to the market trend prediction result to obtain a market trend prediction report.
5. The big data-based pharmaceutical market data analysis method according to claim 1, wherein the step of performing real-time market monitoring and inventory optimization by integrating real-time data flow analysis technology with internet of things data based on the market trend prediction report to generate real-time market monitoring and regional inventory optimization strategies is specifically as follows:
Based on the input real-time market data, adopting an Apache Kafka data stream processing framework to collect and process data streams and generate real-time market data streams;
based on the real-time market data stream, adopting an Apache Flink stream data analysis technology to perform real-time data analysis and processing to generate a real-time market analysis result;
based on the real-time market analysis result, integrating and analyzing inventory data by adopting an Internet of things equipment collection technology and a data fusion algorithm to generate integrated inventory data;
based on the integrated inventory data, adopting a dynamic planning algorithm and an inventory optimization model to optimize and adjust the inventory quantity, and generating a real-time inventory optimization strategy;
based on the real-time inventory optimization strategy and the real-time market analysis result, a regional strategy making model is adopted to carry out comprehensive analysis on regional inventory and market strategies, and real-time market monitoring and regional inventory optimization strategies are generated.
6. The big data based drug market data analysis method according to claim 1, wherein the step of performing a study of a drug interaction network based on the real-time market monitoring and regional inventory optimization strategy by using a multi-level network analysis method, and generating a drug combination therapy plan report is specifically as follows:
Based on the real-time market monitoring and regional inventory optimization strategy, extracting and analyzing the drug interaction data by adopting a network mining technology to generate a primary drug interaction network;
based on the preliminary drug interaction network, adopting a graph theory analysis method and a core node identification algorithm to identify and analyze core nodes and correlations thereof, and generating a core drug interaction network;
based on the core drug interaction network, classifying and clustering the drugs by adopting a modularity analysis technology and a clustering algorithm to generate a drug clustering module;
based on the drug clustering module, a multilevel network analysis method and a therapy optimization algorithm are adopted to conduct research and optimization of drug combination, and a drug combination therapy scheme report is generated.
7. The big data based medicine market data analysis method according to claim 1, wherein based on the medicine combination therapy plan report, a collaborative filtering algorithm of a hybrid model is adopted to execute personalized medicine recommendation, and the step of generating a personalized medicine recommendation plan specifically comprises:
based on the drug combination therapy scheme report, adopting a user historical data analysis technology and a preliminary screening algorithm to analyze and screen user historical medication data, and generating a user drug use candidate set;
Based on the user medicine use candidate set, adopting a collaborative filtering technology and a user preference modeling algorithm to analyze and model user preference, and generating a user preference model;
based on the user preference model and the drug combination therapy scheme report, adopting a mixed model collaborative filtering algorithm to calculate and analyze personalized drug recommendation, and generating a preliminary personalized drug recommendation list;
based on the preliminary personalized medicine recommendation list, a user feedback analysis technology and a model fine adjustment algorithm are adopted to verify the recommendation effect and optimize and adjust the model, and a personalized medicine recommendation scheme is generated.
8. A big data based pharmaceutical market data analysis system, characterized in that the big data based pharmaceutical market data analysis method according to any of claims 1-7, the system comprises a data acquisition module, a data preprocessing module, a data analysis module, a market prediction module, a real-time monitoring module, a pharmaceutical interaction research module and a personalized recommendation module.
9. The big data based pharmaceutical market data analysis system of claim 8, wherein the data acquisition module captures data of medical institutions, medical research centers, and patient self-reports based on web crawler technology, generating an original medical dataset;
The data preprocessing module is used for filling missing values by adopting a K-nearest neighbor filling algorithm based on an original medical data set, converting the data into a unified standard space by a minimum and maximum standardization method and generating a standardized data set;
the data analysis module adopts principal component analysis to perform dimension reduction and feature extraction based on a standardized data set, and then performs data visualization processing through a matplotlib library to generate a market history data feature report;
the market prediction module adopts a Bayesian deep learning model and a PyMC3 probability programming method based on the market history data characteristic report to reveal the probability relation between data and simulate and infer future market change so as to generate a market trend prediction report;
the real-time monitoring module is used for carrying out real-time analysis on data by adopting an Apache Kafka data stream processing frame and an Apache Flink stream data analysis technology based on real-time market data and market trend prediction reports, and generating a real-time market monitoring and regional inventory optimization strategy through a dynamic planning algorithm and an inventory optimization model;
the drug interaction research module analyzes interactions and correlations among drugs based on real-time market monitoring and regional inventory optimization strategies by adopting a network mining technology and a graph theory analysis method to generate a drug combination therapy scheme report;
The personalized recommendation module generates a personalized medicine recommendation scheme for a user by adopting a collaborative filtering technology and a user preference modeling algorithm based on historical data of the user and a medicine combination therapy scheme report.
10. The big data based pharmaceutical market data analysis system of claim 8, wherein the data acquisition module comprises a network crawling sub-module;
the data preprocessing module comprises a data cleaning sub-module and a data standardization sub-module;
the data analysis module comprises a data optimization sub-module, a data visualization sub-module and a first report generation sub-module;
the market prediction module comprises a model training sub-module, a prediction simulation sub-module and a second report generation sub-module;
the real-time monitoring module comprises a data collection sub-module, a first data analysis sub-module and an inventory optimization sub-module;
the medicine interaction research module comprises a network research sub-module, a second data analysis sub-module and a scheme making sub-module;
the personalized recommendation module comprises a user analysis sub-module, a recommendation calculation sub-module and a result optimization sub-module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311444060.8A CN117196694A (en) | 2023-11-02 | 2023-11-02 | Medicine market data analysis method and system based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311444060.8A CN117196694A (en) | 2023-11-02 | 2023-11-02 | Medicine market data analysis method and system based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117196694A true CN117196694A (en) | 2023-12-08 |
Family
ID=88998349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311444060.8A Pending CN117196694A (en) | 2023-11-02 | 2023-11-02 | Medicine market data analysis method and system based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117196694A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102190897B1 (en) * | 2019-09-19 | 2020-12-15 | (주)어반유니온 | Method and Apparatus for analyzing fashion trend based on big data |
| CN116501779A (en) * | 2023-06-26 | 2023-07-28 | 图林科技(深圳)有限公司 | Big data mining analysis system for real-time feedback |
| DE202023103864U1 (en) * | 2023-07-11 | 2023-08-16 | Kishori Jagdish Bhagat | A system for analyzing the role of the national stock exchange in the capital market |
| CN116823498A (en) * | 2023-06-09 | 2023-09-29 | 深圳市邦德布拉泽科技有限公司 | Personalized insurance product recommendation engine system based on big data analysis and machine learning |
-
2023
- 2023-11-02 CN CN202311444060.8A patent/CN117196694A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102190897B1 (en) * | 2019-09-19 | 2020-12-15 | (주)어반유니온 | Method and Apparatus for analyzing fashion trend based on big data |
| CN116823498A (en) * | 2023-06-09 | 2023-09-29 | 深圳市邦德布拉泽科技有限公司 | Personalized insurance product recommendation engine system based on big data analysis and machine learning |
| CN116501779A (en) * | 2023-06-26 | 2023-07-28 | 图林科技(深圳)有限公司 | Big data mining analysis system for real-time feedback |
| DE202023103864U1 (en) * | 2023-07-11 | 2023-08-16 | Kishori Jagdish Bhagat | A system for analyzing the role of the national stock exchange in the capital market |
Non-Patent Citations (1)
| Title |
|---|
| 吴俊杰;刘冠男;王静远;左源;部慧;林浩;: "数据智能:趋势与挑战", 系统工程理论与实践, no. 08, pages 2116 - 2149 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yalcin et al. | The use of multi-criteria decision-making methods in business analytics: A comprehensive literature review | |
| US10977293B2 (en) | Technology incident management platform | |
| Ragan et al. | Characterizing provenance in visualization and data analysis: an organizational framework of provenance types and purposes | |
| Poleto et al. | The roles of big data in the decision-support process: an empirical investigation | |
| Prada et al. | Educational data mining for tutoring support in higher education: a web-based tool case study in engineering degrees | |
| Baek et al. | Educational data mining versus learning analytics: A review of publications from 2015 to 2019 | |
| CN117876016B (en) | Distributed market data acquisition management system | |
| Kienzle et al. | Toward model-driven sustainability evaluation | |
| Li et al. | Regional network education information collection platform for smart classrooms based on big data technology | |
| Li et al. | Study on the interaction between big data and artificial intelligence | |
| Lu et al. | Frontier of information visualization and visual analytics in 2016 | |
| Burmeister et al. | Towards an extended enterprise architecture meta-model for Big Data-A literature-based approach | |
| Joshy et al. | Future educational environment–Identification of smart campus topic trends using text mining | |
| Islam et al. | A framework for effective big data analytics for decision support systems | |
| Karanth et al. | From data to knowledge analytics: capabilities and limitations | |
| CN117196694A (en) | Medicine market data analysis method and system based on big data | |
| Sangeetha et al. | Modelling of E-governance framework for mining knowledge from massive grievance redressal data | |
| Jin et al. | Technology prospecting for high tech companies through patent mining | |
| Chen et al. | On international Chinese education index ranking in a global perspective | |
| Abbas et al. | Improving Smart Grids Security: An Active Learning Approach for Smart Grid-Based Energy Theft Detection | |
| Rincón et al. | Project portfolio risk management. Bibliometry and collaboration Scientometric domain analysis | |
| Prada et al. | Data mining tool for academic data exploitation: Graphical data analysis and visualization | |
| Jiang et al. | A bibliometric analysis of research on organizational resilience | |
| Xie et al. | A sensitivity analysis of factors influential to the popularity of shared data in data repositories | |
| Ghahramani et al. | Visualisation for social media analytics: landscape of R packages |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |