CN114329140A

CN114329140A - Data processing method, device, equipment and storage medium

Info

Publication number: CN114329140A
Application number: CN202111624294.1A
Authority: CN
Inventors: 刘剑; 黄昉; 史亚冰; 蒋烨; 柴春光; 朱勇; 吕雅娟
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-12

Abstract

The present disclosure provides a data processing method, apparatus, device and storage medium, which relate to the technical field of data processing, and in particular to the technical field of artificial intelligence such as natural language processing and knowledge charts. The specific implementation scheme is as follows: acquiring specification data of medicines of a plurality of data sources; and integrating the instruction data of the medicine to obtain integrated data. The embodiment of the disclosure can integrate the specification data of the medicines of a plurality of data sources to obtain richer specification data of the medicines.

Description

Data processing method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of data processing technology, and in particular, to the field of artificial intelligence technology such as natural language processing and knowledge profiles.

Background

The construction of the drug instruction book mainly comprises the technologies of manual work, text recognition or crawler and the like. The manual mode is inefficient, and the maintenance cost is high. Text Recognition relies on the accuracy of Optical Character Recognition (OCR) technology Recognition, and the extracted drug instruction contents are relatively low in quality. The crawler and other technologies crawl a single data source on a network, and the crawled specifications are often incomplete and cannot meet the requirements on business.

Disclosure of Invention

The disclosure provides a data processing method, apparatus, device and storage medium.

According to an aspect of the present disclosure, there is provided a data processing method including:

acquiring specification data of medicines of a plurality of data sources;

and integrating the instruction data of the medicine to obtain integrated data.

According to another aspect of the present disclosure, there is provided a data processing apparatus including:

the acquisition module is used for acquiring the specification data of the medicines of a plurality of data sources;

and the integration module is used for integrating the specification data of the medicine to obtain integrated data.

According to another aspect of the present disclosure, there is provided an electronic device including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to cause the at least one processor to perform a method according to any one of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method according to any one of the embodiments of the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method according to any of the embodiments of the present disclosure.

The embodiment of the disclosure can integrate the specification data of the medicines of a plurality of data sources to obtain richer specification data of the medicines.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of a data processing method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 7 is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure;

FIG. 8 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 9 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 10 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 11 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 12 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a drug order building system according to an embodiment of the present disclosure;

FIG. 14 is a schematic diagram of a data source build according to an embodiment of the present disclosure;

FIG. 15 is a schematic illustration of data pre-processing according to an embodiment of the present disclosure;

FIG. 16 is a schematic illustration of filtration according to an embodiment of the present disclosure;

FIG. 17 is a schematic illustration of disambiguation according to an embodiment of the present disclosure;

FIG. 18 is a schematic diagram of a preference according to an embodiment of the present disclosure;

FIG. 19 is a schematic diagram of attribute level preference according to an embodiment of the present disclosure;

FIG. 20 is a schematic block diagram of an example electronic device that may be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram of a data processing method according to an embodiment of the present disclosure. The method can comprise the following steps:

s101, acquiring specification data of medicines of a plurality of data sources;

and S102, integrating the instruction data of the medicine to obtain integrated data.

In embodiments of the present disclosure, it is possible to obtain specification data for a drug from a number of data sources. The manual data of the medicine may be referred to as a medicine manual, or the like. The data source for the specification data for the pharmaceutical product may include a variety of, for example, a plurality of different web sites belonging to different data sources. Such as a web site for a web pharmacy, a web site for a drug delivery manufacturer, etc. The manual data of the medicine acquired from the web page of each website may also be referred to as web page manual data. The specification data, e.g. the text information in the specification presented on the web page, for the same drug may differ. For example, some web pages have detailed text information in the drug a specification, and some web pages have simple text information in the drug a specification. In addition, the instruction book data of the same drug on different web pages may have partial attribute overlap.

In the embodiment of the disclosure, the instruction book data of a certain medicine is acquired from a plurality of data sources and integrated, and the acquired integrated data is equivalent to the constructed new more abundant and accurate instruction book data. The newly constructed instruction data can be displayed in a webpage mode and the like, and can also be used as basic data of tasks such as medicine searching, medication suggestion and the like.

The embodiment of the disclosure can integrate the specification data of the medicines of a plurality of data sources, obtain richer specification data of the medicines, and meet the requirements of various scenes needing to use the specification data of the medicines.

Fig. 2 is a schematic diagram of a data processing method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the data processing method embodiments described above. In one possible embodiment, the integrating the instruction data of the pharmaceutical product in S102 includes: at least one of pre-processing, filtering, disambiguating, and preferring the instruction data for the drug. .

In one possible embodiment, the integrating the instruction data of the pharmaceutical product in S102 includes: s201, preprocessing the instruction book data of the medicine.

The method is beneficial to keeping the information which is required in the specification data of the medicine and is as rich as possible by preprocessing the specification data of the medicine.

In one possible embodiment, the pre-processing comprises: and S2011, preprocessing at a page level.

In one possible embodiment, the pre-processing comprises: and S2012, preprocessing of the attribute level.

In one possible embodiment, the pre-processing comprises: and S2013, associating the attributes.

In the disclosed embodiment, the preprocessing may include one or more steps of S2011, S2012, and S2013. Further, if the preprocessing includes a plurality of steps, the order of execution between the plurality of steps may not be limited. For example, if two steps are included, S2011 and S2012 may be performed first. As another example, if three steps are included, S2011, S2012 and S2013 may be executed first. For another example, if three steps are included, S2012 may be executed first, then S2011 may be executed, and finally S2013 may be executed. Other execution sequences may be used, and the disclosed embodiments are not limited.

In a possible embodiment, the page-level preprocessing of S2011 includes: and merging the content of the instruction book data of the same medicine distributed on different webpages. For example, if the same drug A is distributed on both web pages P1 and P2, the contents of the two web pages may be combined into a single copy of the drug's instruction sheet data. In the merging process, the repeated content may only retain one item, and the non-repeated content may retain a plurality of items. By combining the contents of the instruction data of different webpages, more complete instruction data of the medicine can be obtained.

In a possible implementation, the attribute-level preprocessing of S2012 includes: and analyzing and cleaning key value pairs in the instruction book data of the medicine. For example, a plurality of key-value pairs may be included in the specification data of a drug, each key-value pair may include an attribute name and an attribute value. By analyzing and cleaning key value pairs in the specification data of the medicine, more accurate attribute names and attribute values can be obtained.

In a possible embodiment, in the preprocessing of the attribute level in S2012, the parsing the key-value pairs in the specification data of the pharmaceutical product includes: and splitting key value pairs included in the attribute values in the specification data of the medicine to obtain new attribute names and attribute values.

In embodiments of the present disclosure, a drug may include multiple attributes. The complete attribute may include a key-value pair consisting of an attribute name and an attribute value. For example, the attribute name of a drug includes drug name, composition, indication, usage amount, contraindications, pharmacological actions, approval date, modification date, approval information, and the like. The attribute names typically have corresponding attribute values. For example, the attribute name "drug name" corresponds to the attribute value "a certain piece"; the attribute name 'usage dosage' corresponds to an attribute value '3 times a day, 1 tablet each time, and is taken in three meals'; the attribute name "approval date" corresponds to an attribute value of "1 month and 1 day 2021.

If a certain attribute value in the specification data of the medicine also comprises a plurality of key value pairs, the key value pairs can be further split into attribute names and attribute values. For example, the attribute value corresponding to the attribute name of a certain medicine being "medicine name" is "common name: a certain tablet; the English name is xxx tables; the commodity name is somehow through. The 3 key value pairs in the attribute values can be split, and the attribute value corresponding to the attribute name of "common name" is "a certain piece", the attribute value corresponding to the attribute name of "english name" is "xxx tables", and the attribute value corresponding to the attribute name of "commodity name" is "a certain communication".

By splitting the key value pairs in the attribute values, richer and more reasonable attribute names and attribute values can be obtained.

In a possible embodiment, in the preprocessing of the attribute level in S2012, the washing key-value pairs in the specification data of the pharmaceutical product includes: at least one of symbols, numbers, and formats in attribute names in the specification data of the medicine is washed.

If the attribute names in the specification data of the medicine have multiple expression modes, different expression modes of the attribute names can be cleaned to obtain a uniform expression mode. For example, after the symbols, spaces, and the like in the attribute names [ drug name ], "drug name", and the like are cleaned, a uniform expression of the attribute names can be obtained: name of the drug.

By cleaning the symbols, numbers and formats in the attribute names, the expression mode of the obtained attribute names is more standard, and the attribute names are beneficial to uniformly showing the effect or performing other processing by using the standard attribute names.

In a possible embodiment, the associating of the attribute at S2013 includes: and associating the attribute name in the instruction book data of the medicine with the attribute name of the medical knowledge map.

In embodiments of the present disclosure, the medical knowledge-map may be pre-constructed. For example, a medical knowledge map including various basic attributes of a drug may be constructed based on data on an authoritative website, such as a drug administration website. In a medical knowledge-graph, an attribute name of a drug may have a corresponding Identification (ID). For example, the ID corresponding to the attribute name "drug name" is "001", the ID corresponding to the attribute name "component" is "002", and the ID corresponding to the attribute name "indication" is "003". Specific IDs in the map may include numbers, characters, symbols, and the like, and the embodiments of the present disclosure are not limited thereto.

In the embodiment of the present disclosure, attribute normalization and format conversion are performed. Attribute name normalization may include normalizing attribute names of different descriptions to the same attribute name. The format conversion may include mapping the attribute names to IDs of the attribute names corresponding to the medical knowledge-graph to ID the attribute names. In this way, the attribute name included in the specification data of each medicine can be associated with the attribute name in the map, and the corresponding ID in the map can be set for the attribute name included in the specification data of the medicine. By associating the attribute names in the specification data of the medicine with the attribute names of the medical knowledge graph, the medical knowledge graph is beneficial to reasonably screening the specification data of the medicine and keeping accurate attributes.

Fig. 3 is a schematic diagram of a data processing method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the data processing method embodiments described above. In a possible embodiment, the integrating the instruction data of the pharmaceutical product in S102 further includes: s301, filtering the instruction book data of the medicine. The instruction book data of the medicine which does not meet the requirements can be removed through filtering, and more accurate instruction book data of the medicine can be obtained.

In one possible embodiment, the filtering comprises: s3011, filtering at page level.

In one possible embodiment, the filtering comprises: and S3012, filtering attribute levels.

In the embodiment of the present disclosure, the instruction manual data of the medicine captured from each website may be filtered, or the instruction manual data of the medicine after the preprocessing may be filtered. The filtering may include at least one of S3011 and S3012. The execution sequence of S3011 and S3012 is not limited, S3011 may be executed first, or S3012 may be executed first.

For example, through page-level filtering, the specification data for the unsatisfactory drug or drugs may be removed. As another example, one or more attributes of the drug's specification data that are not satisfactory may be removed by filtering at the attribute level.

In one possible implementation, the filtering at the page level in S3011 includes: filtering the specification data of the drug according to drug approval information and/or drug name.

For example, the drug approval information may include attributes such as "national drug standard", "original national drug standard", and the like. Generally, the drug approval information is authoritative and can be used as a basis for determining whether the instruction data of the drug meets the requirements. For another example, the name of a drug is generally an attribute of all drugs, and can be used to distinguish different drugs, or can be used as a basis for determining whether the instruction data of the drug meets the requirements. Through the medicine approval information and/or the medicine name, the specification data of the medicine can be accurately filtered, and the accurate specification data of the medicine meeting the requirements is reserved.

In one possible embodiment, in the page-level filtering of S3011, the filtering the instruction book data of the drug according to the drug approval information and/or the drug name includes filtering out the instruction book data of the drug in at least one of the following cases:

the attribute name in the specification data of the drug does not contain drug approval information;

the attribute name in the specification data of the medicine does not contain the medicine name;

the attribute value corresponding to the drug approval information in the specification data of the drug is null;

and the attribute value corresponding to the medicine name in the specification data of the medicine is null.

For example, if the attribute name in the specification data for a drug does not contain drug approval information, the drug may not be approved or drug approval information for the drug may not be captured. In this case, the accuracy of the specification data of the medicine is questioned, and the specification data of the medicine may be filtered out.

For another example, if the attribute name in the specification data of a certain drug does not include a drug name, it may not be possible to determine what drug the drug is specifically, and the specification data of the drug may be filtered out.

For another example, if the attribute name in the specification data of a certain drug includes drug approval information, but the attribute value of the drug approval information is null, the attribute value of the drug approval information may not be successfully captured, and the specification data of the drug may be filtered.

For another example, if the attribute name in the specification data of a certain drug contains a drug name, but the attribute value of the drug name is null, the attribute value of the drug name may not be successfully captured, and the specification data of the drug may be filtered.

The data of the specifications of the medicines which do not accord with the medicine approval information and/or the medicine names are filtered, and the accurate data of the specifications of the medicines which accord with the requirements better can be reserved.

In one possible implementation, the filtering at the attribute level in S3012 includes: and deleting the corresponding key value pair from the specification data of the medicine aiming at the attribute value failed in grabbing through the template.

For example, corresponding crawling templates may be set for the characteristics of the web pages of the respective data sources. If only a certain attribute name is grabbed by using the grabbing template, and the complete attribute value of the attribute name is not grabbed, the key-value pair comprising the attribute name and the attribute value can be deleted. There may be various reasons for the crawling failure, for example, a certain web page needs to be logged in to see the complete attribute value of some attributes. In this case, the captured attribute value may include contents such as "login" and an ellipsis, and if the "login" or the ellipsis is identified, it may be determined that the capture of the attribute value fails. By deleting the key value pair where the attribute value failed to be captured is located, incomplete attributes can be filtered out, and complete and accurate attributes are reserved.

Fig. 4 is a schematic diagram of a data processing method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the data processing method embodiments described above. In a possible embodiment, the integrating the instruction data of the pharmaceutical product in S102 further includes: s401, disambiguating the instruction book data of the medicine. By disambiguating, content that is easy to generate ambiguity can be eliminated, and more accurate specification data of the medicine can be obtained.

In one possible embodiment, the disambiguation comprises: s4011, information splitting is approved.

In one possible embodiment, the disambiguation comprises: and S4012, associating the medicines with the registered map.

In the embodiment of the present disclosure, the specification data of the medicine captured from each website may be disambiguated, the specification data of the preprocessed medicine may be disambiguated, or the specification data of the filtered medicine may be disambiguated. Disambiguation may include at least one of steps S4011 and S4012. The execution sequence of S4011 and S4012 is not limited, and S4011 may be executed first, or S4012 may be executed first.

In one possible embodiment, the splitting of the approval information in S4011 comprises: the method includes the steps of splitting the specification data of the medicine containing a plurality of pieces of medicine approval information into a plurality of pieces of specification data, each piece of specification data containing only one piece of medicine approval information, and copying the other contents in the specification data of the medicine in the split specification data. In embodiments of the present disclosure, a plurality of drug approval information may be included in the specification data of a single drug. For example, some of the drug approval information may be updated without being deleted, and some of the drug approval information may be approved by a different place or institution. The manual data of one medicine can be split into a plurality of manual data according to different medicine approval information. For example, if the specification data of the preprocessed and/or filtered medicine includes a plurality of national drug standards or original national drug standards, the medicine can be split according to the national drug standards or the original national drug standards. And splitting the medicine approval information based on the medicine approval information in the medicine specification data to obtain the medicine specification data unique to the medicine approval information.

In one possible embodiment, S4012 said drug association of entered atlas comprises: correlating the drug approval information with the specification data for the drug via a medical knowledge map.

In one possible embodiment, the drug approval information includes a national drug standard and/or a native national drug standard.

For example, the medical knowledge map may enter various approval information for a drug, such as national drug standards and/or original national drug standards, and so forth. The Chinese medicine standard characters of the specification data of the medicine and the Chinese medicine standard characters of the medical knowledge map can be associated and matched. If the national drug standard of the medicine of the medical knowledge graph comprises the national drug standard of the instruction book data of the medicine, the successful association can be judged. Otherwise, it may be determined to be unassociated. Based on the drug approval information association matching the drug approval information included in the manual data of each drug, it can be determined whether the drug approval information included in the manual data of the drug is accurate.

In a possible embodiment, S4012 said drug association of entered atlas further comprises: the instruction book data of unassociated drugs is discarded. By discarding the instruction book data of the medicine, which is not associated with the approval information of the medicine with the registered map, inaccurate instruction book data of the medicine can be removed, and accurate instruction book data of the medicine can be retained.

Fig. 5 is a schematic diagram of a data processing method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the data processing method embodiments described above. In a possible embodiment, the integrating the instruction data of the pharmaceutical product in S102 further includes: and S501, optimizing the instruction book data of the medicine. By preference, the preferred attributes can be selected from the specification data of the drug.

In one possible embodiment, the preferentially selecting includes: s5011, selecting the source preferentially.

In one possible embodiment, the preferentially selecting includes: s5012, global optimization.

In the embodiment of the present disclosure, the specification data of the medicine captured from each website may be preferred, the specification data of the preprocessed medicine may be preferred, the specification data of the filtered medicine may be preferred, or the specification data of the disambiguated medicine may be preferred. Preferably, at least one of the steps S5011 and S5012 may be included. The execution order of S5011 and S5012 is not limited, and S5011 may be executed first, or S5012 may be executed first.

In one possible embodiment, S5011 preferentially includes within the source: and performing attribute level preference on the specification data of the medicines of the same data source. By means of source interior preference, the optimal attribute values can be selected for all the attribute names in the specification data of a plurality of medicines in the same data source.

In one possible embodiment, the global preference of S5012 includes: for the instruction book data of the medicines of different data sources, the instruction book data of the medicines are selected according to priority and/or authority, and then attribute level preference is performed on the instruction book data of the selected medicines. In embodiments of the present disclosure, the priority and/or authority of the data source may be predetermined, such as based on expert experience to determine whether the data source is authoritative or the priority of the data source. Through global preference, the instruction book data of a plurality of medicines can be selected from different data sources according to priority and/or authority, and then the optimal attribute value is selected for each attribute name in the instruction book data of the selected medicines.

In one possible embodiment, in S5011 and/or S5012, the preference of the attribute level includes at least one of:

counting attribute values corresponding to attribute names of the specification data of the medicines, and setting the attribute value with the largest occurrence frequency as the attribute value corresponding to the attribute name;

calculating the similarity between the instruction book data of each medicine and the medicine name in the medical knowledge graph, and keeping the attribute value in the instruction book data of the medicine with the highest similarity;

selecting an attribute value in the specification data of the medicine with the largest attribute quantity according to the attribute quantity contained in the specification data of each medicine;

and selecting the attribute value in the instruction manual data of the latest medicine according to the updating time of the instruction manual data of each medicine.

In embodiments of the present disclosure, the above-described steps of preference of attribute levels may be performed in one or more. The execution can be performed in a set order every time, and the execution order can also be adjusted.

In one example, the same data source includes the specification data D11, D12, D13, D14, D15, and D16 for 6 medications of the same medication.

If the attribute value corresponding to the attribute name a1 includes a11 and a12, a11 appears 4 times, a12 appears 2 times, and the corresponding attribute value a11 is selected for the attribute name a 1.

If the attribute value corresponding to the attribute name A2 includes A21 and A22, and A21 and A22 both appear 3 times, further judgment is needed. If the medicine name of the instruction book data D12 of the medicine in which a22 is located has the highest similarity with the medicine name in the map, the corresponding attribute value a22 is selected for the attribute name a 2.

If the attribute value corresponding to the attribute name A3 includes A31 and A33, A31 and A33 both appear 3 times. Further, if the medicine names of the data D11 of the manual of the medicine in which a31 is located and the data D13 of the manual of the medicine in which a33 is located are similar to the medicine names in the map, further judgment is required. If D11 includes more attribute names than D13 in the manual data D11 and D13 of the medicine, the corresponding attribute value a33 may be selected for the attribute name A3.

If the attribute value corresponding to the attribute name A4 includes A41 and A44, A41 and A44 both appear 3 times. Further, the medicine name of the manual data D11 of the medicine where a41 exists and the manual data D14 of the medicine where a44 exists are similar to the medicine name in the map. Further, in the manual data D11 and D14 of the medicine, if the number of attribute names included in D11 is the same as that of D14, further judgment is required. If the update time of D14 is up-to-date, the corresponding attribute value A44 may be selected for attribute name A4.

Therefore, the specification data of a plurality of medicines of the same medicine in the same data source can be arranged into one part, so that the attribute name in the arranged specification data corresponds to the unique attribute value.

In another example, the different data sources include the specification data D21, D22, D23, D24, D25, and D26 for 6 medications of the same medication. For example, if D21, D22 came from an authoritative website, D21, D22 may be selected for attribute level preference. As another example, if the priorities of D21, D22, D23, D24, D25, and D26 are decreasing, the higher priority shares, such as D24, D25, and D26, may be selected for attribute level preference.

In one possible embodiment, the drug name similarity is calculated using an edit distance. For example, the edit distance may refer to the minimum number of edit operations required to transition from one string to another string. If they are at greater distances, they are said to be more different. Permitted editing operations may include replacing one character with another, inserting one character, deleting one character, and the like. The similarity of the medicine names can be rapidly and accurately calculated through the editing distance.

Fig. 6 is a schematic diagram of a data processing method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the data processing method embodiments described above. In one possible implementation, S101 of the method includes:

s601, capturing specification data of medicines of a plurality of data sources;

and S602, updating the database in an incremental mode by using the specification data of the captured medicines.

For example, the specification data of the drugs of each data source is downloaded to a local database by using an API provided by a web data crawling platform and a crawling template configured in advance for each data source. And acquiring a batch of latest specification data of the medicines each time, and after the grabbing task is completed, incrementally updating the specification data of the offline medicines and storing the updated specification data into the database. By automatically capturing the specification data of the medicines of a plurality of data sources and updating the increment, the specification data of the medicines to be processed can be quickly acquired, and a rich data base is provided for subsequent processing.

Fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure. The apparatus may include:

an obtaining module 701, configured to obtain specification data of drugs from multiple data sources;

an integration module 702, configured to integrate the specification data of the drug to obtain integrated data.

Fig. 8 is a schematic structural diagram of a data processing apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the data processing apparatus embodiments described above. In one possible embodiment, the integration module 702 is configured to at least one of pre-process, filter, disambiguate, and prefer the pharmaceutical specification data.

In one possible implementation, the integration module 702 includes:

and the preprocessing submodule 801 is used for preprocessing the instruction book data of the medicine.

In one possible implementation, the preprocessing sub-module 801 is configured to perform at least one of page-level preprocessing, attribute-level preprocessing, and attribute association.

In one possible implementation, the pre-processing sub-module 801 includes at least one of:

the page preprocessing submodule 8011 is configured to perform the page level preprocessing, and includes: merging the content of the instruction book data of the same medicine distributed on different webpages;

the attribute preprocessing sub-module 8012 is configured to perform the attribute level preprocessing, including: analyzing and cleaning key value pairs in the instruction book data of the medicine;

the association preprocessing submodule 8013 is configured to perform the attribute association, and includes: and associating the attribute name in the instruction book data of the medicine with the attribute name of the medical knowledge map.

In one possible embodiment, the attribute preprocessing sub-module 8012 is configured to parse key-value pairs in the instruction book data of the medicine, and includes: splitting key value pairs included in attribute values in the specification data of the medicines to obtain new attribute names and attribute values;

the attribute preprocessing submodule 8012 is configured to clean key-value pairs in the specification data of the medicine, and includes: at least one of symbols, numbers, and formats in attribute names in the specification data of the medicine is washed.

Fig. 9 is a schematic structural diagram of a data processing apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the data processing apparatus embodiments described above. In one possible implementation, the integration module 702 includes:

and a filtering submodule 901, configured to filter the specification data of the medicine.

In a possible implementation, the filtering submodule 901 is configured to perform at least one of filtering at a page level and filtering at an attribute level.

In a possible implementation, the filtering submodule 901 includes:

the page filtering sub-module 9011 is configured to perform filtering at the page level, where the filtering includes: filtering the specification data of the drug according to drug approval information and/or drug name;

the attribute filtering submodule 9012 is configured to perform filtering of the attribute level, and includes: and deleting the corresponding key value pair from the specification data of the medicine aiming at the attribute value failed in grabbing through the template.

In a possible embodiment, the page filtering sub-module 901 filters out the instruction book data of the medicine specifically in the case of at least one of the following cases:

Fig. 10 is a schematic structural diagram of a data processing apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the data processing apparatus embodiments described above. In one possible implementation, the integration module 702 includes:

a disambiguation sub-module 1001 for disambiguating the specification data of the drug.

In one possible embodiment, the disambiguation sub-module 1001 is configured to perform at least one of an approval information split and a mapped drug association.

In one possible implementation, the disambiguation module 1001 includes:

the approval information splitting sub-module 10011, configured to split the approval information, includes: splitting the specification data of the drug containing a plurality of pieces of drug approval information into a plurality of pieces of specification data, each piece of specification data containing only one piece of drug approval information, and copying the other contents in the specification data of the drug in the split specification data;

the map medicine association sub-module 10012 is configured to perform medicine association of the entered map, where the medicine association includes: and associating the medicine approval information with the instruction book data of the medicine through the medicine approval information in the medical knowledge map, wherein the medicine approval information comprises national medicine standard words and/or original national medicine standard words.

In a possible embodiment, the map drug association sub-module 10012 is configured to perform the drug association of the entered map further including: the instruction book data of unassociated drugs is discarded.

Fig. 11 is a schematic structural diagram of a data processing apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the data processing apparatus embodiments described above. In one possible implementation, the integration module 702 includes:

the preference sub-module 1101 is configured to select a preference for the instruction book data of the medicine.

In one possible implementation, the preference sub-module 1101 is configured to perform at least one of intra-source preference and global preference.

In one possible implementation, the preference sub-module 1101 includes at least one of:

an in-source preference sub-module 11011 configured to perform the in-source preference includes: performing attribute level preference on the specification data of the medicines of the same data source;

a global preference sub-module 11012 for performing the global preference includes: for the instruction book data of the medicines of different data sources, the instruction book data of the medicines are selected according to priority and/or authority, and then attribute level preference is performed on the instruction book data of the selected medicines.

In one possible embodiment, the preference of the attribute level includes at least one of:

In one possible embodiment, the drug name similarity is calculated using an edit distance.

Fig. 12 is a schematic structural diagram of a data processing apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the data processing apparatus embodiments described above. In a possible implementation, the obtaining module 701 of the apparatus includes:

the data grabbing submodule 1201 is used for grabbing instruction book data of medicines of a plurality of data sources;

an incremental update sub-module 1202 for incrementally updating the database with the manual data of the drug that was taken.

For a description of specific functions and examples of each module and each sub-module of the data processing apparatus in the embodiment of the present disclosure, reference may be made to the related description of the corresponding step in the above data processing method embodiment, and details are not repeated here.

In an application scenario, the embodiment of the present disclosure may provide a method for constructing a drug specification fusing multiple data sources, where a high-quality drug specification data is constructed by using a specification of a drug from multiple data sources, for example, web specification data of the drug, through steps of data preprocessing, web specification filtering, disambiguation, preference, and the like.

In the current market, companies with medical lines are all relying on laying a large amount of manpower to solve the problem of drug knowledge. If the medicine knowledge is automatically constructed, the construction cost of the medicine knowledge can be greatly reduced, and absolute advantages are provided for products, so that the market is rapidly broken through. Knowledge used in pharmacy-related lines includes drug knowledge maps.

The medicine specification data is an important knowledge source for constructing medicine map data and data which can be directly displayed in the service landing process, can reflect the data quality and the professionality and has very important function in related pharmacy services. The medicine specification data is used as a data extraction source to assist the construction of medicine map data, and the accuracy of the map data can be obviously improved.

For example, the overall structure and process flow of the pharmacy specification building system are as shown in fig. 13. The system mainly comprises a data source construction module and a specification construction module. And the data source building module is used for updating the crawled webpage specification data into the database in an incremental mode. And the specification construction module is used for exporting the webpage specification data in the database, and then completing construction of the medicine specification data through preprocessing, filtering, disambiguation, preference and other modes.

1 data source construction module

The functions of the data source building module comprise: and acquiring webpage specification data through a webpage data capturing platform, and updating the acquired webpage specification data into a database in an incremental manner. As shown in FIG. 14, the following steps may be performed sequentially throughout the construction of the data source construction module.

Acquiring webpage specification data: the module can acquire webpage specification data through a webpage data capturing platform. The module can comprise an API provided by a webpage data crawling platform and crawling templates configured in advance for each data source, and is used for downloading webpage specification data of each data source to the local. The crawling template may include configuration of the web page information crawling platform when crawling the web page. For example, different crawling templates are created for different web pages, and in each crawling template, the attribute name and the attribute value of the drug specification are configured as a key-value (key-value) to crawl the form, for example: "drug name: liu Wei Di Huang Wan (pill of six ingredients with rehmannia).

Updating the latest webpage specification data: since a batch of latest webpage specification data can be obtained every time of grabbing, the stored offline webpage specification data can be updated after the grabbing task is completed.

Updating and warehousing the webpage specification data increment: after the latest off-line webpage specification data is updated, the off-line webpage specification data is updated in an incremental mode and stored in the database.

2 description construction module

2.1 data preprocessing Module

The functions of the data preprocessing module comprise: and obtaining a piece of preprocessed webpage specification data from each data source through operations such as preprocessing of page level and attribute level, attribute association and the like on the webpage specification data exported from the database, so as to be used in the subsequent process. As shown in fig. 15, in the data preprocessing module, the following steps may be sequentially performed:

preprocessing at page level: the instruction data from the same webpage can be merged, and the instruction contents of the same medicine distributed on different pages can be merged.

Preprocessing of attribute level: the method can be used for analyzing and cleaning the captured webpage specification key-value pair, and comprises the following steps: parsing the contents of a specification attribute value HyperText Markup Language (HTML) tag, cleaning a specification attribute name, splitting a specific attribute, and the like. For example, other key-value pairs included in the attribute values may be subject to the splitting process. As another example, unnecessary punctuation marks, formatting, etc. in the attribute names may be cleaned.

And (3) attribute association: the preprocessed data needs to be subjected to attribute association operation so as to associate the attribute name of the webpage specification with the attribute name in the medical knowledge map. The attribute association operation mainly comprises attribute name normalization and format conversion. Attribute name normalization may include normalizing attribute names of different descriptions to the same attribute name. The format conversion may include mapping the attribute names to IDs (identifications) of the attribute names corresponding to the medical knowledge-map to ID the attribute names. That is, the same ID as the same attribute name in the medical knowledge map can be added to the attribute name of the web page specification by attribute association. For example, the ID of "date of production" in the map is 26, and the ID of "date of production" in the description of the web page is also set to 26, so that the attribute name of "date of production" has a unique identification, and the attribute name can be conveniently searched by the ID.

2.2 filtration

The filtering module mainly comprises a filtering module of a page level and an attribute level. The filtering module can filter out the webpage specifications which do not meet the requirements and the attributes of the webpage specifications, so that the number of candidate webpage specifications is reduced, and the attributes which do not meet the requirements are filtered out, so that downstream tasks are not influenced. As shown in fig. 16, the filtering method specifically includes:

filtering at page level: and filtering out webpage specification data which do not contain the national standard characters or the medicine names and have empty national standard characters or medicine names.

Filtering of attribute level: for attribute values that fail to grab through the template (e.g., indicating that the grab is not complete if an ellipsis, login, etc. is included in the attribute value). For the attributes, the corresponding key-value pair is directly deleted from the preprocessed webpage specification.

In this example, the "national drug standard" may include a drug manufacturing approval document number obtained by a drug manufacturing unit after being strictly approved by the national food and drug administration before manufacturing a new drug, which corresponds to a human identification card. For example, the format of the national drug standard may be: the Chinese medicine standard characters +1 letter +8 digits, wherein the letter used by the chemical is 'H', and the letter used by the Chinese medicine is 'Z', and the like. Only if the approval document is obtained, the medicine can be produced and sold. The national drug standard of a drug changes for some reason, and the "original national drug standard" may include the national drug standard number of the drug prior to the change.

2.3 disambiguation

The disambiguation module may associate the web page specification data with the drugs in the recorded medical knowledge map. For example, as shown in FIG. 17, disambiguation may include the following steps:

splitting the standard character: the quasi-character splitting can be used for preprocessing the next disambiguation stage, so that the subsequent disambiguation steps can be unified conveniently. The specific method comprises the following steps: splitting a specification containing a plurality of national drug standards into that each specification only contains one national drug standard, and copying other contents of the specification; for example, the specification data including 3 national drug standards is divided into 3 parts of specification data, and each part includes different national drug standards and the rest is the same.

Associating with the drug in the recorded medical knowledge-graph: the relation is realized through the national drug standard of the webpage specification. Since the drugs in the medical knowledge graph may contain two attributes of the national drug standard and the original national drug standard, both of the two attributes can be included in the association policy during association. For example, the Chinese medicine standard characters in the medicine are firstly associated with the webpage specification through the medical knowledge map, and if the Chinese medicine standard characters are not associated, the Chinese medicine standard characters are used for association. After the processing of the module, the webpage specification data which is not related can be directly discarded.

2.4 preferred

Because the webpage specification data acquired by the scheme comes from a plurality of specification data sources, the same medicine may contain a plurality of specification data, and the attributes of the same medicine may be multiple. The data and the attributes of the medicine specification can be normalized through the preference module, and finally each medicine only contains one specification data, and the attribute value in each specification data is unique.

The preference module may contain both intra-source preference and global preference modules, see FIG. 18. The source preference can be in the same specification data source, and for the webpage specification data of the same medicine, the only optimal attribute value can be selected through a strategy. The intra-source preferences primarily include preference for attribute levels. For example, referring to FIG. 19, preference of attribute levels may be performed in the following policy order, terminating when only one attribute value remains:

attribute value statistics: in the multiple specification data, if a certain attribute name includes a plurality of attribute values, the attribute value that appears most frequently is set as the attribute value corresponding to the attribute name.

Similarity to drug name in medical knowledge map: the more similar the drug name and medical knowledge map corresponding to an attribute in a specification data, the greater the likelihood that the attribute will remain. The method for calculating the similarity of the drug names can adopt an edit distance.

According to the number of attributes contained in the specification itself: since the specification data is described in more detail as more attributes are included in one specification data, attribute values included in a specification having a larger number of attributes can be selected.

Updating time according to the specification: the newer the update time of the specification data, the higher the reliability of the specification, and therefore, the attribute value included in the latest specification data can be selected.

The specifications corresponding to the same drug may be duplicated among different specification data sources, and the specification quality of different data sources is also uneven. Thus, after intra-source preference, global preference can be performed. Different specification data sources can be marked by priority, and a plurality of authoritative data sources can be set. The global preference is relative to the intra-source preference, and because the priorities among different data sources are considered, the page level preference can be performed firstly, and then the attribute level preference can be performed. For example, the specific operation of global preference is as follows:

and sequencing the instruction book data corresponding to the same medicine through the priority of the data source, if the instruction book data of the authoritative data source exists, selecting the instruction book data of the authoritative data source, discarding the rest instruction book data, and then transferring the selected instruction book data to the attribute level for preference.

If the drug does not contain the instruction data of the authoritative data source, then all the instruction data is directly passed on to preference at the attribute level.

The preference policy of the attribute level in global preference is consistent with the preference policy of the attribute level in source preference, which can be referred to the above related description.

And after the global preference strategy is executed, the final medicine specification data fusing multiple data sources can be obtained.

The method for constructing the medicine specification fusing multiple data sources provided by the embodiment of the disclosure comprises at least one of the following advantages:

(1) compared with a manual construction mode, the method for constructing the medicine specification can automatically construct medicine knowledge by using technical means, can greatly reduce construction cost of the medicine knowledge, provides absolute advantages for products, and accordingly rapidly breaks through the market.

(2) The information extraction platform is used for acquiring webpage specification data, such as a hundred-degree internal webpage structured information extraction platform, and the key-value pair information of the medicine specification can be completely reserved through data preprocessing and other modes, so that the content in the medicine specification can be accurately acquired no matter the medicine specification is displayed for a client or a downstream task is used as a data source.

(3) The method comprises the steps of obtaining webpage specification data from a plurality of data sources, and processing the webpage specification data of the data sources through technical means. Compared with the medicine specification acquired by a single data source, the medicine specification data quality obtained by applying the medicine specification construction method disclosed by the embodiment of the disclosure is higher, and the specification data is more complete.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

Fig. 20 illustrates a schematic block diagram of an example electronic device 2000, which may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 20, the apparatus 2000 includes a computing unit 2001, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)2002 or a computer program loaded from a storage unit 2008 into a Random Access Memory (RAM) 2003. In the RAM 2003, various programs and data required for the operation of the device 2000 can also be stored. The calculation unit 2001, the ROM 2002, and the RAM 2003 are connected to each other by a bus 2004. An input/output (I/O) interface 2005 is also connected to bus 2004.

A number of components in device 2000 are connected to I/O interface 2005, including: an input unit 2006 such as a keyboard, a mouse, and the like; an output unit 2007 such as various types of displays, speakers, and the like; a storage unit 2008 such as a magnetic disk, an optical disk, or the like; and a communication unit 2009 such as a network card, a modem, a wireless communication transceiver, or the like. The communication unit 2009 allows the device 2000 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 2001 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 2001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 2001 executes the respective methods and processes described above, for example, a data processing method. For example, in some embodiments, a data processing method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 2008. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 2000 via the ROM 2002 and/or the communication unit 2009. When the computer program is loaded into the RAM 2003 and executed by the computing unit 2001, one or more steps of a data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 2001 may be configured to perform a data processing method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method of data processing, comprising:

acquiring specification data of medicines of a plurality of data sources;

and integrating the instruction data of the medicine to obtain integrated data.

2. The method of claim 1, wherein the integrating the instructional data for the pharmaceutical product comprises: at least one of pre-processing, filtering, disambiguating, and preferring the instruction data for the drug.

3. The method of claim 2, wherein the pre-processing comprises: at least one of page level preprocessing, attribute level preprocessing, and attribute association;

the page level preprocessing comprises: merging the content of the instruction book data of the same medicine distributed on different webpages;

the attribute level preprocessing comprises: analyzing and cleaning key value pairs in the instruction book data of the medicine;

the attribute association comprises: and associating the attribute name in the instruction book data of the medicine with the attribute name of the medical knowledge map.

4. The method of claim 3, wherein the parsing key-value pairs in the specification data of the drug comprises: splitting key value pairs included in attribute values in the specification data of the medicines to obtain new attribute names and attribute values;

the cleaning of the key-value pairs in the instruction book data of the drug comprises: at least one of symbols, numbers, and formats in attribute names in the specification data of the medicine is washed.

5. The method of any of claims 2 to 4, wherein the filtering comprises: at least one of page level filtering and attribute level filtering;

the page level filtering includes: filtering the specification data of the drug according to drug approval information and/or drug name;

the filtering of the attribute level includes: and deleting the corresponding key value pair from the specification data of the medicine aiming at the attribute value failed in grabbing through the template.

6. The method of claim 5, wherein the filtering the specification data for the drug according to drug approval information and/or drug name comprises filtering out the specification data for the drug if at least one of:

7. The method of any of claims 2 to 6, wherein the disambiguation comprises at least one of approving information resolution and drug association of an entered profile;

the approval information splitting comprises: splitting the specification data of the drug containing a plurality of pieces of drug approval information into a plurality of pieces of specification data, each piece of specification data containing only one piece of drug approval information, and copying the other contents in the specification data of the drug in the split specification data;

the drug association of the entered map comprises: and associating the medicine approval information with the instruction book data of the medicine through the medicine approval information in the medical knowledge map, wherein the medicine approval information comprises national medicine standard words and/or original national medicine standard words.

8. The method of claim 7, wherein the atlas-entered drug association further comprises: the instruction book data of unassociated drugs is discarded.

9. The method of any of claims 2-8, wherein the preference includes at least one of an intra-source preference and a global preference;

the in-source preference comprises: performing attribute level preference on the specification data of the medicines of the same data source;

the global preference comprises: for the instruction book data of the medicines of different data sources, the instruction book data of the medicines are selected according to priority and/or authority, and then attribute level preference is performed on the instruction book data of the selected medicines.

10. The method of claim 9, wherein the preference of the attribute level comprises at least one of:

11. The method of claim 10, wherein the drug name similarity is calculated using an edit distance.

12. The method of any one of claims 1 to 11, wherein obtaining pharmacy specification data for a plurality of data sources comprises:

capturing drug specification data from web pages of a plurality of data sources;

and performing incremental updating on the database by using the captured medicine specification data.

13. A data processing apparatus comprising:

14. The apparatus of claim 13, wherein the integration module is to at least one of pre-process, filter, disambiguate, and prefer the pharmaceutical specification data.

15. The apparatus of claim 14, wherein the consolidation module comprises a pre-processing submodule to perform at least one of page level pre-processing, attribute level pre-processing, and attribute association, the pre-processing submodule comprising at least one of:

the page preprocessing submodule is used for preprocessing the page level and comprises: merging the content of the instruction book data of the same medicine distributed on different webpages;

the attribute preprocessing submodule is used for preprocessing the attribute level and comprises: analyzing and cleaning key value pairs in the instruction book data of the medicine;

the association preprocessing submodule is used for performing attribute association and comprises: and associating the attribute name in the instruction book data of the medicine with the attribute name of the medical knowledge map.

16. The apparatus of claim 15, wherein the attribute preprocessing submodule for the parsing of key-value pairs in the specification data of the drug comprises: splitting key value pairs included in attribute values in the specification data of the medicines to obtain new attribute names and attribute values;

the attribute preprocessing submodule is used for cleaning key value pairs in the specification data of the medicine and comprises the following steps: at least one of symbols, numbers, and formats in attribute names in the specification data of the medicine is washed.

17. The apparatus of any of claims 14 to 16, wherein the consolidation module includes a filtering submodule to perform at least one of page level filtering and attribute level filtering, the filtering submodule comprising:

the page filtering submodule is used for filtering the page level and comprises: filtering the specification data of the drug according to drug approval information and/or drug name;

an attribute filtering submodule, configured to perform filtering of the attribute level, including: and deleting the corresponding key value pair from the specification data of the medicine aiming at the attribute value failed in grabbing through the template.

18. The apparatus of claim 17, wherein the page filtering sub-module is specifically configured to filter out the instruction book data of the pharmaceutical product if at least one of:

19. The apparatus of any one of claims 14 to 18, wherein the integration module comprises a disambiguation sub-module for performing at least one of approval information splitting and mapped drug association, the disambiguation sub-module comprising:

an approval information splitting sub-module, configured to split the approval information, including: splitting the specification data of the drug containing a plurality of pieces of drug approval information into a plurality of pieces of specification data, each piece of specification data containing only one piece of drug approval information, and copying the other contents in the specification data of the drug in the split specification data;

the map medicine association submodule is used for carrying out medicine association of the input map and comprises the following steps: and associating the medicine approval information with the instruction book data of the medicine through the medicine approval information in the medical knowledge map, wherein the medicine approval information comprises national medicine standard words and/or original national medicine standard words.

20. The apparatus of claim 19, wherein the profile drug association sub-module for performing the entered-profile drug association further comprises: the instruction book data of unassociated drugs is discarded.

21. The apparatus of any of claims 14 to 20, wherein the integration module includes a preference module to perform at least one of intra-source and global preference, the preference module including at least one of:

an in-source preference module configured to perform the in-source preference including: performing attribute level preference on the specification data of the medicines of the same data source;

a global preference sub-module, configured to perform the global preference including: for the instruction book data of the medicines of different data sources, the instruction book data of the medicines are selected according to priority and/or authority, and then attribute level preference is performed on the instruction book data of the selected medicines.

22. The apparatus of claim 21, wherein the preference of the attribute level comprises at least one of:

23. The apparatus of claim 22, wherein the drug name similarity is calculated using an edit distance.

24. The apparatus of any of claims 13 to 23, the obtaining means comprising:

the data capturing submodule is used for capturing specification data of the medicines of a plurality of data sources;

and the increment updating submodule is used for carrying out increment updating on the database by using the specification data of the grabbed medicines.

25. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

26. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-12.

27. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-12.