JP7343585B2 - Identification support system, identification support client, identification support server, and identification support method - Google Patents

Description

The present invention relates to a drug identification support system, an identification support client, an identification support server, and an identification support method.

In medical settings such as hospitals and pharmacies, drugs provided to patients are audited and drugs brought by patients are differentiated. Since manual auditing and differentiation requires a long work time and imposes a heavy burden on users (doctors, pharmacists, etc.), techniques for supporting auditing and differentiation have been proposed. For example, Patent Document 1 describes an image processing device that performs character recognition.

Japanese Patent Application Publication No. 2015-068765

One embodiment of the present invention provides an identification support system, an identification support client, and an identification support method that allow a user to accurately identify drugs. The present invention also provides an identification support server that can be used to identify drugs.

In order to achieve the above-mentioned object, an identification support system according to a first aspect of the present invention includes an image acquisition unit that acquires an image of a drug, and a system that extracts character strings included in the image character by character and indicates the character string. a text conversion unit that generates a first text; a text modification unit that generates a second text by modifying the first text based on an expression used to identify the drug; The identification information shown and the character information showing the characters attached to the drug are compared with the drug master stored as text, and the second text is a candidate for the drug indicated by the second text. The device includes a collation unit that acquires identification information about the candidate drug, and an output unit that outputs the identification information about the candidate drug.

According to the first aspect, the character string included in the image is extracted character by character, the extracted text is modified based on the expression used for drug identification, and the modified text is compared with the drug master. The user can accurately identify the drug. Note that in the first aspect, the components of the system may be housed in one housing, or may be housed separately in a plurality of housings. Further, a plurality of devices may be connected via a network and satisfy the configuration requirements of the first aspect as a whole.

In the identification support system according to the second aspect, in the first aspect, the text conversion unit extracts a character string using a learned model configured by machine learning. The trained model may be a trained model using a neural network.

The identification support system according to the third aspect according to the first or second aspect further includes an emphasis processing unit that emphasizes a character string included in the image to generate a character string emphasized image, and a text conversion unit that emphasizes the character string. Generating first text from the image. According to the third aspect, recognition accuracy when extracting a character string from an image can be improved.

In the identification support system according to a fourth aspect, in any one of the first to third aspects, the collation unit calculates the degree of similarity between the second text and the character information, and identifies the candidate drug based on the degree of similarity. Select. In the fourth aspect, the matching unit may consider the distance between character strings in calculating the degree of similarity.

In the identification support system according to the fifth aspect, in any one of the first to fourth aspects, the drug master has layout information regarding character information, and the collation unit performs collation based on the layout information. The layout information may include information such as the number of columns (two columns, three columns, etc.) and the arrangement (from the upper left to the upper right, then to the lower left, further to the lower right, etc.). The recognition accuracy can be improved by the verification unit performing verification based on the layout information.

In the identification support system according to a sixth aspect, in any one of the first to fifth aspects, the drug master stores the identification information and the image of the drug in association with each other, and the output unit stores the identification information and the image of the candidate drug. to be displayed on the display device in association with the above. According to the sixth aspect, the user can easily visually determine whether the matching result is appropriate. Note that the image of the drug may be an image of the drug packaging (PTP sheet, etc.) instead of the drug itself.

In order to achieve the above-mentioned object, an identification support client according to a seventh aspect of the present invention includes an image acquisition unit that acquires an image of a drug, and a character string that extracts character strings included in the image character by character and indicates the character string. a text conversion unit that generates a first text; a text modification unit that generates a second text by modifying the first text based on an expression used for drug identification; a client-side transmitting unit that transmits to the identification support server; and a client-side receiving unit that receives identification information including a drug code and/or name from the identification support server regarding a candidate drug that is a drug candidate indicated by the second text. , and an output unit that outputs identification information about the candidate drug. According to the seventh aspect, the user can accurately identify drugs. Note that the identification support client according to the seventh aspect may have the configurations according to the second to sixth aspects.

In order to achieve the above-mentioned object, an identification support server according to an eighth aspect of the present invention is configured such that identification information indicating a code and/or name of a drug and character information indicating characters attached to a drug are stored as text. A server-side reception unit that receives the stored drug master and text information about the drug from the identification support client, and compares the text information with the drug master to obtain identification information about the candidate drug that is the drug candidate indicated by the text information. and a server-side transmitter that transmits the acquired identification information to the identification support client. The identification support server according to the eighth aspect can be used for drug identification support. Note that the identification support client according to the eighth aspect may include the configurations according to the second to sixth aspects. Furthermore, an identification support system similar to the identification support system according to the first aspect can be configured by the identification support client according to the seventh aspect and the identification support server according to the eighth aspect.

In order to achieve the above-mentioned object, an identification support method according to a ninth aspect of the present invention includes an image acquisition step of acquiring an image of a drug, and a character string included in the image is extracted character by character to indicate the character string. a text conversion step for generating a first text; a text modification step for generating a second text by modifying the first text based on an expression used to identify the drug; The identification information shown and the character information showing the characters attached to the drug are compared with the drug master stored as text and the second text to determine the candidate drug that is the drug candidate indicated by the second text. and an output step of outputting identification information about the candidate drug. According to the ninth aspect, similarly to the first aspect, the user can accurately identify drugs. Note that the identification support method according to the ninth aspect may have the same configuration as the second to sixth aspects. Furthermore, a program for causing an identification support system or a computer to execute the identification support method of these aspects, and a non-temporary recording medium recording a computer-readable code of such a program can also be mentioned as aspects of the present invention.

Note that the above-described identification support system, identification support client, identification support server, and identification support method can be used for drug identification support and/or audit support.

As described above, according to the identification support system, identification support client, and identification support method of the present invention, a user can accurately identify drugs. Furthermore, the identification support server of the present invention can be used to identify drugs.

FIG. 1 is a diagram showing the configuration of an identification support system according to a first embodiment. FIG. 2 is a functional block diagram of the processing section. FIG. 3 is a diagram showing information stored in the storage section. FIG. 4 is a flowchart showing the processing of the identification support method according to the first embodiment. FIG. 5 is a flowchart showing the matching process. FIG. 6 is a diagram showing an example of a stamp etc. attached to a drug. FIG. 7 is a diagram showing the configuration of an identification support system according to the second embodiment. FIG. 8 is a functional block diagram of the client processing section. FIG. 9 is a diagram showing information stored in the client storage section. FIG. 10 is a functional block diagram of the server processing section. FIG. 11 is a diagram showing information stored in the server storage section. FIG. 12 is a flowchart showing the processing of the identification support method according to the second embodiment. FIG. 13 is another flowchart showing the processing of the identification support method according to the second embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of an identification support system, an identification support client, an identification support server, and an identification support method according to the present invention will be described in detail with reference to the accompanying drawings.

<First embodiment>
FIG. 1 is a block diagram showing the configuration of an identification support system 10 (identification support system) according to the first embodiment. The identification support system 10 is a system that supports drug identification, and can be realized using a computer. As shown in FIG. 1, the identification support system 10 includes a processing section 100, a lighting section 120, a photographing section 130, a storage section 200, a display section 300, and an operation section 400, which are connected to each other to transmit and receive necessary information. . The illumination unit 120 includes a light source that illuminates the drug, and the imaging unit 130 includes a camera that captures images of the drug. The identification support system 10 also connects to an external server, external database, etc. (not shown) via a communication control unit 114 (see FIG. 2) and a network (not shown), and provides information (such as drug identification information and images) as necessary. ) can be obtained.

It should be noted that the identification support system 10 can be applied to assist in the identification of medicines brought by patients, and to assist in the auditing of medicines provided to patients.

<Configuration of processing section>
FIG. 2 is a diagram showing the configuration of the processing section 100. The processing unit 100 includes an image acquisition unit 102 (image acquisition unit), an emphasis processing unit 104 (emphasis processing unit), a text conversion unit 106 (text conversion unit), a text correction unit 108 (text correction unit), and a matching unit 110 (matching unit). ), an output section 112 (output section), and a communication control section 114. The processing unit 100 further includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), which are not shown. Note that processing by each of these units is performed under the control of the CPU.

The functions of each part of the processing unit 100 described above can be realized using various processors. Various types of processors include, for example, a CPU, which is a general-purpose processor that executes software (programs) to realize various functions. In addition, the various processors mentioned above include programmable logic devices (processors whose circuit configuration can be changed after manufacture) such as GPUs (Graphics Processing Units), which are processors specialized in image processing, and FPGAs (Field Programmable Gate Arrays). Programmable Logic Device (PLD) is also included. Furthermore, the various types of processors mentioned above also include dedicated electric circuits, such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration specifically designed to execute specific processing.

The functions of each part may be realized by one processor, or by multiple processors of the same type or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Further, one processor may realize multiple functions. As an example of configuring multiple functions in one processor, first, one processor is configured by a combination of one or more CPUs and software, as typified by computers such as clients and servers, and this processor There is a form in which this is realized as multiple functions. Second, there is a form that uses a processor that realizes the functions of the entire system with one IC (Integrated Circuit) chip, as typified by System On Chip (SoC). In this way, various functions are configured using one or more of the various processors described above as a hardware structure. Furthermore, the hardware structure of these various processors is more specifically an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements.

When the above-mentioned processor or electric circuit executes software (program), the computer-readable code of the software to be executed (for example, the various processors and electric circuits that constitute the processing unit 100, and/or a combination thereof) The software is stored in a non-temporary storage medium such as a ROM, and the processor refers to the software. The software stored in the non-temporary recording medium includes a program (identification support program) for executing the identification support method according to the present invention. The program code may be recorded in a non-temporary recording medium such as various magneto-optical recording devices or semiconductor memories instead of the ROM. During processing using software, for example, a RAM is used as a temporary storage area, and it is also possible to refer to data stored in, for example, an EEPROM (Electronically Erasable and Programmable Read Only Memory), not shown.

<Storage section configuration>
The storage unit 200 includes a non-temporary recording medium such as a DVD (Digital Versatile Disk), a hard disk, and various semiconductor memories, and a control unit thereof, and as shown in FIG. An image 204 (medicine image), a matching result 206, and additional learning data 208 are stored. In addition to this, a drug matching dictionary (described later) in which expressions used for drug identification are learned may be stored. The drug matching dictionary is a dictionary in which, for example, numbers, alphabets, company names and their abbreviations are registered as conversion candidates, and this can increase the possibility that the intended word will be input as a search keyword.

In the drug master 202, identification information including the code and/or name of the drug and character information indicating characters attached to the drug are stored in association with each other as text information. The "code" is, for example, a YJ code (individual drug code consisting of 12 alphanumeric characters), and the name may include the volume of the active ingredient. Furthermore, the "character information" includes stamp information and/or printed information on the drug. Regarding stamping and printing, it is preferable to store information on each of the front and back sides of the drug. The drug master 202 may store the general name of a drug and information on individual products, or the information on original drugs and the information on generic drugs in association with each other.

The storage unit 200 stores the drug image 204 in association with the drug master 202. Preferably, the drug image 204 is also stored for each of the front and back sides of the drug. Note that the storage unit 200 can store a photographed image or a character string emphasized image as a drug image 204.

<Configuration of display section and operation section>
The display unit 300 includes a monitor 310 (display device), and can display information stored in the storage unit 200, results of processing by the processing unit 100, and the like. The operation unit 400 includes a keyboard 410 and a mouse 420 as input devices or pointing devices, and the user performs operations necessary to execute the identification support method according to the present invention via these devices and the screen of the monitor 310. (described later). The monitor 310 may be configured with a touch panel, and the user may perform operations via the touch panel.

<Processing of identification support method>
The identification support method by the identification support system 10 having the above-described configuration will be described below with reference to the flowcharts in FIGS. 4 and 5.

<Image acquisition>
The image acquisition unit 102 controls the illumination unit 120 and the imaging unit 130 to acquire an image of the drug (step S100: image acquisition step). The image acquisition unit 102 takes a plurality of images of the drug while switching the illumination direction, and the emphasis processing unit 104 generates a character string emphasized image by emphasizing character strings (engraved, printed, etc.) included in the taken images. Good too. Further, the image acquisition unit 102 and the communication control unit 114 may acquire images from a device connected via a network. Note that in order to improve the accuracy of identification, it is preferable that images of the drug be acquired on both sides. Further, the image to be photographed may be an image of a drug packaging (PTP, etc.).

<Text generation>
The text conversion unit 106 extracts character strings included in the photographed image or character string emphasized image character by character, and generates first text indicating the character string (step S110: text conversion step). The text conversion (character recognition) method is not particularly limited, but a trained model such as a neural network configured by machine learning, for example, Mask-RCNN (RCNN: Regions with Convolutional Neural Networks) can be used. In Mask-RCNN, a fixed-size feature map obtained by convolution of an input image is divided into a fully connected layer and a convolutional layer for region segmentation, and then input, and the masked region is estimated by the convolutional layer. The text conversion unit 106 may directly estimate the center of a character by machine learning and then specify the range of each character. Note that it is preferable that the text conversion unit 106 detects the layout of the character strings included in the image (for example, "from top left to top right, bottom left, bottom right", "two columns", "free layout", etc.), In this case, it is preferable that the medicine master 202 also has layout information regarding character strings (layout information regarding character information) (described later).

<Text correction>
In the text conversion (character recognition) in step S110, there is a possibility that characters (text) different from the characters that should be originally recognized are output. Therefore, the text modification unit 108 modifies the first text based on the expression used for drug identification to generate a second text (step S120: text modification step). Specifically, the text correction unit 108 corrects the text (generates a second text) using a trained model trained by pairing a character string (text) containing an error with a correct character string (text). )can do. Such a trained model can be constructed from a neural network using RNN (Recurrent Neural Network) or LSTM (Long Short Term Memory), such as seq2seq (sequence to sequence), based on a natural language processing algorithm. I can do it. seq2seq is a model that learns rules for inputting a "sequence of words" (in the present invention, a string containing an error) and outputting (replacing) another "sequence of words" (the correct string). Yes, used for machine translation, etc.

The RNN has an input layer, a hidden layer, and an output layer. It differs from other neural networks (such as convolutional neural networks) in that it has a hidden layer of . A trained model using an RNN retains the state of the hidden layer at time t-1 and uses it for input at the next time t. It is possible to make an estimation using the word context (or word context).

Further, LSTM is a type of RNN, and can be realized by replacing the middle layer unit of RNN with a "block with memory and three gates" called "LSTM block". LSTM can learn long-term dependencies (relationships between distant characters and words) that are difficult to learn with regular RNNs.

For example, if the stamp on the drug is "FF284" (see the example in part (a) of FIG. 6), the text conversion result in step S110 is "F", "F", "2", Assume that they are "日" and "4" ("F" is an alphabet, "2" and "4" are numbers, and "日" is a kanji character). In this case, the text correction unit 108 uses natural language processing to determine that "the words before and after 'day' are numbers, so 'day' is a numeric or alphabetic error," and changes 'day' to '8' or 'B'. It may be possible to modify the

Note that the text modification unit 108 may accept modifications to the first and second texts, and may perform additional learning on the above-mentioned trained model and drug matching dictionary (described later) based on the received modifications.

<Verification>
The collation unit 110 (verification unit) collates the second text with the drug master 202 and obtains identification information about a candidate drug that is a candidate for the drug indicated by the second text (step S130: collation step). The matching unit 110 calculates the degree of similarity between the second text and the character information (third text) of the drug master 202 (step S132 in FIG. 5), and selects a candidate drug based on the degree of similarity. As an index indicating the degree of similarity, Levenshtein distance, Damerau-Levenshtein distance, Hamming distance, Jaro-Winkler distance, etc. can be used. Note that the matching unit 110 may normalize the second text to generate a normalized text, and perform matching using the normalized text (matching step, normalization step). The matching unit 110 can convert, for example, uppercase letters to lowercase letters, full-width letters to half-width letters, and kanji and/or hiragana to katakana (or the reverse of these conversions) as "normalization," thereby changing the representation of the text. It is possible to improve search accuracy by unifying the search results. It is preferable that the collation unit 110 performs conversion in accordance with the expression format of the identification information in the drug master 202 (whether uppercase or lowercase letters are used, etc.).

<Verification according to the structure of the string>
There are various types of configurations (layouts, types of characters, etc.) of character strings (engraved, printed) attached to medicines. Therefore, it is preferable to use a matching method depending on the structure of the character string. FIG. 6 is a diagram showing an example of a character string attached to a drug. Part (a) of FIG. 6 is an example in which the character string is composed of an identification code of about 1 to 4 alphanumeric characters and an identification code of about 2 to 4 digits. In this example, since the length of the characters is short and the amount of information is short as text, it is required that the sequence of character strings be extracted accurately. Therefore, the text conversion unit 106 constructs a character string from each recognized character based on the layout of the character string included in the image. The case of only numbers (or only letters) as in part (b) of FIG. 6 is the same as the case of part (a) of the same figure.

Part (c) of FIG. 6 is an example in which a mark (in this case, an asterisk) and a number are attached to the drug. Since marks are not text, in this case it is preferable to define in advance a rule for expressing marks as special character strings. For example, the star sign (star) can be expressed as "\S" using the escape character "\", and "\S" can be treated as a single character in the same way as a normal character.

Parts (d) and (e) of FIG. 6 are examples in which the character string is composed of multiple columns and has a rich amount of information. In such a case, since there are a large number of characters, it is effective to count them as character types even if the text is extracted in a different order than originally intended. For example, in the example shown in part (e) of the figure, when the character string extracted from the image is "Kasataa OD na 10FF", the matching unit 110 ignores the order of the character strings and replaces "Ka" with "Ka". 1, 1 “sa”, 1 “ta”, 1 “a”, 1 “O”, 1 “D”, 1 “na”, 1 “1” 1 "0" and 2 "F". Similarly, the matching unit 110 checks the consistency of the character information in the medicine master 202, ignoring the order of the character strings.

<Calculation of similarity considering the number of characters of the keyword>
When identifying drugs, the characters may be unidentifiable and cannot be used for verification, or the character string may be shortened due to division of tablets or packaging (PTP sheets, etc.). In this case, short drug names have a relatively higher degree of similarity with keywords than long drug names, and there is a possibility that appropriate search results may not be obtained. Therefore, in the identification support system 10, the degree of similarity can be calculated by considering the number of characters of the keyword as follows. Specifically, when the "number of characters of the corrected text (second text)" is less than "the number of characters of the text information (third text) stored in the drug master 202", the collation unit 110 determines that the A character string having the same length as the second text is extracted from the text No. 3, the degree of similarity between the extracted character string and the second text is calculated, and the value when the degree of similarity is maximum is used (Step S130: verification process). On the other hand, if the "number of characters in the text after correction" is equal to or greater than the "number of characters in the text information stored in the drug master 202", the matching unit 110 calculates the degree of similarity without extracting the character string (step S130: verification process). In this way, by calculating the degree of similarity in consideration of the number of characters of the keyword, it becomes easier to obtain accurate search results.

<Narrowing down candidate drugs>
FIG. 5 is a flowchart showing narrowing down of candidate drugs (step S130 in the flowchart of FIG. 4: collation step). The matching unit 110 calculates the degree of similarity (step S132), and selects a drug for which the degree of similarity is equal to or higher than the threshold value as a candidate drug (step S133). Then, the matching unit 110 determines whether there are multiple candidate drugs (step S134) and whether or not to perform narrowing down when there are multiple candidate drugs (step S136). The matching unit 110 can perform narrowing down, for example, by template matching between a photographed image (or a character string emphasized image) and a master image of the drug master 202 (step S138), but it may also narrow down by other methods. The matching unit 110 can determine whether to perform narrowing down based on the user's operation via the operating unit 400. Alternatively, the matching unit 110 may select all drugs having a degree of similarity equal to or higher than a threshold value as candidate drugs without performing narrowing down (if NO in step S136, proceed to step S139). The matching unit 110 acquires identification information and an image of the final candidate drug (step S139).

<Display of identification information and images>
The output unit 112 (output unit) displays (outputs) the verification result (identification information and image) for the candidate drug on the monitor 310 (display device) (step S140 in FIG. 4: output step). By displaying the identification information (code and/or name of the drug) and the image, the user can easily understand whether the candidate drug is appropriate (whether the verification is appropriate or not). The identification support system 10 (verification unit 110) determines that "the candidate drug is not appropriate" (NO in step S150) and that "verification for all drugs has not been completed" (step S160). If NO), the process returns to step S100 and the process is repeated. The identification support system 10 can make these determinations based on the user's operation via the operation unit 400.

<Generation of additional learning data>
The text correction unit 108 (text correction unit) receives corrections to the first and second texts, and performs additional learning on the trained models for generating the first and second texts based on the received corrections. may be executed. Additionally, the text correction unit 108 may update a drug matching dictionary, which will be described later. When the text correction unit 108 receives corrections to the first and second texts, it generates additional learning data 208 according to the content of the received corrections (step S170: data generation step). The text correction unit 108 may perform additional learning each time additional learning data is generated, or may perform additional learning periodically or at any time in response to a user's instruction via the operation unit 400. Through such additional learning, it is possible to improve the generation accuracy of the first and second texts.

<Effects of the first embodiment>
As explained above, according to the identification support system 10 and the identification support method according to the first embodiment, the user can accurately identify drugs.

<Modify text by referring to dictionary>
In the first embodiment described above, the text correction unit 108 uses the trained model to correct the character recognition result (first text). However, in the identification support system of the present invention, the text correction unit 108 may generate the second text by correcting the first text with reference to a drug matching dictionary in which expressions used to identify drugs have been learned (text correction step). The drug matching dictionary is a conversion dictionary in which words used to identify drugs are registered as conversion candidates; for example, numbers, alphabets, drug names, drug company names, and their trade names and abbreviations are registered. This information may be attached to drugs by engraving and/or printing, printing on packaging, pasting labels, etc. By registering in a drug matching dictionary, you can enter the intended word as a keyword for matching (search). This allows for accurate matching (search).

<Second embodiment>
FIG. 7 is a diagram showing the configuration of an identification support system 20 (identification support system) according to the second embodiment of the present invention. The identification support system 20 has the same functions as the identification support system 10 according to the first embodiment as a whole, but the system includes an identification support client 11 (identification support client) and an identification support server 30 (identification support server). This embodiment differs from the first embodiment in that it is composed of: Regarding the identification support system 20, components common to the identification support system 10 according to the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

<Identification support client configuration>
The identification support client 11 includes a processing section 101, an illumination section 120, a photographing section 130, a storage section 201, a display section 300, and an operation section 400, and is capable of character recognition (text conversion) for drug images, as described later. It transmits and receives data to and from the identification support server 30, displays results, etc. The identification support client 11 can be realized using a computer such as a personal computer or a mobile terminal such as a smartphone, and the display section 300 and the operation section 400 may be integrated by using a touch panel type monitor.

FIG. 8 is a diagram showing the functional configuration of the processing section 101. The processing unit 101 includes an image acquisition unit 102 (image acquisition unit), an emphasis processing unit 104 (emphasis processing unit), a text conversion unit 106 (text conversion unit), a text correction unit 108 (text correction unit), and an output 112 (output section), a client-side transmission section 116 (client-side transmission section), and a client-side reception section 118 (client-side transmission section). Each of these units can be realized using various processors and electric circuits in the same way as described above for the processing unit 100, and when the processor or electric circuit executes software (program), ROM, RAM, etc. are used. It will be done.

FIG. 9 is a diagram showing the configuration of the storage unit 201. The storage unit 201 stores drug images 204 (photographed images, character string emphasized images, images of candidate drugs acquired from a server (described later), etc.) and matching results 206. The storage unit 201 may store the matching result 206 in association with the first and second texts.

<Identification support server configuration>
The identification support server 30 is a server on the cloud CL (see FIG. 7), and includes a server main body 500 and a storage section 510. As shown in FIG. 10, the server main body 500 includes a collation unit 502 (verification unit), a server-side output unit 504 (server-side output unit), a server-side transmission unit 506 (server-side transmission unit), and a server-side reception unit. 508 (server side receiving unit). As shown in FIG. 11, the storage unit 510 stores a drug master 512 (same as the drug master 202 in FIG. 3) and a drug image 514 (same as the drug image 204 in FIG. 3).

<Processing of identification support method>
12 and 13 are flowcharts showing the processing of the identification support method according to the second embodiment. The left side of these figures shows the processing in the identification support client 11, and the right side shows the processing in the identification support server 30. The image acquisition unit 102 and text modification unit 108 of the identification support client 11 process steps S200 to S220 (acquisition of drug images, first text modification by character recognition, etc.) in the same way as steps S100 to S120 described above for the first embodiment generation, generation of a second text by text modification; image acquisition step, text conversion step, text modification step). The text converting unit 106 and the text modifying unit 108 can generate and modify text using a trained model as in the first embodiment, but may also use a conversion dictionary. The client-side transmitter 116 transmits text information about the drug (verification text; second text) to the identification support server 30 (step S230), and the server-side receiver 508 (server-side receiver) of the identification support server 30. receives the text information (step S400).

Similar to steps S130 to S140 described above, the collation unit 502 collates the received text information (second text) with the text information of the drug master 512 to obtain identification information and an image of the candidate drug (step S410; verification step). The server-side transmitting unit 506 transmits the matching result (identification information and image of the candidate drug) to the identification support client 11 (step S420), the client-side receiving unit 118 receives the matching result (step S240), and outputs the matching result to the output unit 112. displays the identification information and image about the candidate drug on the monitor 310 (display device) (step S250: output step). Similar to steps S150 to S160 described above, the identification support client 11 repeats the processing of steps S200 to S250 until the processing for all drugs is completed (until YES in step S270).

Although the second embodiment describes a case in which the identification support server 30 transmits images of candidate drugs to the identification support client 11, other configurations may be adopted for transmitting and receiving images. Specifically, the identification support server 30 transmits a URL (Uniform Resource Locator) indicating the storage location of the image to the identification support client 11, and the output unit 112 of the identification support client 11 downloads the image from the specified URL. can be adopted. In this case, the image storage location may be the storage unit 510 or another storage device. The output unit 112 can store the acquired image in the storage unit 200. Furthermore, in the second embodiment, a case is described in which the storage unit 510 of the identification support server 30 holds (memorizes) a drug image (drug image 514), but the processing capacity of the identification support client 11 is sufficient. In this case, the storage unit 201 of the identification support client 11 may hold (store) the image of the drug.

The text correction unit 108 of the identification support client 11 generates additional learning data similarly to step S170 described above (step S280).

As explained above, in the identification support system (identification support client, identification support server) and identification support method according to the second embodiment, the user can accurately identify drugs as in the first embodiment. Can be done.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

10 Identification support system 11 Identification support client 20 Identification support system 30 Identification support server 100 Processing unit 101 Processing unit 102 Image acquisition unit 104 Emphasis processing unit 106 Text conversion unit 108 Text correction unit 110 Collation unit 112 Output unit 114 Communication control unit 116 Client Side transmitting section 118 Client side receiving section 120 Illumination section 130 Photographing section 200 Storage section 201 Storage section 202 Drug master 204 Drug image 206 Matching result 208 Additional learning data 300 Display section 310 Monitor 400 Operation section 410 Keyboard 420 Mouse 500 Server main body 502 Collation unit 504 Server-side output unit 506 Server-side transmission unit 508 Server-side reception unit 510 Storage unit 512 Drug master 514 Drug image CL Cloud S100 to S420 Each step of the identification support method

Claims (8)

an image acquisition unit that acquires an image of the drug;
a text conversion unit that extracts a character string included in the image character by character and generates a first text indicating the character string;
a text modification unit that generates a second text by modifying the first text based on an expression used for drug identification;
a drug master in which identification information indicating the code and/or name of the drug and character information indicating characters attached to the drug are stored as text;
a collation unit that collates the second text with the drug master to obtain the identification information about a candidate drug that is a candidate drug indicated by the second text;
an output unit that outputs the identification information about the candidate drug;
Equipped with
The text modification unit modifies the first text using the context of the extracted characters,
The drug master has layout information regarding the character information,
The matching unit is an identification support system that performs the matching based on the layout information . The identification support system according to claim 1, wherein the text conversion unit extracts the character string using a learned model configured by machine learning. further comprising an emphasis processing unit that emphasizes the character string included in the image to generate a character string emphasized image,
The identification support system according to claim 1 or 2, wherein the text conversion unit generates the first text from the character string emphasized image. The identification support system according to any one of claims 1 to 3, wherein the matching unit calculates a degree of similarity between the second text and the character information, and selects the candidate drug based on the degree of similarity. . The drug master stores the identification information and a drug image in association with each other,
The identification support system according to any one of claims 1 to 4, wherein the output unit displays the identification information and the image for the candidate drug in association with each other on a display device. an image acquisition unit that acquires an image of the drug;
a text conversion unit that extracts a character string included in the image character by character and generates a first text indicating the character string;
a text modification unit that generates a second text by modifying the first text based on an expression used for drug identification;
a client-side transmitter that transmits information indicating the second text to the identification support server;
a client-side receiving unit that receives identification information including a drug code and/or name from the identification support server for a candidate drug that is a drug candidate indicated by the second text;
an output unit that outputs the identification information about the candidate drug;
Equipped with
The text modification unit is an identification support client that modifies the first text using the context of the extracted characters. a drug master in which identification information indicating the code and/or name of the drug and character information indicating characters attached to the drug are stored as text;
a server-side receiving unit that receives text information about the drug from an identification support client;
a collation unit that collates the text information with the drug master to obtain the identification information about a candidate drug that is a drug candidate indicated by the text information;
a server-side transmitter that transmits the acquired identification information to the identification support client;
Equipped with
The drug master has layout information regarding the character information,
The matching unit is an identification support server that performs the matching based on the layout information . an image acquisition step of acquiring an image of the drug;
a text conversion step of extracting a character string included in the image character by character and generating a first text indicating the character string;
a text modification step of modifying the first text based on an expression used for drug identification to generate a second text;
The identification information indicating the code and/or name of the drug and the character information indicating the characters attached to the drug are compared with the drug master stored as text, and the second text is generated. a collation step of acquiring the identification information about the candidate drug that is the drug candidate indicated by;
an output step of outputting the identification information about the candidate drug;
has
In the text modification step, the first text is modified using the context of the extracted characters;
The drug master has layout information regarding the character information,
In the verification step, the verification is performed based on the layout information .

Images