JP2021189842A - Object recognition method, program, and information processing device - Google Patents

Abstract

To provide a technique for suppressing an increase in the labor of an operator even in the case that an identification part attached to an object gets dirty or is damaged.SOLUTION: A management device 100 recognizes the existence of an analyte to a processing object of a process by reading a sample ID from a code sheet 901 in an image from a camera 200. An image of a specimen is registered. If the sample ID is not read from the code sheet 901, the image of the specimen is extracted, an area corresponding to the code sheet is detected from the image, and a feature of the area is registered. After that, the management device 100 recognizes the existence of the analyte by extracting the feature in the image from the camera 200.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to recognition of an article in an area, and more particularly to recognition of the article using an identification unit representing the identification information of the article.

Conventionally, a technique for managing each article using a control number assigned to each article has been proposed. For example, as described in Japanese Patent Application Laid-Open No. 2008-135072 (Patent Document 1) and Japanese Patent Application Laid-Open No. 2008-235504 (Patent Document 2), an article is provided with an identification unit (bar code or the like) indicating a control number. However, a technique is disclosed in which information contained in an identification unit attached to an article is read by a reading device such as a barcode reader, and a management device acquires a reading result to recognize each article.

Japanese Unexamined Patent Publication No. 2008-135072 Japanese Unexamined Patent Publication No. 2008-235504

In the conventional technique, when the identification unit becomes dirty or missing, the reading device cannot read the identification unit, and the management device cannot obtain the information for recognizing the article. In this case, the worker is required to manually enter the control number on the article and manually input the entered control number into the management device, which requires a lot of labor.

The present disclosure has been devised in view of such circumstances, and the purpose is to suppress an increase in labor of workers even if the identification part attached to the article becomes dirty or damaged. To provide technology.

According to certain aspects of the present disclosure, a method performed by a computer, comprising the step of associating and registering the identification information assigned to the article with the image information of the article, the article is an identification unit representing the identification information. The step of recognizing the presence of the article in the image of the area includes the step of acquiring the image of the area to be worked on with respect to the article, and the step of recognizing the identification information from the image of the identification unit in the image of the area. The image information of the identification unit is recognized when the existence of the article is recognized by using the identification information when it can be read, and when the image information of the article can be recognized even when the identification information cannot be read in the image of the area. Is associated with the identification information and registered, and when the image information of the identification unit is associated with the identification information, the presence of the article is recognized by using the image information of the identification unit in the image of the area. , A method of recognizing an article is provided.

Registering the image information of the identification unit in association with the identification information may include registering the feature amount of the image of the identification unit. Recognizing the presence of an article may include recognizing the features registered in the image of the area.

The article may be a sample or a container for the sample that is the subject of the pathological diagnosis.

The image information of the article may include an image of the container of the sample.

The image information of the article may include an image of the sample.

The step of acquiring an image may include acquiring an image of each of two or more areas. It may further include a step of registering the presence of the article in association with the area in which the article is recognized.

The method of recognizing an article may further include a step of specifying the start timing and the end timing of the work in the area based on the positional relationship between the worker and the article in the image of the area. The step of registering the existence may include registering information specifying the timing of the start and the timing of the end of the work in the area.

The positional relationship between the worker and the article may include the distance between the given part of the worker and the article.

According to other aspects of the disclosure, a program is provided that causes a computer to perform the above method by being executed by a computer processor.

According to still another aspect of the present disclosure, there is provided an information processing apparatus comprising a storage device for storing the above program and a processor for executing the program stored in the storage device.

According to the present disclosure, since the image information of the article and the identification information of the identification unit are registered in association with each other, the computer is associated with the image information of the article when the identification information cannot be read from the identification unit of the article. The image information of the identification unit is associated with the identification information. As a result, when the identification information cannot be read from the identification unit of the article, the computer recognizes the article by acquiring the image information of the identification unit of the article and recognizing the identification information associated with the image information. Can be done. That is, even if dirt or defects occur in the identification portion of the article, the worker can recognize the article without requiring labor for inputting the identification information.

It is a figure which shows the structural example of the article management system. It is a figure which shows an example of the hardware composition of the management apparatus 100. It is a figure which shows the data structure example of a sample database. It is a figure which shows the data structure example of a process database. It is a figure which shows the example of the data structure of an image information database. It is a figure which shows an example of the emergence of a new form of a sample. It is a figure which shows the data structure example of the sample database which registered a new sample ID. It is a figure which shows another example of the emergence of a new form of a sample. It is a figure which shows the data structure example of the sample database which registered a new sample ID. It is a figure which shows an example of a sample which does not include the reason for obstructing the reading of the identification information from a code paper. It is a figure which shows an example of a sample including the reason for obstructing the reading of the identification information from a code paper. It is a figure which shows the other example of a sample including the reason for obstructing the reading of the identification information from a code paper. It is a flowchart of the process executed by the management apparatus 100 when a sample is received in the article management system. It is a flowchart of the process executed by the management apparatus 100 to manage a sample (or a sample) in each process of an article management system. It is a figure which shows the example of the data structure of the sample database which registered the new sample ID. It is a figure which shows the data structure example of the image information database in which a sample ID and an image are registered. It is a figure which shows the specific example of the image used for specifying the timing of the start and end of work. It is a figure which shows the specific example of the image used for specifying the timing of the start and end of work. It is a figure which shows the data structure example of the process database which registered the start time. It is a flowchart of the subroutine of step S222. It is a figure which shows the data structure example of the sample database which registered the new sample ID in step S308. It is a figure which shows the data structure example of the process database which registered the new sample ID in step S310. It is a figure which shows the data structure example of the image information database which registered the new sample ID in step S312. It is a figure which shows the data structure example of the process database which registered the end time. It is a figure which shows the data structure example of a process database. It is a figure for demonstrating the area of a plurality of processes in an article management system. It is a figure which shows an example of the image from a camera.

Hereinafter, an embodiment of the article management method will be described with reference to the drawings. In the following description, the same parts and components are designated by the same reference numerals. Their names and functions are the same. Therefore, these explanations will not be repeated.

[1. Configuration example of a system that realizes an article management method]
FIG. 1 is a diagram showing a configuration example of an article management system. In the article management system, the worker 500 manages the work for pathological diagnosis of the sample. The work includes one or more steps. The steps are, for example, specimen preparation (embedding of specimen, slicing, pasting on a glass plate, etc.), diagnosis (imaging, generation of diagnostic result, etc.). The sample is an example of an article to be managed in an article management system, and is composed of, for example, a cell piece or a tissue piece to be the object of pathological diagnosis.

In one operation, the sample is handled while being attached to the glass plate 900. The glass plate 900 is an example of a sample container. A code sheet 901 representing the identification information of the sample is attached to the glass plate. A code representing identification information such as a QR (Quick Response) code (registered trademark) is printed on the code paper 901. The code sheet 901 is an example of an identification unit that represents identification information of a sample.

In the article management system, the area where the sample is brought in is photographed by the camera 200. The number of cameras 200 in the article management system may be one or a plurality.

In the article management system, information is managed by the management device 100. The management device 100 uses the image taken by the camera 200 for sample management. The article management system includes a terminal 600 that presents the contents of work to the worker 500. The terminal 600 is realized by a general-purpose information processing device such as a tablet terminal.

[2. Management device hardware configuration]
FIG. 2 is a diagram showing an example of the hardware configuration of the management device 100. The management device 100 is realized by, for example, a general-purpose computer, and includes a processor 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a storage device 105, an input device 106, and the like. Includes display device 107.

The processor 101 controls the operation of the management device 100. The ROM 102 stores a program executed by the processor 101. The RAM 103 is used as a work area for program execution by the processor 101. The communication interface 104 is used as an interface in communication with an external device, and is realized by, for example, a network card. The storage device 105 stores various data such as parameters in program execution by the processor 101, and is realized by, for example, a hard disk drive. The input device 106 is used for inputting information to the management device 100, and is realized by, for example, a keyboard and / or a mouse. The display device 107 displays the state of processing executed by the management device 100, and is realized by, for example, a display such as a liquid crystal display device.

In one implementation example, the management device 100 is connected to the camera 200 via a network such as a LAN (Local Area Network). The camera 200 may be directly connected to the management device 100.

[3. Database data configuration example]
The storage device 105 may store various databases for sample management. Hereinafter, each data configuration example will be described.

(1) Sample database FIG. 3 is a diagram showing a data configuration example of the sample database. In the article management system, a plurality of sample IDs may be assigned to one sample. When a plurality of sample IDs are assigned to one sample, the sample database stores the plurality of sample IDs in association with each other.

More specifically, the sample database includes, as items, a sample ID (outpatient), a sample ID (system reception), a sample ID (1), and a sample ID (2).

The "sample ID (outpatient)" represents the identification information given to the sample when the sample is carried into the article management system.

The "sample ID (system reception)" represents the identification information given to the sample when the sample is carried into the article management system.

In the example of FIG. 3, "X0001" is shown as "sample ID (outpatient)" and "P0001" is shown as "sample ID (system reception)". This means that the article management system accepts the sample to which the sample ID "X0001" is given and assigns the sample ID "P0001" to the sample. More specifically, when the worker of the article management system receives the sample, he / she inputs "X0001" as the "sample ID (outpatient)" by using the input device 106, and "sample ID (system reception)". ) ”, Enter“ P0001 ”. The entered data is registered in the sample database.

Each of the "sample ID (1)" and "sample ID (2)" is applied to the new form when a new form of the sample appears due to the preparation of the sample in the article management system. Represents the sample ID given in the above. The assignment of the sample ID will be described later with reference to FIGS. 6 to 9.

(2) Process database FIG. 4 is a diagram showing an example of a data structure of a process database. The process database stores the sample ID of each sample (or sample) in each process in the article management system in association with information indicating the status of work for each sample (or sample).

More specifically, the process database includes, as items, a sample ID, a start, and an end. Each of "start" and "end" represents the start time and the end time of the work in the process.

(3) Image Information Database FIG. 5 is a diagram showing an example of a data structure of an image information database. The image information database associates the sample ID with information about the image of the sample (or sample) specified by the sample ID. The information about the image may be the image itself or features extracted from the image.

In the example of FIG. 5, "IM0001" is associated with the sample ID "P0001". "IM0001" is information for specifying an image file. That is, in the example of FIG. 5, the image file specified by "IM0001" is associated with the sample ID "P0001".

In the article management system, when a new sample ID is given to a sample or a sample, an image including at least a part of the appearance of the sample or the sample may be taken and registered. The example of FIG. 5 means that an image file is generated by photographing the appearance of the sample ID “P0001”, and the image file is specified as “IM0001”. The registered image file is stored in, for example, the storage device 105.

[4. Specific example of assigning sample ID]
With reference to FIGS. 6 to 9, the assignment of the sample ID in the appearance of a new form of the sample will be described.

FIG. 6 is a diagram showing an example of the appearance of a new form of a sample. In the example of FIG. 6, a sample placed in a container 800 is sliced and attached to a glass plate 900 to prepare a sample. Code paper 801 is attached to the container 800. Code paper 901 is attached to the glass plate 900.

As shown in FIG. 6, in response to the new appearance of the sample, the worker assigns a new sample ID to the sample and registers the new sample ID in the sample database.

FIG. 7 is a diagram showing a data configuration example of a sample database in which a new sample ID is registered. In comparison with FIG. 3, in FIG. 7, the sample ID “P0002” is added as the sample ID associated with the sample ID “X0001”. That is, in the example of FIG. 7, a sample with a sample ID "X0001" is given a sample ID "P0001" at the time of acceptance by the article management system, and further, when a sample is prepared from the sample. It means that the sample ID "P0002" was given to the sample.

The code printed on the code paper 801 represents the sample ID "P0001", and the code printed on the code paper 901 represents the sample ID "P0002".

FIG. 8 is a diagram showing another example of the emergence of a new form of specimen. In the example of FIG. 8, three specimens attached to the glass plates 910, 920, and 930 are prepared from the specimens in the container 800. Code papers 911, 921, and 931 are attached to each of the glass plates 910, 920, and 930.

As shown in FIG. 8, in response to the emergence of three new samples, the worker assigns a new sample ID to each of the above samples and puts the new sample ID in the sample database. sign up.

FIG. 9 is a diagram showing a data configuration example of a sample database in which a new sample ID is registered. Compared with FIG. 3, in FIG. 9, three sample IDs “P0011”, “P0012”, and “P0013” are added as sample IDs associated with the sample ID “X0001”. That is, in the example of FIG. 9, the sample with the sample ID "X0001" was given the sample ID "P0001" at the time of acceptance by the article management system, and three specimens were further prepared from the sample. Occasionally, it means that each of the sample IDs "P0011", "P0012", and "P0013" was given to each of the three specimens. The code attached to each of the code sheets 911, 921, 931 represents each of the sample IDs "P0011", "P0012", and "P0013".

[5. Specific example of the reason why the reading of the identification information from the identification unit is hindered]
The article management system extracts the code printed on the code paper from the image generated by the camera 200 taking a picture of the area, and when it is determined that the code represents the identification information, the area taken by the camera 200 is taken. The existence of the article associated with the identification information in the above is recognized. In the present specification, the extraction of the code from the code sheet and the determination that the extracted code represents the identification information are referred to as "reading the identification information from the code sheet" as appropriate.

Reading the identification information from the code sheet can be hindered by various reasons. A specific example of the reason why the reading of the identification information from the code sheet is hindered will be described with reference to FIGS. 10 to 12.

FIG. 10 is a diagram showing an example of a sample that does not include a reason for obstructing the reading of identification information from a code sheet. Code paper 901 is attached to the glass plate 900 shown in FIG. A sample piece 903 is attached to the glass plate 900, and a cover glass 902 is placed on the sample piece 903. In the state shown in FIG. 10, the code sheet 901 can present the entire code.

FIG. 11 is a diagram showing an example of a sample including a reason for obstructing the reading of identification information from a code sheet. In the state shown in FIG. 11, dirt 901A is attached to the code paper 901. By covering a part of the cord with the stain 901A, the cord paper 901 cannot present the entire cord. In the state shown in FIG. 11, the presence of the stain 901A hinders the reading of the identification information from the code paper 901.

FIG. 12 is a diagram showing another example of the sample, including the reason for obstructing the reading of the identification information from the code sheet. In the state shown in FIG. 12, the code paper 901 is partially defective as shown as the defective portion 901B. Since the code sheet 901 lacks a part of the code as shown as the defective portion 901B, the entire code cannot be presented. In the state shown in FIG. 12, the presence of the defective portion 901B hinders the reading of the identification information from the code sheet 901.

In the article management system, the sample (or sample) is basically recognized by the identification information read from the code sheet 901. In addition, [7. As will be described later in the flow of sample management processing in the process], in the article management system, since the image information of the code paper 901 itself is associated with the sample ID, even if the reading of the identification information from the code paper 901 is hindered. , Specimens (or specimens) can be recognized.

[6. Flow of processing when receiving a sample]
FIG. 13 is a flowchart of a process executed by the management device 100 when the sample is received in the article management system. The management device 100 realizes the process of FIG. 13, for example, by causing the processor 101 to execute a given program.

In step S100, the management device 100 acquires the "sample ID (outpatient)". In one implementation example, when a worker who receives a sample from the outside inputs a sample ID assigned to the sample to the management device 100, the management device 100 acquires a "sample ID (outpatient)". ..

In step S102, the management device 100 acquires a "sample ID (system reception)". In one implementation example, the management device 100 acquires the "sample ID (system reception)" by inputting the sample ID newly assigned in the article management system to the sample received from the outside by the worker. ..

In step S104, the management device 100 acquires an image of the sample. In one implementation example, the worker captures a sample received from the outside and registers the image of the sample in the management device 100, so that the management device 100 acquires the image of the sample.

In step S108, the management device 100 registers the sample ID and the image acquired in steps S100 to S104 in the database, and ends the process of FIG. 13.

For example, a sample to which the sample ID "X0001" is given to the article management system from the outside is accepted, the sample ID "P0001" is given to the sample, and the image of the sample is specified by the file name "IM0001". In this case, the data is registered in the sample database as shown in FIG. 3, and the data is registered in the image information database as shown in FIG.

[7. Flow of sample management processing in the process]
FIG. 14 is a flowchart of a process executed by the management device 100 to manage a sample (or a sample) in each step of the article management system. The management device 100 realizes the processing of FIG. 14, for example, by causing the processor 101 to execute a given program.

In step S200, the management device 100 determines whether or not any of the sample IDs registered in the sample database has been acquired. In one implementation example, the management device 100 monitors the image generated by the camera 200 and acquires the sample ID by reading the sample ID from the image of the code paper included in the image.

The camera 200 generates an image of a region targeted by the process corresponding to the process of FIG. The area targeted by one step may be photographed by two or more cameras 200. In this case, the management device 100 acquires images generated by each of the two or more cameras 200, and attempts to read the sample ID from those images.

The management device 100 keeps control in step S200 until it is determined that the sample ID can be acquired (NO in step S200), and when it is determined that the sample ID can be acquired (YES in step S200), the control proceeds to step S202. ..

For example, in step S200, the management device 100 acquires an image of the work area of the process A (an example of the process) from the camera 200, and determines whether or not the sample ID can be read from the image. When the specimen (or specimen) is carried into the imaging target area of the camera 200, the image generated by the camera 200 includes an image of the code paper. As a result, the management device 100 acquires the sample ID from the code sheet. That is, when the sample (or sample) is carried into the area, the management device 100 advances control from step S200 to step S202.

In step S202, the management device 100 determines whether or not a new sample ID is given to the sample. For example, the worker notifies the management device 100 via the terminal 600 that a new specimen is to be prepared from the specimen. In one embodiment, the management device 100 determines that the sample is given a new sample ID upon receiving the notification.

If the management device 100 determines that a new sample ID is assigned to the sample (YES in step S202), the control device advances control to step S204, and if not (NO in step S202), the control device 100 proceeds to control to step S208. ..

In step S204, the management device 100 acquires a new sample ID. In one implementation example, the management device 100 acquires the new sample ID by transmitting the new sample ID from the worker to the management device 100 via the terminal 600.

In step S206, the management device 100 associates the new sample ID with the sample ID acquired in step S200 and registers it in the sample database.

FIG. 15 is a diagram showing an example of a data structure of a sample database in which a new sample ID is registered. In FIG. 15, "P0002" is further registered as the sample ID (1) as compared with FIG. The example of FIG. 15 means that, for example, a new sample was prepared from a sample to which the sample ID "P0001" was given at the time of acceptance in the article management system, and the sample ID "P0002" was given to the sample.

Returning to FIG. 14, in step S208, the management device 100 acquires an image of the sample. The management device 100 acquires an image of a sample by acquiring an image generated by the camera 200, for example, when a worker takes a picture of the sample brought into the process with the camera 200. When the sample is treated as a sample in the process, the worker takes a picture of the sample with the camera 200.

In step S210, the management device 100 registers the sample ID and the image in the image information database.

FIG. 16 is a diagram showing a data configuration example of an image information database in which a sample ID and an image are registered. In FIG. 16, as compared with FIG. 5, data relating to the sample ID “P0002” is added to the image information database. More specifically, the sample ID "P0002" is added to the image information database, and the image information "IM0002" associated with the sample ID "P0002" is added. The image information "IM0002" is an image of a new sample to which a sample ID is added to the sample database in step S206, and represents a file name of the image acquired in step S208.

If a new sample is not prepared in the process targeted by FIG. 14, the sample ID is not newly registered in step S206. In such a case, registration of a new sample ID is omitted in step S210, and the sample ID is associated with the sample ID acquired in step S200 (sample ID “P0001” in the example of FIG. 5) and acquired in step S208. The image is registered. For example, the image information of the sample ID "P0001" is changed from "IM0001" to "IM0002".

Returning to FIG. 14, in step S212, the management device 100 registers the sample ID in the process database. If the management device 100 has acquired a new sample ID in step S204, the new sample ID is registered in the process database in step S212, and if the new sample ID has not been acquired in step S204, the management device 100 has not acquired the new sample ID in step S212. Then, the sample ID acquired in step S200 is registered in the process database.

In the example of FIG. 4, a state in which the sample ID “P0002” is registered in the process database of the process targeted by FIG. 14 is shown.

Returning to FIG. 14, in step S214, the management device 100 determines whether or not the work on the sample has been started in the step targeted by FIG.

In one embodiment, the management device 100 identifies the start and end timings of the work based on the positional relationship between the worker's hand and the sample in the image generated by the camera 200. 17 and 18 are diagrams showing specific examples of images used for specifying the start and end timings of work.

Image 1700 of FIG. 17 includes four elements 1710, 1711, 1712, 1713. Element 1710 represents a workbench. Element 1711 represents a specimen (glass plate). Each of the elements 1712 and 1713 represents a worker's left hand and right hand, respectively. In the management device 100, each element identifies a corresponding object (work table, specimen, left hand, right hand) by using the existing image recognition technology, and from the image 1700, the worker's hand is at a distance D1 from the specimen on the work table. Identify that it is located only far away.

Image 1800 of FIG. 18 includes four elements 1810, 1811, 1812, 1813. Element 1810 represents a workbench. Element 1811 represents a specimen (glass plate). Each of the elements 1812 and 1813 represents the left hand and the right hand of the worker. In the management device 100, each element identifies a corresponding object (work table, specimen, left hand, right hand) by using the existing image recognition technology, and from the image 1800, the worker's hand overlaps with the specimen on the work table. Identify that.

For example, the management device 100 defines the timing at which the worker's hand overlaps with the specimen on the work table as the timing at which the work is started, and the time when the worker's hand is separated from the specimen by a given threshold value or more is a certain time or more. The continuous timing is defined as the timing when the work is completed. The management device 100 may define the start and end timings of the work based on the positional relationship between the specimen and a portion other than the worker's hand (arm, head, etc.).

The method based on the image of the camera is merely an example of a method for specifying the timing of starting and ending work. At least one of the start and end timings may be specified as the timing at which the worker transmits the given data to the management device 100 via the terminal 600.

Returning to FIG. 14, the management device 100 suspends control in step S214 until it is determined that the work has started (NO in step S214), and determines that the work has started (YES in step S214). Control proceeds to step S216.

In step S216, the management device 100 registers the start time (the time when it is determined that the work has started in step S214) in the process database.

FIG. 19 is a diagram showing an example of data configuration of a process database in which a start time is registered. In comparison with FIG. 4, the start time is added in FIG.

Returning to FIG. 14, in step S218, the management device 100 acquires an image of the work target area from the camera 200, and determines whether or not the sample ID registered in the process database can be read from the image. do. If the management device 100 determines that the sample ID can be read (YES in step S218), the control proceeds to step S224, and if not (NO in step S218), the control proceeds to step S220.

In step S220, the management device 100 determines whether or not the feature of the image registered in the image information database can be extracted from the image of the target area of the work for the sample ID registered in the process database. If the management device 100 determines that the above features can be extracted (YES in step S220), the control proceeds to step S224, and if not (NO in step S220), the control proceeds to step S222.

The registration of features in the image information database will be described later with reference to FIG.

In step S222, the management device 100 registers an additional sample ID.

FIG. 20 is a flowchart of the subroutine in step S222. The registration of the additional sample ID in step S222 will be described with reference to FIG. 20.

In step S300, the management device 100 attempts to extract a portion of the image of the target area of the work that matches the image associated with the sample ID registered in the process database by a given ratio or more. If the management device 100 succeeds in extracting the above portion (YES in step S300), the control proceeds to step S304, and if not (NO in step S300), the control device advances to step S302.

In step S302, the management device 100 outputs an error and ends the processing of FIGS. 14 and 20.

In step S304, the management device 100 extracts a portion corresponding to the identification unit (code paper) from the portion extracted in step S300. The portion corresponding to the identification unit (code paper) is extracted as, for example, a region containing a predetermined ratio or more of the features of the code paper (for example, the features defining the image as the QR code).

In step S306, the management device 100 detects the characteristics of the region extracted in step S304. The detected feature may be the shape and / or position of a stain or defect on the code paper, or may be a figure (circle, triangle, quadrangle, etc.) to which the shape is similar, or the stain or stain. It may be the color of the portion corresponding to the defect, or it may be a combination thereof.

In step S308, the management device 100 registers a new sample ID in the sample database.

In step S310, the management device 100 registers a new sample ID (which was registered in the sample database in step S308) in the process database.

In step S312, the management device 100 associates the new sample ID and the feature detected in step S306 with each other and registers them in the image information database. After that, the management device 100 returns the control to FIG.

FIG. 21 is a diagram showing a data configuration example of the sample database in which a new sample ID is registered in step S308. Compared with FIG. 15, in FIG. 21, the sample ID “P0003” is added to the sample database. In the sample database of FIG. 21, since the management device 100 cannot read the sample ID from the image of the target area of the work during the work for the sample specified as the sample ID “P0002”, a new sample is obtained for the sample. It means that the ID "P0003" is given.

FIG. 22 is a diagram showing a data configuration example of the process database in which the new sample ID is registered in step S310. Compared with FIG. 19, in FIG. 22, the sample ID registered in the process database is changed from “P0002” to “P0003”. The process database of FIG. 22 means that the object of the process (specimen or specimen) has been changed from the one specified by the sample ID "P0002" to the sample ID "P0003".

FIG. 23 is a diagram showing a data configuration example of the image information database in which the new sample ID is registered in step S312. In comparison with FIG. 16, in FIG. 23, the sample ID “P0003” is added, and the image information “FT0003” associated with the sample ID “P0003” is added. The image information "FT0003" is data representing features extracted in step S306. That is, in the image information database of FIG. 23, a new object (sample ID representing a sample or a sample) of the process and an appearance feature of a portion corresponding to the code sheet of the new object are newly registered. There is.

Returning to FIG. 14, in step S224, the management device 100 determines whether or not the work on the sample is completed in the process. The management device 100 identifies that the work has been completed, for example, by the method described with reference to FIGS. 17 and 18.

When the management device 100 determines that the work on the sample is completed (YES in step S224), the control proceeds to step S226, and if not (NO in step S224), the control returns to step S218.

In step S226, the management device 100 registers the end time in the process database and returns control to step S200.

FIG. 24 is a diagram showing a data configuration example of the process database in which the end time is registered. In comparison with FIG. 22, in FIG. 24, the end time is added for the sample ID “P0003”.

In the process described above with reference to FIGS. 14 to 24, the management device 100 indicates the existence of the sample to be processed in the process from the image from the camera 200 and the code of the sample (or the glass plate as a container). Recognize by reading the sample ID on the paper.

The management device 100 registers an image of the appearance of the sample in the image information database. When the sample ID on the code sheet cannot be read, the management device 100 extracts from the image from the camera 200 a portion that matches the image of the appearance of the sample by a given ratio or more, and further, from the portion, the code sheet is printed. The corresponding area is extracted, a new sample ID is defined, and the image features of the area are registered in the image information database together with the new sample ID.

If the management device 100 cannot read the sample ID from the image from the camera 200 after registering the above feature, and if the registered feature can be extracted from the image from the camera 200 (YES in step S220), the processing target in the process. Recognize the presence of the sample.

The management device 100 does not output an error as long as the existence of the sample to be processed is recognized in the process, and outputs an error in step S302 if the existence of the sample to be processed cannot be recognized in the process.

The process database manages information on each process. The storage device 105 may store a process database for each of the plurality of processes.

The process database may contain information on a plurality of processes. FIG. 25 is a diagram showing an example of data configuration of a process database.

In the example of FIG. 25, the process database includes start and end times for a plurality of processes (process A and process B). In step S216 and step S226, the management device 100 registers the start time and the end time for the process targeted by the process of FIG. By referring to the process database of FIG. 25, it is possible to track the time when one sample is the target of each of the plurality of processes.

FIG. 26 is a diagram for explaining a plurality of process areas in an article management system. In FIG. 26, it is assumed that the region 2610 is a region for step A and the region 2620 is a region for step B. The camera 200A photographs the area 2610, and the camera 200B photographs the area 2620. The management device 100 may register data in step A for the sample ID read from the image from the camera 200A, and may register data in step B for the sample ID read from the image from the camera 200B.

In FIG. 26, regions 2631 and 2632 are further shown. Each of the regions 2631 and 2632 is a region for passing a sample from the region 2610 to the region 2620 or from the region 2620 to the region 2610.

The article management system may further include a camera for photographing each of the regions 2631 and 2632, and the management device 100 reads the sample ID from the image from the camera so that the sample can capture the regions 2631 and 2632. You may manage the passing time.

When reading the sample ID from the image from the camera, the management device 100 may also read out a range in which the sample can be worked (for example, a workbench).

FIG. 27 is a diagram showing an example of an image from a camera. Element 2710 represents a workbench. The management device 100 may target only the area corresponding to the workbench in the image from the camera, that is, the inside of the element 2710, as the target for reading the sample ID. As a result, in the example of FIG. 27, the element PL1 located inside the element 2710 is the target for reading the sample ID, but the element PL2 located outside the element 2710 is not the target for reading the sample ID.

In the field of pathological diagnosis, it is necessary to manage each article with particular care from the viewpoint of preventing misplacement of test results, but it is assumed that the identification part is often contaminated by the chemicals used to stain the sample. Will be done. According to the present disclosure, even if the identification part is dirty or a part of the identification part is missing, if the appearance of the sample is specified in the image from the camera (YES in step S300), the sample is new. A sample ID is defined, and the new sample ID is stored in association with the image information of the identification unit (code sheet). After that, the sample can be identified by the image information of the identification unit (code paper). As a result, even if the identification portion becomes dirty or damaged, the increase in labor of the worker can be suppressed.

The "appearance of the sample" may be the appearance of the container of the sample or the appearance of the sample itself. That is, the images such as "IM0001" and "IM0002" registered in the image information may be images of the sample placed in the container, or may be images of the sample itself not contained in the container. For example, as the image acquired in step S208, the worker may take an image of the portion covered with the cover glass of the glass plate by the camera. Thereby, in step S210, an image including a sample piece that can be visually recognized through the cover glass can be registered. When the management device 100 fails to read the sample ID due to dirt or a defect in the code paper in step S300, the management device 100 can specify the sample by the pattern of the sample piece on the glass plate.

It should be considered that each embodiment disclosed this time is exemplary in all respects and is not restrictive. The scope of the present invention is shown by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. Further, the inventions described in the embodiments and the modifications thereof are intended to be carried out alone or in combination as much as possible.

100 management device, 200,200A, 200B camera, 500 worker, 600 terminal, 800 container, 801,901,911,921,931 code paper, 900,910,920,930 glass plate, 901A dirt, 901B defect, 902 cover glass, 903 specimen pieces.

Claims (10)

The method performed by the computer
Includes a step of associating and registering the identification information assigned to the article with the image information of the article.
The article includes an identification unit representing the identification information.
A step of acquiring an image of a work target area related to the article, and
Including the step of recognizing the presence of the article in the image of the area.
The recognition step is
In the image of the region, when the identification information can be read from the image of the identification unit, the existence of the article is recognized by using the identification information.
In the image of the area, when the identification information cannot be read and the image information of the article can be recognized, the image information of the identification unit is registered in association with the identification information.
A method for recognizing an article, which comprises recognizing the presence of the article by using the image information of the identification unit in an image of the region when the image information of the identification unit is associated with the identification information. Registering the image information of the identification unit in association with the identification information includes registering the feature amount of the image of the identification unit.
The method for recognizing an article according to claim 1, wherein recognizing the presence of the article includes recognizing a feature amount registered in an image of the region. The method for recognizing an article according to claim 1 or 2, wherein the article is a sample or a container for a sample to be diagnosed with a pathology. The method for recognizing an article according to any one of claims 1 to 3, wherein the image information of the article includes an image of a container of a sample. The method for recognizing an article according to claim 3, wherein the image information of the article includes an image of a sample. The step of acquiring the image includes acquiring each image of two or more regions.
The method for recognizing an article according to any one of claims 1 to 5, further comprising a step of registering the existence of the article in association with an area where the article is recognized. Further including a step of identifying the start timing and the end timing of the work in the area based on the positional relationship between the worker and the article in the image of the area.
The method for recognizing an article according to claim 6, wherein the step of registering the existence includes registering information for specifying the start timing and the end timing of the work in the area. The method for recognizing an article according to claim 7, wherein the positional relationship between the worker and the article includes a distance between a given part of the worker and the article. A program that, executed by a processor of a computer, causes the computer to perform the method of recognizing an article according to any one of claims 1 to 8. A storage device that stores the program according to claim 9,
An information processing device including a processor that executes a program stored in the storage device.

Images