JP6497300B2 - Colony detection system, colony detection method, and program - Google Patents

Description

  The present invention relates to a colony detection system for detecting colonies cultured in a medium.

  In recent years, due to the growth of fast food and other food service industries and the increase in easy processed foods, the opportunity to eat food prepared by others has increased remarkably. On the other hand, when foods such as processed foods and processed foods are mixed with dangerous substances such as poisons or microorganisms, it directly affects the human body. When handling processed foods, not only the safety of raw materials but also the production There is a need for food safety that strictly controls the hygiene of the process and covers the entire production and processing of food.

  In particular, food poisoning incidents that have occurred in various places are the starting point, and it is a proposition to further improve the safety of foods. In addition to complying with the Food Sanitation Law, “HACCP (Hazard Analysis Critical) The number of companies introducing “Control Point System” and “FSSC (Food Safety System Certification) 22000” is also increasing.

  As a system for detecting microorganisms mixed in such processed foods, a film-type medium is used, and colonies generated in a specimen cultured in a medium such as the film-type medium are imaged, and What is counted by performing image analysis on “medium image” is also known (for example, Patent Document 1). In particular, a system that counts colonies based on the colony image (hereinafter referred to as “count system”) analyzes the medium image based on the luminance of each pixel constituting the medium image, and images the medium. It is possible to prevent false detection based on variations in the reading environment and culture conditions such as bacterial species.

JP 2011-2112013 A

  However, since the image analysis of the medium image is performed based on the color, the image region of the medium part (culture layer part) where the specimen is appropriately cultured is detected when performing the image analysis of the medium image. However, in the system of Patent Document 1 as described above, the image analysis is performed including other parts than the culture layer, and the colonies are accurately detected. There are many cases where this is not possible.

  The present invention has been made in order to solve the above-mentioned problems, and its purpose is to appropriately detect a colony image, perform image analysis, and accurately detect a colony. It is to provide a detection system and the like.

  In order to solve the above-described problem, the colony detection system and the like according to the present invention obtains captured image data including a medium image in which a medium on which a specimen is placed and a colony is cultured is imaged as captured image data. Scanning the captured image based on means and a template image for specifying the shape of the culture medium, and specifying a region having a predetermined condition as an image portion of the culture image, Image extracting means for extracting an image portion of the medium image from the inside, and performing image analysis on the extracted medium image, and colony pixels used for imaging the colonies cultured in the medium And a detecting means for detecting the colony in particular.

  Since the colony detection system and the like according to the present invention can accurately extract the culture medium image, the colony detection system can accurately detect the colony and includes information on the accurately detected colonies and the number of the colonies. Can be registered in the database.

It is a lineblock diagram showing the composition of a 1st embodiment of the hygiene management system concerning the present invention. It is an example of the film type culture medium used for the sanitary management system of 1st Embodiment. It is a block diagram which shows the structure of the image reading apparatus used for the sanitary management system of 1st Embodiment. It is a block diagram which shows the structure of the server apparatus used for the sanitary management system of 1st Embodiment. It is a figure for demonstrating the search area | region determination process performed in the server apparatus of 1st Embodiment. It is a figure which shows an example of the template image used for the image extraction process performed in the server apparatus of 1st Embodiment. It is FIG. (1) for demonstrating the template matching process performed in the server apparatus of 1st Embodiment. It is FIG. (2) for demonstrating the template matching process performed in the server apparatus of 1st Embodiment. It is a flowchart which shows the operation | movement of the culture medium information registration process in the image reading apparatus of 1st Embodiment. It is a flowchart which shows the operation | movement of the culture medium information registration process in the server apparatus of 1st Embodiment. It is a flowchart which shows the operation | movement of the image extraction process performed in the registration process of the culture medium information in the server apparatus of 1st Embodiment. It is a figure for demonstrating the skip scanning of the template image of 2nd Embodiment. It is a figure for demonstrating the direction scan of the template image of 2nd Embodiment. It is a figure for demonstrating the horizontal simple scanning of the template image of 2nd Embodiment. It is a flowchart which shows the operation | movement of the culture medium image extraction process based on the skip scanning performed in the server apparatus of 2nd Embodiment. It is a flowchart which shows the operation | movement of the culture medium image extraction process based on the direction scan performed in the server apparatus of 2nd Embodiment. It is a flowchart (the 1) which shows operation | movement of the culture medium image extraction process based on the horizontal simple scanning performed in the server apparatus of 2nd Embodiment. It is a flowchart (the 2) which shows the operation | movement of the culture medium image extraction process based on the horizontal simple scanning performed in the server apparatus of 2nd Embodiment. It is a figure for demonstrating the modification of the template matching process performed in the server apparatus of 1st or 2nd embodiment.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In the following embodiment, a colony detection system according to the present invention, a program thereof, for a hygiene management system using a film-type medium in a food production line or an inspection line (hereinafter referred to as “work line”), And it is embodiment at the time of applying a colony detection method. However, the present invention is not limited to the following embodiments within the scope including the technical idea.

[A] First Embodiment First, a first embodiment of the sanitary management system of the present invention will be described with reference to FIGS.
[A1] Hygiene management system First, the hygiene management system S of this embodiment will be described with reference to FIG. In addition, FIG. 1 is a block diagram which shows the structure of the sanitary management system S of this embodiment.

  The sanitary management system S of the present embodiment uses a film-type culture medium (that is, a film-type culture medium) 60 and performs data management on information on the film-type culture medium 60 (hereinafter referred to as “medium information”). This is a system for inspecting the hygienic state of food production or inspection.

  In particular, the sanitary management system S of the present embodiment extracts a processed or processed food as a specimen in a work line 80 having a manufacturing process for manufacturing a processed or processed food or an inspection process for inspecting the food. In addition, this is a system for managing data on the culture status when a pre-specified bacterium such as general live bacteria or Escherichia coli contained in a specimen is cultured in the film-type medium 60.

  Specifically, the sanitary management system S of the present embodiment identifies the type of each film-type medium 60 using information (hereinafter referred to as “type ID”) 66 indicating the medium type of the film-type medium 60. In order to manage and determine the sanitary state of food on the data, information on the work in the manufacture or inspection of food (hereinafter referred to as “work information”) and the bacteria contained in the extracted specimen are cultured (colony) And the medium information of the film-type medium 60 acquired by the image reading apparatus 10 before and after being associated with each other and registered in the server apparatus 40 (specifically, the database 400). doing.

  In addition, the sanitary management system S of the present embodiment uses a medium portion (that is, a culture layer portion described later) from captured image data (hereinafter referred to as “captured image data”) in which the entire film-type medium 60 is imaged. ) Image (i.e., medium image), and based on the extracted medium image, detection of colonies cultured in the medium, counting of the detected colonies and determination of the presence or absence of abnormality in the medium (hereinafter referred to as "colony") "Detection determination process").

  In particular, the sanitary management system S of the present embodiment scans the captured image based on a template image for specifying the shape of the culture medium (specifically, the shape of the culture layer 63 described later), and is predetermined. By performing template matching that specifies a region having a condition as an image portion of a medium image, the image portion of the medium image is extracted from the captured image.

  Usually, when image analysis is performed on a culture medium image, the analysis is performed based on the luminance of pixels constituting the culture medium image and color feature amounts such as RGB. However, if the medium image includes not only an image of a mount (base material) part on which a medium other than the medium is formed, but also an image of a part where characters or marks exist on the mount, the color of the characters or marks May affect the image analysis.

  Therefore, the hygiene management system S of the present embodiment accurately extracts the image portion of the medium image from the captured image by executing template matching using the template image, and detects colonies based on the extracted medium image. Etc. are to be performed.

  In the present specification, the type of the film-type medium 60 not only indicates the type of the medium itself and the type of medium when the cultivated bacteria are different, but also manufactures the film-type medium 60. The type of culture medium when the manufacturer is different is shown. For example, in the present embodiment, the film type medium manufactured by A company and the film type medium manufactured by B company are described as different types of medium.

  In order to realize such a configuration, the sanitary management system S of the present embodiment, as shown in FIG. 1, includes an image reading device 10 used when registering medium information regarding each film-type medium 60, and the sanitary management. The administrator terminal device 20 that manages the system S, the network 30, and the database 400, and in conjunction with the image reading device 10 or the administrator terminal device 20, performs various processes including registration management of medium information. And a server device 40.

  The image reading apparatus 10 has an ADF (Auto Document Feeder) function for taking in the film-type medium 60, and scans and images the film-type medium 60 according to an application combined with a predetermined process to generate image data. A still imaging device such as an image scanner.

  Specifically, the image reading apparatus 10 images the entire film type culture medium 60 when the film type culture medium 60 is taken in by the ADF or when the film type culture medium 60 is taken in and placed on the mounting table. The captured image data is generated, and the generated captured image data is provided to the server device 40.

  Further, the image reading device 10 has a configuration for transmitting the culture medium information to the server device 40 via the network 30 when transmitting the medium information to the server device 40. However, the image reading apparatus 10 is directly or accessed using a communication standard for short-range wireless such as BLUETOOTH (registered trademark), wireless LAN (WLAN: Wireless Local Area Network), or wireless PAN (WPAN: Wireless Personal Area Network). It may be transmitted to the server device 40 via a point, or may be transmitted to the server device 40 using a public telephone line network via a mobile base station (not shown).

  In addition to the reading operation of the film-type medium 60, the image reading device 10 can also add predetermined information to the generated image data based on the input operation of the operator.

  The administrator terminal device 20 is an information communication terminal device such as a tablet information terminal device, a smartphone, a personal computer, or a workstation. The administrator terminal device 20 manages the administrator identification information (hereinafter referred to as “manager ID”), the administrator password, and the worker identification information (that is, “worker ID”). It functions as a control device capable of performing management of access authority to the device 40, management of the terminal ID of the image reading device 10, and correction and other management of registered medium information.

  In addition, the administrator terminal device 20 has a browser function constructed by a markup language such as XML (extensible Markup Language), and executes an operation input instruction and an operation confirmation of the operator using the browser function. The browser function is used to exchange data with the server device 40, view a report, and the like.

  Furthermore, the administrator terminal device 20 has a remote function that allows remote operation of the image reading device 10.

  Note that the administrator terminal device 20 determines the culture medium information when each operation is interrupted due to a failure of a machine that performs the process or the like in the same lot, or when an error in registration of culture medium information occurs in the operator. It has a configuration that can correct other information.

  The network 30 is, for example, a public telephone line network including a mobile phone network (hereinafter referred to as “long-distance communication network”), an IP (Internet Protocol) network such as a short-range wireless network, or both of them are interconnected. It is configured. However, the configuration of the network 30 is not limited to this.

  The server device 40 is a server device used in conjunction with the image reading device 10 or the administrator terminal device 20 to execute each data process for performing hygiene management of the lot and work line 80.

  Then, the server device 40 operates in conjunction with the image reading device 10 to acquire captured image data and other data transmitted from the image reading device 10, detects colonies in the film-type medium 60 from the acquired data, and It has the structure which registers the information regarding a colony as culture-medium information.

Specifically, the server device 40
(1) Acquiring captured image data including a medium image in which a medium on which a specimen is placed and a colony is cultured is imaged;
(2) Based on a template image for specifying the shape of the culture medium, by scanning the captured image constituted by the captured image data and specifying a region having a predetermined condition as an image portion of the culture medium image , Extract the image part of the medium image from the captured image,
(3) Extracting the color feature amount relating to the color of each pixel constituting the extracted medium image,
(4) Based on the color feature value of each extracted pixel, the colony pixel used to image the colony is identified,
(5) Based on the identified colony pixel, the colonies cultured in the medium are detected, and the detected colonies are counted,
(6) Register information on the detected colonies and the counted number of colonies in the database 400 as medium information.
It has a configuration.

  Specifically, the server device 40 of the present embodiment identifies the type of the film-type medium 60 (hereinafter referred to as “medium type”) as identification information (hereinafter, referred to as a medium type provided in the film-type medium 60). (Referred to as “type ID”) 66, and the medium type of each film type medium 60 is identified based on the acquired type ID 66.

  The server device 40 according to the present embodiment has a configuration in which one template image is selected from a plurality of template images based on the identified medium type, and the medium image is extracted using the selected template image. doing.

  In particular, in this embodiment, the template image is a mask image for masking a medium-shaped part (that is, a part of a culture layer 63 described later), and is obtained by imaging the film-type medium 60. It is an image used for detecting and extracting a region of a medium image that is an image of a medium part from an image (that is, a captured image).

  The captured image includes an image of the medium-shaped part and an image in which the color (background color) of the mount part on which the medium is formed (hereinafter also referred to as “background image”). The server device 40 according to the present embodiment can extract a medium image by removing a background image (that is, each pixel of the mount) from the captured image using a template image.

  In addition, when extracting the medium image, the server device 40 of the present embodiment forms a medium image based on information on the arrangement structure of the medium portion in the film-type medium 60 (hereinafter referred to as “arrangement structure information”). In addition to determining a region with high accuracy assumed to be a search region, a medium image is extracted by scanning a template image selected within the determined search region.

  With such a configuration, in the sanitary management system S of the present embodiment, the culture medium image can be accurately extracted, so that colony detection and counting can be performed accurately and accurately detected. Medium information including information on colonies and their numbers can be registered in the database 400.

  Further, in the sanitary management system S of the present embodiment, it is possible to save resources as compared with the case where various data analyzes and provisions are performed using physical resources such as paper media, and the culture medium. By registering the culture medium information in the database 400, registration errors can be remarkably reduced and management of difficult culture media can be facilitated, so that resource saving can be achieved without wasteful consumption of culture media. ing.

[A2] Film Type Medium Next, the film type medium 60 of the present embodiment will be described with reference to FIG. FIG. 2 is an example of a film-type medium 60 used in this embodiment.

  The film-type medium 60 used in the present embodiment is a medium for culturing bacteria contained in food as a specimen detected from each step of the work line 80 using a film or sheet-like dry medium. The film-type medium 60 is, for example, a medium for culturing general living bacteria, coliforms, Staphylococcus aureus, and the like, and other mediums for culturing various microorganisms such as mold, yeast, Listeria, aquatic bacteria, and lactic acid bacteria. There may be.

  The film-type medium 60 includes, for example, a base sheet 61 formed of a film and a circular frame formed on the base sheet 61 with reference to the center of the base sheet 61 as shown in FIG. (Hereinafter referred to as “circular frame”) 62, a culture layer 63 for culturing bacteria provided in the frame, and a cover sheet 64 formed so as to cover the base material sheet including the culture layer 63 And a type ID 66 formed on the right side of the base sheet 61 and indicating the type of the culture medium.

  For example, the type ID 66 is formed of a bar code such as a two-dimensional bar code or an alphanumeric character, and is imaged together when the film-type medium 60 is read, such as a bar code recognition function or an OCR (Optical Character Recognition) function. It is acquired by the administrator terminal device 20 or the server device 40 using the image analysis function.

  The base material sheet 61 is not particularly limited as long as it is a film-like or sheet-like base material, and for example, a plastic film or paper can be used. Preferred examples of plastic films include resin films such as polyethylene, polypropylene, polyethylene terephthalate, polyethylene naphthalate, polymethacrylate, polymethyl methacrylate, polymethyl acrylate, polyester, and polycarbonate. Preferred examples include polyethylene terephthalate and polypropylene synthetic paper. Note that the polypropylene-based synthetic paper is a film synthetic paper whose main raw material is polypropylene.

  In addition, in type ID66 of this embodiment, the culture medium ID which identifies a culture medium separately may be contained in addition to the information which manages the culture medium type of the film type culture medium 60. FIG. In this case, the culture medium ID is acquired in the administrator terminal device 20 or the server device 40 using the image analysis function together with the culture medium type ID.

[A3] Image Reading Device Next, the configuration of the image reading device 10 of the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating the configuration of the image reading apparatus 10 according to the present embodiment.

  The image reading apparatus 10 of the present embodiment has a data storage unit 100 having a memory function used when various programs are executed, a scanning function (imaging function), image data of the film-type medium 60, and the like. In association with the image data generation unit 110 that generates the image data, and the server device 40, the culture medium information is registered in the server device 40 after initial culture and registration after culture (hereinafter referred to as “medium information registration process”). And an application control unit 120 that executes other processing.

  The image reading device 10 includes a network communication unit 130 that communicates with the server device 40 and other communication devices, an ADF unit 140, a display unit 160, a display control unit 161 that controls the display unit 160, and a user's An operation unit 170 for inputting an operation, a timer 180, and a terminal management control unit 190 for controlling the entire apparatus are provided.

  The data storage unit 100 includes an application storage unit 101 that stores various application programs, an image data storage unit 102 that stores image data read and generated by the image data generation unit 110, and an image reader 10. A program relating to management and control, and a ROM / RAM 103 that is used as a work area during execution of each program and stores data used in each process executed by the image reading apparatus 10 are included.

  In particular, the application storage unit 101 includes an application program (hereinafter referred to as “application”) that is executed by the application control unit 120 in conjunction with the image data generation unit 110, the operation unit 170, the display control unit 161, and the image data storage unit 102. Is recorded).

  The image data storage unit 102 stores imaged captured image data, an image ID for managing each image data, and various metadata corresponding to each image data such as a reading time in association with each other. Is done. The image ID is arbitrary identification information given as appropriate in the image reading apparatus 10.

  The image data generation unit 110 includes an optical system, a CCDI sensor (Charge Coupled Device Image Sensor) that converts an optical image input from the optical system into an electrical signal, and image data based on the electrical signal generated by the CCDI sensor. And a generating unit that generates

  In particular, the image data generation unit 110 images the entire image-scanned film-type medium 60 when the bacteria generated in the specimen are cultured or when the film-type medium 60 cultured is imaged. Image data of the entire film type culture medium 60 is generated as captured image data.

  The application control part 120 implement | achieves culture-medium information registration process with a predetermined application. In particular, the application control unit 120 executes various control programs for controlling each unit of the image reading device 10 using the culture medium registration application stored in the application storage unit 101, while performing the network communication unit 130, the display control unit 161, and the operation. Various processes are executed in conjunction with or under control of the unit 170.

  In particular, the application control unit 120 executes the culture medium registration application recorded in the application storage unit 101, acquires the captured image data of the film type culture medium 60 read by controlling the image data generation unit 110, and stores these data in these data The image data is stored in the image data storage unit 102 as medium information together with metadata of each data while giving an image ID.

  Further, the application control unit 120 acquires the current time from the timer 180 when imaging the film-type medium 60, and stores the image data as metadata of the captured image data together with a predetermined image ID using the current time as a reading time. Store in the unit 102.

  On the other hand, the application control unit 120 executes a medium information registration process for registering the medium information stored in the image data storage unit 102 in the server device 40 after acquiring captured image data or at a predetermined timing.

  Note that the application control unit 120 may be realized as a function when the CPU (central processing unit) constituting the terminal management control unit 190 executes an application in terms of implementation.

  The network communication unit 130 establishes a communication line with the server device 40 connected to the network 30 under the control of the application control unit 120 and the terminal management control unit 190, and exchanges various data such as captured image data. .

  The ADF unit 140 has a configuration for realizing ADF functions such as a mechanical structure and a control circuit. In particular, the ADF unit 140 is a mechanism for taking a plurality of film-type culture media 60 placed one by one in the same place into an area (not shown) where scanning is performed. Drive under control.

  The display unit 160 has an image display area of a predetermined size (for example, 5 inches, W480 × H960 pixels), is configured by a panel of liquid crystal elements or EL (Electro Luminescence) elements, and is generated by the display control unit 161. A predetermined image is displayed based on the display data. In particular, in this embodiment, the display unit 160 displays various displays and images of the film-type culture medium 60 in conjunction with the operation unit 170 when the culture medium registration application is being executed.

  The display control unit 161 generates drawing data necessary for causing the display unit 160 to draw a predetermined image under the control of the application control unit 120 or the terminal management control unit 190, and the generated drawing data is displayed on the display unit 160. To output.

  The operation unit 170 includes various confirmation buttons, operation buttons for inputting operation commands, a number of keys such as a numeric keypad, and a touch sensor provided on the display unit 160, and is used when performing each operation. It has become. Specifically, the operation unit 170 is used when performing operations for executing the various processes described above when the culture medium registration application is activated.

  In the present embodiment, the operation unit 170 is used when directly inputting a work ID or the like directly.

  The timer 180 provides the application control unit 120 with the date and time when the image data generation unit 110 reads the film-type medium.

  The terminal management control unit 190 is mainly configured by a central processing unit (CPU) and includes various input / output ports such as a key input port and a display control port, and provides general functions and information of the image reading device 10. It is designed to control overall functions for executing programs.

[A4] Server Device [A4.1] Configuration Next, the configuration of the server device 40 of this embodiment will be described with reference to FIG. FIG. 4 is a configuration diagram showing the configuration of the server device 40 of this embodiment.

  As shown in FIG. 4, the server device 40 of the present embodiment communicates with a database 400 in which various information such as work information and medium information is stored, and with the image reading device 10 and the administrator terminal device 20. Unit 410, data processing unit 420 that executes various processes such as medium information registration processing, server management control unit 430 that controls each unit of server device 40, ROM / RAM 440 that is used for control of each unit, and time management And a timer 450 used for performing. The above-described units are connected to each other by a bus 41, and data transfer is performed between the components.

  The communication control unit 410 is a predetermined network interface, establishes a communication line with the image reading device 10 or the administrator terminal device 20, and exchanges various data with the image reading device 10 or the administrator terminal device 20.

  The database 400 includes an HDD, a medium information database (hereinafter abbreviated as “medium information DB”) 401, a work management database (hereinafter abbreviated as “work management DB”) 404, and a profile information database (hereinafter “ Profile information DB ”) 405.

The medium information DB 401 is extracted for each process of each work line 80, and the medium information regarding the film-type medium 60 of the specimen acquired by scanning by the image reading apparatus 10 is the medium for each type ID 66 and for each medium ID. A database where information is stored. For example, in the culture medium information DB 401,
(1) Type ID 66
(2) Medium ID
(3) Captured image data (4) Medium image data (5) Scanning time (6) Number of colonies (7) Medium determination result (8) Size (number of pixels), barycentric position, feature amount of each pixel, etc. were detected Information about each colony (hereinafter referred to as “colony information”)
(9) Type of profile information (profile ID) used at the time of colony detection
Are recorded in association with each other.

  As the colony information, the number of colonies, and the medium determination in each medium information, the results determined by the data processing unit 420 are stored. The colony information, the number of colonies, and the medium determination in each medium information may be registered when the medium information is registered, or may be registered at a predetermined timing different from the registration timing of the medium information.

The work management DB 404 is a database that stores various types of information regarding work contents for each culture medium type, and is a database for providing inspection contents for each culture medium type to workers and others by associating with the culture medium information DB 401. For example, in the work management DB 404, (1) medium ID is associated with each medium ID.
(2) Examination type (genus to be detected such as general viable bacteria or E. coli) and color (coloring) information (RGB gradation value or its range) when it is generated on the film-type medium 60
(3) Culture start time (4) Culture time from the start of culture until a predetermined time has elapsed (5) Culture location (for example, the position of the incubator number and the number of steps in the incubator)
(6) Work date and time (work start time and end time)
(7) Worker ID (and / or worker name)
Are recorded in association with each other.

  Such information is registered by an operator or the like. Moreover, only a single time (that is, an end time for ending the culture of bacteria in the specimen) may be set as the culture test time, or after legal passage from the culture start time or after a predetermined time has elapsed, for example, A plurality of times such as 24 hours from the start of culture, 48 hours, or 72 hours may be set.

The profile information DB 405 is a database that stores a plurality of profile information. For example, in the profile information DB 405, (1) a type ID is associated with each type of ID.
(2) Profile information (3) Template image data (hereinafter referred to as “template image data”)
(3) Placement structure information (search area information)
Are recorded in association with each other.
The profile information includes
(2A) the type of feature amount relating to the color when detecting colony pixels from the medium image,
(2B) a detection method for detecting a colony based on the type of medium and the type of specimen, a parameter for executing the detection, and
(2C) A threshold value for pass / fail when performing medium determination is included.

The data processing unit 420 executes various types of data processing according to the application recorded in the ROM / RAM 440. In particular, the data processing unit 420 executes a predetermined program,
(1) Operation management of the communication control unit 410;
(2) Type information acquisition process for acquiring the type ID 66 from the medium information transmitted from the image reading device 10 (3) The arrangement structure information and the template image data are acquired based on the acquired type ID 66, and the acquired arrangement structure information and Based on the template image data, a culture medium image extraction process for extracting a culture medium image in each film type culture medium 60. (4) Profile information is acquired based on the acquired type ID 66, and based on the acquired profile information and the extracted culture medium image. , While detecting colonies imaged in the medium image, the colony detection determination process for performing the medium determination based on the counting of the number of colonies detected and the number of colonies,
(5) Medium information registration processing for registering medium information in each film type medium 60;
(6) Management and control of the database 400;
I do.

  Specifically, the data processing unit 420 includes a registration processing unit 421 that executes a medium information registration process, an image extraction processing unit 422 that executes a medium image extraction process, and a colony detection determination unit 423 that executes a colony detection process. including.

  For example, the registration processing unit 421 of the present embodiment constitutes an acquisition unit according to the present invention, and the image extraction processing unit 422 constitutes an image extraction unit. For example, the colony detection determination unit 423 of the present embodiment constitutes feature amount extraction means, identification means, and detection means according to the present invention. Details of the registration processing unit 421, the image extraction processing unit 422, and the colony detection determination unit 423 of this embodiment will be described later.

  The server management control unit 430 is mainly configured by a central processing unit (CPU), and integrally controls each unit of the server device 40 by executing a program. Specifically, the server management control unit 430 performs various controls related to data exchange with the image reading apparatus 10.

  Various programs necessary for driving the server device 40 are recorded in the ROM / RAM 440. The ROM / RAM 440 is used as a work area during execution of each program.

  The timer 450 is used for managing the time required when the medium information registration process and the lot determination process are executed.

[A4.2] Registration Processing Unit Next, details of the registration processing unit 421 in the server device 40 of the present embodiment will be described.

  The registration processing unit 421 according to the present embodiment performs a medium information registration process for the medium information in each film type medium 60 based on an operator's instruction in conjunction with the image reading device 10.

  In particular, the registration processing unit 421 performs image analysis on each captured image data included in each medium information transmitted from the image reading device 10 to acquire the type ID 66, and the acquired type ID 66 is used as the image extraction processing unit. 422 and the colony detection determination unit 423.

  Particularly, when the registration processing unit 421 acquires the culture medium information, the registration processing unit 421 searches the captured image of the captured image data included in the culture medium information, and the type ID 66 configured by the two-dimensional barcode imaged in the captured image. To get. And the registration process part 421 assign | provides culture-medium ID for identifying a culture medium automatically or under an operator's instruction | indication, or when included in classification ID66, culture | cultivation ID is combined with classification ID66. get.

  Further, the registration processing unit 421 associates the colony information, the number of colonies, and the determination result obtained by the colony detection determination process of the colony detection determination unit 423 with the corresponding type ID 66 and medium ID as the medium information, respectively, and the inspection time. At the same time, it is registered in the culture medium information DB 401.

[A4.3] Image Extraction Processing Unit Next, details of the image extraction processing unit 422 in the server device 40 of the present embodiment will be described with reference to FIGS. FIG. 5 is a diagram for explaining search area determination processing executed in the image extraction processing unit 422 of this embodiment, and FIG. 6 is a diagram illustrating an example of a template image used in this embodiment. . 7 and 8 are diagrams for explaining template matching processing executed in the image extraction processing unit 422 of the present embodiment.

  The image extraction processing unit 422 searches the profile information DB 405 based on the type ID 66 acquired by the registration processing unit 421 and acquires corresponding template image data, thereby obtaining one template image from a plurality of template images. And a medium image is extracted using the selected template image.

In particular, the image extraction processing unit 422
(1) Based on the type ID 66, the profile information DB 405 is searched to obtain arrangement structure information indicating information on the arrangement structure of the medium portion in the film-type medium 60, and a medium image in the corresponding captured image data is formed. A search area determination process for determining a search area assumed to be, and
(2) A template matching process that scans using an image (that is, a template image) configured by template image data selected for the determined search region and extracts a culture medium image;
And the medium image extracted by the template matching process is output to the colony detection determination unit 423.

(Search area determination process)
Usually, as shown in FIG. 2 described above, the film-type medium 60 has a specific structure, and the arrangement position and size of the culture layer 63 are not completely the same as other types of film medium. However, the culture layer 63 is formed within a certain error range.

  On the other hand, when image analysis is performed on a medium image, as described above, not only the image of the mount (base material) part on which the medium other than the medium is formed, but also the characters and marks are present on the mount. In the case where the image of the portion to be included is included, the color of characters or marks may affect and the image analysis may not be performed accurately.

  Further, when executing template matching processing using a template image, the template image is scanned within the search range, and analysis is performed on each image at each scanning position, and whether each image is a culture medium image or not. Therefore, it is desirable that the search area for scanning the template image is narrow from the viewpoint of quick processing and reduction of processing load.

  Therefore, the image extraction processing unit 422 according to the present embodiment acquires the arrangement structure information based on the type ID 66, and determines a search region in which a culture medium image in the corresponding captured image data is assumed to be formed. It has become.

  Specifically, the image extraction processing unit 422 has, for example, arrangement structure information as shown in FIG. 5A, and based on the arrangement image information, from the captured image formed by the captured image data. A search area as shown in FIG. 5B is determined.

(Template image)
The image extraction processing unit 422 according to the present embodiment performs the template matching process on the search region determined using the template image.
(1) It has the same shape as the medium to be extracted (that is, the culture layer 63) such as a circle, an ellipse, or a rectangle, and when superimposed on the captured image, the color feature amount (specifically, each pixel of the captured image) , RGB gradation values and luminance values), the first region having the characteristic of invalidating,
(2) a second region having a predetermined shape on the periphery of the first region;
(3) a third region formed on the outer edge of the second region and having a characteristic of invalidating the color feature amount of each pixel of the captured image when superimposed on the captured image;
A template image having It should be noted that the “same shape” of the first region need not be completely the same as long as the region of the medium image can be determined.

  In particular, in the present embodiment, the image extraction processing unit 422 uses a template image (that is, a mask image) in which a mask (template mask) to be excluded from calculation in template matching is formed in the first region and the third region. Use.

  Usually, when a colony is expressed in a medium image that is an image of the culture layer 63, a colony is formed on the medium image, unlike the background color indicating the color of the mount on which the culture layer 63 is formed. Pixels having the color to be included are included. For this reason, since different color feature amounts exist in the region (region of the medium portion) depending on the individual film-type medium 60 (that is, for each medium image), the medium image is extracted (the medium image). In the case of defining the area of (2), using the color feature amount of each pixel in the area increases the possibility of erroneous detection.

  In addition, the film-type medium 60 includes, for example, as shown in FIG. 2 described above, various types of information (hereinafter referred to as “marking information”) such as information on the type ID 66 and the manufacturer, or characters and other markings entered by the operator. Is often formed with a color such as black, and when used for processing to specify a medium image up to the peripheral part of the film-type medium 60, the medium image is distinguished from other regions. In some cases, the medium image cannot be accurately detected.

  On the other hand, if the boundary between the region of the medium part and the background color part of the film type medium 60 such as a mount or base material of the film type medium 60 can be recognized, the area of the medium part is identified. A media image can be extracted.

  Therefore, the image extraction processing unit 422 of the present embodiment has the same shape as the culture layer 63 and is superimposed on the captured image in order to eliminate the influence of the culture medium part (the culture layer 63 part) and various marking information. The first region having the characteristic of invalidating the color feature amount of each pixel of the captured image in the case of being performed, and the color feature amount of each pixel of the captured image formed at the periphery of the first region A template image having two regions (that is, an effective region) and a third region having the same characteristics as the first region at the outer edge of the second region is used.

  For example, when the first region is superimposed on a captured image including a culture medium image and each pixel in which a background color is imaged (that is, a background image), each of the captured images It is only necessary to have a characteristic that can invalidate the color feature amount of the pixel and exclude it from the calculation by template matching. However, the characteristic of invalidating the color feature of each pixel of the captured image is not only completely invalidated and excluded from the calculation by template matching, but the luminance and color feature of each pixel constituting the background image The gradation value may be a characteristic that allows the second area to be detected so as not to be affected (subtracting (invalidating) a predetermined value from the color feature amount of each pixel). .

  The second area may be an area that enables the color feature amount of each pixel of the captured image. That is, the second area functions as a filter if it can be used for calculation by template matching, for example, invalidates a part of the color characteristics of the background color of one color characteristic (for example, R), or the gradation of RGB It may have a filtering function such as correcting the value.

  Further, for example, the image extraction processing unit 422 of the present embodiment has a first region 71 having the same circular shape as the culture layer 63 and a predetermined width at the periphery of the first region 71 as shown in FIG. A template image 70 having a donut-shaped second region 72 having an annular shape and a third region 73 formed outside the second region 72 is used.

  In the present embodiment, as illustrated in FIG. 6, masks (template masks) are formed in the first region and the third region in order to exclude the first region and the third region from calculations in template matching. The template image (that is, the mask image) is used, but a template image that masks at least the first region may be used. For example, a template image formed by the first region and the second region without the third region may be used. Even in this case, the first region can be detected by template matching.

  In addition, the annular width in the second region is not limited as long as it is a width that can perform a template matching operation, that is, a width that can secure a predetermined number of pixels.

  Further, the first region 71 and the second region 72 are circular or annular according to the shape of the medium part, but the shape of the medium part (culture layer 63) is other shapes such as a rectangular shape. In such a case, the first region 71 is formed in the same shape as the culture layer 63 such as a rectangular shape, and the second shape is a first region 71 such as a rectangular ring having a predetermined width on the periphery of the first region 71 ( In other words, it may be formed in an annular shape having a predetermined width according to the shape of the culture layer 63).

(Template matching)
As described above, the image extraction processing unit 422 according to the present embodiment performs the template matching process on the search region determined on the captured image using the template image specified using the type ID 66.

  Specifically, the image extraction processing unit 422 uses the template image, for example, sequentially from the upper left pixel coordinates (for example, (0, 0)) within the search range on the captured image, as shown in FIG. When the rightmost coordinate (n, j) is reached while shifting in the horizontal scanning direction for each pixel (from “0” to “i”), it is perpendicular to the leftmost coordinate (0, j + 1) of the next pixel row. A raster scan is performed in which scanning is performed up to the rightmost coordinate (n, m) of the last pixel row in the search range, and a second region on the captured image at each scanning position is specified based on the template image. .

  For example, as shown in FIG. 7, the image extraction processing unit 422 arranges the center (xi, yj) of the template image at each scanning position Si i (0 ≦ i ≦ n, 0 ≦ j ≦ m), and the template A second region on the captured image within the template image range in a state where the image is superimposed on the captured image is specified.

  Then, the image extraction processing unit 422 determines the position (specifically, the center position) of the template image having a predetermined condition (hereinafter referred to as “matching condition”) from the scanning position of each template image. ) And a mask in the first area when the template image is formed at the determined center position, that is, the first area when the template image is formed at the determined center position on the captured image. The area that has been specified is identified as a part of the medium image, and the image of the part is extracted as a medium image.

  For example, the image extraction processing unit 422 of the present embodiment integrates the color feature values (luminance values) of the pixels belonging to the second region (that is, the effective region) at each scanning position, and calculates the sum as a matching score. Alternatively, the average value of the color feature amount of the pixel is calculated as a matching score from the summed value, and the scanning position with the highest matching score is set as the matching condition, and the center position where the template image for determining the medium portion is arranged To decide.

  In particular, in the present embodiment, the image extraction processing unit 422, at each scanning position, based on (Expression 1), the luminance value I (xi, yj) of each pixel on the captured image belonging to the second region in the template image. ) And the respective luminance values I (xi, yj) are integrated with respect to the second region based on (Equation 2) or (Equation 3), and each summation value Z (s) or each summation value based on the summation value is calculated. A template image for calculating an average value Z (av) of luminance values of pixels, and determining a culture portion at a scanning position having the total value Z (s) max or the average value Z (av) max at which these values become maximum Is determined as the center position.

  In the above formula, R (xi, yj) is the R gradation value in each pixel, G (xi, yj) is the G gradation value in each pixel, and B (xi, yj) is , B gradation values in each pixel are shown. “P” indicates the number of pixels in the second region.

Then, for example, as illustrated in FIG. 8B, the image extraction processing unit 422 performs processing on the captured image based on the determined coordinates S _(maximum) = (x _(maximum) , y _(maximum) ) of the center position. A region in which a first region (for example, a circle with a radius r from the center coordinates (x _(maximum) , y _(maximum) ) shown in FIG.

  In particular, in the case where the background color of the film-type medium 60 is “white” and the medium color of the culture layer 63 is a color other than white, such as “skin color”, as shown in FIG. If the template images do not match, the medium image portion (culture layer 63) enters the second region 72 that transmits the color of the film-type medium 60 of the template image. Therefore, in this case, the image extraction processing unit 422 of the present embodiment causes the culture medium image portion to enter the second region for the sum of the color feature values (the above-described luminance values) of the second region or the average value thereof. As a result, a value that is lower than the highest value is calculated.

  On the other hand, when the medium image portion and the template image match as shown in FIG. 8 (B) under the same conditions as in FIG. 8 (A), the color of the film-type medium 60 of the template image is transmitted. The medium image portion (culture layer 63) does not enter the second region. Therefore, in this case, the image extraction processing unit 422 according to the present embodiment calculates the maximum value for the average value of the color feature amount, and determines the area masked by the first area of the template image as the medium image. A portion is specified, and an image of the region portion is extracted as a medium image.

  On the other hand, the above calculation method and matching conditions based on the calculation method differ depending on the background color of the film-type medium 60 (the color of the base material) and the color of the culture layer 63. For example, when the background color (base material color) of the film type culture medium 60 is “white” and the culture medium color of the culture layer 63 is a color other than white, such as “skin color”, the image extraction processing unit 422 As the matching condition, a scanning position where the difference between the color feature amount of the pixel in the second region (luminance in the case of the present embodiment) and the background color is the smallest is a scanning position having a predetermined condition, That is, it determines as a center position which arrange | positions the template image for determining a culture medium part.

  Specifically, when the background color of the film-type culture medium 60 (the color of the base material) is not “white”, the image extraction processing unit 422 performs color feature amount of each pixel belonging to the second region at each scanning position. (For example, the gradation value of each color component of RGB) and a color feature amount of a predetermined background color are calculated, and an SSD (Sum of Squared Distance) or SAD (Sum (Sum)) is calculated based on these differences. a predetermined value is calculated by, for example, Absolute Absolute Distance), and the sum of the predetermined values of each pixel in the second region is calculated as a matching score, or the average value of the color feature amount of the pixel is matched from the sum In order to calculate the score, and to determine the medium part, the scanning position with the lowest matching score as the matching condition Determining the center position to place the template image.

  That is, in this case, the image extraction processing unit 422 sets the scanning position where the color feature amount of the pixel belonging to the second region is closest to the background color as the center position where the template image for determining the medium portion is arranged. Will be decided. In particular, when the background color is “colored”, the image extraction processing unit 422 causes the scanning position where the average of the color feature amount in each pixel in the second region at each scanning position is closest to the value of colored RGB. Is determined as a scanning position having a predetermined condition (a central position where a template image for determining a medium portion is arranged).

  In the case of SSD, the image extraction processing unit 422, for each scanning position, each color feature amount (R, G, B color components) of pixels belonging to the second region of the template image in the captured image and the background color. The sum of squares (squares of Euclidean distance) of differences from the color feature quantities (color components of R, G, and B) is calculated and integrated, and the scanning position with the smallest integrated value is determined in advance as a predetermined condition. Is determined as a scanning position.

  More specifically, as shown in (Equation 4), the image extraction processing unit 422, as shown in (Expression 4), the gradation value of each RGB color component of each pixel at each scanning position Sij and a predetermined background color Sij (SSD) indicating the sum of squares of the Euclidean distance is calculated on the basis of the difference from the color feature amount, and the calculation similar to the above (Expression 2) or (Expression 3) is performed using the value. The sum or average value of the predetermined values of each pixel in the second region is calculated as a matching score. However, in (Expression 4), “Rb”, “Gb”, and “Bb” indicate the gradation values of the RGB color components in the background color stored in advance.

  In the case of SAD, the image extraction processing unit 422, for each scanning position, each color feature amount (R, G, B color components) of the pixels belonging to the second region of the template image in the captured image and the background color. The sum of absolute values of differences from the color feature quantities (R, G, B color components) is calculated and integrated, and the scanning position with the smallest integrated value is set as a scanning position having a predetermined condition. decide.

  More specifically, as shown in (Equation 5), the image extraction processing unit 422, as shown in (Expression 5), the gradation value of each RGB color component of each pixel at each scanning position Sij and a predetermined background color Sij (SAD) indicating the absolute sum of the pixel value difference is calculated based on the difference from the color feature amount of the image, and the same calculation as the above (Expression 2) or (Expression 3) is performed using the value. And a sum or average value of predetermined values of each pixel in the second region is calculated as a matching score. However, in (Expression 5), “Rb”, “Gb”, and “Bb” indicate the gradation values of the RGB color components in the background color stored in advance, as in (Expression 4).

  Note that the image extraction processing unit 422 may perform template matching by enlarging and reducing the template image. For example, when the size of the captured image changes due to the imaging magnification or other influences, the image extraction processing unit 422 recognizes the size of the culture layer 63 portion in advance before performing template matching, and the template The image is enlarged or reduced according to the size of the culture layer 63. Then, the image extraction processing unit 422 may perform template matching using the enlarged or reduced template image.

  Further, the image extraction processing unit 422 records data that can be converted into a template image in the profile information DB 405 in advance, and when executing template matching, reads the data from the profile information DB and converts it into a template image. Template matching may be performed using the converted template image.

[A4.4] Colony Detection Determination Unit Next, details of the colony detection determination unit 423 in the server device 40 of the present embodiment will be described.

  As a colony detection determination process, the colony detection determination unit 423 searches the profile information DB 405 based on the type ID 66 provided from the registration processing unit 421, acquires the corresponding profile information, and obtains the corresponding medium from the image extraction processing unit 422. Get an image.

  The colony detection determination unit 423 sets the various thresholds or parameters for each film-type medium 60 based on the profile information and the medium image provided from the image extraction processing unit 422, A colony detection process is performed on each captured image data while determining a colony candidate detection technique and a colony identification technique, and the result is provided to the registration processing unit 421, respectively.

Specifically, the colony detection determination unit 423 is based on the provided profile information.
(1) In order to detect pixels assumed to form an imaged colony from the provided medium image, a feature amount relating to a color having a predetermined condition is selected from each pixel constituting the medium image. A pixel having the same (hereinafter referred to as a “colony pixel”),
(2) Concatenating colony pixels and performing labeling to give an ID, and detecting a connected colony pixel group as one colony candidate,
(3) While identifying colony candidates having predetermined conditions as colonies, the number is counted,
(4) Perform medium determination based on the counted number of colonies.

  In particular, when the type of feature quantity is luminance or saturation, the colony detection determination unit 423 obtains a value obtained by executing a predetermined calculation based on the gradation value of each RGB sub-pixel of each pixel. And detecting colony pixels based on the parameters defined in the profile information, identifying colony candidates based on the detected colony pixels, detecting colonies from the identified colony candidates, and counting the number .

  The colony detection determination unit 423 performs labeling by connecting adjacent colony pixels while performing image filter processing including expansion processing and reduction processing for reducing image filter noise, and assigning an ID. Colony candidates are detected based on the connection method and the parameters based on the profile information.

  That is, the colony detection determination unit 423 is configured to connect the colony pixels with respect to adjacent colony pixels based on a range that should be determined to be the same as the feature amount related to the color of each colony pixel specified by the profile. Concatenate those determined to be.

  Furthermore, the colony detection determination unit 423 connects adjacent colony pixels to form a colony pixel group, and uses the color characteristics for each colony pixel group or the barycentric position on the medium image to form the same colony pixel Connect them together.

  Then, the colony detection determination unit 423 sets, for each detected colony candidate, feature amounts such as color characteristics, roundness, shape, and size determined by the profile information, based on the feature amounts specified in the specified profile information. However, the likelihood indicating the likelihood of colony is calculated based on the set feature amount.

  Finally, based on the profile information, the colony detection determination unit 423 identifies colony candidates having a predetermined condition such as a calculated likelihood equal to or greater than a threshold value as a colony, and counts the number as the number of colonies.

  The colony detection determination unit 423 detects the detected colony pixel (for example, the position on the medium image and the ID of the colony candidate to which the own pixel belongs), the detected colony candidate (for example, the arrangement position of the colony pixel on the medium image, the color characteristic, or (Predetermined information such as the center of gravity position), the identified colony (for example, predetermined information such as the arrangement position of the colony pixel on the medium image, color characteristics, or the position of the center of gravity), and the counted number of colonies via the registration processing unit 421 And stored in the medium information DB 401.

  In addition, the expansion process is one of image filter processes, and is a process of replacing all pixels adjacent to a colony pixel with colony pixels when other colony pixels exist adjacent to each colony pixel. .

  Further, the contraction process is a process of replacing all pixels adjacent to the colony pixel with non-colony pixels when there is a non-detected pixel adjacent to the colony pixel (hereinafter referred to as “non-colony pixel”). .

  On the other hand, the colony detection determination unit 423 of the present embodiment performs medium determination based on the number of colonies counted by the colony detection process. In particular, the colony detection determination unit 423 compares the threshold value set in advance with the detected colony number based on the profile information acquired in the colony detection process as the medium determination process, and the detected colony number is greater than or equal to the threshold value. In this case, it is determined that the sample is abnormal, and if it is smaller than the threshold value, it is determined that the sample is normal, and the medium determination of whether the sample of each film-type medium 60 is abnormal or normal (that is, the sample) ) Is executed.

[A5] Hygiene management system operation process [A5.1] Medium information registration process (image reading apparatus)
Next, the operation of the medium information registration process for the film-type medium 60 in the image reading apparatus 10 of the present embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing the registration processing operation of the medium information in the image reading apparatus 10 of the present embodiment.

  In this operation, a plurality of profile information, template image data, and arrangement structure information are stored in advance in the profile information DB 405 based on the type ID.

  In this operation, the plurality of film-type culture media 60 to be registered are assumed to be already placed in a predetermined location of the ADF unit 140, for example, and the number of colonies in each film-type culture media 60 is detected by the culture media. It is assumed that it is executed when information is registered in the database 400.

  In this operation, it is assumed that the image reading apparatus 10 performs calibration for RGB gradation values or luminance values for reading a captured image.

  First, in the image reading apparatus 10, when the application control unit 120 detects an activation instruction for the culture medium registration application via the operation unit 170 (step S <b> 101), the application control unit 120 reads the medium registration application from the application storage unit 101 and activates it (step S <b> 102). ). At this time, the application control unit 120 executes a display for confirming whether or not the plurality of film-type culture media 60 to be registered in the display unit 160 are already placed on the ADF unit 140 in conjunction with the display control unit 161. At the same time, under the control of the terminal management control unit 190, initialization of the work memory and other necessary processes are executed.

  Next, the application control unit 120 causes the display unit 160 to display a screen that prompts the reading start of each film-type medium 60 (hereinafter also referred to as a “reading start confirmation screen”), and in conjunction with the operation unit 170, image data. It waits for reading by the generation unit 110 (step S106).

  Next, when the application control unit 120 detects the start of reading of the film-type medium 60 by the operation unit 170 in cooperation with the image data generation unit 110 and the display control unit 161 (step S107), the application control unit 120 controls the ADF unit 140 to The plurality of film-type culture media 60 set in the ADF unit 140 are sequentially moved to the scanning area, and the image data generation unit 110 scans the plurality of film-type culture media 60 to form an image, and generates each captured image data as an image. The data storage unit 102 stores a predetermined image ID while giving it (step S108).

  When a plurality of film-type culture media 60 are placed on the ADF unit 140, the application control unit 120 continuously images each film-type culture media 60 and stores each of them in the image data storage unit 102. The image ID is stored while being given.

  Next, the application control unit 120 acquires the current time from the timer 180 as the reading time of the captured image data in each film type culture medium 60, and stores it in the image data storage unit 102 as metadata in association with each captured image data. (Step S109).

  Next, the application control unit 120 transmits each captured image data and its metadata as one medium information to the server device 40 via the network communication unit 130 based on the operator's instruction (step S118). The medium information registration process of the reader 10 is finished.

  At this time, when transmitting the culture medium information, the network communication unit 130 executes login for accessing the server device 40 based on the terminal ID and the input ID and password, and the server after the login is completed. A communication line with the apparatus 40 is established, and medium information is transmitted.

[A5.2] Medium information registration process (server device)
Next, operation | movement of the culture medium information registration process of the film type culture medium 60 in the server apparatus 40 of this embodiment is demonstrated using FIG. FIG. 10 is a flowchart showing the operation of registering medium information in the server device 40 of the present embodiment.

  This operation is a process executed in conjunction with the culture medium information registration process executed by the image reading device 10, and the server device 40 picks up each captured image of each film type culture medium 60 transmitted from the image reading device 10. This is a process for registering information corresponding to each data in the culture medium information DB 401 as culture medium information.

  First, in the server device 40, when the communication control unit 410 receives one piece of medium information including one piece of captured image data transmitted from the image reading device 10 (step S300), the registration processing unit 421 uses the captured image data. Image analysis is performed on the configured captured image, the type ID 66 is specified, and provided to the image extraction processing unit 422 and the colony detection determination unit 423 (step S301).

  Next, the registration processing unit 421 acquires the medium ID based on the image analysis of the type ID 66 or the operator's instruction, or automatically assigns the medium ID (step S302).

  Next, the registration processing unit 421 specifies the culture examination time when registering the culture medium information in the captured image data based on the reading time included in the culture medium information (step S305). At this time, the registration processing unit 421 may be registered by designating the culture examination time for registering the culture medium information in each captured image data in conjunction with the image reading device 10 or included in the received culture medium information. The reading time may be registered as it is as the culture examination time.

  Next, the image extraction processing unit 422 searches the profile information DB 405 based on the type ID 66 specified by the registration processing unit 421, acquires template image data and arrangement structure information (step S306), and provides the provided template. Based on the image and the arrangement structure information, a medium image extraction process for extracting a medium image from the captured image is executed, and the extracted medium image is provided to the colony detection determination unit 423 (step S307). Details of the medium image extraction process in step S307 will be described later.

  Next, the colony detection determination unit 423 searches the profile information DB 405 based on the type ID 66 specified by the registration processing unit 421, acquires corresponding profile information (step S308), and acquires the acquired profile information and image extraction processing. Based on the medium image extracted in the unit 422, the color feature amount (luminance value) of each pixel of the medium image is extracted (step S310).

  Next, the colony detection determination unit 423 specifies a colony pixel in the medium image based on the provided profile information, and executes a colony detection process for detecting and counting colonies (step S321). Based on the profile information thus obtained, medium determination (pass / fail determination) is executed (step S322).

  Note that the colony detection determination unit 423, in the processing of step S321 and step S322, causes the display unit 160 of the image reading device 10 to display the detected number of colonies and the result of the pass / fail determination in conjunction with the image reading device 10, You may make a user browse and confirm.

  In addition, the colony detection determination unit 423 determines pass (normal) when the predetermined condition is satisfied in the medium determination of the process of step S322, and rejects when the predetermined condition is not satisfied. (Abnormal) is determined.

  Next, the registration processing unit 421 is included in captured image data, medium image data (hereinafter referred to as “medium image data”), type ID 66, medium ID, determined pass / fail, the number of colonies detected, and medium information. The culture start time information with the read time as the culture start time is registered in the culture medium information DB 401 (step S323), and this operation ends.

  The registration processing unit 421 displays line information and information to be registered on the display unit 160 of the image reading device 10 and registers medium information and the like based on a user instruction input via the image reading device 10. You may make it do.

  Moreover, although the above-mentioned operation | movement is performed when culture medium information is acquired by the image reading apparatus 10, the culture medium information acquired by the image reading apparatus 10 may be recorded on culture medium information DB401, and may be performed after that.

[A5.3] Medium image extraction processing (server device)
Next, operation | movement of the culture medium image extraction process performed in the server apparatus 40 of this embodiment is demonstrated using FIG. In addition, FIG. 11 is a flowchart which shows the operation | movement of the culture medium image extraction process performed in the server apparatus 40 of this embodiment.

  This operation is a process executed in the process of step S307 in the server device 40. In this operation, pixel coordinates are used as coordinates.

  First, when the image extraction processing unit 422 acquires the template image data and the arrangement structure information (step S306), the search area determination that determines the search area in which the culture medium image is assumed to be formed based on the arrangement structure information. Processing is executed (step S501).

  At this time, the image extraction processing unit 422 determines the scanning range of the center position of the template image in the determined search area. That is, the image extraction processing unit 422 sets the center coordinates (x, y) of the template image when the upper left pixel (upper left as viewed in the drawing) is aligned with the upper left pixel in the search range to (0, 0). And the center coordinate (x, y) of the template image when the lower right corner of the template image is aligned with the lower right pixel of the search range is set to (Xmax, Ymax).

  Next, the image extraction processing unit 422 starts scanning the template image in order from the upper right of the determined search area based on the acquired template image (step S502).

  First, the image extraction processing unit 422 identifies a region belonging to the second region on the captured image by superimposing the template image on the corresponding search region (step S503), and the color feature amount in the second region. Is calculated (step S504).

  Specifically, when specifying the second region immediately after the start of scanning, the image extraction processing unit 422 arranges the center coordinates (xi, yj) of the template image at (0, 0). Further, the image extraction processing unit 422 is a case where the process has shifted from the process of step S505, and in the previous process of step 503, the center coordinates of the template image are not (n, yj) (n, yj) (n = Xmax), that is, when the right end of the template image has not reached the right end of the search area, the center coordinate is scanned horizontally by one pixel from the position where the template image is superimposed in the processing of the previous step 503 (x The center of the template image is placed at (i + 1), yj). Further, the image extraction processing unit 422 has shifted from the process of step S505, and the center coordinate of the template image is (n, yj) is (n, yj) in the previous process of step 503. Scans one line vertically and places the center of the template image at the center coordinates (x, y) of (0, (yj + 1)).

  Next, the image extraction processing unit 422 determines whether scanning of the search area based on the template image is completed (Step S505). At this time, if the image extraction processing unit 422 determines that scanning of the search area based on the template image has not ended, the image extraction processing unit 422 proceeds to the process of step S503, and if determined to have ended, The process proceeds to step S506.

  For example, as the processing in step S505, the image extraction processing unit 422 ends scanning of the search area when the center coordinate of the template image is (n, m) (m = ymax) in the processing in step S503. It is determined that

  Next, the image extraction processing unit 422 has a predetermined position with respect to the color feature amount of the pixel belonging to the second region at all scanning positions (specifically, belongs to the second region at all scanning positions). (Scanning position where the integrated value of the color feature amount of the pixel is maximized) is determined (step S506), and a first region (mask portion) is formed when the template image is formed around the determined scanning position. Is determined to be a part of the culture medium image (step S507), and this operation is terminated.

  As described above, in the sanitary management system S of the present embodiment, since the medium image can be accurately extracted, the colonies can be accurately detected and counted, and the accurately detected colonies and the number thereof can be detected. The culture medium information including the information on can be registered in the database.

[B] Second Embodiment First, a second embodiment of the sanitary management system of the present invention will be described with reference to FIGS.

[B1] Hygiene management system In the first embodiment, the hygiene management system S performs template matching at each scanning position while performing horizontal scanning and vertical scanning within the search area for each pixel and for each row. Instead of a point, there is a feature in that an area with high accuracy as a medium portion is scanned efficiently in advance in the search area.

  Specifically, the sanitary management system S of the present embodiment, when scanning the selected template image within the determined search area, is adjacent to the scanning position where the template image is superimposed on the captured image and the scanning position. A medium image by scanning the template image while limiting the search area based on the relationship between the scanning position on the search area and the color feature amount of each pixel belonging to the second area of the template image on the captured image. Is extracted.

  In particular, the hygiene management system S of the present embodiment does not perform horizontal scanning and vertical scanning in the search area for each pixel and for each row, and matches based on the matching condition (match indicating whether the image is a medium portion image). In the search region, a region with high accuracy as a medium portion is limited in advance, and a range in which horizontal scanning or vertical scanning is performed is limited, and the template image is efficiently scanned. .

  The present embodiment has the same configuration as that of the first embodiment except for the above-described feature points, and the same members are denoted by the same reference numerals and description thereof is omitted. In the present embodiment, pixel coordinates are used as coordinates, as in the first embodiment.

[B2] Server Device Next, the server device 40 of the present embodiment will be described with reference to FIGS. FIG. 12 is a diagram for explaining template image scanning (skip scanning) of the present embodiment, and FIG. 13 is a diagram for explaining template image scanning (directional scanning) of the present embodiment. is there. FIG. 14 is a diagram for explaining scanning of the template image (horizontal simple scanning) according to the present embodiment, and FIG. 15 is a diagram illustrating a configuration of the server device 40 according to the present embodiment.

  The server device 40 according to the present embodiment uses a template image at a scan position in which a template image is superimposed on a captured image and a scan position on a search region adjacent to the scan position with reference to each scan position in horizontal scanning. The search area is limited based on the relationship between the color feature amounts of the pixels belonging to a predetermined area (specifically, the second area) on the captured image specified by the above.

  Specifically, as shown in FIG. 12, the server device 40 according to the present embodiment uses a matching degree in template matching (that is, a matching degree indicating whether the image is a medium portion image) or the first embodiment. Similarly, when it is determined that the degree of matching of the previous scanning position (coordinate P) is decreasing during the horizontal scanning while performing horizontal scanning and vertical scanning in the search area for each pixel and for each row, the scanning position ( The horizontal scanning is stopped at the coordinate Z), the vertical scanning (skipping) is performed on the next line, and the scanning for executing the horizontal scanning similarly for the next scanning line (hereinafter simply referred to as “skip scanning”) is performed. The search area is limited.

  On the other hand, the server device 40 replaces the skip scanning with a specific scanning position in which the template image is superimposed on the captured image, and a plurality of scanning positions on the search region adjacent to the specific scanning position. The search area is limited based on the relationship between the color feature amounts of the pixels belonging to the second area of the template image on the captured image.

  Specifically, the server device 40 according to the present embodiment uses the matching degree in template matching (that is, the matching degree indicating whether the image is a medium portion image) or the center position of the template image as shown in FIG. Based on the above, the degree of matching is calculated when the center position is moved from a predetermined scanning start position to an adjacent pixel position (specifically, 8 directions or 4 directions), and the direction with the highest degree of matching is calculated. The search area is limited by performing scanning that moves the scanning direction (hereinafter simply referred to as “directional scanning”).

  On the other hand, the server device 40 according to the present embodiment replaces the skip scanning or the directional scanning with a scanning position in which a template image is superimposed on a captured image in an arbitrary horizontal scanning line, and a search adjacent to the scanning position. Based on the relationship between the scanning position on the region and the color feature amount of each pixel belonging to the second region of the template image on the captured image, the range of the search region for performing horizontal scanning is specified, and the specified range The medium image is extracted by scanning the template image while restricting horizontal scanning.

  Specifically, as shown in FIG. 14, the server device 40 of the present embodiment uses a matching level in template matching (that is, a matching level indicating whether or not it is an image of a culture medium portion), as shown in FIG. A scan that scans a scan line and performs a scan along the vertical scan line while performing a horizontal scan with a predetermined amplitude for the vertical scan line at the scan position with the highest degree of matching (hereinafter simply referred to as “horizontal simple scan”). ”) And the extraction of the medium image is executed.

  Then, the server device 40 according to the present embodiment determines a scanning position having the maximum matching degree from the scanning positions in the skip scanning and the horizontal simple scanning, or matches the center position in the direction scanning. The scanning position when the degree is the highest is determined, and the first area (that is, the mask) of the template image at the determined scanning position is formed, that is, on the captured image centering on the determined scanning position. The template image is arranged, the area covered by the first area of the arranged template image is specified as a part of the medium image, and the image of the part is extracted as the medium image. .

  Specifically, the server device 40 includes a data processing unit 420 similar to that of the first embodiment, and by executing a predetermined program, in addition to each process of the first embodiment, skip scanning and direction scanning. And a medium extraction process based on any one of the horizontal simple scans. In particular, the data processing unit 420 uses the image extraction processing unit 422 to realize a medium extraction process based on any one of skip scanning, direction scanning, and horizontal simple scanning.

  The image extraction processing unit 422 performs matching conditions (captured image) except that the template image is subjected to image scanning by performing any one of skip scanning, direction scanning, and horizontal simple scanning in template matching. (The example of the degree of match based on the average value of each pixel color feature amount (in the case of the present embodiment, the luminance) belonging to the second region of the template image is included.) This is the same as each process in one embodiment.

[B3] Medium image extraction processing (server device)
Next, the culture medium image extraction process executed in the server device 40 of the present embodiment will be described with reference to FIGS. The culture medium image extraction process of the present embodiment is a process executed as a part of the same culture medium information registration process of the first embodiment, and the culture medium information registration executed in the image reading device 10 and the server device 40. The description of the processing is omitted.

[B3.1] Medium Image Extraction Process Based on Skip Scan First, the operation of the medium image extraction process based on the skip scan executed in the server device 40 of the present embodiment will be described with reference to FIG. FIG. 15 is a flowchart showing the operation of the medium image extraction process based on the skip scanning executed in the server device 40 of the present embodiment.

  In this operation, the degree of matching in template matching, that is, the degree of matching indicating whether or not the color feature amount of the pixel belonging to the second area in the template image superimposed on the captured image on the search area is the maximum value, The pixel color feature amount (total value) H in the second region is used, and a scanning position having a high average value in the pixel color feature amount (luminance in the present embodiment) H in the second region is used. A scanning position with a high degree of matching is used.

  First, when the image extraction processing unit 422 acquires the template image data and the arrangement structure information (step S306), the search area determination that determines the search area in which the culture medium image is assumed to be formed based on the arrangement structure information. Processing is executed (step S601).

  At this time, the image extraction processing unit 422 determines the scanning range of the center position of the template image in the determined search area. That is, the image extraction processing unit 422 sets the center coordinates (xi, yj) of the template image when the upper left pixel (upper left as viewed in the drawing) is aligned with the upper left pixel in the search range to (0, 0). And the center coordinate (xi, yj) of the template image when the lower right corner of the template image is aligned with the lower right pixel of the search range is set to (n, m) = (Xmax, Ymax).

  Next, the image extraction processing unit 422 performs initial setting (step S602). Specifically, the image extraction processing unit 422 sets the coordinates (xi, yj) in the search area for setting the center coordinates of the template image to (0, 0). Further, the image extraction processing unit 422 sets the reference color feature amount H0 to “0” and “z” to “1” in the color feature amount Hz.

  Next, the image extraction processing unit 422 sets the center coordinates of the template image to the coordinates (xi, yj) in the set search region, thereby superimposing the template image on the search region and performing the second on the captured image. An area belonging to the area is specified (step S611), and a color feature amount Hz (xi, yj) indicating an integrated value of each pixel of the captured image in the second area is calculated (step S612).

  Next, the image extraction processing unit 422 performs the color feature amount Hz (xi, yj) calculated in the process of step S612 and the color feature amount H (z-1) at the previous coordinate (x (i-1), yj). Alternatively, the color feature amount H (z−1) as an initial value is compared (step S613), and it is determined whether or not the color feature amount Hz tends to decrease (step S614).

  At this time, if the image extraction processing unit 422 determines that the color feature amount Hz does not tend to decrease, the image extraction processing unit 422 proceeds to the process of step S615, and if it determines that the color feature amount Hz tends to decrease, step S621. Move on to processing.

  Next, when the image extraction processing unit 422 determines that the color feature amount Hz does not tend to decrease, the x coordinate of the coordinates (xi, yj) in the search area where the center coordinate of the template image is set is set to “ When the coordinate obtained by adding “1” is specified as the coordinate serving as the center coordinate of the next template image (step S615), the process returns to step S611, and it is determined that the x coordinate is the maximum value (n). Moves to step S621.

  Next, when the image extraction processing unit 422 determines that the color feature amount Hz is decreasing, the image feature processing unit 422 uses the color feature amount H (z−1) when the center coordinate of the template image is the immediately preceding coordinate as the scan line. The maximum color feature amount Hzmax (yj) of (yj) is stored in the ROM / RAM 440 together with the coordinates (x (i−1), yj) (step S621).

  In particular, as illustrated in FIG. 12, when the image extraction processing unit 422 determines that the color feature amount Hz is in a decreasing tendency, the image extraction processing unit 422 calculates the color feature amount Hz (xi, yj) in the process of step S613. The color feature amount H (z−1) of the immediately preceding coordinate V is stored in the ROM / RAM 440 together with the coordinate V (x (i−1), yj) as the maximum color feature amount Hzmax (yj).

  Next, the image extraction processing unit 422 determines whether or not the y coordinate is the maximum value (n) among the coordinates (xi, yj) in the search region where the center coordinate of the template image is set (step S622). ).

  At this time, if the image extraction processing unit 422 determines that the y coordinate of the coordinates (xi, yj) in the search area where the center coordinate of the template image is set is not the maximum value (n), the image extraction processing unit 422 sets the y coordinate to the y coordinate. “1” is added, the x coordinate is set to “0”, a new coordinate (0, y (j + 1)) is set, and “z” is set to “1” (step S623). Returning to the process of step S611, if it is determined that the y coordinate is the maximum value (m), the process proceeds to step S631.

  Next, the image extraction processing unit 422 uses the coordinates (xi, yj) having the maximum color feature value among the color feature values Hzmax (y: y is “0” to “m”) stored in the ROM / RAM 440. ) Is determined as a scanning position that matches the matching condition (step S631).

  Finally, the image extraction processing unit 422 identifies the first region of the template image formed at the determined scanning position as a part of the culture medium image (step S632), and ends this operation.

  In this operation, the horizontal scanning and the vertical scanning when scanning the template image may be reversed. In other words, the image extraction processing unit 422 performs horizontal scanning at the time when the color feature amount shows a decreasing tendency while performing vertical scanning, and performs the next vertical scanning, and when the X coordinate becomes the maximum value. In addition, the scanning position of the template image formed on the basis of the coordinates (xi, yj) having the maximum color characteristic amount among the color characteristic amounts Hzmax (xi) is determined as the scanning position that matches the matching condition. May be.

[B3.2] Medium Image Extraction Process Based on Direction Scan Next, the operation of the medium image extraction process based on the direction scan executed in the server device 40 of the present embodiment will be described with reference to FIG. FIG. 16 is a flowchart showing the operation of the culture medium image extraction process based on the direction scanning executed in the server device 40 of the present embodiment.

  In this operation, the degree of matching in template matching, that is, the degree of matching indicating whether or not the color feature amount of the pixel belonging to the second area in the template image superimposed on the captured image on the search area is the maximum value, The pixel color feature amount (total value) H in the second region is used, and a scanning position having a high average value in the pixel color feature amount (luminance in the present embodiment) H in the second region is used. A scanning position with a high degree of matching is used.

  First, when the image extraction processing unit 422 acquires the template image data and the arrangement structure information (step S306), the search area determination that determines the search area in which the culture medium image is assumed to be formed based on the arrangement structure information. Processing is executed (step S701).

  At this time, the image extraction processing unit 422 determines the scanning range of the center position of the template image in the determined search area. That is, the image extraction processing unit 422 sets the center coordinates (xi, yj) of the template image when the upper left pixel (upper left as viewed in the drawing) is aligned with the upper left pixel in the search range to (0, 0). And the center coordinate (xi, yj) of the template image when the lower right corner of the template image is aligned with the lower right pixel of the search range is set to (n, m) = (Xmax, Ymax).

  Next, the image extraction processing unit 422 performs initial setting (step S702). Specifically, the image extraction processing unit 422 sets the center coordinates of the culture medium portion in the search region and the expected reference coordinates (xi, yj) based on the arrangement structure information. For example, as illustrated in FIG. 13, the image extraction processing unit 422 sets center coordinates P that are reference coordinates.

  Next, the image extraction processing unit 422 sets the center coordinates of the template image to the coordinates (xi, yj) in the set search region, thereby superimposing the template image on the search region and performing the second on the captured image. An area belonging to the area is specified (step S711), and a color feature value Ha (xi, yj) indicating an integrated value of each pixel of the captured image in the second area is calculated (step S712).

  Next, the image extraction processing unit 422 includes eight-direction coordinates (x (i−1), y (j−1)), (x (i−−) adjacent to the coordinates (xi, yj) in the set search region. 1), yj), (xi-1, y (j + 1)), (xi, y (j-1)), (xi, y (j + 1)), (x (i + 1), y (j-1)) , (X (i + 1), yj) and (x (i + 1), y (j + 1)), the template image is superimposed on the search region by setting the center coordinates of the template image, An area belonging to the second area is specified (step S713), and color feature amounts H1 to H8 indicating the integrated values of the pixels of the captured image in the second area are calculated (step S714).

  For example, as illustrated in FIG. 13, the image extraction processing unit 422 sets the center coordinates of the template image for each of the eight directions based on the center coordinates P serving as the reference coordinates. In FIG. 13, only the coordinates of (x (i−1), y (j + 1)) and the coordinates of (x (i + 1), y (i−1)) are displayed.

Next, the image extraction processing unit 422 compares the color feature value Ha (xi, yj) with each of the color feature values H1 to H8 (step S715), and increases from the color feature value Ha (xi, yj). The presence / absence of existing color feature amounts H1 to H8 is determined (step S716). That is, the image extraction processing unit 422 includes the color feature amount Ha (xi, yj),
(1) Compare the color feature amount H1 of the coordinates (x (i-1), y (j-1)),
(2) Compare the color feature amount H2 of the coordinates (x (i-1), yj),
(3) Compare the color feature amount H3 of the coordinates (x (i-1), y (j + 1)),
(4) Compare with the color feature amount H4 of the coordinates (xi, y-1),
(5) The color feature amount H5 at the coordinates (xi, y + 1) is compared,
(6) Compare the color feature quantity H6 of the coordinates (x (i + 1), (j-1)),
(7) Compare the color feature amount H7 of the coordinates (x (i + 1), yj), and
(8) Compare the color feature amount H8 of the coordinates (x (i + 1), y (j + 1)),
The presence / absence of color feature amounts H1 to H8 that increase from the color feature amount Ha (xi, yj) is determined (step S715).

  At this time, if the image extraction processing unit 422 determines that there is at least one color feature amount H1 to H8 that has increased from the color feature amount Ha (xi, yj), the process proceeds to step S717. If it is determined that none of the increasing color feature amounts H1 to H8 is present, the process proceeds to step S721.

  Next, when the image extraction processing unit 422 determines that there is at least one color feature amount H1 to H8 that has increased from the color feature amount Ha (xi, yj), any of the color feature amounts H1 to H8 is determined. In this case, the coordinates of the color feature amount that is larger than the color feature amount Ha (xi, yj) and has the largest difference from the color feature amount Ha (xi, yj) The central coordinates are set (step S717), and the process returns to step S711. For example, as illustrated in FIG. 13, the image extraction processing unit 422 scans the template image in the direction in which the difference from the color feature amount Ha (xi, yj) is the maximum among the eight directions indicated by the arrows.

  Next, when the image extraction processing unit 422 determines that none of the color feature values H1 to H8 has increased from the color feature value Ha (xi, yj), the center coordinates (xi, yj) in step S711. Is determined to be a scanning position that matches the matching condition (step S721).

  Finally, the image extraction processing unit 422 identifies the area where the first area of the template image on the captured image arranged at the determined scanning position is a part of the culture medium image (step S722), End the operation.

[B3.3] Medium Image Extraction Process Based on Horizontal Simple Scan Next, the operation of the medium image extraction process based on the horizontal simple scan executed in the server device 40 of the present embodiment will be described with reference to FIGS. 17 and 18. . FIGS. 17 and 18 are flowcharts showing the operation of the medium image extraction process based on the horizontal simple scan executed in the server device 40 of the present embodiment.

  In this operation, the degree of matching in template matching, that is, the degree of matching indicating whether or not the color feature amount of the pixel belonging to the second area in the template image superimposed on the captured image on the search area is the maximum value, The pixel color feature amount (total value) H in the second region is used, and a scanning position having a high average value in the pixel color feature amount (luminance in the present embodiment) H in the second region is used. A scanning position with a high degree of matching is used.

  First, when the image extraction processing unit 422 acquires the template image data and the arrangement structure information (step S306), the search area determination that determines the search area in which the culture medium image is assumed to be formed based on the arrangement structure information. Processing is executed (step S801).

  At this time, the image extraction processing unit 422 determines the scanning range of the center position of the template image in the determined search area. That is, the image extraction processing unit 422 sets the center coordinates (xi, yj) of the template image when the upper left pixel (upper left as viewed in the drawing) is aligned with the upper left pixel in the search range to (0, 0). And the center coordinate (x, y) of the template image when the lower right corner of the template image is aligned with the lower right pixel of the search range is set to (n, m) = (Xmax, Ymax).

  Next, the image extraction processing unit 422 performs initial setting (step S802). For example, as illustrated in FIG. 14, the image extraction processing unit 422 sets a scanning line (Yb) that is expected to scan the center coordinates of the culture medium portion in the search region, based on the arrangement structure information, and the template image. The coordinates (xi, yj) in the search area for setting the center coordinates of are set to the start coordinates (0, Yb) of the scanning line. Further, the image extraction processing unit 422 sets the reference color feature amount H0 to “0” and “z” to “1” in the color feature amount Hz.

  Next, the image extraction processing unit 422 sets the center coordinates of the template image to the coordinates (xi, Yb) in the set search area, thereby superimposing the template image on the search area and performing the second on the captured image. An area belonging to the area is specified (step S811), and a color feature amount Hz (xi, Yb) indicating an integrated value of each pixel of the captured image in the second area is calculated (step S812). At this time, the image extraction processing unit 422 stores the calculated color feature amount Hz (xi, Yb) in the ROM / RAM 440.

  Next, the image extraction processing unit 422 determines whether or not the x coordinate is the maximum value (n) (step S813). At this time, if the image extraction processing unit 422 determines that the x coordinate of the coordinates (xi, Yb) in the search region where the center coordinate of the template image is set is not the maximum value (n), Of the coordinates (xi, Yb) in the search area where the center coordinates are set, the coordinates obtained by adding “1” to the x coordinates are specified as the coordinates that become the center coordinates of the next template image (step S814). Returning to the process of step S811, if it is determined that the x-coordinate is the maximum value (n), the process proceeds to step S821.

  Next, when the image extraction processing unit 422 determines that the x coordinate of the coordinates (xi, Yb) in the search area where the center coordinates of the template image are set is the maximum value (n), for example, FIG. As shown in FIG. 14, the x coordinate (Xb) of the coordinate B when the maximum value of each color feature amount Hz in the scanning line calculated in step S812 is specified (step S821).

  Next, the image extraction processing unit 422 is based on the resolution of the medium image and the arrangement structure information (a range in which the center of the medium in the medium image is shifted, for example, 4 mm) based on the coordinates (Xb, Yb) in the specified search region. Then, the x coordinate range (± R pixel, for example, R = 10 to 20) that covers the range in which the center of the medium is shifted in the medium image is determined as the range for performing horizontal simple scanning (step width) (step S822). .

  Next, the image extraction processing unit 422 sets the coordinates (xi, yj) in the search area where the center coordinates of the template image are set to the start coordinates (Xb-R pixel, 0) of the horizontal simple scanning line (step S823). . As shown in FIG. 14, “−R” to “+ R” indicate an x-coordinate range (swing width) that covers a range in which the center of the medium in the medium image is shifted.

  Next, the image extraction processing unit 422 sets the center coordinates of the template image to the coordinates (xi, yj) in the set search region, thereby superimposing the template image on the search region and performing the second on the captured image. An area belonging to the area is specified (step S824), and a color feature amount Hz (x, y) in the composite image is calculated (step S825). At this time, the image extraction processing unit 422 stores a color feature amount Hz (xi, yj) indicating the integrated value of each pixel of the captured image in the second region in the ROM / RAM 440.

  Next, the image extraction processing unit 422 determines whether or not the x coordinate is the maximum value (Xb + R) of the cover range (step S826). At this time, if the image extraction processing unit 422 determines that the x coordinate of the coordinates (xi, yj) in the search area where the center coordinate of the template image is set is not the maximum value (Xb + R) of the cover range, Of the coordinates (xi, yj) in the search area where the center coordinate of the template image is set, the coordinate obtained by adding “1” to the x coordinate is specified as the coordinate serving as the center coordinate of the next template image (step S827), the process returns to step S824, and if it is determined that the x coordinate is the maximum value (Xb + R) of the cover range, the process proceeds to step S831.

  Next, when the image extraction processing unit 422 determines that the x coordinate of the coordinates (xi, yj) in the search area where the center coordinates of the template image are set is the maximum value (Xb + R) of the cover range, It is determined whether or not the y coordinate of the coordinates (xi, yj) in the search area is the maximum value (Ymax) (step S831).

  At this time, if the image extraction processing unit 422 determines that the y coordinate of the coordinates (xi, yj) in the search region where the center coordinate of the template image is set is not the maximum value (Ymax), the image extraction processing unit 422 uses the y coordinate. "1" is added to set a new coordinate (xi, yj) (step S832), and the process returns to step S824, and if it is determined that the y coordinate is the maximum value (Ymax), step The process shifts to S833.

  Next, the image extraction processing unit 422 performs scanning in which the center of the template image is set at coordinates (xi, yj) having the color feature amount Hzmax of the maximum value among the color feature amounts Hz stored in the ROM / RAM 440. The position is determined to be a scanning position that matches the matching condition (step S833).

  Finally, the image extraction processing unit 422 identifies the first region of the template image formed at the determined scanning position as a part of the culture medium image (Step S834), and ends this operation.

  In this operation, the horizontal scanning and the vertical scanning when scanning the template image may be reversed. That is, the image extraction processing unit 422 scans the template image with respect to any of the vertical scanning lines, determines the x coordinate of the coordinate having the maximum color feature amount, sets the swing width with reference to the x coordinate, and performs vertical simple processing. A scan may be performed.

  As described above, in the sanitary management system S of the present embodiment, in addition to the effects of the first embodiment, the scanning time for performing template matching can be greatly reduced, so that the medium image can be quickly and efficiently obtained. Can be extracted.

[C] Modification [C1] Modification 1
In each of the above-described embodiments, instead of performing the medium information registration process using the image reading device 10 such as an image scanner alone, the laptop or desktop personal computer and the image reading device of the image scanner Or you may implement | achieve the registration process of the said culture medium information by image input devices, such as a personal computer and a tablet-type information terminal device, a digital camera, or a smart phone.

  In this case, the personal computer and the image input device may be connected according to a predetermined communication standard, and the registration process of the culture medium information may be realized by using the personal computer and the image input device integrally, or a memory card. The captured image data previously acquired by the image input device may be stored in another physical memory, and the registration process of the culture medium information may be realized while the stored image data is captured by a personal computer.

  In this case, the functions of the server device 40 may be provided in a personal computer to execute the various processes described above.

[C2] Modification 2
In each of the embodiments described above, the image reading device 10, the administrator terminal device 20, and the server device 40 may be installed or used in the same site, or each may be installed in a remote place such as outside the country, or remotely It may be used on the ground and each process described above may be executed.

  The server device 40 may read and record the medium information and other information using a database connected via a network, or may be formed by one or a plurality of devices.

[C3] Modification 3
The colony detection determination unit 423 of each of the above embodiments determines a color feature amount, an inspection method, a parameter, a threshold value, and the like for specifying a colony pixel based on profile information. In the process, under a predetermined condition such as a colony not detected, a re-detection process for performing colony detection again with higher sensitivity for detecting colony pixels or colonies themselves may be added. .

  For example, the execution of the re-detection process may be defined in the profile information in the case of a specific bacterium that fails the pass / fail determination of the medium determination when even one colony such as Escherichia coli or coliform is detected. .

  Note that the profile information is not limited to the re-detection process, and is a notification for notifying the administrator of a specific bacterium under a predetermined condition such as failure of the medium determination in the case of specific bacteria. Execution of various processes such as processes may be defined.

[C4] Modification 4
In each of the above-described embodiments, registration of medium information in a specimen cultured on the film-type medium 60 using the film-type medium 60 is performed, but the bacteria cultured on the agar medium using the agar medium In addition, registration of medium information on the specimen may be executed.

  In this case, each agar medium is imaged by using a portable communication terminal device with an imaging function such as a digital camera, a smart phone, or a mobile phone instead of the image reading device 10, and captured image data is generated.

[C5] Modification 5
In the first embodiment, the template image is raster scanned.
(1) A scanning method in which horizontal scanning is started for each row from the lower right in the search region, and vertical scanning is performed in the positive direction of the y-axis (upward in the drawing in FIG. 6).
(2) a scanning method in which vertical scanning is started for each column from the upper left in the search region and horizontal scanning is performed in the positive direction of the x axis (right direction in FIG. 6), or
(3) a scanning method for randomly scanning the scanning position;
To execute template matching.

  In other words, any scanning method may be used as long as matching scores at all scanning positions in the search region can be calculated.

[C6] Modification 6
In the first embodiment and the second embodiment, as illustrated in FIG. 19, a weighting region 79 is provided in the central region (that is, the first region 71 side) of the second region 72 of the template image, and template matching is performed. When calculating the matching score indicating the integrated value of the color feature amount of the pixel belonging to the second region 72 on the captured image, the color feature amount of the pixel in the weight region 79 is adjusted using a predetermined weight. Also good.

  That is, on the captured image, the image extraction processing unit 422 accumulates the color feature amounts of the pixels adjusted by weighting for pixels belonging to the weighted region 79, and for pixels belonging to the second region 72 other than the weighted region 79. Further, the matching score may be calculated by performing integration based on the color feature amount of each pixel.

  For example, the image extraction processing unit 422 calculates the luminance value I (w) ij of the pixel belonging to the weighted region 79 based on the following (Expression 6), and the second region other than the weighted region 79 is calculated according to (Expression 7). The matching score of the average value Z (av) of the sum value is calculated by the sum value Z (s) with the luminance value I (p) ij of the pixel belonging to 72 or the above (Formula 5). “W” indicates a weighting coefficient.

  Usually, in the film-type culture medium 60, the background color part and the culture medium part are not strictly divided into regions because of its structure. Therefore, for example, in the template matching process such as when the color of the medium portion is close to the background color, noise may occur at the boundary between the first region and the second region, and the matching score may not be calculated accurately.

  Therefore, in this modification, in order to more accurately extract the culture medium image from the captured image, the color feature amount (luminance value) of each pixel is used in the boundary region between the first region and the second region using weighting. By adjusting (that is, executing an operation for decreasing the value), an accurate matching score can be calculated and a medium portion can be extracted accurately.

[C7] Modification 7
In the above modified example 6, the weighting area is provided in the area on the center side of the second area of the template image (that is, the first area side) to calculate the matching score. However, each pixel belonging to the second area is calculated. Each of the color feature amounts may be adjusted using weighting.

S ... Hygiene management system 10 ... Image reading device 20 ... Administrator terminal device 30 ... Network 40 ... Server device 60 ... Film type culture medium 80 ... Work line 100 ... Data storage unit 101 ... Application storage unit 102 ... Image data storage unit 110 ... Image data generation unit 120 ... application control unit 130 ... network communication unit 140 ... ADF unit 160 ... display unit 170 ... operation unit 180 ... timer 190 ... terminal management control unit 400 ... database 401 ... medium information DB
404 ... Work management DB
405 ... Profile information DB
410 ... Communication control unit 420 ... Data processing unit 421 ... Registration processing unit 422 ... Image extraction processing unit 423 ... Colony detection determination unit 430 ... Server management control unit

Claims (16)

An acquisition means for acquiring captured image data as captured image data including a culture medium image in which a culture medium in which a specimen is placed and a colony is cultured is imaged;
Scanning on the captured image based on a template image for specifying the shape of the culture medium, and specifying an area having a predetermined condition as an image portion of the culture medium image, from among the captured images Image extracting means for extracting an image portion of the medium image;
Detecting means for performing image analysis on the extracted medium image, identifying a colony pixel used for imaging the colony cultured in the medium, and detecting the colony;
With
The template image includes a first region having the same shape as the medium to be extracted;
The colony detection system, wherein the first region is a mask region having a characteristic of invalidating a color feature amount of each pixel of the captured image when the first region is superimposed on the captured image. . In the colony detection system according to claim 1,
The template image further includes a second region having a predetermined shape on the periphery of the first region,
The second region is a region having characteristics that enable the color feature amount of each pixel of the captured image when the first region is superimposed on the captured image,
The image extracting means;
While masking the region superimposed on the first region at each scanning position, based on the color feature amount of each pixel belonging to the region superimposed on the second region, the medium image is selected from the captured images. A colony detection system characterized by extracting an image portion. In the colony detection system according to claim 2,
The template image includes a third region formed at an outer edge of the second region;
The colony detection system, wherein the third region is a mask region having a characteristic of invalidating a color feature amount of each pixel of the captured image when the third region is superimposed on the captured image. In the colony detection system of any one of Claims 1-3,
The acquisition means acquires type information indicating the type of the medium,
The colony detection system, wherein the image extraction unit selects one template image from the plurality of template images based on the acquired type information, and extracts the culture medium image using the selected template image . In the colony detection system according to claim 4,
The image extracting means determines a search area where the medium image is assumed to be formed from the captured image based on the type information, and scans the selected template image within the determined search area. A colony detection system that extracts the medium image by In the colony detection system of any one of Claims 1-3,
The acquisition means acquires type information indicating the type of the medium,
The image extraction means determines a search area where the culture medium image is assumed to be formed from the captured image based on the type information, and scans the template image within the determined search area. Colony detection system that extracts media images. In the colony detection system of any one of Claims 1-6,
The detecting means;
Feature quantity extraction means for extracting color feature quantities relating to the colors of pixels constituting the extracted medium image;
Identifying means for identifying a colony pixel used to image the colony based on the color feature of each extracted pixel;
Based on the identified colony pixels, colony detection means for detecting the colonies cultured in the medium;
A colony detection system comprising: In the colony detection system according to claim 7,
The acquisition means acquires type information indicating the type of the medium,
The colony detection system in which the specifying unit specifies the colony pixel based on the acquired type information together with the extracted color feature amount of each pixel. In the colony detection system according to claim 7 or 8,
The acquisition means acquires type information indicating the type of the medium,
The colony detection system in which the colony detection unit detects the colonies cultured in the medium based on the acquired type information together with the identified colony pixels. In the colony detection system of any one of Claims 1-9,
A colony detection system further comprising registration means for registering information on the detected colonies as culture medium information in a database. Computer
An acquisition means for acquiring captured image data as captured image data including a culture medium image in which a culture medium in which a specimen is placed and a colony is cultured is imaged;
Scanning on the captured image based on a template image for specifying the shape of the culture medium, and specifying an area having a predetermined condition as an image portion of the culture medium image, from among the captured images Image extracting means for extracting an image portion of the medium image, and performing image analysis on the extracted medium image to identify a colony pixel used for imaging the colonies cultured in the medium Detection means for detecting the colony;
The template image includes a first region having the same shape as the medium to be extracted,
The program according to claim 1, wherein the first area is a mask area having a characteristic of invalidating a color feature amount of each pixel of the captured image when the first area is superimposed on the captured image. The program according to claim 11,
The template image further includes a second region having a predetermined shape on the periphery of the first region,
The second region is a region having characteristics that enable the color feature amount of each pixel of the captured image when the first region is superimposed on the captured image,
The image extracting means;
While masking the region superimposed on the first region at each scanning position, based on the color feature amount of each pixel belonging to the region superimposed on the second region, the medium image is selected from the captured images. A program that extracts image parts. The program according to claim 12,
The template image includes a third region formed at an outer edge of the second region;
The program, wherein the third region is a region having a characteristic of invalidating a color feature amount of each pixel of the captured image when the third region is superimposed on the captured image. An acquisition step of acquiring, as captured image data, data of a captured image including a culture medium image on which a culture medium for placing a specimen and culturing colonies is imaged;
Scanning on the captured image based on a template image for specifying the shape of the culture medium, and specifying an area having a predetermined condition as an image portion of the culture medium image, from among the captured images An image extraction step of extracting an image portion of the medium image;
A detection step of performing image analysis on the extracted medium image, identifying a colony pixel used for imaging the colony cultured in the medium, and detecting the colony;
Including
The template image includes a first region having the same shape as the medium to be extracted;
The colony detection method, wherein the first region is a mask region having a characteristic of invalidating a color feature amount of each pixel of the captured image when the first region is superimposed on the captured image. . In the colony detection method according to claim 14,
Before SL template image, further comprises a second region having a predetermined shape on the periphery of the first region,
The second region is a region having characteristics that enable the color feature amount of each pixel of the captured image when the first region is superimposed on the captured image,
In the image extraction step ,
While masking the region superimposed on the first region at each scanning position, based on the color feature amount of each pixel belonging to the region superimposed on the second region, the medium image is selected from the captured images. A colony detection method for extracting an image portion. In the colony detection method of Claim 15,
The template image includes a third region formed at an outer edge of the second region;
The colony detection method, wherein the third region is a region having a characteristic of invalidating a color feature amount of each pixel of the captured image when the third region is superimposed on the captured image.