CN111240004A - System and method for automatically identifying two insects by microscope - Google Patents
System and method for automatically identifying two insects by microscope Download PDFInfo
- Publication number
- CN111240004A CN111240004A CN202010165944.XA CN202010165944A CN111240004A CN 111240004 A CN111240004 A CN 111240004A CN 202010165944 A CN202010165944 A CN 202010165944A CN 111240004 A CN111240004 A CN 111240004A
- Authority
- CN
- China
- Prior art keywords
- microscope
- picture
- program
- software
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12Q—MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
- C12Q1/00—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
- C12Q1/02—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving viable microorganisms
- C12Q1/04—Determining presence or kind of microorganism; Use of selective media for testing antibiotics or bacteriocides; Compositions containing a chemical indicator therefor
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/361—Optical details, e.g. image relay to the camera or image sensor
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
Abstract
The invention discloses an automatic amphibian recognition system and method for a microscope, the automatic amphibian recognition system for the microscope comprises a fluorescence microscope and a CCD camera, the fluorescence microscope is provided with a fluorescence power supply, the control system comprises a motion control card and a motion controller, the data processing system is a computer, a control program and a recognition program run on the computer, the fluorescence microscope is provided with the CCD camera, the CCD camera is connected with the computer, the computer and the motion controller are both connected with the motion control card, the motion control card is connected with the fluorescence microscope, and the control program is used for collecting data of the CCD camera. Has the advantages that: the detection efficiency is greatly improved, the automatic identification system does not need professional operation, the detection threshold is greatly reduced, the machine identification accuracy can be continuously improved through continuous training and upgrading of the identification algorithm, the problem of large human error is thoroughly solved, and the two-insect detection is more concise and efficient.
Description
Technical Field
The invention relates to the field of giardia and cryptosporidium identification, in particular to a microscope system and a microscope method for automatically identifying two worms.
Background
Giardia and cryptosporidium (abbreviated as 'two worms') are pathogenic microorganisms widely existing in nature, and the detection methods of the two worms in China comprise an immunomagnetic separation fluorescent antibody method in the national standard 'domestic drinking water standard inspection method' (GB5750-2006) and a filter membrane concentration/density gradient separation fluorescent antibody method and a filter capsule concentration/density gradient separation fluorescent antibody method in the industry standard 'town water quality standard inspection method' (CJT 141-2008). The three methods all comprise a process of identifying the two insects by a fluorescent microscope, the process completely depends on manual identification at present, the requirement on the professional level of an identifier is high, and a detector can accurately identify the two insects only after long-term training. In the identification process, the detection personnel need to stare at the microscope for a long time, so that eye fatigue is easily caused, identification errors are caused, the accuracy is low, in addition, manual identification cannot be continuously operated for a long time, and the effective identification time causes the reduction of the whole detection working efficiency.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a system and a method for automatically identifying two insects by a microscope, so as to overcome the technical problems in the prior related art.
The technical scheme of the invention is realized as follows:
according to one aspect of the invention, a microscope automatic identification two-insect system is provided.
The microscope automatic two-insect recognition system comprises an imaging system, a control system and a data processing system, the imaging system includes a fluorescence microscope configured with a fluorescence power source and a CCD camera, the control system comprises a motion control card and a motion controller, the data processing system is a computer, the computer runs a control program and an identification program, the fluorescence microscope is provided with a CCD camera, the CCD camera is connected with the computer, the computer and the motion controller are both connected with the motion control card, the motion control card is connected with a fluorescence microscope, the control program is used for collecting the CCD camera data, the control program is used for controlling the movement of the objective lens and the objective table of the fluorescence microscope, and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
Further, the CCD camera is connected with the computer through a USB data line.
Furthermore, the motion control card is connected with the computer through a USB data line.
Furthermore, the objective lens and the objective table of the fluorescence microscope are both connected with the motion control card.
According to another aspect of the present invention, a method for a microscope to automatically identify a two-insect system is provided.
The microscope automatic identification system for the two insects comprises the following steps:
opening a fluorescence microscope switch, a fluorescence power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
setting a port number com of a USB line of the motion control card, wherein a specific checking path is as follows: this computer-property-device manager-port;
the software end clicks the 'connection microscope', when the connection is successful, the interface pops up a 'connection success' word, a 'confirmation' dialog box is clicked to disappear, and then a picture storage path is set;
observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the lower right corner position of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
before scanning is started, software can detect the multiple of the objective lens used at present, if the multiple is not 20 times of the objective lens, the objective lens is automatically switched into the 20 times of the objective lens, snake-shaped scanning is started from a starting point position defined by a user, the CCD shoots at the same time, the shot pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all working software can be automatically completed;
after scanning under the 20-time objective lens is finished, the software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like is carried out until photographing of the Nth coordinate is finished;
after the 40-time objective lens is photographed, the software can recognize the photographed picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is automatically completed;
the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to quickly move to the coordinate position, and the user can verify the result through the eyepiece.
Further, the image recognition program is developed based on the python language, and the software used includes python, pytorech, scimit-spare, numpy, and opencv-python.
Further, after the 20 times objective lens finishes scanning, the software can identify the scanned picture, the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, according to the characteristic, the program takes the green channel value in the RGB three-color channel as the judgment basis, and the suspected region selection algorithm is formulated as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
Further, the coordinate point determination specifically includes: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
After the shooting under the 40-time objective lens is completed, the software can recognize the shot picture again, the recognition program reads the high-magnification picture, the two insects are recognized by adopting an image classification algorithm based on a convolutional neural network, the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, the output is a one-dimensional array which comprises three data, and the three data respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-amphibian. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
The invention has the beneficial effects that: the automatic identification speed of present solitary two worm slides is about 60 minutes/piece, and is equivalent with skilled detection personnel naked eye identification speed, but automatic identification system can continuous automatic work, does not have the tired problem that needs the rest of eye, consequently detection efficiency improves greatly, and automatic identification system does not need professional's operation, greatly reduced detects the threshold, through the continuous training and the upgrading of recognition algorithm, machine identification rate of accuracy can constantly improve, thoroughly solved the big problem of human error, two worm detects and will become more succinct high-efficient.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram of the general architecture of an automatic two-insect recognition system for a microscope in accordance with the present invention;
FIG. 2 is an isometric view of a fluorescent power supply;
FIG. 3 is an isometric view of the fluorescence microscope according to FIG. 1;
FIG. 4 is an isometric view of the motion controller of FIG. 1;
FIG. 5 is a flow chart of a method for using a microscope system for automatically identifying two insects, in accordance with an embodiment of the present invention;
FIG. 6 is a table of software and development kit version information used by an automatic two-bug identification system for a microscope according to an embodiment of the invention;
FIG. 7 is a flowchart of a suspected area selection algorithm in an automatic microscope identification system according to an embodiment of the present invention;
fig. 8 is a parameter table of a convolutional neural network structure in a microscope automatic recognition two-insect system according to an embodiment of the invention.
In the figure:
1. a fluorescence microscope; 2. a CCD camera; 3. a computer; 4. a motion control card; 5. a motion controller; 6. a fluorescent power supply; 7. an object stage; 8. an eyepiece.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
According to an embodiment of the invention, a microscope automatic identification two-insect system is provided.
As shown in fig. 1-4, the microscope system for automatically identifying two insects comprises an imaging system, a control system and a data processing system, wherein the imaging system comprises a fluorescence microscope (1) and a CCD camera (2), the fluorescence microscope (1) is provided with a fluorescence power supply (6), the control system comprises a motion control card (4) and a motion controller (5), the data processing system is a computer (3), the computer (3) runs a control program and an identification program, the fluorescence microscope (1) is provided with the CCD camera (2), the CCD camera (2) is connected with the computer (3), the computer (3) and the motion controller (5) are both connected with the motion control card (4), the motion control card (4) is connected with the fluorescence microscope (1), and the control program is used for collecting data of the CCD camera (2), the control program is used for controlling the movement of an objective lens and an objective table (7) of the fluorescence microscope (1), and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
Wherein, the CCD camera (2) is connected with the computer (3) through a USB data line.
The motion control card (4) is connected with the computer (3) through a USB data line.
The objective lens and the objective table (7) of the fluorescence microscope (1) are both connected with the motion control card (4).
For the convenience of understanding the above technical solution of the present invention, the following detailed description is made on the flow of the above solution of the present invention with reference to the accompanying drawings, and specifically is as follows:
according to the embodiment of the invention, a method for automatically identifying two insect systems by a microscope is also provided.
As shown in fig. 5-8, in actual use, the microscope automatically recognizes the use of the two-worm system, comprising the steps of:
step S101, turning on a fluorescent microscope switch, a fluorescent power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
step S103, setting a com port number of the USB line of the motion control card, wherein the specific checking path is as follows: this computer-property-device manager-port;
step S105, clicking a 'connection microscope' on the software end, popping up a 'connection success' word on the interface after the connection is successful, clicking a 'confirmation' dialog box to disappear, and then setting a picture storage path;
step S107, observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the position of the lower right corner point of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
step S109, before scanning, software detects the multiple of the objective lens used at present, if the multiple is not 20 times, the objective lens is automatically switched to be 20 times, then serpentine scanning is started from the position of a starting point defined by a user, the CCD takes pictures at the same time, the taken pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all the working software can be automatically completed;
step S111, after the 20-time objective lens is scanned, software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like are carried out until the photographing of the Nth coordinate is completed;
step S113, after the shooting under the 40-time objective lens is finished, the software can recognize the shot picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is finished automatically;
step S115, the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to rapidly move to the coordinate position, and the user can verify the result through the eyepiece.
In one embodiment, the image recognition program is developed based on the python language, and the software used includes python, pytorech, scimit-spare, numpy, and opencv-python, see FIG. 6 in particular.
In an embodiment, after the 20 times objective lens finishes scanning, the software identifies the scanned picture, the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, according to the characteristic, the program uses the green channel value in the RGB three-color channel as the judgment basis, and makes the suspected region selection algorithm as shown in fig. 7, which is as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
In one embodiment, the coordinate point determination is specifically: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
After the shooting under the 40-time objective lens is completed, the software can recognize the shot picture again, the recognition program reads the high-magnification picture, the two insects are recognized by adopting an image classification algorithm based on a convolutional neural network, the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, the output is a one-dimensional array which comprises three data, and the three data respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-amphibian. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
For the convenience of understanding the technical solutions of the present invention, the following detailed description will be made on the working principle or the operation mode of the present invention in the practical process.
In practical application, a sample slide to be identified is placed on a fluorescence microscope objective table, a motion controller controls the fluorescence microscope objective table to move in three axes through a motion control card, a diagonal line of a rectangular area of the sample slide to be identified is taken as an identification starting point and an identification end point, a control program on a computer is connected with the motion control card in a serial port communication mode, the control program obtains coordinate point data of the identification starting point and the end point through the motion control card and checks whether a fluorescence microscope objective is in low magnification, if not, the identification starting point and the end point are automatically switched to the low magnification, then the fluorescence microscope objective table moves in a mode from an X-axis starting point to an X-axis end point and from a Y-axis starting point to a Y-axis end point, the step distance is the minimum pixel distance each time, the fluorescence microscope objective table moves up and down to complete automatic focusing, and, and when the fluorescence microscope objective table moves to the scanning end point, the control program finishes moving and stops photographing.
The identification program reads the low-magnification picture, and in order to improve the identification speed, the picture is firstly subjected to size compression because the background of the picture in the real object is black in a FITC mode of a fluorescence microscope, two insects are usually displayed as green after fluorescence dyeing is excited, according to the characteristic, the picture is subjected to filtering processing to shield other mixed colors except the green, the green channel numerical value in an RGB three-color channel is taken as a judgment basis, the picture is scanned pixel by pixel from the horizontal direction and the vertical direction respectively, a green channel numerical value threshold value is set, pixel point coordinates meeting conditions are stored into a list through comparison, all pixel points in the list are subjected to density-based DBSCAN clustering, and each category represents a suspicious region. And averaging the coordinate values of the pixel points in each category of the plurality of categories obtained after clustering to obtain a central coordinate point of the suspicious region, and selecting a square frame as the suspicious region by taking the coordinate point as the center. And the recognition program recognizes the complete picture and feeds back all the coordinate points of the suspicious region to the control program.
The control program controls the microscope to switch the objective lens to high magnification through the motion control card, controls the fluorescence microscope objective table to automatically move the coordinate points of each suspicious region in sequence, controls the CCD camera to photograph and store the coordinate points of each suspicious region until all the coordinate points of all the suspicious regions are photographed.
The recognition program reads the high-magnification picture, and the two worms are recognized by adopting an image classification algorithm based on a convolutional neural network, wherein the convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, and the output is a one-dimensional array which comprises three data which respectively represent the probability that the recognized objects are giardia, cryptosporidium and non-two worms. Before the application of the program, a designed neural network is trained, thousands of pictures of two worms with artificial labels are used as a training data set, and the recognition precision is ensured to reach more than 95%. After training is finished, the high-magnification picture information is input into a neural network to obtain accurate prediction. And after the recognition program finishes the content recognition of the high-magnification photo, feeding back the recognition result to the control program.
The control program arranges the data to give an automatic identification result report, simultaneously the result supports manual recheck, besides supporting manual checking of the original record photo of the result, the control program controls the fluorescence microscope objective table to automatically move to a result coordinate point through the motion control card, automatically switches to high magnification through controlling the fluorescence microscope objective lens, and realizes the purpose of manual naked eye recheck through switching a DAPI mode and a DIC mode.
The automatic identification control software is compiled through Visual C + +2015, is controlled through communication between a USB (universal serial bus) line and a hardware microscope, and is connected with the CCD camera through the USB line;
the control motion principle is as follows: the software is connected with the microscope control platform by setting a 'com' port. The position of the sample in the visual field is manually adjusted through an ocular lens, then the software end clicks a set starting point and an equipment end point, and the software records the coordinates of the starting point and the end point set by a user. Clicking 'start scanning' again, the software will self-check the objective lens multiple, automatically switch 20 times objective lens for use (if it is 20 times, then directly scan), at this time, the software sends a command to the platform, starts scanning from the starting position, meanwhile, the software calls the CCD to take a picture, the taken picture is stored in a file folder with a specified path, after all the pictures are taken, the image recognition module carries out image recognition, the image recognition module finds out suspected positions in the picture and records the coordinates in a list form, after scanning is finished, the software sends a command to switch the 40-time objective lens, starts sending the command to move to each point in the suspected list for re-photographing, photographs are taken and stored in a folder with a specified path, after all photographs are taken, the image recognition module will perform image recognition and will record the coordinates of the two worms (giardia or cryptosporidium) in a list. An operator can click the list coordinate moving platform to a corresponding position through a mouse to perform manual verification;
the principle of image recognition: the image recognition module is developed by adopting python3.6 language, and simultaneously, a pitorch, scimit-spare, numpy and opencv-python development toolkit is used. The background of the picture taken by the microscope is usually black, while the bug is usually green. According to the characteristics, the program takes the value of a green channel in an RGB three-color channel as a judgment basis to formulate a suspected area selection algorithm. For the confirmation of the coordinate points of the suspected area, DBSCAN clustering is carried out on the coordinates of the pixel points in the list, a plurality of categories are obtained after clustering, the coordinate values of the pixel points in each category are averaged to obtain the central coordinate of the suspected area, a square frame is selected as the suspected area by taking the coordinate as the center, and the default size of the square frame is 60 x 60 (unit: pixel);
for the pictures shot under the 40-time objective lens, the image recognition module classifies the bugs through an image classification algorithm based on a convolutional neural network;
how to determine giardia or cryptozoa: the program classifies the bugs using an image classification algorithm based on a convolutional neural network. The convolutional neural network comprises 1 input layer, 4 hidden layers and 1 output layer, specific parameters of a network structure are shown in fig. 8, the convolutional neural network inputs a picture with the size of 60 × 60 pixels, outputs a group of one-dimensional arrays and comprises three data which respectively represent the probability of belonging to three categories (other, giardia and hidden insects). The designed network is first trained. 1598 small insect pictures with artificial labels are collected in the early stage to serve as a training set, a cross entropy loss function is adopted to train the network, the optimization algorithm is Adam, the learning rate is 0.001, the batch size is 5, and 150 cycles of iteration are performed in total. And selecting 100 small insect pictures with artificial labels as a test set to verify the training effect of the network. After 150 iterations, the classification accuracy of the network can reach 95%, and the trained network parameters are stored.
Then, according to the coordinate center of the suspected area, adjusting a microscope lens, and taking an image enlarged by 400 times by taking the suspected area as the center. And selecting an area which possibly contains the bugs in the center from the pictures magnified by 400 times as an object for image classification.
And finally, compressing the selected picture into 60 × 60 pixels, and inputting the pixels into a trained network to obtain three predicted values. The maximum value among the three predicted values represents the category to which the picture belongs, for example, the output value is [0.1,0.8,0.1], the probability of belonging to giardia is 0.8, and the picture is the giardia.
The invention provides an automatic two-insect recognition system of a microscope, which utilizes the strong calculation and control capability of a computer, adopts automatic motion control to replace manual operation, and adopts artificial intelligent image processing to replace artificial visual recognition. The problem that a tester is required to have high professional level and cannot manually observe a microscope for a long time is thoroughly solved, the two-worm recognition threshold is reduced, recognition errors are reduced, the recognition accuracy is improved, the recognition efficiency is improved through continuous automatic recognition, and the overall detection working efficiency is obviously improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (8)
1. The microscope automatic amphibian recognition system is characterized by comprising an imaging system, a control system and a data processing system, wherein the imaging system comprises a fluorescence microscope (1) and a CCD (charge coupled device) camera (2), the fluorescence microscope (1) is provided with a fluorescence power supply (6), the control system comprises a motion control card (4) and a motion controller (5), the data processing system is a computer (3), the computer (3) runs a control program and an identification program, the fluorescence microscope (1) is provided with the CCD camera (2), the CCD camera (2) is connected with the computer (3), the computer (3) and the motion controller (5) are both connected with the motion control card (4), the motion control card (4) is connected with the fluorescence microscope (1), the control program is used for collecting data of the CCD camera (2), the control program is used for controlling the movement of an objective lens and an objective table (7) of the fluorescence microscope (1), and the identification program can identify the authenticity of the two insects and acquire position coordinates based on artificial intelligence image processing.
2. The microscope automatic two-worm recognition system according to claim 1, characterized in that the CCD camera (2) is connected with the computer (3) through a USB data line.
3. The microscope system for automatically identifying two insects according to claim 2, wherein the motion control card (4) is connected to the computer (3) through a USB data line.
4. The system for microscope automatic identification of two worms according to claim 3 characterised by the fact that the objective and the stage (7) of the fluorescence microscope (1) are both connected to a motion control card (4).
5. A method for microscope automatic identification of two-worm system, characterized in that for use of the microscope automatic identification of two-worm system of claim 4, comprising the steps of:
opening a fluorescence microscope switch, a fluorescence power switch and a motion control card switch, and connecting a motion control card USB line and a CCD data USB line to a computer;
setting a port number com of a USB line of the motion control card, wherein a specific checking path is as follows: this computer-property-device manager-port;
the software end clicks the 'connection microscope', when the connection is successful, the interface pops up a 'connection success' word, a 'confirmation' dialog box is clicked to disappear, and then a picture storage path is set;
observing through an ocular lens, moving the position of a sample through a motion controller, generally defining the upper left corner point of a circumscribed rectangle of a circle drawn by the sample as a starting point, clicking a 'set starting point' button of a software interface after moving in place, then observing through the ocular lens, moving the sample to the lower right corner position of the rectangle, clicking a 'set end point' button of the software interface, switching a microscope from ocular lens observation to CCD observation after finishing setting, and then clicking a 'start scanning' button;
before scanning is started, software can detect the multiple of the objective lens used at present, if the multiple is not 20 times of the objective lens, the objective lens is automatically switched into the 20 times of the objective lens, snake-shaped scanning is started from a starting point position defined by a user, the CCD shoots at the same time, the shot pictures are stored in a path folder designated by the user, and at the moment, the user only needs to wait, and all working software can be automatically completed;
after scanning under the 20-time objective lens is finished, the software can identify the scanned picture, circle out suspicious pictures and coordinates, generate a suspicious picture parameter list in a suspicious object list on the left upper side of the software, then the software enables the microscope to automatically switch the 40-time objective lens through a command, the objective table moves to a first suspicious position, focusing and photographing are carried out again, and the like is carried out until photographing of the Nth coordinate is finished;
after the 40-time objective lens is photographed, the software can recognize the photographed picture again, the two-insect result and the coordinate list are in the microorganism list on the left lower side of the software to generate a microorganism parameter list, and all work is automatically completed;
the user can double click certain parameter information in the two lists through the mouse to enable the microscope objective table to quickly move to the coordinate position, and the user can verify the result through the eyepiece.
6. The method for microscope to automatically identify two-insect system according to claim 5, wherein the image recognition program is developed based on python language, and the software used includes python, pyroch, scimit-spare, numpy and opencv-python.
7. The method for automatically identifying two worms by a microscope as claimed in claim 5, wherein the background of the picture display picture collected by the fluorescence microscope is usually black, and the bug is usually green, and according to the characteristic, the program takes the value of the green channel in the RGB three-color channel as the judgment basis, and the suspected area selection algorithm is formulated as follows:
in image preprocessing, in order to increase the execution speed of the program, the input picture is first subjected to size compression, and the picture is compressed to 1/4 of the original picture by default, and the ratio can be adjusted by modifying the "rate" parameter in the program. Then, since the background color of part of the picture is not black, in order to avoid the influence caused by the background color, the picture is subjected to mean filtering. Finally, the program only focuses on the green channel value, so all other color channel values are set to 0.
Scanning the picture pixel by pixel, respectively scanning the picture from the horizontal direction and the vertical direction by adopting a loop nesting mode, and adjusting the loop step length by modifying a step parameter in a program to increase the scanning speed, wherein the default is 2. The larger the number, the shorter the scanning time, but since the smaller the size of the bug, it is easy to cause omission, and therefore it needs to be selected properly.
Judging whether the green channel numerical value of the pixel point meets the condition or not, setting a threshold value [35, 50] for the green channel numerical value through experiments, if the green channel numerical value of the pixel point is in the threshold value range, storing the coordinate of the point into a bug _ cord list, and if not, carrying out no treatment on the point. The threshold range can be adjusted by modifying a 'threshold' parameter in a program, and the larger the range is, the more suspected areas are found, which brings burden to the subsequent identification work; the smaller the range, the greater the probability of missed detection.
The clustering of the coordinates of the pixel points in the list is performed by DBSCAN, the clustering of the DBSCAN is a clustering algorithm based on density, adjacent and close points in low-latitude data can be aggregated into one class, and the class number and the clustering center do not need to be specified in advance, so that the method is very suitable for determining a suspected area. The pixel coordinates stored in the bug _ code list are clustered, and each category represents a suspected area.
8. The method for microscope to automatically identify two-insect system according to claim 5, wherein the coordinate point determination is specifically: and clustering to obtain a plurality of categories, averaging the coordinate values of the pixel points in each category to obtain the central coordinate of the suspected area, and selecting a square box as the suspected area by taking the coordinate as the center, wherein the default size of the square box is 60 pixels by 60 pixels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010165944.XA CN111240004A (en) | 2020-03-11 | 2020-03-11 | System and method for automatically identifying two insects by microscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010165944.XA CN111240004A (en) | 2020-03-11 | 2020-03-11 | System and method for automatically identifying two insects by microscope |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111240004A true CN111240004A (en) | 2020-06-05 |
Family
ID=70866333
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010165944.XA Pending CN111240004A (en) | 2020-03-11 | 2020-03-11 | System and method for automatically identifying two insects by microscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111240004A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112098383A (en) * | 2020-09-21 | 2020-12-18 | 郑州中普医疗器械有限公司 | Method for identifying microorganism bacteria and electronic device |
| CN112150446A (en) * | 2020-09-27 | 2020-12-29 | 郑州中普医疗器械有限公司 | Microscope examination and film reading scanning method, computer equipment and system |
| CN114859539A (en) * | 2022-05-24 | 2022-08-05 | 宾盛科技(武汉)有限公司 | Fluorescent intelligent identification sensor and multi-channel fluorescent scanning system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101799438A (en) * | 2010-03-10 | 2010-08-11 | 广州市番禺区胆囊病研究所 | Method for detecting clonorchis sinensis eggs in gallstone |
| CN106841072A (en) * | 2017-03-30 | 2017-06-13 | 莱森光学（深圳）有限公司 | A kind of method differentiated for algae with algae proliferation non-destructive monitoring situation |
| CN110672608A (en) * | 2019-10-15 | 2020-01-10 | 南京泰立瑞信息科技有限公司 | Dynamic planning method and system for full-slice scanning path |
-
2020
- 2020-03-11 CN CN202010165944.XA patent/CN111240004A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101799438A (en) * | 2010-03-10 | 2010-08-11 | 广州市番禺区胆囊病研究所 | Method for detecting clonorchis sinensis eggs in gallstone |
| CN106841072A (en) * | 2017-03-30 | 2017-06-13 | 莱森光学（深圳）有限公司 | A kind of method differentiated for algae with algae proliferation non-destructive monitoring situation |
| CN110672608A (en) * | 2019-10-15 | 2020-01-10 | 南京泰立瑞信息科技有限公司 | Dynamic planning method and system for full-slice scanning path |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112098383A (en) * | 2020-09-21 | 2020-12-18 | 郑州中普医疗器械有限公司 | Method for identifying microorganism bacteria and electronic device |
| CN112150446A (en) * | 2020-09-27 | 2020-12-29 | 郑州中普医疗器械有限公司 | Microscope examination and film reading scanning method, computer equipment and system |
| CN114859539A (en) * | 2022-05-24 | 2022-08-05 | 宾盛科技(武汉)有限公司 | Fluorescent intelligent identification sensor and multi-channel fluorescent scanning system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111240004A (en) | System and method for automatically identifying two insects by microscope | |
| CA2219432C (en) | Cytological specimen analysis system with prescreening and generation of viewing path information | |
| US8116543B2 (en) | System for and method of intelligently directed segmentation analysis for automated microscope systems | |
| CN110146974B (en) | Intelligent biological microscope | |
| CN110793946A (en) | Fungus sample microscopic imaging and intelligent identification system and method | |
| CA2905730A1 (en) | System and method for reviewing and analyzing cytological specimens | |
| JP5274731B1 (en) | Observation system, program, and control method for observation system | |
| JPH0484884A (en) | Image recognizing system and image recognizing control system | |
| WO2011030481A1 (en) | Device for bacteria classification and pretreatment device for bacteria test | |
| US11762185B2 (en) | Method and system for identifying objects in a blood sample | |
| CN113631982A (en) | Automated micro-scanning system and method | |
| CN113139485B (en) | Bone marrow cell classification and identification method, device and system based on deep learning | |
| CN111079831A (en) | Intelligent optical detection sample characteristic and flaw automatic marking method and device | |
| US20200074628A1 (en) | Image processing apparatus, imaging system, image processing method and computer readable recoding medium | |
| CN111797706A (en) | Image-based parasite egg shape recognition system and method | |
| CN115079393B (en) | Chromosome karyotype analysis microscopic shooting device and method based on 10-fold objective lens | |
| WO2023031622A1 (en) | System and method for identifying and counting biological species | |
| CN110739051A (en) | Method for establishing eosinophilic granulocyte proportion model by using nasal polyp pathological picture | |
| CN209803001U (en) | Chromosome scanning imaging system | |
| CN113330311A (en) | Cell image analysis device and sample analysis method | |
| CN114829944A (en) | Sample analysis system and method | |
| CN111369553A (en) | Sample slide scanning and image processing method and digital microscope system | |
| CN100483432C (en) | Genital-infection inteeligent identification system and method | |
| JPH03123860A (en) | Chromosome inspecting device | |
| US20230291992A1 (en) | Method and system for event-based imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200904 Address after: 100076 No. 10, golden street, Xihongmen Town, Beijing, Daxing District Applicant after: HKY TECHNOLOGY Co.,Ltd. Address before: 5 / F (05) 501-7, building 75, yard 6, Jiuwen Road, Chaoyang District, Beijing 100020 Applicant before: BEIJING AILUKE TECHNOLOGY TESTING Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200605 |