JP4740686B2 - Protein crystallization state discrimination system and method - Google Patents

Description

  TECHNICAL FIELD The present invention relates to a system and method for discriminating a protein crystal state for automating protein crystal formation in a protein solution, and an electronic image discrimination system, and more particularly to a system and method for updating a discriminant criterion for a protein crystal state and an electronic The present invention relates to an image discrimination system.

  Conventionally, X-ray crystal structure analysis methods have been used to analyze protein crystal structures. In such a structural analysis method, in order to obtain a high quality crystal of a protein to be a structural analysis target sample, as a preparation for preparing the structural analysis sample, a method of generating crystals by evaporating water from an aqueous protein solution (crystallization method) ) Is used. At this time, a screening operation is performed to select whether crystals can be obtained from a protein solution whose structure and properties are unknown.

In this conventional screening operation, a crystal state is sequentially examined with a human eye using a microscope, and the state of crystal formation is discriminated. In this discrimination, it is discriminated whether the protein solution of the sample remains as a solution, precipitates, microcrystals are obtained, or crystals are obtained. Table 1 shows the discrimination criteria used in this screening.

  In Table 1, “transparent” means that the protein solution does not produce any crystals. Further, “precipitation” is classified into four. That is, those that are not seen in the grain and are dark or brown (precipitation (i)), those in which a point-like structure is observed and appearing white, with some dots (precipitation (ii)), amorphous-like It is classified into one that may appear transparent in tissue (precipitation (iii)) and one that may show particles in large amorphous-like tissue (precipitation (iv)). Moreover, what has produced | generated the crystal | crystallization is classified into five. That is, crystals with a peak of about 50 μm or less (microcrystals), needle-like crystals (crystal (i)), plate-like crystals (crystal (ii)), overlapping Are observed (crystal (iii)), and good quality crystals are observed (crystal (iv)). For convenience, a score number may be assigned to each classification. However, this score does not necessarily change so as to increase sequentially according to the growth process of the protein crystal. A schematic diagram of images classified in this way is shown in FIG. 7 for each score. In particular, in the crystallization experiment of an unknown protein solution, since the number of samples to be classified is large, assigning a score is a very useful means.

  In the conventional screening work based on human observation, the observer is trained by training and can perform such classification empirically. A trained observer observes the microscopic images obtained for a large number of samples and performs selection. The classification determined at this time is itself useful information about the sample for X-ray structural analysis, and is an important index for structural analysis and property determination of proteins.

  The methods for obtaining this crystal can be broadly divided into a method in which a protein solution is dropped on the lower surface of a cover glass to evaporate (hanging drop vapor diffusion method), and a protein solution is placed on an evaporation vessel. An evaporation method (sitting drop vapor diffusion method) is used. In order to obtain microscopic images of a large number of protein solutions, it is effective to automatically record microscopic images, and a sitting drop vapor diffusion method that is easy to mechanize is desirable.

  In particular, samples of unknown protein solutions are often classified into transparent, precipitate (i), precipitate (ii), and precipitate (iii) shown in Table 1, but samples classified into these still have crystals. It is not efficient that an observer observes all of them because they are not generated.

  Therefore, as disclosed in Patent Document 1, a protein solution crystallization state discrimination system that creates a discrimination program using a known image when the system is activated has been proposed. The discriminating system differentiates the observed image and extracts a feature amount from the differentiated image. Subsequently, the crystallization state of the protein solution is determined by comparing the extracted feature quantity with the feature quantity of a known image. When this system is used, the crystallization state can be discriminated more efficiently and reliably than the observation by visual observation.

  Furthermore, an example of a conventional texture analysis method is described in Non-Patent Document 1.

JP 2004-345867 A R.M.Haralick, K. Shanmugam and I. Dinstein, "Texture features for image classification" IEEE Trans. Syst., Man, Cybern., Vol.SMC-3, no6, pp.610-621, 1973

  However, the conventional protein solution crystallization state determination system needs to analyze a large number of images in order to adjust the system, and thus it takes time to determine the crystallization state using the system. Furthermore, since the discrimination program cannot be changed during operation, for example, when the discrimination result is incorrect, when an unknown crystal is observed, or when the imaging environment of a protein crystal changes, the analysis must be performed again. .

The present invention provides a protein solution crystallization state discrimination system that includes a user interface and can change the discrimination program during system operation. Furthermore, by using the user interface, it is possible to easily determine the crystallization state, and it is possible to provide a state determination system that reduces determination errors.
More specifically, according to the present invention, an original image acquisition unit that acquires an electronic image of a protein solution as an original image, and an original image matrix that calculates a first simultaneous generation matrix for at least a part of the original image A calculation unit; an original image statistic calculation unit that calculates a first set of image statistics from the first simultaneous generation matrix; an image differentiation unit that performs a differentiation process on the original image to obtain a differential image; A differential image matrix calculation unit that calculates a second simultaneous generation matrix for at least a portion of the image, a differential image statistic calculation unit that calculates a second set of image statistics from the second simultaneous generation matrix, and the second Using the set of image statistics, the electronic image of the protein solution is classified into at least two types, and the electronic image of the protein solution determined to belong to the first type of the two types is further added to the first set of images At least two types using statistics A discrimination criterion setting unit for setting a discrimination criterion for classification, a discrimination unit for discriminating an electronic image of the protein solution using the discrimination criterion to obtain a discrimination result, and a display unit for displaying the discrimination result, A system for discriminating the crystallization state of a protein solution is provided, which can accept a change in the discrimination result from the outside and automatically update the discrimination standard to become a new discrimination standard corresponding to the change. Here, the discrimination system of the present invention is configured to display the discrimination result on the display unit and easily change the discrimination result. Further, when the determination result is changed, the determination system can update the determination criterion and display a new determination result on the display unit.

Furthermore, in the present invention, the display unit displays a tree diagram showing the classification of the electronic image of the protein solution, a graph expressed using two or more feature quantities that determine the discrimination criterion, and the acquired protein solution electrons. Display images.
Furthermore, the display unit displays an electronic image of the protein solution belonging to each node of the tree diagram in association with the node.

  For image analysis, a technique generally called texture analysis can be used. This analysis target includes an original image of the protein solution (an image that has been electronically acquired and preliminarily adjusted for brightness, region extraction, etc., and an image that has not been differentiated) and a derivative that is obtained by differentiating the original image. By using the images together, the state of crystallization in the unknown protein solution can be well distinguished. Further, by performing the determination in two stages, the work for determining the crystallization state of the protein solution can be easily mechanized. According to this method, the transparency and precipitation (i) shown in Table 1 can be appropriately classified from others.

  Further, the discrimination system of the present invention sets a discrimination criterion for further classifying the electronic image of the protein solution classified into the species other than the first species into at least two types using the second set of image statistics. be able to. Thereby, in addition to classification with transparency and precipitation (i), precipitation (ii) and precipitation (iii) can be classified from other things.

  Further, in the above method, the discrimination criterion setting unit converts the first discrimination criterion and the second discrimination criterion into the second set of image statistics using an electronic image of a protein crystal state that has been discriminated in advance. A linear discriminant based on the first set of image statistics is used to establish a third discriminant criterion using the electronic image of the crystal state of the protein that has been discriminated in advance. Using the electronic image of the crystal state of the protein being discriminated, a fourth discrimination criterion is established by a linear discriminant method based on the second set of image statistics, and the discriminator comprises the first discriminator. According to the discrimination criterion and the second discrimination criterion, the electronic image is classified into three types using the second set of image statistics, and the first set of image statistics is used according to the third discrimination criterion. Belongs to the first of the three types The child images are further classified into two types, and electronic images belonging to any species other than the first type are further classified into two types using the second set of image statistics according to the fourth discrimination criterion. .

  If the discriminator is constructed so that four appropriately selected criteria are used in an appropriate order, errors in discrimination are reduced, and good discrimination that can sufficiently reproduce discrimination by humans does not go through classification by humans. You can do it too.

Determination system of the present invention classifies a transparent state, the first to fourth precipitation state, microcrystalline state, an electronic image of the protein solution to one of the plurality of states formed a crystalline state as elements Therefore, the first discrimination criterion is that the protein solution is either a transparent state and a state consisting of the first to third precipitation states, or a fourth precipitation state, a microcrystalline state, and a crystalline state. The second determination criterion is that the protein solution is either a transparent state or a first precipitation state, or the second to second determination criteria. Whether to be in a fourth precipitation state, a microcrystalline state, or a crystalline state is determined. The third determination criterion is that the protein solution is in a transparent state or a first state. Either in the precipitation state The fourth determination criterion is to determine whether the protein solution is in the second precipitation state or the third precipitation state, and the first determination unit is According to the first discrimination criterion and the second discrimination criterion, the protein solution is either in a transparent state and a first precipitation state, or in a second to third precipitation state. Or the fourth precipitation state, the microcrystalline state, or the crystalline state, and the second determination unit is transparent according to the third determination criterion. Determining whether the protein solution in either the state or the first precipitation state is in the transparent state or the first precipitation state, and the third determination unit includes the first determination unit According to the discrimination criterion of 4, second to third precipitation Protein solution in either the state in which the state is the second precipitation state or a system to determine whether either of the third precipitation state.

Further, in the present invention, an original image acquisition step of acquiring an electronic image of a protein solution to obtain an original image, an original image matrix calculation step of calculating a first simultaneous generation matrix for at least a part of the original image, An original image statistic calculating step for calculating a first set of image statistics from one simultaneous generation matrix, an image differentiating step for performing differential processing on the original image to obtain a differential image, and at least a part of the differential image. A differential image matrix calculation step for calculating two simultaneous generation matrices, a differential image statistic calculation step for calculating a second set of image statistics from the second simultaneous generation matrix, and a second set of image statistics. The electronic image of the protein solution is classified into at least two types, and the electronic image of the protein solution determined to belong to the first of the two types is further classified into the first set of image statistics A setting step for setting a discrimination criterion for classifying into at least two types by using a discrimination step, a discrimination step for discriminating an electronic image of the protein solution using the discrimination criterion and obtaining a discrimination result, and a display step for displaying the discrimination result And an update step of automatically updating the discrimination criterion so that it becomes a new discrimination criterion according to the change when the discrimination result received from the outside is changed, A determination method is provided.

Furthermore, in the present invention, an original image acquisition unit that acquires a target electronic image as an original image, an original image matrix calculation unit that calculates a first simultaneous generation matrix for at least a part of the original image, An original image statistic calculating unit that calculates a first set of image statistics from one simultaneous generation matrix, an image differentiating unit that performs differential processing on the original image to obtain a differential image, and at least a part of the differential image A differential image matrix calculation unit for calculating two simultaneous generation matrices, a differential image statistic calculation unit for calculating a second set of image statistics from the second simultaneous generation matrix, and the second set of image statistics. The target electronic images are classified into at least two types, and the target electronic images determined to belong to the first type of the two types are further classified into at least two types using the first set of image statistics. Set the discriminant criteria to classify Comprising the parts, and using said discrimination criterion to obtain the discrimination results to determine the electronic image of the target determination unit, and a display unit for displaying the determination result, reception of a change in the determination result from the outside, the change There is provided a protein crystallization state discrimination system capable of automatically updating the discrimination criteria so as to become a new discrimination criterion according to the above. Here, the target indicates, for example, the shape of the object, or the shape, pattern or the like of the object surface.

  In texture analysis, discrimination criteria can generally be established based on various images. However, in the present invention, an original image and a differential image are used, and the type of image suitable for the feature determined by each criterion is selected. This can be selected. Thereby, discrimination errors are reduced, and good discrimination mechanization capable of sufficiently reproducing discrimination by humans becomes possible.

  The analysis system of the present invention can determine the crystallization state of a sample for determining the crystallization of a protein solution using an electronic image. As a result, it is possible to mechanize the crystallization state determination work that has conventionally relied on the observer. Furthermore, if the analysis system of the present invention is combined with an automatic observation means for a crystallized sample of a protein solution, the protein crystallization state in which the observation of the unknown protein and the determination of the crystallization state of the unknown protein are performed automatically and continuously. An observation analysis system can be constructed.

  The system for discriminating the crystallization state of a protein solution according to the present invention can change and update the discrimination standard for classifying the electronic image of the protein solution according to the discrimination result, thereby improving the discrimination rate of the electronic image of the protein crystal. There is an effect.

  Furthermore, in the system for discriminating the crystallization state of the protein solution of the present invention, the discrimination criterion can be changed and updated at any time, so that the discrimination rate is good even during operation of the system. Can do.

  The discrimination system of the present invention can make the user easily understand the discrimination result when a dendrogram showing the classification of the electronic image of the protein solution is displayed on the display unit.

  Furthermore, a graph represented by using two or more feature quantities for determining the discrimination criterion can be displayed on the display unit, and in this case, the user can easily understand the detailed discrimination result. be able to.

  Next, when the electronic image of the acquired protein solution is displayed on the display unit, the discrimination result by this system can be compared with the discrimination by visual observation, and the discrimination result can be changed. can do.

  In addition, when an electronic image of the acquired protein solution is displayed on the display unit corresponding to each node of the dendrogram, there is an effect that the user can instantly understand the determination result.

  Since the system for determining the crystallization state of the protein solution of the present invention includes a display unit that exhibits the above-described effects, it does not require advanced knowledge or technology when changing the determination program.

  In addition, the system for discriminating the crystallization state of a protein solution according to the present invention, when combined with an automatic observation unit for a crystallized sample of a protein solution, continuously uses a discrimination standard for classifying an electronic image of a protein solution according to a discrimination result. Can be updated automatically or manually.

  The present invention is implemented using a computer device. An appropriate image input unit is connected to the computer. The image input unit acquires an electronic image by an imaging device such as a CCD (charge coupled device) provided in the microscope, an A / D converter, an appropriate memory, and the like, and inputs the electronic image to a computer. Further, the computer is provided with ordinary devices such as an arithmetic device, a storage device, a display device, and a communication device. Each step or each part of the present invention is processed as a functional block implemented in the arithmetic unit, thereby realizing a function. For example, the determination step and the determination unit are realized as a function of performing a determination by a storage operation and an arithmetic processing operation constituted by a register or a memory that requires a program read from the storage device in the arithmetic device. The present invention provides a method and means for discriminating the crystallization state of a protein solution using image processing and image classification using such a computer.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a cross-sectional view showing a prepared state of a sample manufactured for crystallizing (depositing) a protein in a solution to be observed in the present invention. The observation object of the present invention is preferably an observation image of an optical microscope for observing the protein solution 1, but other targets can also be used as objects for discrimination. The protein solution 1 is a solution containing moisture. The protein solution 1 is prepared by being dropped onto a dropping stage 5 disposed in the container 3. The opening of the container 3 is covered with the cover slip 2, and the contact portion between the container 3 and the cover slip 2 is sealed, so that airtightness is maintained thereafter. A precipitating material 4 is disposed in the container 3 in advance and absorbs moisture in the container.

  In such a sample, the water content in the solution decreases over a period of several days to several years, and when the saturation solubility of the protein is reached, protein crystals may gradually grow. Depending on the type of protein, no crystal may be formed or precipitation may occur.

  FIG. 2 is a configuration diagram illustrating the configuration of the crystallization state determination system 100 of the present embodiment. The system includes an original image acquisition unit 12, an original image matrix calculation unit 14, an original image statistic calculation unit 20, an image differentiation unit 22, a differential image matrix calculation unit 24, a differential image statistic calculation unit 26, and a discrimination criterion setting unit. 28, a discrimination reference storage unit 32, a discrimination unit 34, and a display unit 38 are provided. Although not shown, the determination unit 34 may include first to third determination units. Further, in this system, in order to establish a discrimination criterion, electronic image data 18 from an external imaging device 16 is acquired for a protein image in which the crystallization state is classified in advance, and the crystal state data 30 is obtained. You may acquire. This system acquires electronic image data 18 from an external imaging device 16 for an image of an unknown protein solution, discriminates the crystallization state of the unknown protein solution, and displays the crystal state data 36 on a display unit. 38.

  The original image acquisition unit 12 acquires the electronic image data 18 photographed and digitized by the imaging device 16 provided in the microscope, for example, and adjusts the electronic image to obtain the original image data 122. The adjustment is a general process such as a process of adjusting the brightness of the digitized data to an appropriate range, reducing a gradation within a range sufficient for calculation, and cutting out a necessary area.

  The original image matrix calculation unit 14 scans each pixel of at least a part of the adjusted original image data 122 and calculates a first simultaneous generation matrix 142 based on the original image data 122.

ここでは、処理対象の範囲の画素が走査されて、同時生成行列の行列要素に加算されている。 Here, the simultaneous generation matrix has, as an element, the probability that the density at a point separated by a certain displacement δ = (r, θ) (see FIG. 3a) from the point of image density (pixel value) i is j. It is a matrix like this. For example, for an image as shown in FIG. 3b, a simultaneous generation matrix such as the following equation is calculated for each (r, θ).

Here, the pixels in the range to be processed are scanned and added to the matrix elements of the simultaneous generation matrix.

  The element appearing in each matrix represents the appearance frequency of the pixel for each combination of (r, θ) and (i, j). Normalization processing is performed to convert the appearance frequency of this pixel into a probability. The matrix element of the simultaneous generation matrix having this probability as an element is hereinafter referred to as E (i, j). In addition, although this element is calculated for every (r, θ), the description is omitted. The original image matrix calculation unit 14 has a function of obtaining such a probability, calculates a frequency using an appropriate memory, and outputs a simultaneous generation matrix having the probability as an element by normalizing the frequency. Have.

ここで、この各画像統計量の計算には、次の関係式が用いられている。

なお、これらの画像統計量は、非特許文献１によって上記と同様に導入されている。 Based on the simultaneous generation matrix E (i, j) determined in this way, the original image statistic calculation unit 20 calculates the first set of image statistic 202. This set of image statistics is expressed by the following equation.

Here, the following relational expressions are used for the calculation of each image statistic.

Note that these image statistics are introduced by Non-Patent Document 1 in the same manner as described above.

  The image differentiation unit 22 performs a differentiation process on the original image 122 output from the original image acquisition unit 12. In the present embodiment, this differential processing can be performed by applying a Sobel filter (horizontal differential filter: see FIG. 4a, vertical differential filter: see FIG. 4b) known as a primary differential filter. At this time, in FIG. 4, the process of adding the surrounding pixels in consideration of the weights shown in the drawing to the central pixel as the target pixel is performed by sequentially scanning the target pixel. Thereby, the differential image 222 is obtained by the image differential unit 22.

  The differential image matrix calculation unit 24 calculates the second simultaneous generation matrix 242 based on the differential image 222 by the same process as the process performed on the original image 122 by the original image matrix calculation unit 14. In addition, the differential image statistic calculation unit 26 calculates the second set of image statistic 262 from the second simultaneous generation matrix 242 by the same processing as the original image statistic calculation unit 20.

  The discrimination criterion setting unit 28 accepts as the crystal state data 30 whether the original image 122 is in the crystal state of Table 1. This crystal state data 30 is data indicating which of 10 levels of scores 0 to 9 in Table 1. Further, the discrimination criterion setting unit 28 receives the first set of image statistics 202 and the second set of image statistics 262.

  The discrimination criterion setting unit 28 establishes four types of discrimination criteria from these received data by the linear discrimination method. The linear discriminant method is a method for identifying to which group (class) a statistical sample belongs from several characteristics of the statistical sample. At this time, as the statistical sample, the crystal state data 30 is used, and the space (characteristic space) in which the identification is performed is the 14-dimensional space of the first image statistic based on the original image that does not use differentiation, or A 14-dimensional space of second image statistics based on the differential image is selected.

  The discrimination criterion is a criterion that is expressed as a function called a linear discriminant function, and classifies discrimination targets into two by one linear discriminant function. This linear discriminant function is determined as a hyperplane in the 14-dimensional space (a surface having a 13-dimensional degree of freedom that partitions the 14-dimensional space into two half-spaces). In other words, the linear discriminant function can be classified by data dispersed in the 14-dimensional space (each of which is information on the crystal state of the protein that has been discriminated in advance, the first image statistic or the second image It is defined as the hyperplane when dividing the statistical data into coordinate values.

The index for determining the linear discriminant function is to intuitively determine a hyperplane (linear discriminant function) that is a partitioning method that separates the data classified in advance as much as possible. Specifically, the inner product of data u = (u1,..., U14) T having the first image statistic or the second set of image statistics as an element and a coefficient matrix (coefficient vector) A is taken. Thus, the scalar quantity z is calculated (Formula 15). Here, the coefficient matrix A includes unknown parameters to be fitted later as elements. In addition, a large number of data u are used as the data u. Each of the data u has a crystal state determined in advance and crystal state data 30 is obtained, and the first image statistic and the second image statistic are calculated. It is data of.

ここで、Ｐ（ωi）は２つに分類された分類の各クラスωiの事前確率、Σiはクラスωiの共分散行列、ｍiはクラスωiの特徴データ（１４次元の座標値）の単純平均によって作られるベクトル、ｍは全パターンにおける同様の平均のベクトルを表わす。 Next, an intra-class covariance matrix ΣZ and an inter-class covariance matrix ΣB are obtained by Equations 16 and 17.

Here, P (ωi) is a prior probability of each class ωi of the two classifications, Σi is a covariance matrix of class ωi, mi is a simple average of feature data (14-dimensional coordinate values) of class ωi. The generated vector, m, represents the same average vector in all patterns.

のようになる。これを用いて、評価関数ＪΣ（Ａ）を式２０のように定める。

この評価関数は、分類されるクラス間の分離度を表わしている。従って、最適となる係数行列Ａは、この評価関数ＪΣの最大値を与えるものである。 Corresponding to the transformation of the data u with the coefficient matrix A, the variance of the scalar quantity z within and between classes is obtained.

become that way. Using this, the evaluation function JΣ (A) is defined as in Expression 20.

This evaluation function represents the degree of separation between classes to be classified. Therefore, the optimum coefficient matrix A gives the maximum value of the evaluation function JΣ.

In order to calculate A that gives the maximum value of the evaluation function JΣ, an eigenvector that gives the maximum eigenvalue of the matrix given by Equation 21 may be obtained.

なお、式２２のｕ0は、１４元のベクトルｕにｍを代入した場合にｇ（ｍ）＝０が成立するように定められる。 The optimum A determined in this way gives the hyperplane normal vector of the discriminant function defined in Equation 22.

Note that u0 in Expression 22 is determined so that g (m) = 0 holds when m is substituted into the 14-element vector u.

  The discrimination criterion setting unit 28 executes the above processing. For this purpose, the discrimination criterion setting unit 28 is provided with a processing unit (not shown) for appropriately performing matrix calculation. Specifically, it has a calculation function capable of processing inner product calculation (Equations 15 and 17, etc.), product-sum operation (Equations 16 and 17), and eigenvalue calculation (eigenvalue calculation and eigenvector determination). In the description of the above formulas, the order of formula transformation is described in order, but the discrimination criterion setting unit 28 does not need to perform special calculations for the formulas 18 to 20.

  Specific processing of the discrimination reference setting unit 28 as a function of the crystallization state discrimination system 100 of the present embodiment will be described with reference to FIG. First, the discrimination criterion setting unit 28 inputs the crystal state data 30 (S2 in FIG. 5) for the observation image (the teaching observation image) in which the crystal state is discriminated in advance, and the first calculated for the image. The input of the set of image statistics and the second set of image statistics is accepted (S4). Next, the discrimination criterion to be established is selected (S6). Since the set of image statistics used for processing depends on the discrimination criterion, the discrimination criterion to be established is selected, and each image is classified based on the data of the crystal state discriminated in advance (S8). This classification includes two classifications each having the classification shown in Table 1 as an element, each configured by a combination of the elements. This classification is determined by the discrimination criterion to be established at that time, as will be described below.

  In the present embodiment, discrimination criteria g1 to g4 are established and used for discrimination. g1 determines whether the protein solution is in a state consisting of a transparent state and first to third precipitation states, or a state consisting of a fourth precipitation state, a microcrystalline state, and a crystalline state. (First discrimination criterion). g2 determines whether the protein solution is in a state consisting of a transparent state and a first precipitation state, or a state consisting of a second to fourth precipitation state, a microcrystalline state and a crystalline state. (Second discrimination criterion). g3 determines whether the protein solution is in a transparent state or in a first precipitation state (third determination criterion). g4 determines whether the protein solution is in the second precipitation state or the third precipitation state (fourth determination criterion). Each discrimination criterion determines the coefficient of the polynomial in Equation 22, and the value of the polynomial of each discrimination criterion is calculated based on the image statistic (14 statistics) u for the image to be discriminated. Classification is done by As to how to determine the positive / negative of the discrimination criteria g1 to g4, the eigenvector of the maximum eigenvalue described above can be arbitrarily determined by utilizing the arbitrary property that becomes the eigenvector of the same eigenvalue even if it is inverted. In the present embodiment, the classification on the side with the smaller score in Table 1 is determined so that the evaluation value of the polynomial of each discrimination criterion is positive.

  For example, in the discrimination based on the discrimination criterion g1, a state (first state) consisting of a transparent state and a first precipitation state, a state consisting of second to fourth precipitation states, a microcrystalline state and a crystalline state (first state) This corresponds to the classification of the state (2). In this way, classification is performed according to different classification methods depending on each discrimination criterion.

  Then, using the classification classified according to the discrimination criterion in step S8, an intra-class covariance matrix and an inter-class covariance matrix are calculated according to equations 16 and 17 (S10). Then, the matrix of Expression 21 is calculated (S12), and the eigenvector corresponding to the maximum eigenvalue of the matrix is calculated (S14). For this calculation, any calculation method for numerically calculating eigenvalues and eigenvectors of a matrix can be used. When the eigenvector is obtained, a polynomial (equation 22) using the eigenvector as a coefficient substitutes m (m is an average vector of feature data in all patterns) for the vector u so that g (m) = 0 holds. Then, u0 is determined (S16).

  Thereby, all the coefficients of Formula 22 are calculated, and the discrimination criterion is established. By repeating such a process for all the discrimination criteria g1 to g4 (S18), all discrimination criteria are established. Note that g3 and g4 are used only for discrimination after being already classified by g1 and g2, so that the classification according to the discrimination criteria in step S8 can be performed for those classified by established g1 and established g2. Good. As described above, the discrimination criterion setting unit 28 establishes a discrimination criterion.

  The discrimination criterion setting unit 28 stores the discrimination criteria g 1 to g 4 in the discrimination criterion storage unit 32. The classifications A to E based on the calculation shown in the rightmost column of Table 1 are classified based on the discrimination criteria g1 to g4. Also, those that do not fall into any classification are classified as F.

  In the present embodiment, discrimination criteria g1 to g4 are established using images that are actually classified into scores 0 to 9. The images used for establishing the discrimination criterion are about 20 to 60 images classified in advance by a skilled observer for each of the scores 0 to 9.

  Next, a method for discriminating a sample of an unknown protein solution by the system 100 of the present embodiment will be described. In this determination, the determination unit 34 reads the determination criteria g1 to g4 from the determination criterion storage unit 32.

  For the unknown protein solution, a sample is prepared as shown in FIG. Then, an observation image is acquired as an optical image, and the same part as when establishing a discrimination criterion (original image acquisition part, original image matrix calculation part, original image statistic calculation part, image differentiation part, differential image matrix calculation part, The same processing is performed by the differential image statistic calculation unit). This provides a first set of image statistics and a second set of image statistics for the sample of unknown protein solution.

  Since the first set and the second set of image statistics are vectors each consisting of 14-dimensional numerical data, the four types of discrimination criteria g1 to g4 established by the discrimination criteria setting unit 28 are applied to them. To make a decision. That is, the discrimination is performed by substituting the image statistic vector obtained from the unknown protein solution into the polynomial indicating the discrimination criterion and evaluating the value. The determination result data 36 is displayed on the user interface 38 or stored in the storage unit.

  Next, specific steps for applying the first to fourth discrimination criteria (g1 to g4) will be described with reference to FIG. The discriminating unit first discriminates using the first discrimination criterion g1 and the second discrimination criterion g2 (FIG. 6, S20). Here, the fact that g1 is positive corresponds to the state consisting of the transparent state and the first to third precipitation states, and that g1 is negative means that the fourth precipitation state and the microcrystalline state. This corresponds to a state consisting of a crystalline state. Similarly, the positive and negative values of g2 are in a state consisting of a transparent state and a first precipitation state, and are in a state consisting of second to fourth precipitation states, a microcrystalline state and a crystalline state, respectively. Corresponding to Therefore, when both g1 and g2 are positive, the unknown protein solution is in a state consisting of a transparent state and a first precipitation state (FIG. 6, branch 202; Table 1, A or B). When g1 is positive and g2 is negative, it is in a state consisting of the second to third precipitation states (branch 204; Table 1, C or D), and when g1 and g2 are both negative, 4 is a state consisting of a precipitation state, a microcrystalline state, and a crystalline state (branch 206; Table 1, E). It should be noted that for the combination of g1 being negative and g2 being positive in the discrimination criterion, an image showing the actual crystallization state of the protein solution is hardly like this. In this case, processing not shown as an error is performed. It is classified as F in Table 1.

  Next, when the image is determined to be in a transparent state and a first precipitation state (branch 202; Table 1, A or B), determination is performed using the third determination criterion g3 (S22). The sign of g3 corresponds to whether the unknown protein solution is in a clear state (branch 222; Table 1, A) or a first precipitation state (branch 224; Table 1, B), respectively.

  When the image is determined to be in the state of the second to third precipitation states (branch 204; Table 1, C or D), the determination is performed using the fourth determination criterion g4 (S24). The sign of g4 indicates whether the unknown protein solution is in the second precipitation state (branch 242; Table 1, C) or the third precipitation state (branch 244; Table 1, D), respectively. Correspond. As described above, after it is determined which of A to E in Table 1, the result is stored in the variable X holding the determination result according to each determination result (S26a to S26e). Then, the discrimination result is displayed (S28).

このように、人によって分類された分類のうち、透明（表１、Ａ）、沈殿（ｉ）（Ｂ）、沈殿（ｉｉ）（Ｃ）、沈殿（ｉｉｉ）（Ｄ）が、これらの相互からも、そして、残りの分類（Ｅ）からも良好に分類されている。 Table 2 shows the result of each classification of the image of the unknown protein obtained in the present embodiment. Each row of the table is a crystallization state in which the same image is classified by a person, and each column is a classification obtained by processing by the system of the present embodiment. Each data is the actual number of images that apply to human classification and calculation classification. The “result” column indicates the ratio at which images classified by people are appropriately classified by calculation.

Thus, among the classifications classified by humans, transparent (Table 1, A), precipitation (i) (B), precipitation (ii) (C), precipitation (iii) (D) are derived from each other. And it is classified well from the remaining classification (E).

  A specific display method of the display unit 38 in the crystallization state determination system 100 of the present embodiment will be described with reference to FIGS. 8 and 9A to 9D. FIG. 8 is an explanatory diagram showing an outline of a display image on the display unit 38. The display unit 38 includes an image display unit 84, a discrimination tree diagram display unit 82, and a discriminant display unit 86 on the result display unit 80. Then, the observed image is displayed on the image display unit 84, the tree diagram 90 created from the discrimination criteria of the observed image is displayed on the discriminant tree diagram display unit 82, and the discriminant display unit 86 is displayed. The discriminant is displayed.

  Next, how the display changes in the display image shown in FIG. 8 according to the change of the determination result will be described with reference to FIGS. In the tree diagram 90, a node (all) including all images and a plurality of sub-nodes branching from the node are displayed. For example, in FIG. 9A, the subnode (1 and 2) branches from the node (all) to which all the images of the tree diagram 90 are associated, and the subnode (1 or 2) further subdivides the subnode ( 1/1 and 1/2) are branched.

  When the node 92 in the tree diagram 90 is selected by the mouse cursor, the system displays an image belonging to the node 92 on the image display unit 84 (FIG. 9A). Further, in the vicinity of the selected node, the first to third principal components are extracted and expressed from the division command for dividing the set of images belonging to the node based on the discrimination criterion and the 14 feature amounts by the principal component analysis. A 3D graph command for displaying a 3D graph is displayed. Here, when a split command is selected, as shown in FIG. 9B, subnodes (1/2/1 and 1/2 / generated from the selected node (1/2) according to the split are generated. 2) is added to the tree diagram 90, and an image belonging to the subnode is displayed. In FIG. 9B, an image belonging to the sub node (1/2/1) is displayed on the first image display unit 84a, and an image belonging to the sub node (1/2/2) is displayed on the second image display unit 84b. . Then, by moving the discriminated image from the first image display portion 84a to the second image display portion 84b, the discrimination criterion used for discrimination by the present system can be changed (FIG. 9 (c)). . Furthermore, when the discriminated image cannot be discriminated by the first image display unit 84a or the second image display unit 84b, it is temporarily stored in the third image display unit 84c or the fourth image display unit 84d, and then It can be moved to other subnodes or destroyed. On the other hand, when the 3D graph command is selected, the 3D graph 94 is displayed on the result display unit 80 (FIG. 9D). The 3D graph is created from the first to third principal components from the 14 feature amounts by principal component analysis. Furthermore, the discrimination plane 96 determined by the discrimination criteria can be displayed on the 3D graph, and the discrimination criteria can be shown in detail. Further, by changing the position and angle of the determination surface 96, the determination reference can be changed.

  In addition, “1/2” and “clsC” displayed on each node may be input by the user or automatically provided by the system. Here, “clsC” means that the classification of calculation shown in Table 1 is C. Furthermore, the first to fourth image display units can change the display size by moving the boundary. Furthermore, the edge of the node 88 and the edge of the image corresponding to the node 88 are displayed using the same color so that each node 88 of the dendrogram 86 and the image corresponding to the node can be easily identified. can do. Further, for example, it may be possible to easily identify using a notation of characters or symbols.

  Furthermore, the crystallization state determination system 100 of the present embodiment can display the identification rate on the display unit 38 in order to evaluate the determination criterion. This discrimination rate is a numerical value obtained by the user evaluating the discrimination result on the display unit or using an image in which the presence or absence of the crystallization state is discriminated in advance, and is accurately discriminated out of the discriminated images. It shows the ratio of images. Further, when the identification rate satisfies a predetermined condition, the crystallization state determination system 100 of the present invention may be configured to update the determination criterion.

  In addition, although the display method of the display unit 38 in the crystallization state determination system 100 of the present invention has been described, the structure of the tree diagram created by the division is not limited to the structure described above, and the present invention It is possible to make changes based on the technical idea.

  From the configuration of the present embodiment, the method for determining the crystallization state of the protein solution provided by the present invention is summarized as shown in FIG. That is, an electronic image of the protein solution is acquired and used as an original image (original image acquisition step, S42). Then, a first simultaneous generation matrix is calculated for at least a part of the original image (original image matrix calculation step, S44). Further, a first set of image statistics is calculated from the first simultaneous generation matrix (original image statistics calculation step, S46). The original image is subjected to differentiation processing to obtain a differential image (image differentiation step, S48). A second simultaneous generation matrix is calculated for at least a part of the differential image (differential image matrix calculation step, S50). A second set of image statistics is calculated from the second simultaneous generation matrix (differential image statistics calculation step, S52). According to the second set of image statistics, the electronic image of the protein solution is classified into at least two types, and the electronic image of the protein solution determined to belong to the first type of the two types is used as the first set of images. A discrimination criterion for classifying into at least two types using statistics is set. (Setting step, S54). Here, the discrimination criteria are g1 and g2 in step S20 in FIG. 6 and g3 in step S22. This process is executed by the operation of the computer using the discrimination criteria established as in the present embodiment in the first discrimination step and the second discrimination step. Next, an electronic image of the protein solution is discriminated using the discrimination criterion (discrimination step, S56). Subsequently, the discrimination result obtained by the discrimination criterion is displayed on the display unit (display step, S58). The discrimination criterion can be updated using the display result (update step, S60).

It is sectional drawing of the sample container which shows the structure of the sample in embodiment of this invention. It is a block diagram which shows the structure of the structural sample of the crystallization state discrimination | determination system in embodiment of this invention. It is explanatory drawing explaining the method of the texture analysis in embodiment of this invention. It is explanatory drawing explaining the differentiation method in the texture analysis in embodiment of this invention. It is a flowchart which shows the method of establishment of the discrimination | determination reference | standard in embodiment of this invention. It is a flowchart which shows the discrimination method in embodiment of this invention. It is explanatory drawing which shows typically the classification | category by the image of the crystal state of a protein solution. It is explanatory drawing explaining the display image of the display part in embodiment of this invention. It is explanatory drawing explaining the change method of the discrimination | determination result in embodiment of this invention. It is explanatory drawing explaining the change method of the discrimination | determination result in embodiment of this invention. It is explanatory drawing explaining the change method of the discrimination | determination result in embodiment of this invention. It is explanatory drawing explaining the 3D graph which shows the discrimination | determination result in embodiment of this invention. 3 is a flowchart showing a method for determining a crystal state of a protein solution provided by the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Protein solution 2 Cover slip 3 Container 4 Precipitating material 5 Dropping stage 100 Crystallization state discrimination system 12 Original image acquisition part 14 Original image matrix calculation part 16 Imaging device 20 Original image statistic calculation part 22 Image differentiation part 24 Differential image matrix calculation Unit 26 Differential Image Statistics Calculation Unit 28 Discrimination Criteria Setting Unit 32 Discrimination Criteria Storage Unit 34 Discrimination Unit 38 Display Unit

Claims (17)

An original image acquisition unit that acquires an electronic image of the protein solution and uses it as an original image;
An original image matrix calculation unit for calculating a first simultaneous generation matrix for at least a part of the original image;
An original image statistic calculator that calculates a first set of image statistics from the first simultaneous generation matrix;
An image differentiating unit that performs a differentiation process on the original image to obtain a differentiated image;
A differential image matrix calculation unit for calculating a second simultaneous generation matrix for at least a part of the differential image;
A differential image statistic calculator that calculates a second set of image statistics from the second simultaneous generation matrix;
Using the second set of image statistics, the electronic image of the protein solution is classified into at least two types, and the electronic image of the protein solution determined to belong to the first type of the two types is further converted into the first image. A discrimination criterion setting unit that sets discrimination criteria to be classified into at least two types using a set of image statistics,
A discriminator for discriminating an electronic image of the protein solution using the discriminant criterion to obtain a discrimination result;
A display unit that displays the discrimination result, and accepts a change in the discrimination result from the outside , and can automatically update the discrimination standard to become a new discrimination standard according to the change. Crystallization state discrimination system.   The system according to claim 1, wherein the display unit displays a dendrogram showing classification of electronic images of the protein solution.   The system according to claim 1, wherein the display unit displays a graph represented by using two or more feature amounts for determining the discrimination criterion.   The system according to claim 1, wherein the display unit displays an electronic image of the acquired protein solution. The system according to claim 2 , wherein the display unit can display an electronic image of a protein solution belonging to each node of the dendrogram in association with the node.   The discrimination criterion setting unit sets a discrimination criterion for further classifying an electronic image of a protein solution classified into a species other than the first species into at least two species using the second set of image statistics. The system according to 1. The discrimination criterion setting unit
Establishing a first discriminant criterion and a second discriminant criterion by a linear discriminant method based on the second set of image statistics, using an electronic image of a protein crystallized state that has been discriminated in advance,
Using the electronic image of the protein crystallographic state that has been previously determined, a third determination criterion is established by a linear determination method based on the first set of image statistics,
Using the electronic image of the crystal state of the protein that has been discriminated in advance, a fourth discrimination criterion is established by a linear discriminant based on the second set of image statistics,
The discriminator classifies electronic images into three types using the second set of image statistics based on the first discrimination criterion and the second discrimination criterion, and the third discrimination criterion Using the first set of image statistics, the electronic images belonging to the first type of the three types are further classified into two types, and the second set of image statistics is used according to the fourth discrimination criterion. The system according to claim 6, wherein electronic images belonging to any species other than the first type are further classified into two types. In order to classify the electronic image of the protein solution into any one of a plurality of states constituted by the transparent state , the first to fourth precipitation states, the microcrystalline state, and the crystalline state,
The first discriminating criterion is that the protein solution is one of a transparent state and a first to third precipitation state, or a fourth precipitation state, a microcrystalline state, and a crystalline state. To determine which one is in,
The second discriminating criterion is that the protein solution is one of a state consisting of a transparent state and a first precipitation state, or a state consisting of a second to fourth precipitation state, a microcrystalline state and a crystalline state. To determine which one is in,
The third discrimination criterion is to discriminate whether the protein solution is in a transparent state or a first precipitation state,
The fourth discrimination criterion is to discriminate whether the protein solution is in the second precipitation state or the third precipitation state,
According to the first discrimination criterion and the second discrimination criterion, the first discriminating unit is one of a state in which the protein solution has a transparent state and a first precipitation state, or a second to third precipitation state. Or any one of the states consisting of the fourth precipitation state, the microcrystalline state, and the crystalline state,
The second discriminating unit determines whether the protein solution in the transparent state or the first precipitation state is in the transparent state or the first precipitation state according to the third discrimination criterion. Is to determine
The third discriminating unit determines whether the protein solution in any of the second to third precipitation states is in the second precipitation state or the third precipitation state according to the fourth discrimination criterion. The system according to claim 7, wherein the system determines whether or not the An original image acquisition step of acquiring an electronic image of the protein solution and using it as an original image;
An original image matrix calculating step of calculating a first simultaneous generation matrix for at least a part of the original image;
An original image statistic calculating step of calculating a first set of image statistic from the first simultaneous generation matrix;
An image differentiation step of performing a differentiation process on the original image to obtain a differential image;
A differential image matrix calculating step of calculating a second simultaneous generation matrix for at least a part of the differential image;
A differential image statistic calculating step of calculating a second set of image statistics from the second simultaneous generation matrix;
Using the second set of image statistics, the electronic image of the protein solution is classified into at least two types, and the electronic image of the protein solution determined to belong to the first type of the two types is further converted into the first image. A setting step for setting a discrimination criterion for classifying into at least two types using a set of image statistics;
A discrimination step for discriminating an electronic image of the protein solution using the discrimination criteria and obtaining a discrimination result;
A display step for displaying the determination result;
A method for determining the crystallization state of a protein solution, comprising: an update step of automatically updating a determination criterion so as to become a new determination criterion according to the change when the determination result received from the outside is changed .   The method according to claim 9, wherein the displaying step further comprises displaying a dendrogram showing a classification of an electronic image of the protein solution.   The method according to claim 9, wherein the displaying step further includes a step of displaying a graph represented using two or more feature quantities that determine the discrimination criterion.   The method according to claim 9, wherein the displaying step further includes a step of displaying an electronic image of the acquired protein solution. The method according to claim 10 , wherein the displaying step further includes a step of displaying an electronic image of a protein solution belonging to each node of the dendrogram in association with the node.   The setting step includes a step of setting a discrimination criterion for further classifying an electronic image of a protein solution classified into a species other than the first type into at least two types using the second set of image statistics, The method of claim 9. The setting step includes
Establishing a first discrimination criterion and a second discrimination criterion by a linear discrimination method based on the second set of image statistics, using an electronic image of a crystal state of a protein that has been discriminated in advance;
Establishing a third discrimination criterion by a linear discriminant based on the first set of image statistics, using the pre-discriminated electronic image of the crystalline state of the protein;
Using the electronic image of the crystalline state of the protein that has been discriminated in advance to establish a fourth discriminant criterion by a linear discriminant method based on the second set of image statistics, and the discriminating step comprises: According to the first discrimination criterion and the second discrimination criterion, the electronic images are classified into three types using the second set of image statistics, and the first set of images is classified according to the third discrimination criterion. The electronic images belonging to the first type of the three types are further classified into two types using the statistics, and other than the first type using the second set of image statistics according to the fourth discrimination criterion The method according to claim 14, wherein the electronic image belonging to any one of the following types is further classified into two types. In order to classify the electronic image of the protein solution into any one of a plurality of states constituted by the transparent state , the first to fourth precipitation states, the microcrystalline state, and the crystalline state,
The first discriminating criterion is that the protein solution is one of a transparent state and a first to third precipitation state, or a fourth precipitation state, a microcrystalline state, and a crystalline state. To determine which one is in,
The second discriminating criterion is that the protein solution is one of a state consisting of a transparent state and a first precipitation state, or a state consisting of a second to fourth precipitation state, a microcrystalline state and a crystalline state. To determine which one is in,
The third discrimination criterion is to discriminate whether the protein solution is in a transparent state or a first precipitation state,
The fourth discrimination criterion is to discriminate whether the protein solution is in the second precipitation state or the third precipitation state,
In the first determination step, according to the first determination criterion and the second determination criterion, the protein solution is either in a transparent state or a first precipitation state, or second to third precipitations. Determining whether one of the states consisting of the state, or any of the state consisting of the fourth precipitation state, the microcrystalline state, and the crystalline state,
In the second determination step, the protein solution in either the transparent state or the first precipitation state is in the transparent state or the first precipitation state according to the third determination criterion. Is a step to determine whether
In the third determination step, the protein solution in any of the second to third precipitation states is in the second precipitation state or the third precipitation state according to the fourth determination criterion. The method of claim 15, wherein the method is a step of determining which one is present. An original image acquisition unit that acquires an electronic image of a target and uses it as an original image;
An original image matrix calculation unit for calculating a first simultaneous generation matrix for at least a part of the original image;
An original image statistic calculator that calculates a first set of image statistics from the first simultaneous generation matrix;
An image differentiating unit that performs a differentiation process on the original image to obtain a differentiated image;
A differential image matrix calculation unit for calculating a second simultaneous generation matrix for at least a part of the differential image;
A differential image statistic calculator that calculates a second set of image statistics from the second simultaneous generation matrix;
Using the second set of image statistics, the target electronic images are classified into at least two types, and the target electronic images determined to belong to the first type of the two types are further classified into the first set of image statistics. A discrimination criterion setting unit that sets discrimination criteria to be classified into at least two types using image statistics;
A discriminator for discriminating the target electronic image using the discrimination criteria and obtaining a discrimination result;
A display unit for displaying the discrimination result, and accepting a change in the discrimination result from the outside , and automatically updating the discrimination standard to become a new discrimination standard according to the change . State determination system.

Images