WO2019073312A1

WO2019073312A1 - Method and device for integrating image channels in a deep learning model for classification

Info

Publication number: WO2019073312A1
Application number: PCT/IB2018/056266
Authority: WO
Inventors: Harshit Pande; Abdul Aziz; Bharath Cheluvaraju; Tathagato Rai Dastidar; Apurv Anand; Rohit Kumar Pandey
Original assignee: Sigtuple Technologies Private Limited
Priority date: 2017-10-13
Filing date: 2018-08-20
Publication date: 2019-04-18

Abstract

Embodiments of present disclosure disclose method and device for integrating image channels in a deep learning model for classification of objects in a sample. Initially, at least one microscopic image of a sample comprising plurality of objects is received. Plurality of image channels is generated for the at least one microscopic image using at least one image operator. The plurality of image channels comprises at least one colour image channel and at least one hand-crafted image channel. Upon the generation, the plurality of image channels is provided to a deep learning model to integrate the plurality of image channels with the deep learning model for classification of the plurality of objects.

Description

TITLE: "METHOD AND DEVICE FOR INTEGRATING IMAGE CHANNELS IN A DEEP LEARNING MODEL FOR CLASSIFICATION"

TECHNICAL FIELD

The present subject matter relates to field of artificial intelligence, more particularly, but not exclusively to a method and device for integrating image channels in a deep learning model for classification of objects in a sample.

BACKGROUND

Classification of one or more objects in a sample may be necessary for analysis and examination of the sample. For example, the sample may be body fluid. Existing classification systems include statistical models, also referred to as learning models, relating to neural networks. Such learning models may be configured for automated extraction of features of the one or more objects from an image of the sample. The extracted features are used for the classification of the one or more objects. Such learning models may not be able to extract all features of the one or more objects. By which, performance of such learning models for the classification may be affected or reduced. In some learning models, the extraction of features may be independent of task associated with the classification. In such cases, output of the learning models may not be accurate for the classification. The information disclosed in this background of the disclosure section is only for enhancement of understanding of the general background of die invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art. SUMMARY

In an embodiment, the present disclosure relates to a method for integrating image channels in a deep learning model for classification of objects in a sample. Initially, at least one microscopic image of a sample comprising plurality of objects is received. Plurality of image channels is generated for the at least one microscopic image using at least one image operator. The plurality of image channels comprises at least one colour image channel and at least one hand-crafted image channel. Upon the generation, the plurality of image channels is provided to a deep learning model to integrate the plurality of image channels with the deep learning model for classification of the plurality of objects. Tn an embodiment, the present disclosure relates to an integration unit for integrating image channels in a deep learning model for classification of objects in a sample. The integration unit comprises a processor and a memory communicatively coupled to the processor. The memory stores processor-executable instructions which on execution cause the processor to perform the hierarchical classification. Initially, at least one microscopic image of a sample comprising plurality of objects is received. Plurality of image channels is generated for the at least one microscopic image using at least one image operator. The plurality of image channels comprises at least one colour image channel and at least one hand-crafted image channel . Upon the generation, the plurality of image channels is provided to a deep learning model to integrate the plurality of image channels with the deep learning model for classification of the plurality of objects.

The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described, by way of example only, and regarding the accompanying figures, in which:

Figure 1 illustrates an exemplary environment for integrating image channels in a deep learning model for classification of objects in a sample, in accordance with some embodiments of the present disclosure;

Figure 2 shows a detailed block diagram of an integration unit for integrating image channels in a deep learning model for classification of objects in a sample, in accordance with some embodiments of the present disclosure; Figure 3 shows a detailed block diagram comprising an integration unit for integrating image channels in a deep learning model for classification of objects in a sample, in accordance with some embodiments of the present disclosure;

Figure 4 illustrates a flowchart showing an exemplary method for integrating image channels in a deep learning model for classification of objects in a sample, in accordance with some embodiments of present disclosure;

Figures 5a illustrates an exemplary image of a urine sample for classification of plurality of objects in the urine sample, in accordance with some embodiments of the present disclosure;

Figures 5b shows exemplary images of plurality of objects in a urine sample for classification of the plurality of objects in accordance with some embodiments of the present disclosure;

Figures 6 shows exemplary images for edge as a hand-crafted image channel generated for classification of plurality of objects in a sample in accordance with some embodiments of the present disclosure;

Figures 7 shows exemplary images for sharpness as a hand-crafted image channel extracted for classification of plurality of objects in a sample in accordance with some embodiments of the present disclosure;

Figures 8a shows plot of precision of learning model with hand-crafted image channel and without hand-crafted image channel in accordance with some embodiments of the present disclosure;

Figures 8b shows plot of sensitivity of learning model with hand-crafted image channel and without hand-crafted image channel in accordance with some embodiments of the present disclosure; and Figure 9 illustrates a block diagram of an exemplary computer system for implementing embodiments consistent with the present disclosure.

It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and executed by a computer or processor, whether such computer or processor is explicitly shown.

DETAILED DESCRIPTION

In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the disclosure to the forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the spirit and the scope of the disclosure.

The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by "comprises ... a" does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

The terms "includes", "including", or any other variations thereof, are intended to cover a non- exclusive inclusion, such that a setup, device, or method that includes a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. Tn other words, one or more elements in a system or apparatus proceeded by "includes... a" does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

In the following detailed description of the embodiments of the di sclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

Microscopic examination of a sample is an important part of analysis of the sample in a pathology laboratory. Analysis of the sample may be required to provide an opinion or report on the sample. Classification of plurality of objects in the sample may be an essential step in the analysis. The classification may include to determine or to identify class (also referred as label, category, and the like) associated with each of the plurality of objects in the sample. One or more deep learning models may be used for performing the classification. The one or more deep learning models may be arranged in a hierarchical manner for extracting features which are used for the classification. The present disclosure discloses a method and device for integrating image channels in the one or more deep learning models for performing classification of the plurality of objects. The image channels include colour image channels and additionally, hand-crafted image channels, which are generated using image operators. The image channels are integrated with the deep learning model, by which, contextual features of the plurality of objects may be retrieved. Thus, when the integrated deep learning model is used for the classification, an efficient and accurate classification of the plurality of objects may be performed. Figure 1 illustrates an exemplary environment 100 for integrating image channels in a deep learning model for classification of objects in a sample. The exemplary environment 100 may comprise an integration unit 101, an image acquisition unit 102, amicroscopic unit 103, a stage 104, a sample 105 and a deep learning model 106 for performing the integration. In an embodiment, the sample 105 may also be referred to as a specimen. In an embodiment, the sample 105 may be a drop of centrifuged sample from a subject. In an embodiment, the sample 105 may be associated with, but is not limited to, urine, blood, semen, tissue, smear, body fluid, biological fluid, cells, biopsy and so on, obtained from the subject. The subject may be a human, an animal, or a plant. The sample 105 may be one of a mono-layer sample and a multilayer sample placed on the stage 104 under the microscopic unit 103. The mono -layer sample includes a single layer comprising plurality of objects. The multi-layer sample includes two or more layers, each at different focal depth. In the multi-layer sample, each of the plurality of objects may be present in one of the two or more layers or may be suspended across the two or more layers. In an embodiment, the plurality of objects in the sample 105 may include, but are not limited to, at least one of Red Blood Cell (RBC), White Blood Cell (WBC), RBC clump, WBC clump, epithelial cell (also referred as epithelial), cast, bacteria, yeast, parasite, mucus, sperm, crystal, artefact, malignant cell and so on.

The microscopic unit 103 may be any system which is configured to provision focus of a region of interest in the sample 105. The image acquisition unit 102 may be configured to capture the microscopic image of the region of interest of the sample 105. In an embodiment, the image acquisition unit 102 may be a device, such as camera and the like, which is configured to capture an image of the sample 105. Such image may be referred to as the at least one microscopic image in the present disclosure.

Further, the integration unit 101 in the exemplary environment 100 may be configured to integrate plurality of image channels in the deep learning model 106. The integration unit 101 may comprise a processor 107, an Input/Output (I/O) interface 108and a memory 110 for the integration. The memory may be communicatively coupled to the processor 107 and the memory may store processor-executable instructions which on execution cause the processor 107 to integrate the plurality of image channels with the deep-learning model.

For the integration, initially, the integration unit 101 may be configured to receive at least one microscopic image of the sample 105 comprising the plurality of objects. In an embodiment, the at least one microscopic image may be received from the image acquisition unit 102. In an embodiment, the image acquisition unit 102 may be an integral part of the integration unit 101 (not shown in figure). In an embodiment, the at least one microscopic image may be at least one optimal image associated with the sample 105. The at least one optimal image may be image with optimal representation of the plurality of objects in the sample 105. One or more techniques, known to a person skilled in the art, may be implemented for acquiring the at least one optimal image. In an embodiment, when the integration unit 101 receives plurality of microscopic images, each of the plurality of microscopic images may be associated with each of one or more Field of Views (FOVs) of the sample 105. In an embodiment, each of the plurality of microscopic images may be image captured at a predefined focal depth for the multiple layers of the sample 105. In an embodiment, the image acquisition unit 102 may be configured to perform global threshold masking on the plurality of microscopic images to separate FOV of the sample 105 from dark surrounding regions. One or more techniques, known to a person skilled in the art, may be implemented for performing the global threshold masking. Upon receiving the at least one microscopic image, the integration unit 101 may be configured to generate the plurality of image channels for the at least one microscopic image. At least one image operator may be used for generating the at least one microscopic image. In an embodiment, the plurality of image channels may be referred to as plurality of image features. The plurality of image channels comprises at least one colour image channel and at least one hand-crafted image channel. In an embodiment, the plurality of image channels may include features associated with the at least one microscopic image. The at least one colour image channel may be one of colour models, known in the art. The colour models may include, but are not limited to, a Red Blue Green (RBG) model, a Cyan Magenta Yellow and Key (CMYK) model, a grey-scale model, Hue Saturation Value (HSV) model, YUV model and the like. In an embodiment, the at least one colour image channel may be in a form of any digital image format, known in the art. The at least one hand-crafted channel may be selected from attributes associated with the plurality of objects in the sample 105. The attributes of the plurality of objects may include, but are not limited to, sharpness, edge, shape, texture and so on, of the plurality of objects.

In an embodiment, the integration unit 101 may select the plurality of image channels based on domain knowledge relating to the classification. In an embodiment, the integration unit 101 may select the at least one hand-crafted image channel from at least one of morphological information, textual information and sharpness information derived from the domain knowledge. The selection of the at least one colour image channel and the at least one handcrafted image channel may be based on task that is to be performed during the classification of the plurality of objects. In an embodiment, the classification of the plurality of objects may include one or more tasks. The one or more tasks may include, but are not limited to, detecting blurry object, identifying class of an object, determined number of objects in the sample 105, determining size of an object, determining location of an object (also referred as localization of the object) and so on. It may be required that at least one of the one or more tasks is to be performed for the classification. Each of the one or more tasks may be associated with the attributes associated with the plurality of objects. In an embodiment, the at least one hand- crafted image channel for a task from the one or more tasks, may be associated with corresponding attributes. For example, the task for detecting blurry objects may be associated with the sharpness and the at least one hand-crafted image channel may be sharpness channel. Similarly, the task of identifying class of the object m ay be associated with the edge and the at least one hand-crafted image channel may be edge channel.

In an embodiment, the integration unit 101 may select the plurality of image channels based on a cross-validation technique. The cross-validation techniques may be implemented on the deep learning model 106 for validating the deep learning model 106. Output of the cross- validation technique may be used for selecting the plurality of image channels. One or more techniques, known to a person skilled in the art, may be implemented for cross-validating the deep-learning model. In an embodiment, the integration unit 101 may select the plurality of image channels from Z- stack of images captured from plurality of layers of the sample 105. In an embodiment, optimal images may be selected from the Z-stack of images and provided as the plurality of image channels for the integration. One or more techniques, known to a person skilled in the art, may be implemented for selecting the plurality of image channels from the Z-stack of images.

Upon the generation, the integration unit 101 may provide the plurality of image channels to the deep learning model to integrate the plurality of image channels with the deep learning model. The deep learning model 106 may be one of Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and so on. In an embodiment, combination, or fusion of CNN and RNN may be implemented as the deep learning model 106. In an embodiment, variant of said deep learning model 106 may be implemented, hi an embodiment, the deep learning model 106, with integrated plurality of image channels may be implemented for classification of the plurality of objects in the sample 105. In an embodiment, the integration unit 101 may be configured to integrate plurality of image channels with each of plurality of deep learning models. In such a case, each of the plurality of deep learning models may be associated with each of the one or more tasks relating to the classification of the plurality of objects. The plurality of image channels may vary for each of the plurality of deep learning models, based on the task to be performed. Upon said integration, the plurality of deep learning models may be implemented in a hierarchical manner, for the classification.

Further, in an embodiment, the deep learning model 106 may be trained using the plurality of image channels. One or more techniques, known to a person skilled in the art may be implemented for training the deep learning model 106. Tn an embodiment, back-propagation learning may be implemented for the training. In an embodiment, by the back-propagation learning, optimization for loss minimization may be achieved.

Figure 2 shows a detailed block diagram of the integration unit 101 for integrating the plurality of image channels in the deep learning model 106 for classification of the plurality of objects in the sample 105.

Data 202 and one or more modules 201 in the memory 110 of the integration unit 101 may be described herein in detail.

In one implementation, the one or more modules 201 may include, but are not limited to, a microscopic image receive module 203, an image channel generate module 204, an image channel provide module 205, a model train module 206, and one or more other modules 207, associated with the integration unit 101.

In an embodiment, the data 202 in the memory 110 may comprise microscopic image data 208 (also referred to as at least one microscopic image 208), image channel data 209 (also referred to as plurality of image channels 209), training data 210, and other data 211 associated with the integration unit 101.

In an embodiment, the data 202 in the memory 110 may be processed by the one or more modules 201 of the integration unit 101. Tn an embodiment, the one or more modules 201 may be implemented as dedicated units and when implemented in such a manner, said modules maybe configured with the functionality defined in the present disclosure to result in a novel hardware. As used herein, the term module may refer to an Application Specific Integrated Circuit (ASIC), an electronic circuit, a Field-Programmable Gate Arrays (FPGA), Programmable System-on-Chip (PSoC), a combinational logic circuit, and/or other suitable components that provide the described functionality. Figure 3 shows a detailed block diagram comprising the one or more modules 201 of the integration unit 101 for integrating the plurality of image channels 209 in the deep learning model 106.

The microscopic image receive module 203 may be configured to receive the at least one microscopic image 208 of the sample 105 comprising the plurality of objects. The at least one microscopic image 208 may be received from the image acquisition unit 102. In an embodiment, the microscopic image receive module 203 may be configured to capture the at least one microscopic image 208 using one or more techniques known in the art. In one embodiment, the microscopic image receive module 203 may receive images of a sample from a database or a storage unit associated with at least one of the integration unit 101, the image acquisition unit 102 and the microscopic unit 103.

Consider, the sample 105 may be a urine sample 105 for urinalysis. Figures 5a illustrates an exemplary image 500 of the urine sample acquired by the image acquisition unit 102. The exemplary image 500 may be received by the microscopic image receive module 203 as the at least one microscopic image 208. In an embodiment, the plurality of objects associated with the urine sample may be bacteria 501, crystal 502, RBC 503, WBC 504, yeast 505, WB clumps 506, RBC clumps 507, epithelial 508 and cast 509, as shown in Figure 5b. In the exemplary image 500, a global threshold mask may be performed to obtain circular FOV of the urine sample, as shown in Figure 5a.

Upon receiving the at least one microscopic image 208, the image channel generate module 204 may be configured to generate the plurality of image channels 209 for the at least one microscopic image 208. In an embodiment, the image channel generate module 204 may implement the at least one image operator for generating the plurality of image channels 209. The plurality of image channels 209 comprises the at least one colour image channel and the at least one hand-crafted image channel. The at least one colour image channel may be generated using image operators configured to extract colour image channels from the at least one microscopic image 208. Such image operators may be operators known to a person skilled in the art. The at least one colour image channel may be selected from the colour models such as RBG model, CMYK model, a grey-scale model, HSV model, YUV model and the like. The at least one hand-crafted image channel may be generated using image operators configured to extract hand-crafted image channels from the at least one microscopic image 208. Such image operators may be operators known to a person skilled in the art. The at least one hand crafted image channel may be selected from sharpness channel, edge channel, shape channel, texture channel and so on.

In an embodiment, the image channel generate module 204 may be configured to select the plurality of image channels 209 based on the domain knowledge relating to the classification. The selection of the at least one colour image channel and the at least one hand-crafted image channel may be based on task that is to be performed during the classificati on of the plurality of objects. For example, consider the task in the classification is to classify blurry objects and sharper objects from the plurality of objects in order to reject the blurry objects, hi such case, the at least one hand-crafted image channel may be selected to be the sharpness channel. Consider the RGB is selected as the at least one colour image channel. Therefore, for said task, the plurality of image channels 209 to be generated by the image channel generate module 204 may be red channel, green channel, blue channel, and sharpness channel. Hence, number of the plurality of image channels 209 to be generated may be four. The number of the plurality of image channels 209 to be generated may vary based on selection of the at least one colour image channel and the at least one hand-crafted image channel. In an embodiment, the at least one hand-crafted image channel may be selected based on the task. For example, along with the RGB model as the at least one colour image channel, the at least one hand-crafted image channel may be the sharpness channel and the texture channel. In such case, the number of the plurality of image channels 208 is five. For above example, the least one image operator may be a variance of Laplacian operator for generating the sharpness channel for the at least one microscopic image 208. The variance of Laplacian operator may be indicated as given in equation 1.

where, is discrete convolution operator;

L(x, y) is Laplacian of one of the at least one colour image channel; and

m(x, y) is mean filter.

In an embodiment, using the variance of Laplacian operator on the image 500, sharpness value associated with each of image pixel in the image 500 may be computed. Figures 6 shows exemplary images for sharpness as the at least one hand-crafted image channel generated by the image channel generate module 204. In Figure 6, the plurality of objects may be the crystal 502 and a blurry object 602. In an embodiment, sharpness channel 601 of the crystal 502 and sharpness channel 603 of the blurry object 602 may be obtained using equation 1 on the image 500. One or more other techniques, known to a person skilled in the art, may also be implemented for obtaining sharpness channel of each of the plurality of objects in the image 500. In an embodiment, the sharpness channel associated with the plurality of objects may be localized sharpness. In the localized sharpness, the sharpness values within each of the plurality of objects may be extracted.

In another example, consider the task for the classification may be to identify the plurality of objects based on edges of each of the plurality of objects. For such case, the at least one colour image channel may be selected from one of the colour models and the at least one hand-crafted image channel may be selected to be the edge channel. Consider the at least one colour image channel is selected to be the CMYK model. The plurality of image channels 209 for said task may be the cyan channel, the magenta channel, the yellow channel, and the key channel, along with the edge channel. Therefore, the total number of the plurality of image channels 209 which are to be generated by the image channel generate model 204 are five. At least one image operator configured to generate the edge channel for the image 500 may be implemented in the image channel generate module. Said image operator may be one of Roberts cross edge detector operator, Sobel edge detector operator, Canny edge detector operator, Compass edge detector operator and so on. Any image operator configured to generate the edge channel for an image, known in the art, may be implemented in the image channel generate module 204. Figure 7 shows exemplary images for edge as the at least one hand-crafted image channel generated by the image channel generate module 204. In Figure 7, the plurality of objects may be the crystal 502 and the bacteria 501. In an embodiment, edge channel 701 of the crystal 502 and edge channel 702 of the bacteria 501 may be obtained by using one of the Canny edge detection operator technique and the Sobel edge detector operator on the image 500. One or more other techniques, known to a person skilled in the art, may also be implemented for obtaining edge channels of the plurality of objects in the sample 105.

In another example, consider the task for the classification may include to identify the plurality of objects based on shape associated with each of the plurality of objects. For such case, the at least one colour image channel may be selected from the colour models and the at least one hand-crafted image channel may be selected to be the shape channel. Consider the at least one colour image channel may be selected to be the RGB model. The plurality of image channels 209 for said task may be the red channel, the green channel, the blue channel, and the shape channel associated with each of the plurality of objects. Therefore, the total number of the plurality of image channels 209 which are to be generated by the image channel generate module 204 is four. In an embodiment, the image channel generate module 204 may be configured to select the plurality of image channels 209 based on a cross-validation technique. The cross-validation technique may be implemented on the deep learning model 106 for evaluating results of the deep learning model 106. Output of the cross-validation technique may be used for selecting the plurality of image channels 209. In an embodiment, the cross-validation technique may be implemented in the deep-learning model 106, upon the integration of the plurality of image channels 209. By this, one or more image channels from the plurality of image channels 209 which provide highest performance on validation data, are used for training the deep-learning model. One or more techniques, known to a person skilled in the art, may be implemented for cross-validating the deep-learning model. Said one or more techniques, may include, but are not limited to validation set approach, leave out one cross validation, k-fold cross validation, stratified k-fold cross validation, adversarial validation, cross validation for time series, custom cross validation techniques and so on.

In an embodiment, consider the sample 105 is a multi-layer sample, the image channel generate module 202 may be configured to select the plurality of image channels 209 from the Z-stack of images captured from the plurality of layers of the multi-layer sample. The Z-stack of images may be acquired at a predefined focal depth of the multi-layer sample. In such case, each of the Z-stack of images include different information associated with the plurality of objects, hi an embodiment, said information may be used to extract contextual features associated with the plurality of objects. A context function may be used to determine the relevance of the information of each of the Z-stack of images. For example, the contextual function may be configured to determine optimal images from the Z-stack of images. In an embodiment, the optimal images may be determined based on the one or more attributes associated with the plurality of objects in the multi-layer sample. The optimal images may be further used for extracting the contextual features of the plurality of objects.

In an embodiment, the image channel generate module 204 may generate the plurality of image channels 209 for each of the optimal images which are determined. For example, if five optimal images are determined with highest sharpness values, then the plurality of image channels 209 may be generated for each of the five optimal images. Consider, the at least one colour image channel to be the RGB model and the at least one hand-crafted image channel to be the sharpness channel. In such case, the number of the plurality of image channels 209 to be generated are four for each of the five optimal images. Therefore, total number of the plurality of image channels 209 to be generated are twenty. Tn an embodiment, the generated image channels may be represented in 4-Dimensional (4D) tensor format. In an embodiment, the 4D tensor format of the plurality' of image channels 209 may include dimensions as first dimension to be height, second dimension to be width, third dimension to be the extracted hand-crafted features and fourth dimension to be one of depth associated with the optimal images and the number of the optimal images.

Upon the generation, the image channel provide module 205 may be configured to provide the plurality of image channels 209 to the deep learning model to integrate the plurality of image channels 209 with the deep learning model. In an embodiment, the deep learning model 106 may be one of CNN, RNN, and fusion of CNN and RNN. One or more techniques, known to a person skilled in the art may be used for integrating the plurality of image channels 209 with the deep learning model 106.

Further, in an embodiment, the model train module 206 may be configured to train the deep learning model 106 using the training data 209. The training of the deep-learning model 106 may be performed upon the integration of the plurality of image channels 209. In an embodiment, the training data 209 may include, but is not limited to, the plurality of image channels 208. One or more techniques, known to a person skilled in the art may be implemented for training the deep learning model 106. As described previously, the deep learning model 106 may be implemented to perform the classification of the plurality of objects. The CNN, upon the integration and the training as proposed in the present disclosure, may be configured to perform one or more tasks associated with the classification. In an embodiment, one or more CNN may be implemented for performing each of the one or more tasks in the classification. In an embodiment, the deep learning model 106 may be the RNN. In an embodiment, the deep learning model 106 may be fusion of the CNN and the RNN . When the plurality of image channels 209 is provided in the 4D tensor format, the fusion of the CNN and the RNN may be implemented as the deep learning model 106 for the classification. In an embodiment, when the one or more deep learning models are implemented for the classification, each of the one or more learning models may be one of the CNN model and the RNN model.

Figures 8a shows plot of precision of the deep learning model 106 with and without the plurality of image channels 209 proposed in the present disclosure. The precision of the deep learning model 106 for each of the plurality of objects is evaluated for the deep learning model 106 which are trained using the plurality of image channels 209. Said precision is plotted to compare with precision of the deep learning model 106 which are trained using the plurality of image channels 209. The precision for the deep learning model 106 with the plurality of image channels 209 may be improved for some of the plurality of objects. Similarly, Figures 8b shows plot of sensitivity of the deep learning model 106 with and without the plurality of image channels 209 proposed in the present disclosure. The sensitivity of the deep learning model 106 for each of the plurality of objects is evaluated for the deep learning model 106 which are trained using the plurality of image channels 209. Said sensitivity is plotted to compare with sensitivity of the deep learning model 106 which are trained using the plurality of image channels 209. The sensitivity for the deep learning model 106 with the plurality of image channels 209may be improved for some of the plurality of objects.

Figure 4 illustrates a flowchart showing an exemplary method for integrating the plurality of image channels 209 in the deep learning model 106 for classification of the plurality of objects in the sample 105.

At block 401, the microscopic image receive module 203 may be configured to receive the at least one microscopic image 208 of the sample 105 comprising the plurality of objects. In an embodiment, the plurality of objects in the microscopic image may be observed at the relative natural scale for capturing the at least one microscopic image 208 and receiving the at least one microscopic image 208 for the integration.

At block 402, the image channel generate module 204 may be configured generate to the plurality of image channels 209 for the at least one microscopic image 207. The plurality of image channels 209 may be generated using at least one image operator. The plurality' of image channels 209 includes the at least one colour image channel and the at least one hand -crafted image channel. In an embodiment, the plurality of image channels 209 is selected based on the domain knowledge relating to the classification. In an embodiment, the plurality of image channels 209 is selected based on the cross-validation technique. In an embodiment, the plurality of image channels 209 is selected from the Z-stack of images captured from the plurality of layers of the sample 105.

At block 403, the image channel provide module 205 may be configured to provide the plurality of image channels 209 to the deep learning model 106 to integrate the plurality of image channels 209 with the deep learning model 106 for classification of the plurality of objects. In an embodiment, the deep learning model 106 may be one of CNN, RNN, and fusion of CNN and RNN.

At block 404, the model train module 205 may be configured to train the deep learning model 106 using the training data 210. The training data 210 may include the plurality of image channels 209. One or more techniques, known in the art, may be implemented for training the deep learning model 106.

As illustrated in Figure 4, the method 400 may include one or more blocks for executing processes in the integration unit 101. The method 400 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions or implement particular abstract data types. The order in which the method 400 are described may not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method. Additionally, individual blocks may be deleted from the methods without departing from the scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

Computing System

Figure 9 illustrates a block diagram of an exemplary computer system 900 for implementing embodiments consistent with the present disclosure. In an embodiment, the computer system 900 is used to implement the integration unit 101. The computer system 900 may include a central processing unit ("CPU" or "processor") 902. The processor 902 may include at least one data processor for executing processes in Virtual Storage Area Network. The processor 902 may include specialized processing units such as, integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc.

The processor 902 may be disposed in communication with one or more input/output (I/O) devices 909 and 910 via I/O interface 901. The I/O interface 901 may employ communication protocols/methods such as, without limitation, audio, analog, digital, monaural, RCA, stereo, IEEE- 1394, serial bus, universal serial bus (USB), infrared, PS/2, BNC, coaxial, component, composite, digital visual interface (DVT), high-definition multimedia interface (HDMI), RF antennas, S-Video, VGA, IEEE 702.n /b/g/n/x, Bluetooth, cellular (e.g., code-division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), WiMax, or the like), etc.

Using the I/O interface 901, the computer system 900 may communicate with one or more I/O devices 909 and 910. For example, the input devices 909 may be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touchpad, trackball, stylus, scanner, storage device, transceiver, video device/source, etc. The output devices 910 may be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, Plasma display panel (PDP), Organic light-emitting diode display (OLED) or the like), audio speaker, etc. In some embodiments, the computer system 900 may consist of the integration unit 101. The processor 902 may be disposed in communication with the communication network 911 via a network interface 903. The network interface 903 may communicate with the communication network 911. The network interface 903 may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE 702.11a/b/g/n/x, etc. The communication network 911 may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface 903 and the communication network 91 1, the computer system 900 may communicate with a deep learning model 912 for integrating plurality of image channels 208 in the deep learning model 106 for the classification of plurality of objects in a sample. The network interface 903 may employ connection protocols include, but not limited to, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base T), transmission control protocol/internet protocol (TCP IP), token ring, IEEE 702.1 la/b/g/n/x, etc.

The communication network 911 includes, but is not limited to, a direct interconnection, an e- commerce network, a peer to peer (P2P) network, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, Wi-Fi, and such. The first network and the second network may either be a dedicated network or a shared network, which represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), etc., to communicate with each other. Further, the first network and the second network may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, etc.

In some embodiments, the processor 902 may be disposed in communication with a memory 905 (e.g., RAM, ROM, etc. not shown in Figure 9) via a storage interface 904. The storage interface 904 may connect to memory 905 including, without limitation, memory drives, removable disc drives, etc., employing connection protocols such as, serial advanced technology attachment (SATA), Integrated Drive Electronics (IDE), IEEE-1394, Universal Serial Bus (USB), fibre channel, Small Computer Systems Interface (SCSI), etc. The memory drives may further include a drum, magnetic disc drive, magneto-optical drive, optical drive, Redundant Array of Independent Discs (RAID), solid-state memory devices, solid-state drives, etc.

The memory 905 may store a collection of program or database components, including, without limitation, user interface 906, an operating system 907 etc. In some embodiments, computer system 900 may store user/application data 906, such as, the data, variables, records, etc., as described in this disclosure. Such databases may be implemented as fault-tolerant, relational, scalable, secure databases such as Oracle ® or Sybase®.

The operating system 907 may facili tate resource management and operation of the computer system 900. Examples of operating systems include, without limitation, APPLE MACINTOSH® OS X, UNIX®, UNIX-like system distributions (E.G., BERKELEY SOFTWARE DISTRIBUTION™ (BSD), FREEBSD™, NETBSD™, OPENBSD™, etc), LINUX DISTRIBUTIONS™ (E.G., RED HAT™, UBUNTU™, KUBUNTU™, etc.), IBM™ OS/2, MICROSOFT™ WINDOWS™ (XP™, VISTA™/7/8, 10 etc.), APPLE® IOS™, GOOGLE® ANDROID™, BLACKBERRY® OS, or the like.

Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perfonn steps or stages consistent with the embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include Random Access Memory (RAM), Read-Only Memory (ROM), volatile memory, non-volatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.

Advantages

Embodiments of the present disclosure discloses provision automated integration of image channels with a deep learning model.

Embodiments of the present disclosure provide to additional image channels along with colour image channels to the deep learning model, by which contextual features of the plurality of objects may be extracted. Thereby, accurate and efficient classification of plurality of objects may be achieved.

Embodiments of the present disclosure provide higher precision and higher sensitivity in the deep learning model for the classification.

Embodiments of the present disclosure enables deep learning model to classify plurality of objects even with usage of minimal data of objects.

The described operations may be implemented as a method, system or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The described operations may be implemented as code maintained in a "non-transitory computer readable medium", where a processor may read and execute the code from the computer readable medium. The processor is at least one of a microprocessor and a processor capable of processing and executing the queries. A non- transitory computer readable medium may include media such as magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, DVDs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, Flash Memory, firmware, programmable logic, etc.), etc. Further, non-transitory computer-readable media may include all computer-readable media except for a transitory. The code implementing the described operations may further be implemented in hardware logic (e.g., an integrated circuit chip, Programmable Gate Array (PGA), Application Specific Integrated Circuit (ASIC), etc.).

Still further, the code implementing the described operations may be implemented in "transmission signals", where transmission signals may propagate through space or through a transmission media, such as, an optical fibre, copper wire, etc. The transmission signals in which the code or logic is encoded may further comprise a wireless signal, satellite transmission, radio waves, infrared signals, Bluetooth, etc. The transmission signals in which the code or logic is encoded is capable of being transmitted by a transmitting station and received by a receiving station, where the code or logic encoded in the transmission signal may be decoded and stored in hardware or a non-transitory computer readable medium at the receiving and transmitting stations or devices. An "article of manufacture" includes non- transitory computer readable medium, hardware logic, and/or transmission signals in which code may be implemented. A device in which the code implementing the described embodiments of operations is encoded may include a computer readable medium or hardware logic. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the invention, and that the article of manufacture may include suitable information bearing medium known in the art.

The terms "an embodiment", "embodiment", "embodiments", "the embodiment", "the embodiments", "one or more embodiments", "some embodiments", and "one embodiment" mean "one or more (but not all) embodiments of the invention(s)" unless expressly specified otherwise.

The terms "including", "comprising", having" and variations thereof mean "including but not limited to", unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise. The terms "a", "an" and "the" mean "one or more", unless expressly specified otherwise.

A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary a variety of optional components are described to illustrate the wide variety of possible embodiments of the invention. When a single device or article is described herein, it will be readily apparent that more than one device/article (whether or not they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device/article may be used in place of the more than one device or article or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the invention need not include the device itself.

The illustrated operations of Figure 4 shows certain events occurring in a certain order. In alternative embodiments, certain operations may be performed in a different order, modified, or removed. Moreover, steps may be added to the above described logic and still conform to the described embodiments. Further, operations described herein may occur sequentially or certain operations may be processed in parallel. Yet further, operations may be performed by a single processing unit or by distributed processing units.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims. While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims. Referral numerals;

Claims

We claim:

11. A method for integrating image channels in a deep learning model (106) for classification of objects in a sample (105), comprising:

receiving, by an integration unit (101), at least one microscopic image (208) of a sample (105) comprising plurality of objects;

generating, by the integration unit (101), plurality of image channels (209) for the at least one microscopic image (208) using at least one image operator, wherein the plurality of image channels (209) comprises at least one colour image channel and at least one hand-crafted image channel; and

providing, by the integration unit (101), the plurality of image channels (209) to a deep learning model to integrate the plurality of image channels (209) with the deep learning model for classification of the plurality of objects.

2. The method as claimed in claim 1 further comprising training, by the integration unit (101), the deep learning model using the plurality of image channels (209).

3. The method as claimed in claim 1, wherein the plurality of image channels (209) is selected based on domain knowledge relating to the classification.

4. The method as claimed in claim 3, wherein the at least one hand-crafted image channel is selected from at least one of morphological information, textual information and sharpness information derived from the domain knowledge.

5. The method as claimed in claim 1, wherein the plurality of image channels (209) is selected based on a cross-validation technique.

6. The method as claimed in claim 1 , wherein the plurality of image channels (209) is selected from Z-stack of images captured from plurality of layers of the sample (105).

7. The method as claimed in claim 1, wherein the deep learning model may be one of Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and fusion of CNN and RNN.

8. An integration unit for integrating image channels in a deep learning model (106) for classification of objects in a sample (105), comprises: a processor (107); and

a memory (110) communicatively coupled to the processor (107), wherein the memory (109) stores processor-executable instructions, which, on execution, cause the processor (107) to:

receive at least one microscopic image (208) of a sample (105) comprising plurality of objects;

generate plurality of image channels (209) for the at least one microscopic image (208) using at least one image operator, wherein the plurality of image channels (209) comprises at least one colour image channel and at least one hand-crafted image channel; and

provide the plurality of image channels (209) to a deep learning model to integrate the plurality of image channels (209) with the deep learning model for classification of the plurality of objects.

9. The integration unit as claimed in claim 8 further comprises the processor (107) configured to train the deep learning model using the plurality of image channels (209).

10. The integration unit as claimed in claim 8, wherein the plurality of image channels (209) is selected based on domain knowledge relating to the classification.

11. The integration unit as claimed in claim 10, wherein the at least one hand-crafted image channel is selected from at least one of morphological information, textual information and sharpness information derived from the domain knowledge.

12. The integration unit as claimed in claim 8, wherein the plurality of image channels (209) is selected based on a cross-validation technique.

13. The integration unit as claimed in claim 8, wherein the plurality of image channels (209) is selected from Z-stack of images captured from plurality of layers of the sample (105).

14. The integration unit as claimed in claim 8, wherein the deep learning model may be one of Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and fusion of CNN and RNN.