US20190122104A1

US20190122104A1 - Building a binary neural network architecture

Info

Publication number: US20190122104A1
Application number: US15/855,015
Authority: US
Inventors: Min Zhang; Gopal Biligeri Avinash; Sharath Pasupunuti
Original assignee: General Electric Co
Current assignee: General Electric Co
Priority date: 2017-10-19
Filing date: 2017-12-27
Publication date: 2019-04-25
Also published as: WO2019078914A1; US20190122364A1; CN111492372B; US10796221B2; US20210374505A1; JP2021500662A; EP3698277A1; WO2019078923A1; US10607135B2; CN111433785B; CN111433785A; US20190122075A1; US20190122074A1; EP3698276A1; JP7053818B2; CN111492372A; US11093820B2; WO2019078924A1

Abstract

Systems and techniques for building a binary neural network architecture are presented. In one example, a system trains a neural network based on a data set to form a first neural network of a binary neural network architecture and determine whether a first class exists. The system also trains a copy of the first neural network based on the data set to form a second neural network of the binary neural network architecture and determine whether a second class exists. Furthermore, the system trains a copy of the second neural network based on the data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, where M is an integer greater than or equal to three.

Description

TECHNICAL FIELD

This application claims priority to U.S. Provisional Application No. 62/574,333, filed Oct. 19, 2017, and entitled “DEEP LEARNING ARCHITECTURE FOR AUTOMATED IMAGE FEATURE EXTRACTION”, the entirety of which is incorporated herein by reference.

RELATED APPLICATION

This disclosure relates generally to artificial intelligence.

BACKGROUND

Artificial Intelligence (AI) can be employed for classification and/or analysis of digital images. For instance, AI can be employed for image recognition. In certain technical applications, AI can be employed to enhance imaging analysis. In an example, region-of-interest based deep neural networks can be employed to localize a feature in a digital image. However, accuracy and/or efficiency of a classification and/or an analysis of digital images using conventional artificial techniques is generally difficult to achieve. Furthermore, conventional artificial techniques for classification and/or analysis of digital images generally requires labor -intensive processes such as, for example, pixel annotations, voxel level annotations, etc. As such, conventional artificial techniques for classification and/or analysis of digital images can be improved.

SUMMARY

The following presents a simplified summary of the specification in order to provide a basic understanding of some aspects of the specification. This summary is not an extensive overview of the specification. It is intended to neither identify key or critical elements of the specification, nor delineate any scope of the particular implementations of the specification or any scope of the claims. Its sole purpose is to present some concepts of the specification in a simplified form as a prelude to the more detailed description that is presented later.
According to an embodiment, a system includes a neural network training component and a neural network duplication component. The neural network training component trains a neural network based on a data set to form a first neural network of a binary neural network architecture and determine whether a first class exists. The neural network duplication component trains a copy of the first neural network based on the data set to form a second neural network of the binary neural network architecture and determine whether a second class exists. The neural network duplication component also trains a copy of the second neural network based on the data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, where M is an integer greater than or equal to three.
According to another embodiment, a method is provided. The method provides for using a processor operatively coupled to memory to execute computer executable components to perform acts such as training a neural network based on an image data set to generate a first neural network of a binary neural network architecture and determine whether a first class exists. The method also provides for acts such as training a copy of the first neural network based on the image data set to generate a second neural network of the binary neural network architecture and determine whether a second class exists. Furthermore, the method provides for acts such as training a copy of the second neural network based on the image data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, where M is an integer greater than or equal to three.
According to yet another embodiment, a computer readable storage device comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising: training a neural network based on an image data set to generate a first neural network and determine whether a first class exists, training a copy of the first neural network based on the image data set to generate a second neural network and determine whether a second class exists, training a copy of the second neural network based on the image data set to form an Mth neural network and determine whether an Mth class exists, where M is an integer greater than or equal to three, and generating a neural network architecture that includes the first neural network, the second neural network and the Mth neural network.
The following description and the annexed drawings set forth certain illustrative aspects of the specification. These aspects are indicative, however, of but a few of the various ways in which the principles of the specification may be employed. Other advantages and novel features of the specification will become apparent from the following detailed description of the specification when considered in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Numerous aspects, implementations, objects and advantages of the present invention will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:

FIG. 1 illustrates a high-level block diagram of an example deep neural network builder component, in accordance with various aspects and implementations described herein;

FIG. 2 illustrates an example system for building a binary neural network architecture, in accordance with various aspects and implementations described herein;

FIG. 3 illustrates an example neural network architecture, in accordance with various aspects and implementations described herein;

FIG. 4 illustrates an inference phase associated with an example neural network architecture, in accordance with various aspects and implementations described herein;

FIG. 5 illustrates a high-level block diagram of an example deep learning component, in accordance with various aspects and implementations described herein;

FIG. 6 illustrates a high-level block diagram of an example spring block associated with a neural network, in accordance with various aspects and implementations described herein;

FIG. 7 illustrates an example user interface, in accordance with various aspects and implementations described herein;

FIG. 8 depicts a flow diagram of an example method for building a binary neural network architecture, in accordance with various aspects and implementations described herein;

FIG. 9 is a schematic block diagram illustrating a suitable operating environment; and

FIG. 10 is a schematic block diagram of a sample-computing environment.

DETAILED DESCRIPTION

Various aspects of this disclosure are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It should be understood, however, that certain aspects of this disclosure may be practiced without these specific details, or with other methods, components, materials, etc. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects.
Systems and techniques for building a binary neural network architecture are presented. For example, as compared to conventional artificial intelligence (AI) techniques, the subject innovations provide for a novel AI framework that includes a novel binary neural network architecture. In an embodiment, a binary neural network architecture can be built by training a neural network based on a data set to form a first neural network of the binary neural network architecture and determine whether a first class exists. The data set can be, for example, an image data set (e.g., a set of digital images, a set of medical imaging data, etc.). Furthermore, a copy of the first neural network can be trained based on the data set to form a second neural network of the binary neural network architecture determine whether a second class exists. A copy of the second neural network can also be trained based on the data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, where M is an integer greater than or equal to three. As such, the binary neural network architecture can include the first neural network, the second neural network and the Mth neural network. In an implementation, respective neural networks of the binary neural network architecture can generate mutually exclusive outputs. Additionally, in an aspect, training of the neural network(s) for the binary neural network architecture can be performed in a concatenating manner on respective classes. Therefore, by employing the novel AI framework as described herein, detection and/or localization of one or more features associated with a data set (e.g., detection and/or localization of one or more diseases for a patient associated with medical imaging data) can be improved. Furthermore, accuracy and/or efficiency for classification and/or analysis of a data set (e.g., a set of digital images, medical imaging data, etc.) can be improved. Moreover, effectiveness of a machine learning model for classification and/or analysis of a data set (e.g., a set of digital images, medical imaging data, etc.) can be improved, performance of one or more processors that execute a machine learning model for classification and/or analysis of a data set (e.g., a set of digital images, medical imaging data, etc.) can be improved, and/or efficiency of one or more processors that execute a machine learning model for classification and/or analysis of a data set (e.g., a set of digital images, medical imaging data, etc.) can be improved.
Referring initially to FIG. 1, there is illustrated an example system 100 for building a binary neural network architecture, according to an aspect of the subject disclosure. The system 100 can be employed by various systems, such as, but not limited to medical device systems, medical imaging systems, medical diagnostic systems, medical systems, medical modeling systems, enterprise imaging solution systems, advanced diagnostic tool systems, simulation systems, image management platform systems, care delivery management systems, artificial intelligence systems, machine learning systems, neural network systems, modeling systems, aviation systems, power systems, distributed power systems, energy management systems, thermal management systems, transportation systems, oil and gas systems, mechanical systems, machine systems, device systems, cloud-based systems, heating systems, HVAC systems, medical systems, automobile systems, aircraft systems, water craft systems, water filtration systems, cooling systems, pump systems, engine systems, prognostics systems, machine design systems, and the like. In one example, the system 100 can be associated with a classification system to facilitate visualization and/or interpretation of medical imaging data. Moreover, the system 100 and/or the components of the system 100 can be employed to use hardware and/or software to solve problems that are highly technical in nature (e.g., related to processing digital data, related to processing medical imaging data, related to medical modeling, related to medical imaging, related to artificial intelligence, etc.), that are not abstract and that cannot be performed as a set of mental acts by a human.
The system 100 can include a deep neural network builder component 102 that can include a neural network training component 104 and a neural network duplication component 106. Aspects of the systems, apparatuses or processes explained in this disclosure can constitute machine-executable component(s) embodied within machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described. The system 100 (e.g., the deep neural network builder component 102) can include memory 110 for storing computer executable components and instructions. The system 100 (e.g., the deep neural network builder component 102) can further include a processor 108 to facilitate operation of the instructions (e.g., computer executable components and instructions) by the system 100 (e.g., the deep neural network builder component 102).
The deep neural network builder component 102 can receive medical imaging data (e.g., MEDICAL IMAGING DATA shown in FIG. 1). The medical imaging data can be two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by one or more medical imaging devices. For instance, the medical imaging data can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the medical imaging data can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The medical imaging data can be received directly from one or more medical imaging devices. Alternatively, the medical imaging data can be stored in one or more databases that receives and/or stores the medical imaging data associated with the one or more medical imaging devices. A medical imaging device can be, for example, an x-ray device, a computed tomography (CT) device, another type of medical imaging device, etc. It is to be appreciated that, in certain embodiments, the medical imaging data can be a different type of data set (e.g., different type of images, etc.).
The neural network training component 104 can train a neural network based on the medical imaging data to determine whether a first class exists in the medical imaging data. Furthermore, the neural network training component 104 can train the neural network based on the medical imaging data to form a first neural network of a neural network architecture. The neural network architecture can be, for example, a binary neural network architecture that performs machine learning associated with one or more binary classifications for the medical imaging data. In an aspect, the first neural network can generate mutually exclusive outputs. The mutually exclusive outputs of the first neural network can be related to a feature detector of the first neural network. For example, the mutually exclusive outputs of the first neural network can provide a condition vs non-condition prediction (e.g. a yes/no prediction) for the first class.
In an embodiment, the first neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the first neural network can be, for example, a spring network of convolutional layers. For instance, the first neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the first neural network. In an example, the first neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the first neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the first neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
The neural network duplication component 106 can train a copy of the first neural network based on the medical imaging data to determine whether a second class exists in the medical imaging data. Furthermore, the neural network duplication component 106 can train the copy of the first neural network based on the medical imaging data to form a second neural network of the neural network architecture (e.g., the binary neural network architecture). In an aspect, the second neural network can generate mutually exclusive outputs. The mutually exclusive outputs of the second neural network can be related to a feature detector of the second neural network. For example, the mutually exclusive outputs of the second neural network can provide a condition vs non-condition prediction (e.g. a yes/no prediction) for the second class.
In an embodiment, the second neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the second neural network can be, for example, a spring network of convolutional layers. For instance, the second neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the second neural network. In an example, the second neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the second neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the second neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
Additionally, the neural network duplication component 106 can train a copy of the second neural network based on the medical imaging data to determine whether an Mth class exists in the medical imaging data. Furthermore, the neural network duplication component 106 can train the copy of the second neural network based on the medical imaging data to form an Mth neural network of the neural network architecture (e.g., the binary neural network architecture), where M is an integer greater than or equal to three. For example, the neural network duplication component 106 can train a copy of the second neural network based on the medical imaging data to determine whether a third class exists in the medical imaging data and to form a third neural network of the neural network architecture (e.g., the binary neural network architecture), the neural network duplication component 106 can train a copy of the third neural network based on the medical imaging data to determine whether a fourth class exists in the medical imaging data and to form a fourth neural network of the neural network architecture (e.g., the binary neural network architecture), etc. In an aspect, the Mth neural network can generate mutually exclusive outputs. The mutually exclusive outputs of the Mth neural network can be related to a feature detector of the Mth neural network. For example, the mutually exclusive outputs of the Mth neural network can provide a condition vs non-condition prediction (e.g. a yes/no prediction) for the Mth class. In an embodiment, the Mth neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the Mth neural network.
In an embodiment, the Mth neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the Mth neural network can be, for example, a spring network of convolutional layers. For instance, the Mth neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the Mth neural network. In an example, the Mth neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the Mth neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the Mth neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
In another embodiment, training of the neural network with respect to the first class, training of the copy of the neural network with respect to the second class, and training of the copy of the second neural network with respect to the Mth class are performed in a concatenating manner can be performed in a concatenating manner For example, training of the neural network with respect to the first class and training of the copy of the neural network with respect to the second class are performed in a concatenating manner Furthermore, training of the copy of the first neural network with respect to the second class and training of the copy of the second neural network with respect to the Mth class are performed in a concatenating manner. As such, training of the neural network(s) for the neural network architecture (e.g., the binary neural network architecture) can be staggered to provide an improved neural network architecture. In yet another embodiment, the deep neural network builder component 102 can generate network architecture data (e.g., NETWORK ARCHITECTURE DATA shown in FIG. 1). The network architecture data can include data associated with the first neural network, the second neural network and/or the Mth neural network of the neural network architecture (e.g., the binary neural network architecture). For example, network architecture data can include a set of weights and/or a set of biases for the first neural network, the second neural network and/or the Mth neural network of the neural network architecture (e.g., the binary neural network architecture).
It is to be appreciated that technical features of the deep neural network builder component 102 are highly technical in nature and not abstract ideas. Processing threads of the deep neural network builder component 102 that process and/or analyze the medical imaging data, determine outlier medical imaging data, etc. cannot be performed by a human (e.g., are greater than the capability of a single human mind). For example, the amount of the medical imaging data processed, the speed of processing of the medical imaging data and/or the data types of the medical imaging data processed by the deep neural network builder component 102 over a certain period of time can be respectively greater, faster and different than the amount, speed and data type that can be processed by a single human mind over the same period of time. Furthermore, the medical imaging data processed by the deep neural network builder component 102 can be one or more medical images generated by sensors of a medical imaging device. Moreover, the deep neural network builder component 102 can be fully operational towards performing one or more other functions (e.g., fully powered on, fully executed, etc.) while also processing the medical imaging data.
Referring now to FIG. 2, there is illustrated a non-limiting implementation of a system 200 in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.
The system 200 can include a deep neural network builder 202. The deep neural network builder 202 can correspond to functionality of the deep neural network builder component 102. In an aspect, the deep neural network builder 202 can receive medical imaging data 204. The medical imaging data 204 can correspond to the medical imaging data received by the deep neural network builder component 102 in FIG. 1. For instance, the medical imaging data 204 can be two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by one or more medical imaging devices. For instance, the medical imaging data 204 can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the medical imaging data 204 can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The medical imaging data 204 can be received directly from one or more medical imaging devices. Alternatively, the medical imaging data 204 can be stored in one or more databases that receives and/or stores the medical imaging data 204 associated with the one or more medical imaging devices. A medical imaging device can be, for example, an x-ray device, a CT device, another type of medical imaging device, etc. It is to be appreciated that, in certain embodiments, the medical imaging data 204 can be a different type of data set (e.g., different type of images, etc.).
In an embodiment, a neural network 206 can receive the medical imaging data 204. The neural network 206 can be trained based on the medical imaging data 204 to determine whether a first class exists in the medical imaging data 204. For example, the neural network 206 can be trained based on the medical imaging data 204 to determine a first output category classification for the medical imaging data 204. The neural network 206 can also be trained based on the medical imaging data 204 to form a first neural network 208. The first neural network 208 can be, for example, a trained version of the neural network 206. In an aspect, the first neural network 208 can generate mutually exclusive outputs based on the first class. For example, the first neural network 208 can generate either a yes output or a no output related to the first class.
In another embodiment, the first neural network 208 can be copied to generate a first neural network copy 210. The first neural network copy 210 can be trained based on the medical imaging data 204 to determine whether a second class exists in the medical imaging data 204. For example, the first neural network copy 210 can be trained based on the medical imaging data 204 to determine a second output category classification for the medical imaging data 204. In an aspect, the first neural network copy 210 can be pre-trained based on the neural network 206. The first neural network copy 210 can also be trained based on the medical imaging data 204 to form a second neural network 212. The second neural network 212 can be, for example, a trained version of the first neural network copy 210. In an aspect, the second neural network 212 can generate mutually exclusive outputs based on the second class. For example, the second neural network 212 can generate either a yes output or a no output related to the second class.
In yet another embodiment, the second neural network 212 can be copied to generate a second neural network copy 214. The second neural network copy 214 can be trained based on the medical imaging data 204 to determine whether an Mth class exists in the medical imaging data 204. For example, the second neural network copy 214 can be trained based on the medical imaging data 204 to determine an Mth output category classification for the medical imaging data 204. In an aspect, the second neural network copy 214 can be pre-trained based on the first neural network 208. The second neural network copy 214 can also be trained based on the medical imaging data 204 to form an Mth neural network 216. The Mth neural network 216 can be, for example, a trained version of the second neural network copy 214. In an aspect, the Mth neural network 216 can generate mutually exclusive outputs based on the Mth class. For example, the Mth neural network 216 can generate either a yes output or a no output related to the Mth class.
Referring now to FIG. 3, there is illustrated a non-limiting implementation of a system 300 in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.
The system 300 can include a neural network architecture 302. The neural network architecture 302 can be, for example, a binary neural network architecture. The neural network architecture 302 can include the first neural network 208, the second neural network 212 and the Mth neural network 216. In an aspect, the first neural network 208 can be a first feature detector of the neural network architecture 302, the second neural network 212 can be a second feature detector of the neural network architecture 302, and the Mth neural network 216 can be an Mth feature detector of the neural network architecture 302. For instance, the first neural network 208 can be trained to identify a first feature from medical imaging data, the second neural network 212 can be trained to identify a second feature from medical imaging data, and the Mth neural network 216 can be trained to identify an Mth feature from medical imaging data. As such, a more robust, trained neural network architecture can be provided.
Referring now to FIG. 4, there is illustrated a non-limiting implementation of a system 400 in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.
The system 400 can include the neural network architecture 302. The neural network architecture 302 can include the first neural network 208, the second neural network 212 and the Mth neural network 216. The neural network architecture 302 can be employed for an inference phase. For example, the first neural network 208 can perform a machine learning process (e.g., an artificial intelligence process for machine learning) related to feature detection in medical imaging data 402. The medical imaging data 402 can be two-dimensional medical imaging data and/or three -dimensional medical imaging data generated by one or more medical imaging devices. For instance, the medical imaging data 402 can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the medical imaging data 402 can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The medical imaging data 402 can be received directly from one or more medical imaging devices. Alternatively, the medical imaging data 402 can be stored in one or more databases that receives and/or stores the medical imaging data 402 associated with the one or more medical imaging devices. A medical imaging device can be, for example, an x -ray device, a CT device, another type of medical imaging device, etc. It is to be appreciated that, in certain embodiments, the medical imaging data 402 can be a different type of data set (e.g., different type of images, etc.).
In an embodiment, the first neural network 208 can be, for example, a spring network of convolutional layers. For instance, the first neural network 208 can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data 402 associated with convolutional layers of the first neural network 208. In an example, the first neural network 208 can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data 402 and a second convolutional layer process associated with sequential upsampling of the medical imaging data 402. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the first neural network 208 can alter convolutional layer filters similar to functionality of a spring. For instance, the first neural network 208 can analyze the medical imaging data 402 for feature detection based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
Additionally or alternatively, the second neural network 212 can perform a machine learning process (e.g., an artificial intelligence process for machine learning) related to feature detection in the medical imaging data 402. In an aspect, the second neural network 212 can be, for example, a spring network of convolutional layers. For instance, the second neural network 212 can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data 402 associated with convolutional layers of the second neural network 212. In an example, the second neural network 212 can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data 402 and a second convolutional layer process associated with sequential upsampling of the medical imaging data 402. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the second neural network 212 can alter convolutional layer filters similar to functionality of a spring. For instance, the second neural network 212 can analyze the medical imaging data 402 for feature detection based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
Additionally or alternatively, the Mth neural network 216 can perform a machine learning process (e.g., an artificial intelligence process for machine learning) related to feature detection in the medical imaging data 402. In an aspect, the Mth neural network 216 can be, for example, a spring network of convolutional layers. For instance, the Mth neural network 216 can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data 402 associated with convolutional layers of the Mth neural network 216. In an example, the Mth neural network 216 can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data 402 and a second convolutional layer process associated with sequential upsampling of the medical imaging data 402. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the Mth neural network 216 can alter convolutional layer filters similar to functionality of a spring. For instance, the Mth neural network 216 can analyze the medical imaging data 402 for feature detection based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
Referring now to FIG. 5, there is illustrated a non-limiting implementation of a system 500 in accordance with various aspects and implementations of this disclosure. Repetitive description of like elements employed in other embodiments described herein is omitted for sake of brevity.
The system 500 can include a deep learning component 502 that can include a machine learning component 504, a medical imaging diagnosis component 506 and a visualization component 508. Aspects of the systems, apparatuses or processes explained in this disclosure can constitute machine-executable component(s) embodied within machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described. The system 500 (e.g., the deep learning component 502) can include memory 512 for storing computer executable components and instructions. The system 500 (e.g., the deep learning component 502) can further include a processor 510 to facilitate operation of the instructions (e.g., computer executable components and instructions) by the system 500 (e.g., the deep learning component 502).
The deep learning component 502 (e.g., the machine learning component 504) can receive medical imaging data (e.g., MEDICAL IMAGING DATA shown in FIG. 5). The medical imaging data can be two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by one or more medical imaging devices. For instance, the medical imaging data can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the medical imaging data can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The medical imaging data can be received directly from one or more medical imaging devices. Alternatively, the medical imaging data can be stored in one or more databases that receives and/or stores the medical imaging data associated with the one or more medical imaging devices. A medical imaging device can be, for example, an x-ray device, a CT device, another type of medical imaging device, etc. The machine learning component 504 can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the machine learning component 504 can perform deep learning to facilitate classification and/or localization of one or more diseases associated with the medical imaging data. In another aspect, the machine learning component 504 can perform deep learning based on a convolutional neural network that receives the medical imaging data.
In an embodiment, the machine learning component 504 can perform an inference phase using the neural network architecture 302. For example, the medical imaging data can be a medical image for an anatomical region of a patient associated with the medical image. For the inference phase associated with the neural network architecture 302, the machine learning component 504 can generate learned medical imaging output regarding an anatomical region based on the neural network architecture 302 that receives medical imaging data. In an aspect, the machine learning component 504 can employ the first neural network 208, the second neural network 212 and/or the Mth neural network 216. In certain embodiments, the machine learning component 504 can extract information that is indicative of correlations, inferences and/or expressions from the medical imaging data based on the neural network architecture 302. The machine learning component 504 can generate the learned medical imaging output based on the execution of the neural network architecture 302. The learned medical imaging output generated by the machine learning component 504 can include, for example, learning, correlations, inferences and/or expressions associated with the medical imaging data. In an aspect, the machine learning component 504 can perform learning with respect to the medical imaging data explicitly or implicitly using the neural network architecture 302. The machine learning component 504 can also employ an automatic classification system and/or an automatic classification process to facilitate analysis of the medical imaging data. For example, the machine learning component 504 can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to learn and/or generate inferences with respect to the medical imaging data. The machine learning component 504 can employ, for example, a support vector machine (SVM) classifier to learn and/or generate inferences for medical imaging data. Additionally or alternatively, the machine learning component 504 can employ other classification techniques associated with Bayesian networks, decision trees and/or probabilistic classification models. Classifiers employed by the machine learning component 504 can be explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via receiving extrinsic information). For example, with respect to SVM's, SVM's can be configured via a learning or training phase within a classifier constructor and feature selection module. A classifier can be a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class—that is, f(x)=confidence(class).
The medical imaging diagnosis component 506 can employ information provided by the machine learning component 504 (e.g., the learned medical imaging output) to classify and/or localize a disease associated with the medical imaging data. In an embodiment, the medical imaging diagnosis component 506 can determine a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the neural network architecture 302. In certain embodiments, the medical imaging diagnosis component 506 can determine one or more confidence scores for the classification and/or the localization. For example, a first portion of the anatomical region with a greatest likelihood of a disease can be assigned a first confidence score, a second portion of the anatomical region with a lesser degree of likelihood of a disease can be assigned a second confidence score, etc. A disease classified and/or localized by the medical imaging diagnosis component 506 can include, for example, a lung disease, a heart disease, a tissue disease, a bone disease, a tumor, a cancer, tuberculosis, cardiomegaly, hypoinflation of a lung, opacity of a lung, hyperdistension, a spine degenerative disease, calcinosis, or another type of disease associated with an anatomical region of a patient body. In an aspect, the medical imaging diagnosis component 506 can determine a prediction for a disease associated with the medical imaging data. For example, the medical imaging diagnosis component 506 can determine a probability score for a disease associated with the medical imaging data (e.g., a first percentage value representing likelihood of a negative prognosis for the disease and a second value representing a likelihood of a positive prognosis for the disease).
The visualization component 508 can generate deep learning data (e.g., DEEP LEARNING DATA shown in FIG. 5) based on the classification and/or the localization for the portion of the anatomical region. In an embodiment, the deep learning data can include a classification and/or a location for one or more diseases located in the medical imaging data. In certain embodiments, the deep learning data can include probability data indicative of a probability for one or more diseases being located in the medical imaging data. The probability data can be, for example, a probability array of data values for one or more diseases being located in the medical imaging data. In another embodiment, the visualization component 508 can generate a multi-dimensional visualization associated with the classification and/or the localization for the portion of the anatomical region. The multi-dimensional visualization can be a graphical representation of the medical imaging data that shows a classification and/or a location of one or more diseases with respect to a patient body. The visualization component 508 can also generate a display of the multi -dimensional visualization of the diagnosis provided by the medical imaging diagnosis component 506. For example, the visualization component 508 can render a 2D visualization of the portion of the anatomical region on a user interface associated with a display of a user device such as, but not limited to, a computing device, a computer, a desktop computer, a laptop computer, a monitor device, a smart device, a smart phone, a mobile device, a handheld device, a tablet, a portable computing device or another type of user device associated with a display. In an aspect, the multi-dimensional visualization can include the deep learning data. The deep learning data associated with the multi-dimensional visualization can be indicative of a visual representation of the classification and/or the localization for the portion of the anatomical region. The deep learning data can also be rendered on the 3D model as one or more dynamic visual elements. In an aspect, the visualization component 508 can alter visual characteristics (e.g., color, size, hues, shading, etc.) of at least a portion of the deep learning data associated with the multi-dimensional visualization based on the classification and/or the localization for the portion of the anatomical region. For example, the classification and/or the localization for the portion of the anatomical region can be presented as different visual characteristics (e.g., colors, sizes, hues or shades, etc.), based on a result of deep learning and/or medical imaging diagnosis by the machine learning component 504 and/or the medical imaging diagnosis component 506. In another aspect, the visualization component 508 can allow a user to zoom into or out with respect to the deep learning data associated with the multi-dimensional visualization. For example, the visualization component 508 can allow a user to zoom into or out with respect to a classification and/or a location of one or more diseases identified in the anatomical region of the patient body. As such, a user can view, analyze and/or interact with the deep learning data associated with the multi-dimensional visualization.
It is to be appreciated that technical features of the deep learning component 502 are highly technical in nature and not abstract ideas. Processing threads of the deep learning component 502 that process and/or analyze the medical imaging data, determine deep learning data, etc. cannot be performed by a human (e.g., are greater than the capability of a single human mind). For example, the amount of the medical imaging data processed, the speed of processing of the medical imaging data and/or the data types of the medical imaging data processed by the deep learning component 502 over a certain period of time can be respectively greater, faster and different than the amount, speed and data type that can be processed by a single human mind over the same period of time. Furthermore, the medical imaging data processed by the deep learning component 502 can be one or more medical images generated by sensors of a medical imaging device. Moreover, the deep learning component 502 can be fully operational towards performing one or more other functions (e.g., fully powered on, fully executed, etc.) while also processing the medical imaging data.
Referring now to FIG. 6, there is illustrated a non-limiting implementation of a system 600 in accordance with various aspects and implementations of this disclosure. The system 600 can illustrate an example spring block for a deep learning architecture associated with a neural network. For example, the system 600 can correspond to a neural network included in the neural network architecture 302. In another example, the system 600 can correspond to the first neural network 208, the second neural network 212 and/or the Mth neural network 216. The spring block associated with the system 600 can be associated with sequential upsampling and downsampling for a spring deep learning network. In an aspect, the spring block associated with the system 600 can consist of connected pair down sampling/up sampling layers and convolutional layers. The spring block associated with the system 600 can also be very flexible in terms of depth (e.g., number of paired up/down sampling convolutional layers) and/or size of convolutional filters (e.g. a convolutional filter size equal to 3×3, a convolutional filter size equal to 5×5, a convolutional filter size equal to 7×7, etc.).
In an embodiment, the system 600 can include a convolutional layer 611. The convolutional layer 611 can be a first convolutional layer of a convolutional neural network that processes imaging data. Furthermore, the convolutional layer 611 can be associated with a first filter size. The convolutional layer 611 can be followed by a pooling layer (down) 612. The pooling layer (down) 612 can be associated with downsampling. For instance, the pooling layer (down) 612 can reduce dimensionality of data generated by the convolutional layer 611. In one example, the pooling layer (down) 612 can reduce dimensionality of a feature map for imaging data processed by the convolutional layer 611. The pooling layer (down) 612 can be followed by a convolutional layer 613. The convolutional layer 613 can be a second convolutional layer of the convolutional neural network that processes the imaging data. Furthermore, the convolutional layer 613 can be associated with a second filter size that is different than the first filter size associated with the convolutional layer 611. For example, the second filter size associated with the convolutional layer 613 can be smaller than the first filter size associated with the convolutional layer 611. The convolutional layer 613 can be followed by a pooling layer (down) 614. The pooling layer (down) 614 can be associated with downsampling. For instance, the pooling layer (down) 614 can reduce dimensionality of data generated by the convolutional layer 613. In one example, the pooling layer (down) 614 can reduce dimensionality of a feature map for imaging data processed by the convolutional layer 613. The pooling layer (down) 614 can be followed by a convolutional layer (not shown), which, in turn, can be followed by a pooling layer (up) 615. However, in certain embodiments, the pooling layer (down) 614 can be followed by one or more other convolutional layers and/or one or more other pooling layers (down) prior to the pooling layer (up) 615 to further process imaging data with different filter sizes and/or further reduction to dimensionality of data. The pooling layer (up) 615 can be associated with upsampling. For instance, the pooling layer (up) 615 can increase dimensionality of data generated by one or more convolutional layers. In one example, the pooling layer (up) 615 can increase dimensionality of a feature map for imaging data processed by one or more convolutional layers. The pooling layer (up) 615 can be followed by a convolutional layer 616. The convolutional layer 616 can be, for example, a third convolutional layer of the convolutional neural network that processes the imaging data. Furthermore, the convolutional layer 616 can be associated with the second filter size associated with the convolutional layer 613.
The convolutional layer 616 can be followed by a pooling layer (up) 617. The pooling layer (up) 617 can be associated with upsampling. For instance, the pooling layer (up) 617 can increase dimensionality of data generated by the convolutional layer 616. In one example, the pooling layer (up) 617 can increase dimensionality of a feature map for imaging data processed by the convolutional layer 616. The pooling layer (up) 617 can be followed by a convolutional layer 618. The convolutional layer 618 can be, for example, a fourth convolutional layer of the convolutional neural network that processes the imaging data. Furthermore, the convolutional layer 618 can be associated with the first filter size associated with the convolutional layer 616. As such, the spring block associated with the system 600 can behave similar to functionality of a spring where a filter size for one or more convolutional layers are repeated while processing imaging data via a neural network.
FIG. 7 illustrates an example user interface 700, in accordance with various aspects and implementations described herein. The user interface 700 can be a display environment for medical imaging data and/or deep learning data associated with medical imaging data. The user interface 700 can include medical imaging data 702. In one embodiment, the medical imaging data 702 can be displayed as a multi -dimensional visualization that presents a medical imaging diagnosis for a patient. For example, in certain embodiments, the medical imaging data 702 can be displayed as a multi-dimensional visualization that presents one or more classifications and/or one or more localizations for one or more diseases identified in medical imaging data 702. In certain embodiments, the medical imaging data 702 can be displayed as a multi -dimensional visualization that presents localization data for a medical imaging diagnosis. In another embodiment, the user interface 700 can include a heat bar 704. The heat bar 704 can include a set of colors that correspond to different values for the localization data. The user interface 700 can also include a prediction section 706 to present one or more predictions associated with the medical imaging data 702. The prediction section 706 can include a patient name 708 for a patient (e.g., a patient body) associated with the medical imaging data 702. The prediction section 706 can also include a condition portion 710 and a prediction portion 712. In an embodiment, information included in the prediction portion 712 can be determined by the first neural network 208, the second neural network 212 and/or the Mth neural network 216. The condition portion 710 can include one or more conditions such as, for example, a tuberculosis condition 710 a, a lateral view condition 710 b, a cardiomegaly condition 710 c, an opacity/lung condition 710 d, a lung/hypoinflation condition 710 e, a hyperdistention condition 710 f, a spine degenerative condition 710 g, a calcinosis condition 710 h and/or another type of condition. The prediction portion 712 can include corresponding predictions 712 a-h for the conditions included in the condition portion 710. For example, the prediction 712 a can include a prediction for the medical imaging data 702 being associated with tuberculosis (e.g., a 38.42% chance of a negative prognosis for tuberculosis and a 61.58% chance of a positive prognosis for tuberculosis). In another example, the prediction 712 h can include a prediction for the medical imaging data 702 being associated with calcinosis (e.g., a 40.99% chance of a negative prognosis for calcinosis and a 59.01% chance of a positive prognosis for calcinosis). In certain embodiments, the prediction section 706 can also include a patient age 714, a patient gender 716 and/or other information regarding a patient associated with the patient name 708. As such, in certain embodiments, the medical imaging data 702 can be associated with multiple diseases. Furthermore, multiple inferencing models can be employed and aggregated as deep learning data shown in the user interface 700.
FIG. 8 illustrates a methodology and/or a flow diagram in accordance with the disclosed subject matter. For simplicity of explanation, the methodology is depicted and described as a series of acts. It is to be understood and appreciated that the subject innovation is not limited by the acts illustrated and/or by the order of acts, for example acts can occur in various orders and/or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts may be required to implement the methodology in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methodology could alternatively be represented as a series of interrelated states via a state diagram or events. Additionally, it should be further appreciated that methodologies disclosed hereinafter and throughout this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methodologies to computers. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media.
Referring to FIG. 8, there is illustrated a non-limiting implementation of a methodology 800 for building a binary neural network architecture, according to an aspect of the subject innovation. At 802, a neural network is trained (e.g., by neural network training component 104) based on an image data set to generate a first neural network of a binary neural network architecture and determine whether a first class exists. In an embodiment, the image data set can be medical imaging data. For instance, the image data set can be two-dimensional medical imaging data and/or three-dimensional medical imaging data generated by one or more medical imaging devices. In one example, the image data set can be electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In certain embodiments, the image data set can be a series of electromagnetic radiation imagery captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device) during an interval of time. The image data set can be received directly from one or more imaging devices (e.g., one or more medical imaging devices). Alternatively, the image data set can be stored in one or more databases that receives and/or stores the image data set associated with the one or more imaging devices. An imaging device can be, for example, a camera, an x-ray device, a CT device, another type of imaging device, etc.
In an aspect, the first neural network can be generated with mutually exclusive outputs. The mutually exclusive outputs of the first neural network can be related to a feature detector of the first neural network. For example, the mutually exclusive outputs of the first neural network can provide a condition vs non-condition prediction (e.g. a yes/no prediction) for the first class. In an embodiment, the first neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the first neural network can be, for example, a spring network of convolutional layers. For instance, the first neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the first neural network. In an example, the first neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the first neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the first neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
At 806, a copy of the first neural network is trained (e.g., by neural network duplication component 106) based on the image data set to generate a second neural network of the binary neural network architecture and determine whether a second class exists. In an embodiment, the copy of the first neural network can be trained with respect to training of the neural network in a concatenating manner Furthermore, in an aspect, the second neural network can be generated with mutually exclusive outputs. The mutually exclusive outputs of the second neural network can be related to a feature detector of the second neural network. For example, the mutually exclusive outputs of the second neural network can provide a condition vs non-condition prediction (e.g. a yes/no prediction) for the second class.
In an embodiment, the second neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the second neural network can be, for example, a spring network of convolutional layers. For instance, the second neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the second neural network. In an example, the second neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the second neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the second neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
At 808, a copy of the second neural network is trained (e.g., by neural network duplication component 106) based on the image data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, where M is an integer greater than or equal to three. In an embodiment, the copy of the second neural network can be trained with respect to training of the copy of the first neural network in a concatenating manner. Furthermore, in an aspect, the Mth neural network can be generated with mutually exclusive outputs.
In an embodiment, the Mth neural network can perform a machine learning process (e.g., an artificial intelligence process for machine learning) based on the medical imaging data. In an aspect, the Mth neural network can be, for example, a spring network of convolutional layers. For instance, the Mth neural network can perform a plurality of sequential and/or parallel downsampling and upsampling of the medical imaging data associated with convolutional layers of the Mth neural network. In an example, the Mth neural network can perform a first convolutional layer process associated with sequential downsampling of the medical imaging data and a second convolutional layer process associated with sequential upsampling of the medical imaging data. The spring network of convolutional layers can include the first convolutional layer process associated with the sequential downsampling and the second convolutional layer process associated with sequential upsampling. The spring network of convolutional layers associated with the Mth neural network can alter convolutional layer filters similar to functionality of a spring. For instance, the Mth neural network can analyze the medical imaging data based on a first convolutional layer filter that comprises a first size, a second convolutional layer filter that comprises a second size that is different than the first size, and a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.
The aforementioned systems and/or devices have been described with respect to interaction between several components. It should be appreciated that such systems and components can include those components or sub-components specified therein, some of the specified components or sub-components, and/or additional components. Sub-components could also be implemented as components communicatively coupled to other components rather than included within parent components. Further yet, one or more components and/or sub-components may be combined into a single component providing aggregate functionality. The components may also interact with one or more other components not specifically described herein for the sake of brevity, but known by those of skill in the art.
In order to provide a context for the various aspects of the disclosed subject matter, FIGS. 9 and 10 as well as the following discussion are intended to provide a brief, general description of a suitable environment in which the various aspects of the disclosed subject matter may be implemented.
With reference to FIG. 9, a suitable environment 900 for implementing various aspects of this disclosure includes a computer 912. The computer 912 includes a processing unit 914, a system memory 916, and a system bus 918. The system bus 918 couples system components including, but not limited to, the system memory 916 to the processing unit 914. The processing unit 914 can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as the processing unit 914.
The system bus 918 can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, Industrial Standard Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Firewire (IEEE 1394), and Small Computer Systems Interface (SCSI).
The system memory 916 includes volatile memory 920 and nonvolatile memory 922. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer 912, such as during start -up, is stored in nonvolatile memory 922. By way of illustration, and not limitation, nonvolatile memory 922 can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory 920 includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM.
Computer 912 also includes removable/non-removable, volatile/non -volatile computer storage media. FIG. 9 illustrates, for example, a disk storage 924. Disk storage 924 includes, but is not limited to, devices like a magnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zip drive, LS-100 drive, flash memory card, or memory stick. The disk storage 924 also can include storage media separately or in combination with other storage media including, but not limited to, an optical disk drive such as a compact disk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatile disk ROM drive (DVD-ROM). To facilitate connection of the disk storage devices 924 to the system bus 918, a removable or non-removable interface is typically used, such as interface 926.
FIG. 9 also depicts software that acts as an intermediary between users and the basic computer resources described in the suitable operating environment 900. Such software includes, for example, an operating system 928. Operating system 928, which can be stored on disk storage 924, acts to control and allocate resources of the computer system 912. System applications 930 take advantage of the management of resources by operating system 928 through program modules 932 and program data 934, e.g., stored either in system memory 916 or on disk storage 924. It is to be appreciated that this disclosure can be implemented with various operating systems or combinations of operating systems.
A user enters commands or information into the computer 912 through input device(s) 936. Input devices 936 include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit 914 through the system bus 918 via interface port(s) 938. Interface port(s) 938 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB). Output device(s) 940 use some of the same type of ports as input device(s) 936. Thus, for example, a USB port may be used to provide input to computer 912, and to output information from computer 912 to an output device 940. Output adapter 942 is provided to illustrate that there are some output devices 940 like monitors, speakers, and printers, among other output devices 940, which require special adapters. The output adapters 942 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device 940 and the system bus 918. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 944.
Computer 912 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 944. The remote computer(s) 944 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer 912. For purposes of brevity, only a memory storage device 946 is illustrated with remote computer(s) 944. Remote computer(s) 944 is logically connected to computer 912 through a network interface 948 and then physically connected via communication connection 950. Network interface 948 encompasses wire and/or wireless communication networks such as local-area networks (LAN), wide-area networks (WAN), cellular networks, etc. LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ring and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).
Communication connection(s) 950 refers to the hardware/software employed to connect the network interface 948 to the bus 918. While communication connection 950 is shown for illustrative clarity inside computer 912, it can also be external to computer 912. The hardware/software necessary for connection to the network interface 948 includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.
FIG. 10 is a schematic block diagram of a sample-computing environment 1000 with which the subject matter of this disclosure can interact. The system 1000 includes one or more client(s) 1010. The client(s) 1010 can be hardware and/or software (e.g., threads, processes, computing devices). The system 1000 also includes one or more server(s) 1030. Thus, system 1000 can correspond to a two-tier client server model or a multi-tier model (e.g., client, middle tier server, data server), amongst other models. The server(s) 1030 can also be hardware and/or software (e.g., threads, processes, computing devices). The servers 1030 can house threads to perform transformations by employing this disclosure, for example. One possible communication between a client 1010 and a server 1030 may be in the form of a data packet transmitted between two or more computer processes.
The system 1000 includes a communication framework 1050 that can be employed to facilitate communications between the client(s) 1010 and the server(s) 1030. The client(s) 1010 are operatively connected to one or more client data store(s) 1020 that can be employed to store information local to the client(s) 1010. Similarly, the server(s) 1030 are operatively connected to one or more server data store(s) 1040 that can be employed to store information local to the servers 1030.
It is to be noted that aspects or features of this disclosure can be exploited in substantially any wireless telecommunication or radio technology, e.g., Wi-Fi; Bluetooth; Worldwide Interoperability for Microwave Access (WiMAX); Enhanced General Packet Radio Service (Enhanced GPRS); Third Generation Partnership Project (3GPP) Long Term Evolution (LTE); Third Generation Partnership Project 2 (3GPP2) Ultra Mobile Broadband (UMB); 3GPP Universal Mobile Telecommunication System (UMTS); High Speed Packet Access (HSPA); High Speed Downlink Packet Access (HSDPA); High Speed Uplink Packet Access (HSUPA); GSM (Global System for Mobile Communications) EDGE (Enhanced Data Rates for GSM Evolution) Radio Access Network (GERAN); UMTS Terrestrial Radio Access Network (UTRAN); LTE Advanced (LTE-A); etc. Additionally, some or all of the aspects described herein can be exploited in legacy telecommunication technologies, e.g., GSM. In addition, mobile as well non-mobile networks (e.g., the Internet, data service network such as internet protocol television (IPTV), etc.) can exploit aspects or features described herein.
While the subject matter has been described above in the general context of computer-executable instructions of a computer program that runs on a computer and/or computers, those skilled in the art will recognize that this disclosure also can or may be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks and/or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods may be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as personal computers, hand-held computing devices (e.g., PDA, phone), microprocessor-based or programmable consumer or industrial electronics, and the like. The illustrated aspects may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. However, some, if not all aspects of this disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
As used in this application, the terms “component,” “system,” “platform,” “interface,” and the like, can refer to and/or can include a computer -related entity or an entity related to an operational machine with one or more specific functionalities. The entities disclosed herein can be either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
In another example, respective components can execute from various computer readable media having various data structures stored thereon. The components may communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal). As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry, which is operated by a software or firmware application executed by a processor. In such a case, the processor can be internal or external to the apparatus and can execute at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electronic components without mechanical parts, wherein the electronic components can include a processor or other means to execute software or firmware that confers at least in part the functionality of the electronic components. In an aspect, a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system.
In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. Moreover, articles “a” and “an” as used in the subject specification and annexed drawings should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form.
As used herein, the terms “example” and/or “exemplary” are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as an “example” and/or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.
Various aspects or features described herein can be implemented as a method, apparatus, system, or article of manufacture using standard programming or engineering techniques. In addition, various aspects or features disclosed in this disclosure can be realized through program modules that implement at least one or more of the methods disclosed herein, the program modules being stored in a memory and executed by at least a processor. Other combinations of hardware and software or hardware and firmware can enable or implement aspects described herein, including a disclosed method(s). The term “article of manufacture” as used herein can encompass a computer program accessible from any computer-readable device, carrier, or storage media. For example, computer readable storage media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical discs (e.g., compact disc (CD), digital versatile disc (DVD), blu-ray disc (BD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ), or the like.
As it is employed in the subject specification, the term “processor” can refer to substantially any computing processing unit or device comprising, but not limited to, single-core processors; single-processors with software multithread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi-core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Further, processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment. A processor may also be implemented as a combination of computing processing units.
In this disclosure, terms such as “store,” “storage,” “data store,” data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component are utilized to refer to “memory components,” entities embodied in a “memory,” or components comprising a memory. It is to be appreciated that memory and/or memory components described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
By way of illustration, and not limitation, nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory can include RAM, which can act as external cache memory, for example. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memory components of systems or methods herein are intended to include, without being limited to including, these and any other suitable types of memory.
It is to be appreciated and understood that components, as described with regard to a particular system or method, can include the same or similar functionality as respective components (e.g., respectively named components or similarly named components) as described with regard to other systems or methods disclosed herein.
What has been described above includes examples of systems and methods that provide advantages of this disclosure. It is, of course, not possible to describe every conceivable combination of components or methods for purposes of describing this disclosure, but one of ordinary skill in the art may recognize that many further combinations and permutations of this disclosure are possible. Furthermore, to the extent that the terms “includes,” “has,” “possesses,” and the like are used in the detailed description, claims, appendices and drawings such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.

Claims

What is claimed is:

1. A deep neural network system, comprising:

a memory that stores computer executable components;

a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:

a neural network training component that trains a neural network based on a data set to form a first neural network of a binary neural network architecture and determine whether a first class exists; and

a neural network duplication component that trains a copy of the first neural network based on the data set to form a second neural network of the binary neural network architecture and determine whether a second class exists, wherein the neural network duplication component trains a copy of the second neural network based on the data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, and wherein M is an integer greater than or equal to three.

2. The deep neural network system of claim 1, wherein the first neural network generates mutually exclusive outputs.

3. The deep neural network system of claim 1, wherein the second neural network generates mutually exclusive outputs.

4. The deep neural network system of claim 1, wherein the Mth neural network generates mutually exclusive outputs.

5. The deep neural network system of claim 1, wherein training of the neural network with respect to the first class and training of the copy of the neural network with respect to the second class are performed in a concatenating manner.

6. The deep neural network system of claim 1, wherein training of the copy of the first neural network with respect to the second class and training of the copy of the second neural network with respect to the Mth class are performed in a concatenating manner.

7. The deep neural network system of claim 1, where the first neural network performs a plurality of sequential and/or parallel downsampling and upsampling of the data set associated with convolutional layers of the first neural network.

8. The deep neural network system of claim 1, where the second neural network performs a plurality of sequential and/or parallel downsampling and upsampling of the data set associated with convolutional layers of the second neural network.

9. A method, comprising using a processor operatively coupled to memory to execute computer executable components to perform the following acts:

training a neural network based on an image data set to generate a first neural network of a binary neural network architecture and determine whether a first class exists;

training a copy of the first neural network based on the image data set to generate a second neural network of the binary neural network architecture and determine whether a second class exists; and

training a copy of the second neural network based on the image data set to form an Mth neural network of the binary neural network architecture and determine whether an Mth class exists, wherein M is an integer greater than or equal to three.

10. The method of claim 9, wherein the training the neural network comprises generating the first neural network with mutually exclusive outputs.

11. The method of claim 9, wherein the training the copy of the first neural network comprises generating the second neural network with mutually exclusive outputs.

12. The method of claim 9, wherein the training the copy of the second neural network comprises generating the Mth neural network with mutually exclusive outputs.

13. The method of claim 9, wherein the training the copy of the first neural network comprises training the copy of the first neural network with respect to the training of the neural network in a concatenating manner.

14. The method of claim 9, wherein the training the copy of the second neural network comprises training the copy of the second neural network with respect to the training of the copy of the first neural network in a concatenating manner.

15. A computer readable storage device comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:

training a neural network based on an image data set to generate a first neural network and determine whether a first class exists;

training a copy of the first neural network based on the image data set to generate a second neural network and determine whether a second class exists;

training a copy of the second neural network based on the image data set to form an Mth neural network and determine whether an Mth class exists, wherein M is an integer greater than or equal to three; and

generating a neural network architecture that includes the first neural network, the second neural network and the Mth neural network.

16. The computer readable storage device of claim 15, wherein the training the neural network comprises generating the first neural network with a first output and a second output that are mutually exclusive.

17. The computer readable storage device of claim 15, wherein the training the copy of the first neural network comprises generating the second neural network with a first output and a second output that are mutually exclusive.

18. The computer readable storage device of claim 15, wherein the training the copy of the second neural network comprises generating the Mth neural network with a first output and a second output that are mutually exclusive.

19. The computer readable storage device of claim 15, wherein the training the copy of the first neural network comprises training the copy of the first neural network with respect to the training of the neural network in a concatenating manner.

20. The computer readable storage device of claim 15, wherein the training the copy of the second neural network comprises training the copy of the second neural network with respect to the training of the copy of the first neural network in a concatenating manner.