WO2020118614A1

WO2020118614A1 - Image identification method and device for patches on head and neck

Info

Publication number: WO2020118614A1
Application number: PCT/CN2018/120876
Authority: WO
Inventors: 肖韬辉; 王珊珊; 郑海荣; 刘新; 梁栋
Original assignee: 深圳先进技术研究院
Priority date: 2018-12-13
Filing date: 2018-12-13
Publication date: 2020-06-18

Abstract

The present invention provides an image identification method and device for patches on head and neck. The method comprises the following steps: obtaining the head-and-neck magnetic resonance image to be identified, inputting the image into a constructed patch identification module, and obtaining a patch identification result of the image. The patch identification module comprises a U-type convolution neural network module, the convolution layer of which uses dense connection blocks in a dense convolution neural network module. The embodiment of the description realizes automatic identification of patches on head and neck, thereby improving the accuracy of identification result.

Description

Head and neck plaque image recognition method and device

Technical field

This specification belongs to the technical field of image processing, and particularly relates to a head and neck plaque image recognition method and device.

Background technique

Stroke has now become one of the diseases with the highest fatality rate and disability rate in adults. Among stroke patients in my country, more than 70% of them are ischemic strokes. The main pathogenic factor of ischemic stroke is thromboembolism caused by rupture of atherosclerotic plaque. Examining the structure of blood vessel wall can effectively improve the diagnosis rate of stroke. Perform identification diagnosis.

Since the head and neck plaques are very small, the smaller the object recognition difficulty, the more difficult it is to segment or detect the head and neck plaques. In the prior art, the detection of plaque is mainly visually recognized by a clinical imaging doctor, and the doctor reads a large number of films every day, which will cause visual fatigue, and also have certain diagnostic results due to some personal subjective factors such as experience and experience. error. Therefore, a technical solution capable of accurately identifying head and neck plaques is urgently needed in the art.

Summary of the invention

The purpose of this specification is to provide a head and neck plaque image recognition method and device, which realizes the automatic identification of head and neck plaques and improves the accuracy of the head and neck plaque recognition results.

On the one hand, the embodiments of the present specification provide a head and neck plaque image recognition method, including:

Obtain the head and neck magnetic resonance image to be recognized;

Input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model to obtain the plaque recognition result in the head and neck magnetic resonance image to be recognized;

Wherein, the plaque recognition model uses a U-shaped convolutional neural network model, and the convolutional layer of the U-shaped convolutional neural network model uses a dense connection block in a dense convolutional neural network model.

Further, in another embodiment of the method, the number of feature maps output by each of the densely connected blocks in the patch identification model is different.

Further, in another embodiment of the method, the densely connected blocks in the patch identification model include multiple densely connected layers, and densely connected between the densely connected layers.

Further, in another embodiment of the method, there are multiple dense connection blocks, and the number of dense connection layers in each dense connection block is the same.

Further, in another embodiment of the method, the plaque recognition model is constructed using the following method:

Acquiring multiple sample data, the sample data including: a head and neck magnetic resonance image and a plaque marker in the head and neck magnetic resonance image;

Establishing the plaque recognition model, using the head and neck magnetic resonance image in the sample data as input data of the plaque recognition model, and using the corresponding plaque marker in the head and neck magnetic resonance image as the plaque recognition The output label of the model trains the plaque recognition model until the plaque recognition model meets the preset requirements.

Further, in another embodiment of the method, the method further includes: optimizing the plaque recognition model using a cross-validation method.

Further, in another embodiment of the method, the acquiring multiple sample data includes:

Acquiring multiple head and neck magnetic resonance images, and normalizing the head and neck magnetic resonance images;

Plaque labeling the acquired head and neck magnetic resonance images to obtain plaque marks in the head and neck magnetic resonance images;

Correspondingly, the use of the normalized head and neck magnetic resonance image as input data of the plaque recognition model includes:

The normalized head and neck magnetic resonance image is used as input data of the plaque recognition model.

On the other hand, this specification provides a head and neck plaque image recognition device, including:

An image acquisition module for acquiring head and neck magnetic resonance images to be identified;

An image recognition module, used to input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model, and obtain the plaque recognition result in the head and neck magnetic resonance image to be recognized;

Further, in another embodiment of the device, the number of feature maps output by each of the densely connected blocks in the patch identification model is different.

Further, in another embodiment of the apparatus, the densely connected blocks in the patch identification model include multiple densely connected layers, and densely connected between the densely connected layers.

Further, in another embodiment of the apparatus, the number of the dense connection blocks is multiple, and the number of dense connection layers in each dense connection block is the same.

Further, in another embodiment of the device, the device further includes: a model building module for building the plaque recognition model using the following method:

Acquiring multiple sample data, the sample data including: a head and neck magnetic resonance image and plaques marked in the head and neck magnetic resonance image;

Establishing the plaque recognition model, using the head and neck magnetic resonance image in the sample data as input data of the plaque recognition model, and using the corresponding plaque marked in the head and neck magnetic resonance image as the plaque recognition The output label of the model trains the plaque recognition model until the plaque recognition model meets the preset requirements.

Further, in another embodiment of the device, the model building module is also used to:

A cross-validation method is used to optimize the plaque recognition model.

Further, in another embodiment of the device, the model building module is specifically used to:

Acquiring multiple head and neck magnetic resonance images, and normalizing the head and neck magnetic resonance images, and using the normalized head and neck magnetic resonance images as input data of the plaque recognition model;

Plaque annotation is performed on the acquired multiple head and neck magnetic resonance images to obtain plaque marks in the head and neck magnetic resonance images.

In still another aspect, this specification provides a head and neck plaque image recognition processing device, including: at least one processor and a memory for storing processor executable instructions, and the processor executes the instructions to implement the Head and neck plaque image recognition method.

In another aspect, this specification provides a head and neck plaque image recognition system, including:

A data acquisition module for acquiring sample data, the sample data including: head and neck magnetic resonance images and plaques marked in the head and neck magnetic resonance images;

The detection model building module is used to build a plaque recognition model, and perform model training and model optimization, wherein the plaque recognition model uses a U-shaped convolutional neural network model, and a convolutional layer of the U-shaped convolutional neural network model Use dense connection blocks in the dense convolutional neural network model;

The model test module is used for inputting the head and neck magnetic resonance images to be recognized to obtain the head and neck plaque recognition results.

The head and neck plaque image recognition method, device, processing equipment, and system provided in this specification, based on deep learning, combine the U-shaped convolutional neural network model with the dense convolutional neural network model to construct a plaque recognition model, and then The recognized head and neck magnetic resonance image is input into the constructed plaque recognition model, that is, the plaque recognition result of the head and neck magnetic resonance image to be recognized can be obtained, and the automatic recognition of the head and neck plaque is realized, without the need for artificial naked eye recognition, and the head and neck are improved Plaque recognition results. In addition, the plaque recognition model in the embodiment of this specification combines the U-shaped convolutional neural network model with the dense convolutional neural network model, which can reduce training sample data, better retain image information, and improve the number of feature map multiplexing To further improve the efficiency and accuracy of plaque recognition.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the specification or the technical solutions in the prior art, the following will briefly introduce the drawings required in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only These are some of the embodiments described in this specification. For those of ordinary skill in the art, without paying any creative labor, other drawings can also be obtained based on these drawings.

FIG. 1 is a schematic flowchart of a head and neck plaque image recognition method in an embodiment of this specification;

2 is a schematic diagram of a network architecture of a plaque recognition model in an embodiment of this specification;

3 is a schematic diagram of a module structure of an embodiment of a head and neck plaque image recognition device provided in this specification;

4 is a schematic structural diagram of a head and neck plaque image recognition device according to another embodiment of this specification;

5 is a schematic diagram of a workflow of a head and neck plaque image recognition system in an embodiment of this specification;

6 is a block diagram of a hardware structure of a head and neck plaque recognition server applying an embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be described clearly and completely in conjunction with the drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments of this specification, but not all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this specification.

An embodiment of the present specification provides a head and neck plaque image recognition method, which combines a U-shaped convolutional neural network with a secret convolutional neural network model to construct a plaque recognition model. Using the constructed plaque recognition model to perform plaque recognition on the head and neck magnetic resonance images, the automatic recognition of head and neck plaques is realized, and the plaque regions in the head and neck magnetic resonance images are identified, which provides a data basis for the diagnosis of stroke.

The head and neck plaque image recognition method in this manual can be applied to the client or server. The client can be a smartphone, tablet, smart wearable device (smart watch, virtual reality glasses, virtual reality helmet, etc.), smart vehicle equipment, etc. Electronic equipment.

Specifically, FIG. 1 is a schematic flowchart of a head and neck plaque image recognition method in an embodiment of the present specification. As shown in FIG. 1, the overall process of the head and neck plaque image recognition method provided in an embodiment of the present specification may include:

Step 102: Acquire a head and neck magnetic resonance image to be identified.

Magnetic resonance examination is currently a relatively common medical examination method. In the embodiments of the present specification, the head and neck magnetic resonance image of the user, that is, the head and neck integrated blood vessel wall magnetic resonance image can be obtained, and the head and neck plaque can be performed based on the acquired head and neck magnetic resonance image to be identified Block identification.

Step 104: Input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model to obtain a plaque recognition result in the head and neck magnetic resonance image to be recognized;

In the specific implementation process, a plaque recognition model can be constructed based on deep learning methods. For example, the existing head and neck magnetic resonance image data of stroke patients can be used for model training to learn from input magnetic resonance images to output head and neck A functional mapping relationship between the results of patch segmentation to build a patch recognition model. FIG. 2 is a schematic diagram of the network architecture of the plaque recognition model in an embodiment of the present specification. As shown in FIG. 2, the plaque recognition model in the embodiment of the present specification can be a U-shaped convolutional neural network model (ie U-Net) and Dense convolutional neural network model (DenseNet) is combined. As shown in Figure 2, the overall architecture of the plaque recognition model can be the structure of a U-shaped convolutional neural network model. The convolutional layer in the U-shaped convolutional neural network model can use the densely connected blocks in the dense convolutional neural network model (Ie dense block).

U-Net can be understood as a variant of the convolutional neural network, whose structure is mainly like the letter U, hence the name U-Net. The entire neural network of U-Net is mainly composed of two parts: the contraction path and the expansion path. The contraction path is mainly used to capture the context information in the picture, and the expansion path commensurate with it is to segment the need in the picture. For precise positioning. DenseNet can be understood as a convolutional neural network with dense connections. In this network, there is a direct connection between any two layers, that is, the input of each layer of the network is the union of the outputs of all previous layers. , And the feature map learned by this layer will be directly passed to all subsequent layers as input. As shown in FIG. 2, in the embodiment of the present specification, each convolutional layer of the U-shaped convolutional neural network model may use dense connection blocks, and dense connections exist in each dense connection block. Dense connections can alleviate the problem of gradient disappearance, strengthen feature propagation, encourage feature reuse, greatly reduce the amount of parameters, and improve the accuracy of image recognition.

After the plaque recognition model is constructed, the head and neck magnetic resonance image to be recognized is input into the constructed plaque recognition model, and the plaque recognition result of the head and neck magnetic resonance image to be recognized can be obtained, such as: the head and neck magnetic to be recognized can be recognized Whether there is plaque in the resonance image, and if so, it can also identify the area where the plaque is located, or the shape and size of the plaque.

As shown in FIG. 2, the number above each dense connection block may indicate the number of feature maps output by the dense connection block. In an embodiment of this specification, the number of feature maps output by each dense connection block may be different, as shown in FIG. As shown in 2, the number of feature maps output by each dense connection block may be: 32, 64, 128, 256, 512. Of course, other numbers of feature maps may be used according to actual needs, which is not specifically limited in the embodiments of this specification. The dense connection block is located at different convolution layers of the U-shaped convolutional neural network model. The number of feature maps output by different dense connection blocks is set to be different, which can adapt to the structural needs of the U-shaped convolutional neural network model and better retain the image. Information to improve image recognition results.

As shown in FIG. 2, the lower left corner of FIG. 2 is a schematic structural diagram of each dense connection block. In one embodiment of this specification, each dense connection block may include multiple dense connection layers, and each small connection block in FIG. 2 A circle can represent a densely connected layer in a densely connected block, and densely connected between each densely connected layer. As shown in Figure 2, there is a direct connection between any two densely connected layers in the densely connected block, that is, the input of each layer in the densely connected block is the union of the outputs of all previous layers, and the layer is The learned feature map will also be directly passed to all densely connected layers behind it as input. The dense connection method of multiple dense connection layers in the dense connection block can improve the reusability of the feature map and improve the accuracy of the patch recognition result.

In some embodiments of this specification, there are multiple dense connection blocks, and the number of dense connection layers in each dense connection block is the same. As shown in Figure 2, the number of output features of the second to fifth densely connected blocks in the U-shaped convolutional neural network model is larger than that set in the original dense block, which may result in a rapid increase in the number of parameters. In the embodiment of the present specification, the number of dense connection layers in each dense connection block is set to be the same, which can maintain an appropriate calculation amount, reduce the calculation amount of the network model, and improve the efficiency of image recognition. As shown in FIG. 2, each dense connection block can be set to 5 dense connection layers. Of course, according to actual needs, it can also be set to other numbers of dense connection layers, which is not specifically limited in the embodiments of this specification.

In addition, as shown in FIG. 2, in some embodiments of this specification, the size of the convolution kernel in the plaque recognition model can all be set to 3*3, and the activation function can be all set to ReLU (Rectified Linear Units). The plaque recognition model The other connection methods in can be consistent with the U-Net structure. The encoding process is actually a downsampling layer. The downsampling layer can use a maximum pooling operation of 2*2; the decoding process is actually an upsampling process, which can be 2*2. In the deconvolution operation, the output feature map of the corresponding layer in the encoding and decoding is stitched and fused in the middle, and the image recognition result is finally output.

In the embodiment of this specification, based on deep learning, a U-shaped convolutional neural network model and a dense convolutional neural network model are combined to construct a plaque recognition model, and then the head and neck magnetic resonance image to be recognized is input to the constructed plaque recognition In the model, the plaque recognition results of the head and neck magnetic resonance images to be recognized can be obtained, which realizes the automatic recognition of the head and neck plaques, does not require artificial naked eye recognition, and improves the recognition results of the head and neck plaques. In addition, the plaque recognition model in the embodiment of this specification combines the U-shaped convolutional neural network model with the dense convolutional neural network model, which can reduce training sample data, better retain image information, and improve the number of feature map multiplexing To further improve the efficiency and accuracy of plaque recognition.

Based on the above embodiments, in some embodiments of this specification, the plaque recognition model may be constructed using the following method:

In a specific embodiment, a head and neck magnetic resonance image of a historical user may be acquired as sample data, and the sample data may be a head and neck magnetic resonance image of a user diagnosed as having a stroke. The sample data may also include the acquired plaque markers in the head and neck magnetic resonance images as training labels, and the specific number of sample data may be selected according to actual needs, which is not specifically limited in the embodiments of this specification. In an embodiment of this specification, after acquiring multiple head and neck magnetic resonance images, the acquired head and neck magnetic resonance images may be normalized, that is, the pixels of the head and neck magnetic resonance images in the sample data may be processed in a unified manner, such as : Normalize the pixel points of the head and neck magnetic resonance image to 0-1, and use the normalized head and neck magnetic resonance image as the input data of the model to facilitate subsequent model training. The plaques of the acquired head and neck magnetic resonance images can be labeled by professional doctors or according to the user's diagnosis results. Specifically, the location and size of the plaque can be marked.

After the sample data preparation is completed, a plaque recognition model may be constructed, such as: constructing a network architecture of the plaque recognition model, etc. For the network architecture of the plaque recognition model, reference may be made to the records of the foregoing embodiments, and details are not described here. Among them, the patch identification model may also include multiple model parameters, such as: the size of the convolution kernel, the number of densely connected blocks, and so on. After the plaque recognition model is constructed, the head and neck magnetic resonance image in the sample data can be used as the input data of the plaque recognition model, and the plaque marker in the corresponding head and neck magnetic resonance image can be used as the output data of the plaque recognition model. The block recognition model performs model training until the plaque recognition model meets the preset requirements, for example, if the model output accuracy meets the requirements or the model training times meet the requirements, it can be considered that the model training is completed.

The embodiment of the present specification uses deep learning training to construct a plaque recognition model, which can realize automatic recognition of head and neck plaques without manual recognition, and improves the accuracy of head and neck plaque recognition.

In some embodiments of this specification, after the model training is completed, the cross-validation method may also be used to optimize the plaque recognition model to improve the accuracy of the model recognition results. Cross-validation can be regarded as a practical method of statistically cutting data samples into smaller subsets. In the sample data, most of the samples can be taken out to build the model, and a small part of the sample can be used for prediction with the newly established model. Find the forecast errors of these small samples and record the sum of their squares. This process continues until all samples are forecasted once and only once.

It should be noted that the head and neck plaque image recognition method in the embodiment of the present specification may not be limited to the identification of head and neck plaques, and may also be used in other image recognition processes, such as: identifying other focus areas (such as brain tumors). You can use the magnetic resonance image training of other parts to build a corresponding recognition model to complete the automatic recognition of other lesion areas.

The head and neck plaque image recognition method provided by the embodiment of the present specification builds a plaque recognition model based on the deep learning method, and uses deep learning technology to achieve the purpose of automatically detecting the head and neck plaque of stroke patients, and realizes the automatic detection of the magnetic resonance vascular wall The plaque in the image improves the accuracy and prevention of stroke disease diagnosis.

The embodiments of the above method in this specification are described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. For the relevant parts, please refer to the description of the method embodiments.

Based on the above-described head and neck plaque image recognition method, one or more embodiments of the present specification further provide a head and neck plaque image recognition device. The device may include a system (including a distributed system), software (applications), modules, components, servers, clients, etc. using the method described in the embodiments of the present specification in combination with necessary hardware implementation devices. Based on the same innovative concept, the devices in one or more embodiments provided by the embodiments of this specification are as described in the following embodiments. Since the implementation solution of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiments of the present specification may refer to the implementation of the foregoing method, and the repetition is not repeated. As used below, the term "unit" or "module" may implement a combination of software and/or hardware that achieves a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation of hardware or a combination of software and hardware is also possible and conceived.

Specifically, FIG. 3 is a schematic diagram of a module structure of an embodiment of the head and neck plaque image recognition device provided in this specification. As shown in FIG. 3, the head and neck plaque image recognition device provided in this specification includes: an image acquisition module 31, image recognition Module 32, where:

The image acquisition module 31 can be used to acquire the head and neck magnetic resonance image to be identified;

The image recognition module 32 may be used to input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model to obtain the plaque recognition result in the head and neck magnetic resonance image to be recognized;

The head and neck plaque image recognition device provided by the embodiment of the present specification combines the U-shaped convolutional neural network model and the dense convolutional neural network model based on deep learning to construct a plaque recognition model, and then the head and neck magnetic resonance to be recognized When the image is input into the constructed plaque recognition model, the plaque recognition results of the head and neck magnetic resonance images to be recognized can be obtained, and the automatic recognition of the head and neck plaques is realized without artificial visual recognition, which improves the recognition results of the head and neck plaques . In addition, the plaque recognition model in the embodiment of this specification combines the U-shaped convolutional neural network model with the dense convolutional neural network model, which can reduce training sample data, better retain image information, and improve the number of feature map multiplexing To further improve the efficiency and accuracy of plaque recognition.

On the basis of the above embodiments, the number of feature maps output by each of the densely connected blocks in the patch recognition model is different.

The head and neck plaque image recognition device provided by the embodiment of the present specification sets the number of feature maps output by different dense connection blocks to be different, which can adapt to the structural needs of the U-shaped convolutional neural network model, better retain image information, and improve image recognition result.

Based on the above embodiments, the densely connected blocks in the patch identification model include multiple densely connected layers, and densely connected between the densely connected layers.

The embodiment of the present specification adopts the dense connection mode of multiple dense connection layers in the dense connection block, which can improve the multiplexing of the feature map and improve the accuracy of the patch recognition result.

Based on the above embodiment, the number of the dense connection blocks is multiple, and the number of dense connection layers in each of the dense connection blocks is the same.

In the embodiment of the present specification, setting the number of dense connection layers in each dense connection block to be the same can maintain an appropriate calculation amount, reduce the calculation amount of the network model, and improve the efficiency of image recognition.

FIG. 4 is a schematic structural diagram of a head and neck plaque image recognition device in still another embodiment of the present specification. As shown in FIG. 4, based on the above embodiment, the device further includes: a model building module 41 for adopting the following method Construct the plaque recognition model:

In the embodiment of the present specification, a plaque recognition model is constructed by using deep learning training, which can realize automatic recognition of head and neck plaques without manual recognition, and improves the accuracy of head and neck plaque recognition.

Based on the above embodiments, the model building module is also used to:

A cross-validation method is used to optimize the plaque recognition model.

In the embodiment of the present specification, the cross-validation method is used to optimize the model, improve the accuracy of model construction, and further improve the accuracy of the model recognition result.

Based on the above embodiments, the model building module is specifically used to:

In the embodiment of the present specification, the pixels of the head and neck magnetic resonance image in the sample data are uniformly processed, which facilitates subsequent model training and improves the accuracy of model construction.

It should be noted that the above description of the device according to the method embodiment may also include other implementations. For a specific implementation manner, reference may be made to the description of related method embodiments, and details are not repeated herein.

An embodiment of the present specification also provides a head and neck plaque image recognition processing device, including: at least one processor and a memory for storing processor executable instructions, and the processor implements the instructions to implement the head and neck plaques of the foregoing embodiments Block image recognition methods, such as:

Obtain the head and neck magnetic resonance image to be recognized;

The storage medium may include a physical device for storing information, usually after the information is digitized and then stored in a medium using electrical, magnetic, or optical means. The storage medium may include: devices that use electrical energy to store information, such as various types of memory, such as RAM, ROM, etc.; devices that use magnetic energy to store information, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, and bubble memories, U disk; a device that uses optical means to store information such as CD or DVD. Of course, there are other ways of readable storage media, such as quantum memory, graphene memory, and so on.

It should be noted that the above description of the processing device according to the method embodiment may also include other implementation manners. For a specific implementation manner, reference may be made to the description of related method embodiments, and details are not repeated herein.

5 is a schematic diagram of a workflow of a head and neck plaque image recognition system in an embodiment of the present specification. As shown in FIG. 5, an embodiment of the present specification also provides a head and neck plaque image recognition system, which may include:

The data collection module 51 may be used to collect sample data, the sample data including: a head and neck magnetic resonance image and plaques marked in the head and neck magnetic resonance image;

The detection model construction module 52 can be used to build a plaque recognition model, and perform model training and model optimization, wherein the plaque recognition model uses a U-shaped convolutional neural network model, and the U-shaped convolutional neural network model The accumulation layer uses dense connection blocks in the dense convolutional neural network model;

The model test module 53 may be used to input a head and neck magnetic resonance image to be recognized to obtain a head and neck plaque recognition result.

As shown in FIG. 5, the data collection module 51 can use the magnetic resonance image of the integrated blood vessel wall of the head and neck of a historical user, and perform plaque annotation to obtain sample data as input and output for model training. The detection model construction module 52 can be used to construct the network structure of the deep convolution recognition model, that is, the patch recognition model. The network structure of the model can refer to the records of the foregoing embodiments, and details are not described here. The designed deep convolution recognition model network is used to train the pre-processed sample data, and after a lot of training and cross-validation, the optimization model is continuously improved. Finally, the model with better training is selected for model testing and result display. The model test module 53 can perform an online plaque test on the head and neck integrated magnetic resonance vessel wall image, and can display the identified plaque results online. In addition, the head and neck plaque image recognition system can also include a model application module, which can be used in clinical diagnosis to help doctors identify plaque that may cause stroke, thereby improving the early detection rate of stroke patients.

The head and neck plaque image recognition system provided by the embodiments of the present specification builds a plaque recognition model based on the deep learning method, and uses deep learning technology to achieve the purpose of automatically detecting head and neck plaques of stroke patients, and realizes the automatic detection of the magnetic resonance vascular wall The plaque in the image improves the accuracy and prevention of stroke disease diagnosis.

The head and neck plaque image recognition system provided in this specification may be a separate head and neck plaque image recognition system, and may also be applied to various data analysis and processing systems. The system may include any one of the head and neck plaque image recognition devices in the above embodiments. The system may be a separate server, or it may include a server cluster, system (including distributed system), software (application) using one or more of the methods or one or more embodiments of this specification. Terminal devices that actually operate devices, logic gate devices, quantum computers, etc., combined with the necessary implementation hardware. The detection system for checking the difference data may include at least one processor and a memory storing computer-executable instructions. When the processor executes the instructions, the steps of the method in any one or more of the above embodiments are implemented.

The method embodiments provided in the embodiments of this specification can be executed in a mobile terminal, a computer terminal, a server, or a similar computing device. Taking an example of running on a server, FIG. 6 is a block diagram of a hardware structure of a head and neck plaque recognition server applying an embodiment of the present application. As shown in FIG. 6, the server 10 may include one or more (only one is shown in the figure) processor 100 (the processor 100 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), A memory 200 for storing data, and a transmission module 300 for communication functions. A person of ordinary skill in this neighborhood can understand that the structure shown in FIG. 6 is merely an illustration, which does not limit the structure of the foregoing electronic device. For example, the server 10 may also include more or fewer components than those shown in FIG. 6, for example, it may also include other processing hardware, such as a database or a multi-level cache, a GPU, or have a configuration different from that shown in FIG.

The memory 200 can be used to store software programs and modules of application software, such as program instructions/modules corresponding to the head and neck plaque image recognition method in the embodiments of the present specification. The processor 100 runs the software programs and modules stored in the memory 200, thereby Perform various functional applications and data processing. The memory 200 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 200 may further include memories remotely provided with respect to the processor 100, and these remote memories may be connected to a computer terminal through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

The transmission module 300 is used to receive or send data via a network. The specific example of the network described above may include a wireless network provided by a communication provider of computer terminals. In one example, the transmission module 300 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through the base station to communicate with the Internet. In one example, the transmission module 300 may be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The method or apparatus described in the above embodiments provided in this specification can implement business logic through a computer program and be recorded on a storage medium, and the storage medium can be read and executed by a computer to achieve the effects of the solutions described in the embodiments of this specification.

The above-mentioned head and neck plaque image recognition method or device provided in the embodiments of the present specification can be implemented by a processor executing corresponding program instructions in a computer, such as using a Windows operating system C++ language to implement on a PC side, a Linux system, or other, for example Use android, iOS system programming language to realize in the intelligent terminal, and realize the processing logic based on quantum computer.

It should be noted that the description of the device, computer storage medium, and system described above in the specification according to the related method embodiments may also include other implementation manners. For specific implementation manners, reference may be made to the description of the corresponding method embodiments, and details are not repeated here. .

The embodiments in this specification are described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method embodiment.

The embodiments of this specification are not limited to those that must comply with industry communication standards, standard computer data processing and data storage rules, or those described in one or more embodiments of this specification. Some industry standards or implementations described in a custom manner or embodiments based on slightly modified implementations can also achieve the same, equivalent, or similar, or predictable implementation effects of the foregoing embodiments. Examples obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc., can still fall within the scope of optional implementations of the examples in this specification.

In the 1990s, the improvement of a technology can be clearly distinguished from the improvement in hardware (for example, the improvement of circuit structures such as diodes, transistors, and switches) or the improvement in software (the improvement of the process flow). However, with the development of technology, the improvement of many methods and processes can be regarded as a direct improvement of the hardware circuit structure. Designers almost get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware physical modules. For example, a programmable logic device (Programmable Logic Device, PLD) (such as a field programmable gate array (Field Programmable Gate Array, FPGA)) is such an integrated circuit, and its logic function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a PLD without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is also mostly implemented with "logic compiler" software, which is similar to the software compiler used in program development and writing, but before compilation The original code must also be written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), and HDL is not only one kind, but there are many kinds, such as ABEL (Advanced Boolean Expression) Language , AHDL (AlteraHardwareDescriptionLanguage), Confluence, CUPL (CornellUniversityProgrammingLanguage), HDCal, JHDL (JavaHardwareDescriptionLanguage), Lava, Lola, MyHDL, PALASM, RHDL (RubyHardwareDescription) It is VHDL (Very-High-Speed Integrated Circuit Hardware Description) and Verilog. Those skilled in the art should also be clear that by simply programming the method flow in the above hardware description languages and programming into the integrated circuit, the hardware circuit that implements the logic method flow can be easily obtained.

The controller may be implemented in any suitable manner, for example, the controller may take a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor , Logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program method steps to make the controller use logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded The same function is realized in the form of a microcontroller or the like. Therefore, such a controller can be regarded as a hardware component, and the device for implementing various functions included therein can also be regarded as a structure within the hardware component. Or even, the means for realizing various functions can be regarded as both a software module of an implementation method and a structure within a hardware component.

The system, device, module or unit explained in the above embodiments may be specifically implemented by a computer chip or entity, or implemented by a product with a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an on-board human-machine interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet A computer, a wearable device, or any combination of these devices.

Although one or more embodiments of this specification provide method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of the steps listed in the embodiment is only one way among the order of execution of many steps, and does not represent a unique order of execution. When the actual device or terminal product is executed, it may be executed sequentially or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "include", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, product, or device that includes a series of elements includes not only those elements, but also others that are not explicitly listed Elements, or also include elements inherent to such processes, methods, products, or equipment. Without more restrictions, it does not exclude that there are other identical or equivalent elements in the process, method, product or equipment including the elements. The first and second words are used to indicate names, but do not indicate any particular order.

For the convenience of description, when describing the above device, the functions are divided into various modules and described separately. Of course, when implementing one or more of this specification, the functions of each module may be implemented in the same or more software and/or hardware, or the modules that achieve the same function may be implemented by a combination of multiple submodules or subunits, etc. . The device embodiments described above are only schematic. For example, the division of the unit is only a division of logical functions. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or integrated To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The present invention is described with reference to flowcharts and/or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and/or block in the flowchart and/or block diagram and a combination of the flow and/or block in the flowchart and/or block diagram may be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processing machine, or other programmable data processing device to produce a machine that enables the generation of instructions executed by the processor of the computer or other programmable data processing device An apparatus for realizing the functions specified in one block or multiple blocks of one flow or multiple flows of a flowchart and/or one block or multiple blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and/or block diagrams.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device The instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and/or block diagrams.

In a typical configuration, the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-permanent memory, random access memory (RAM) and/or non-volatile memory in computer-readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer-readable media, including permanent and non-permanent, removable and non-removable media, can store information by any method or technology. The information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic cassette tapes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. As defined in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

Those skilled in the art should understand that one or more embodiments of this specification may be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may employ computer programs implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code The form of the product.

One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

The embodiments in this specification are described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method embodiment. In the description of this specification, the description referring to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means specific features described in conjunction with the embodiment or examples , Structure, material or characteristic is included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, without contradicting each other, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.

The above is only an embodiment of one or more embodiments of this specification, and is not intended to limit one or more embodiments of this specification. For those skilled in the art, various modifications and changes can be made to one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this specification shall be included in the scope of the claims.

Claims

A head and neck plaque image recognition method, characterized in that it includes:

Obtain the head and neck magnetic resonance image to be recognized;

Input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model to obtain the plaque recognition result in the head and neck magnetic resonance image to be recognized;

Wherein, the plaque recognition model uses a U-shaped convolutional neural network model, and the convolutional layer of the U-shaped convolutional neural network model uses a dense connection block in a dense convolutional neural network model.
The method according to claim 1, wherein the number of feature maps output by each of the densely connected blocks in the patch recognition model is different.
The method according to claim 1, wherein the densely connected blocks in the patch identification model include multiple densely connected layers, and densely connected between the densely connected layers.
The method according to claim 3, wherein the number of the dense connection blocks is plural, and the number of dense connection layers in each of the dense connection blocks is the same.
The method of claim 1, wherein the plaque recognition model is constructed using the following method:

Acquiring multiple sample data, the sample data including: a head and neck magnetic resonance image and a plaque marker in the head and neck magnetic resonance image;

Establishing the plaque recognition model, using the head and neck magnetic resonance image in the sample data as input data of the plaque recognition model, and using the corresponding plaque marker in the head and neck magnetic resonance image as the plaque recognition The output label of the model trains the plaque recognition model until the plaque recognition model meets the preset requirements.
The method of claim 5, wherein the method further comprises: optimizing the plaque recognition model using a cross-validation method.
The method of claim 5, wherein the acquiring multiple sample data includes:

Acquiring multiple head and neck magnetic resonance images, and normalizing the head and neck magnetic resonance images;

Plaque labeling multiple acquired head and neck magnetic resonance images to obtain plaque marks in the head and neck magnetic resonance images;

Correspondingly, the use of the normalized head and neck magnetic resonance image as input data of the plaque recognition model includes:

The normalized head and neck magnetic resonance image is used as input data of the plaque recognition model.
A head and neck plaque image recognition device, characterized in that it includes:

An image acquisition module for acquiring head and neck magnetic resonance images to be identified;

An image recognition module, used to input the head and neck magnetic resonance image to be recognized into the constructed plaque recognition model, and obtain the plaque recognition result in the head and neck magnetic resonance image to be recognized;

Wherein, the plaque recognition model uses a U-shaped convolutional neural network model, and the convolutional layer of the U-shaped convolutional neural network model uses a dense connection block in a dense convolutional neural network model.
The apparatus according to claim 8, wherein the number of feature maps output by each of the densely connected blocks in the patch identification model is different.
The device according to claim 8, wherein the densely connected blocks in the patch identification model include multiple densely connected layers, and densely connected between the densely connected layers.
The apparatus of claim 10, wherein the number of the dense connection blocks is plural, and the number of dense connection layers in each of the dense connection blocks is the same.
The apparatus of claim 8, wherein the apparatus further comprises: a model building module for building the plaque recognition model using the following method:

Acquiring multiple sample data, the sample data including: a head and neck magnetic resonance image and plaques marked in the head and neck magnetic resonance image;

Establishing the plaque recognition model, using the head and neck magnetic resonance image in the sample data as input data of the plaque recognition model, and using the corresponding plaque marked in the head and neck magnetic resonance image as the plaque recognition The output label of the model trains the plaque recognition model until the plaque recognition model meets the preset requirements.
The apparatus of claim 12, wherein the model building module is further used to:

A cross-validation method is used to optimize the plaque recognition model.
The apparatus of claim 12, wherein the model building module is specifically used to:

Acquiring multiple head and neck magnetic resonance images, and normalizing the head and neck magnetic resonance images, and using the normalized head and neck magnetic resonance images as input data of the plaque recognition model;

Plaque annotation is performed on the acquired multiple head and neck magnetic resonance images to obtain plaque marks in the head and neck magnetic resonance images.
A head and neck plaque image recognition processing device, characterized in that it includes: at least one processor and a memory for storing processor-executable instructions, and the processor implements any one of claims 1-7 when executing the instructions The method.
A head and neck plaque image recognition system, characterized in that it includes:

A data acquisition module for acquiring sample data, the sample data including: head and neck magnetic resonance images and plaques marked in the head and neck magnetic resonance images;

The detection model building module is used to build a plaque recognition model, and perform model training and model optimization, wherein the plaque recognition model uses a U-shaped convolutional neural network model, and a convolutional layer of the U-shaped convolutional neural network model Use dense connection blocks in the dense convolutional neural network model;

The model test module is used for inputting the head and neck magnetic resonance images to be recognized to obtain the head and neck plaque recognition results.