CN110838118A

CN110838118A - System and method for anomaly detection in medical procedures

Info

Publication number: CN110838118A
Application number: CN201911114375.XA
Authority: CN
Inventors: 阿伦·因南耶; 吴子彦; 阿比舍克·沙玛; 斯里克里希纳·卡拉南
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2019-09-24
Filing date: 2019-11-14
Publication date: 2020-02-25
Anticipated expiration: 2039-11-14
Also published as: US20210090736A1; CN110838118B

Abstract

The present application relates to systems and methods for anomaly detection in medical procedures. The method may include obtaining image data collected by one or more vision sensors through monitoring a medical procedure. The method may include determining a detection result of the medical procedure based on the image data by using a trained machine learning model for anomaly detection. The detection result may include whether an abnormality exists in the medical procedure. In response to a detection of the presence of the anomaly, the method may further include providing feedback relating to the anomaly.

Description

System and method for anomaly detection in medical procedures

Cross-referencing

This application claims priority to U.S. application No.16/580,053, filed 24.9.2019, the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates generally to the field of anomaly detection, and more particularly to a system and method for anomaly detection in medical procedures.

Background

Medical procedures (e.g., medical scanning, surgery) in hospitals are often sensitive to foreign bodies. For example, metal objects within a Magnetic Resonance (MR) scan room can cause damage to the scanner and patient and result in poor scan results (e.g., artifacts in images generated based on the MR scan). As another example, in a surgical environment, objects (e.g., sponges, needles, etc.) used during a surgical procedure may be inadvertently left in the patient. Conventionally, to detect and/or track such objects, magnetically active elements may be detected using a magnetic tracker during a medical scan, or such objects may be marked with a Radio Frequency Identification (RFID) tag, bar code, or the like. Trackers or marked objects are highly susceptible to human error. For example, an operator (e.g., a nurse, a technician, etc.) forgets to push the wheelchair out of the MRI room, the RFID tag is damaged, and so forth. Accordingly, there is a need to provide a system and method for efficiently and universally detecting objects of interest during a medical procedure.

Disclosure of Invention

One embodiment of the present application provides a method for anomaly detection in a medical procedure. The method includes obtaining image data collected by one or more vision sensors through monitoring a medical procedure. The method includes determining a detection result of the medical procedure based on the image data using a trained machine learning model for anomaly detection, the detection result including whether an anomaly exists in the medical procedure. The method further includes providing feedback relating to the anomaly in response to the detection of the presence of the anomaly.

One of the embodiments of the present application provides a system for anomaly detection in a medical procedure. The system comprises an acquisition module, a determination module and a feedback module. The acquisition module is used to obtain image data collected by one or more vision sensors through monitoring a medical procedure. The determination module is configured to determine a detection result of the medical procedure using the trained machine learning model for anomaly detection based on the image data, the detection result including whether the medical procedure is anomalous. The feedback module is configured to provide feedback relating to the anomaly in response to the detection of the presence of the anomaly.

One of the embodiments of the present application provides an apparatus for abnormality detection in a medical procedure, the apparatus including a processor and a memory, the memory storing instructions. The instructions, when executed by the processor, cause the apparatus to implement a method for anomaly detection in a medical procedure.

One embodiment of the present application provides a computer-readable storage medium storing computer instructions, which when read by a computer, cause the computer to perform a method for abnormality detection in a medical procedure.

Drawings

The present application may be further described in terms of exemplary embodiments. The exemplary embodiments may be described in detail with reference to the accompanying drawings. The described embodiments are not limiting exemplary embodiments in which like reference numerals represent similar structures throughout the several views of the drawings and wherein:

FIG. 1 is a schematic view of an exemplary anomaly detection system shown in accordance with some embodiments of the present application;

FIG. 2 is a schematic diagram of hardware components and/or software components of an exemplary computing device according to some embodiments of the present application;

FIG. 3 is a schematic diagram of exemplary hardware and/or software components of a mobile device according to some embodiments of the present application;

FIG. 4A is a block diagram of an exemplary processing device shown in accordance with some embodiments of the present application;

FIG. 4B is a block diagram of another exemplary processing device, shown in accordance with some embodiments of the present application;

FIG. 5 is a flow diagram of an exemplary process of anomaly detection shown in accordance with some embodiments of the present application;

FIG. 6 is an exemplary flow diagram illustrating training of a machine learning model according to some embodiments of the present application;

FIG. 7 is a schematic illustration of test results relating to an exemplary medical procedure, shown in accordance with some embodiments of the present application;

FIG. 8 is a schematic illustration of a test result relating to another exemplary medical procedure, shown according to some embodiments of the present application; and

fig. 9 is a schematic illustration of anomaly detection for an exemplary surgical procedure, shown in accordance with some embodiments of the present application.

Detailed Description

The following description is presented to enable one of ordinary skill in the art to make and use the application and is provided in the context of a particular application and its requirements. It will be apparent to those skilled in the art that various modifications to the disclosed embodiments are possible, and the general principles defined in this application may be applied to other embodiments and applications without departing from the spirit and scope of the application. Thus, the present application is not limited to the described embodiments, but should be accorded the widest scope consistent with the claims.

One aspect of the present application relates to methods and systems for anomaly detection in medical procedures. The system may obtain image data collected by one or more vision sensors by monitoring a medical procedure. The system may obtain a trained machine learning model for anomaly detection. Based on the image data, the system may determine a detection result of the medical procedure using the trained machine learning model. The detection result may include whether an abnormality exists in the medical procedure. In response to the detection of the presence of the anomaly, the system may provide feedback relating to the anomaly. In this way, the abnormality detection system can universally and efficiently detect whether there is an abnormality in a medical procedure. As used herein, the term "generically" means that the anomaly detection system may be applied to monitoring anomalies caused by the presence of various foreign objects that may cause damage to or cause anomalies in medical equipment, individuals, etc. that are associated with medical procedures, rather than being applied only to monitoring of particular types of foreign objects. The anomaly detection system may further determine positional information of one or more objects causing anomalies in the medical procedure and provide feedback. The method and system for abnormality detection according to some embodiments of the present application may reduce the risk of abnormalities caused by foreign objects present in a medical procedure to individuals, medical equipment, etc. associated with the medical procedure. Thus, the systems and methods described herein may perform automated anomaly detection based on image processing. For example, the system and method may input images relating to a medical procedure into a trained machine learning model. By processing the images, the trained machine learning model can directly and automatically output the detection result. The detection result may include whether an abnormality exists in the medical procedure. Although the anomaly-causing object is diverse, the systems and methods described herein can identify anomalies in real-time relating to medical procedures and anomaly-causing objects.

FIG. 1 is a schematic diagram of an exemplary anomaly detection system 100, shown in accordance with some embodiments of the present application. In some embodiments, the anomaly detection system 100 may be used in an Intelligent Transportation System (ITS), a security system, a transportation management system, a prison system, an astronomical observation system, a monitoring system, a species identification system, an industrial control system, an Identification (ID) system, a medical process system, a retrieval system, or the like, or any combination thereof. The anomaly detection system 100 can include a platform for data and/or information processing, e.g., for training anomaly detection and/or data classification (e.g., image classification, text classification, etc.). The anomaly detection system 100 can be applied to intrusion detection, fault detection, network anomaly traffic detection, fraud detection, behavioral anomaly detection, and the like, or a combination thereof. "outliers" may also be referred to as outliers, noise, deviations, exceptions, and the like. As used herein, "abnormal" refers to a behavior or event that is determined to be unusual or abnormal according to known or inferred conditions. For example, for the auditing process of a police station, prison, etc., an "anomaly" may include an anomaly due to the presence of a foreign object. For another example, for a medical procedure, "abnormalities" may include abnormalities caused by personal behavior, abnormalities caused by the presence of foreign objects, and the like.

For convenience of description, the abnormality detection system 100 for a medical procedure will be described as an example. As shown in fig. 1, the anomaly detection system 100 can include a medical device 110, a monitoring device 120, one or more terminals 140, a processing device 130, a storage device 150, and a network 160. In some embodiments, the medical device 110, the monitoring device 120, the processing device 130, the terminal 140, and/or the storage device 150 may be connected and/or communicate via a wireless connection (e.g., the network 160), a wired connection, or a combination thereof. The connections between components in the anomaly detection system 100 may vary. As shown in fig. 1, the monitoring device 120 may be connected to the processing device 130 via a network 160. Storage device 150 may be connected to processing device 130 through network 160 or directly to processing device 130. The terminal 140 may be connected to the processing device 130 through the network 160 or may be directly connected to the processing device 130.

Medical device 110 may include any device used in a medical procedure. A medical procedure may refer to an activity or series of actions performed to obtain a result of healthcare, for example, to measure, diagnose, and/or treat a subject (e.g., a patient). Exemplary medical procedures may include point-of-care examinations, diagnostic examinations, therapeutic procedures, autopsy, and the like. An immediate examination is one that is performed to check the overall health of an individual before the disease can treat the disease or condition. When the immediate check is performed, the result of the immediate test can be obtained in real time. For example, the immediate check may include a blood pressure test. Diagnostic tests may be performed to check for certain conditions or diseases or to test physical endurance. For example, the diagnostic examination may include a cardiac stress test for testing cardiac intensity, an imaging scan of a portion or the entire body of a patient, a procedure for diagnosis, and the like. A treatment program may include a series of operations to treat a problem or disease in a subject (e.g., a patient). For example, the treatment procedure may include surgery, radiation therapy, and the like. The subject may include biological or non-biological. For example, the subject may include a patient, an artificial object, and the like. For example, a subject may include a particular part, organ, and/or tissue of a patient.

The medical device 110 can include an imaging device, a therapy device (e.g., a surgical device), a multi-modality device, and the like to obtain one or more images of different modalities or to obtain images related to at least a portion of a subject, to treat at least a portion of a subject, and the like. The imaging device may be configured to generate an image including at least a portion of the subject. Exemplary imaging devices may include, for example, a Computed Tomography (CT) device, a cone-beam CT device, a Positron Emission Tomography (PET) device, a volume CT device, a Magnetic Resonance Imaging (MRI) device, or the like, or combinations thereof. The treatment device may be configured to treat at least a portion of the subject. Exemplary treatment devices may include radiation therapy devices (e.g., linear accelerators), X-ray therapy devices, surgical devices, and the like. Exemplary surgical devices may include anesthesia machines, ventilators, operating tables, lights, infusion pumps, surgical consumables (e.g., tourniquets, sponges, etc.), and the like, or any other tool, such as scalpels, hemostats, and the like.

The monitoring device 120 may be located at any location that enables the monitoring device 120 to monitor a region of interest (AOI) or an object of interest. The monitoring device 120 may include one or more acoustic sensors, one or more visual sensors, and the like. One or more acoustic sensors may be configured to collect audio signals and/or generate audio data during a medical procedure. A visual sensor may refer to a device for visual recording. The vision sensor may capture image data to record the medical procedure. The image data may include a still image, a video, an image sequence including a plurality of still images, and the like. In some embodiments, the vision sensor may include a stereo camera for capturing still images or video. In some embodiments, the visual sensor may comprise a digital camera. In some embodiments, the monitoring device 120 may transmit the collected image data and/or audio data to the processing device 130, the storage device 150, and/or the terminal 140 via the network 160.

The processing device 130 may process data and/or information obtained from the medical device 110, the monitoring device 120, the terminal 140, the storage device 150, and/or the monitoring device 120. For example, the processing device 130 may process image data captured by the monitoring device 120. For another example, the processing device 130 may train the machine learning model to obtain a trained machine learning model for anomaly detection. As yet another example, processing device 130 may use a trained machine learning model for anomaly detection and determine a detection result of a medical procedure based on image data. In some embodiments, the determination and/or updating of the trained machine learning model may be performed on a processing device, while the application of the trained machine learning model may be performed on a different processing device. In some embodiments, the determination and/or updating of the trained machine learning model may be performed on a processing device of a system other than the anomaly detection system 100 or on a server other than the processing device 130 that includes the trained machine learning model application. In some embodiments, the determination and/or updating of the trained machine learning model may be performed offline.

In some embodiments, the processing device 130 may be a single server or a group of servers. The server groups may be centralized or distributed. In some embodiments, the processing device 130 may be local or remote. For example, the processing device 130 may access information and/or data from the medical device 110, the terminal 140, the storage device 150, and/or the monitoring device 120 via the network 160. As another example, the processing device 130 may be directly connected to the medical device 110, the monitoring device 120, the terminal 140, and/or the storage device 150 to access information and/or data thereof. In some embodiments, the processing device 130 may be implemented on a cloud platform. In some embodiments, the processing device 130 may be implemented by a mobile device 300 having one or more components as described in fig. 3.

The terminal 140 may be connected to and/or in communication with the medical device 110, the processing device 130, the storage device 150, and/or the monitoring device 120. For example, the terminal 140 may obtain the processed image from the processing device 130. As another example, one or more terminals 140 may obtain image data obtained by monitoring device 120 and send the image data to processing device 130 for processing. In some embodiments, terminal 140 may include a mobile device 141, a tablet computer 142, …, a handheld computer 143, or the like, or any combination thereof. In some embodiments, the terminal 140 may include an input device, an output device, and the like. In some embodiments, the terminal 140 may be part of the processing device 130.

The storage device 150 may store data, instructions, machine learning models (e.g., initial machine learning models, trained machine learning models, etc.), and/or any other information. In some embodiments, the storage device 150 may store data obtained from the medical device 110, the terminal 140, the processing device 130, and/or the monitoring device 120. In some embodiments, storage device 150 may store data and/or instructions that processing device 130 may perform or be used to perform the exemplary methods described herein. In some embodiments, storage device 150 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), and the like, or any combination thereof. In some embodiments, the storage device 150 may be implemented on a cloud platform as described elsewhere in this application. Network 160 may include any suitable network that may facilitate the exchange of information and/or data for anomaly detection system 100. In some embodiments, one or more components of the abnormality detection system 100 (e.g., the medical device 110, the terminal 140, the processing device 130, the storage device 150, the monitoring device 120, etc.) may communicate information and/or data with one or more other components of the abnormality detection system 100 via the network 160. Network 160 may be and/or include a public network (e.g., the internet), a private network (e.g., a Local Area Network (LAN), a Wide Area Network (WAN)), etc.), a wired network (e.g., an ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi network, etc.), a cellular network (e.g., a Long Term Evolution (LTE) network), a frame relay network, a Virtual Private Network (VPN), a satellite network, a telephone network, a router, a hub, a switch, a server computer, etc., and/or any combination thereof. In some embodiments, the network 160 may include one or more network access points (e.g., wired and/or wireless network access points such as base stations and/or internet exchange points) through which one or more components of the anomaly detection system 100 may be connected to the network 160 to exchange data and/or information.

FIG. 2 is a schematic diagram of hardware and/or software components of an exemplary computing device, shown in accordance with some embodiments of the present application. As shown in FIG. 2, computing device 200 may include a processor 210, memory 220, input/output (I/O)230, and communication ports 240. In some embodiments, processing device 130 and/or terminal 140 may be implemented on computing device 200.

FIG. 3 is a schematic diagram of exemplary hardware and/or software components of a mobile device according to some embodiments of the present application. In some embodiments, the processing device 130 and/or the terminal 140 may be implemented on a mobile device 300.

FIG. 4A is a block diagram of an exemplary processing device, shown in accordance with some embodiments of the present application. In some embodiments, processing device 130 may be implemented on computing device 200 (e.g., processor 210) shown in fig. 2 or CPU 340 shown in fig. 3. As shown in fig. 4A, the processing device 130 may include an acquisition module 410, a determination module 420, a feedback module 430, and a storage module 440.

The acquisition module 410 may be configured to obtain information and/or data for anomaly detection of a medical procedure. For example, the acquisition module 410 may acquire image data collected by one or more vision sensors in monitoring a medical procedure. As another example, the acquisition module 410 may obtain a trained machine learning model for anomaly detection. The trained machine learning model for anomaly detection may be configured to detect anomalies with respect to a particular medical procedure based on particular image data associated with the particular medical procedure. Based on a determination that an abnormality exists for a particular medical procedure, the trained machine learning model may be used to identify and/or determine location information for one or more objects of interest in the input particular image data. An object of interest refers to an object that causes an abnormality in a particular medical procedure. In some embodiments, the acquisition module 410 may obtain image data or a trained machine learning model from the monitoring device 120, the storage device 150, the terminal 140, or any other storage device, periodically or in real-time. For example, the image data may be collected by the monitoring device 120 and sent to one or more components of the anomaly detection system 100.

The determination module 420 may use a trained machine learning model to determine the detection of the medical procedure based on the image data. The determination module 420 may input the image data into a trained machine learning model. The determination module 420 may obtain detection results generated using the trained machine learning model based on the input image data. In some embodiments, the detection result of the medical procedure may include a positive result or a negative result. A positive result may indicate the presence of an abnormality associated with the medical procedure. In some embodiments, based on the image data, the presence of abnormalities in the image data relating to the medical procedure may be determined using a trained machine learning model. In response to determining that the image data includes an abnormality, the determination module 420 may identify and/or determine location information of one or more objects of interest in the image data that cause the abnormality in the medical procedure.

In response to the detection of the presence of an anomaly, the feedback module 430 may be configured to provide feedback relating to the anomaly. In some embodiments, the feedback provided by the feedback module 430 may include detection results regarding the abnormal presence of the medical procedure. For example, the feedback module 430 may generate a notification to inform that an exception exists. A notification informing that there is an abnormality is transmitted to the device (e.g., the terminal 140). The device may play and/or display a notification to the relevant individual (e.g., patient, physician) to inform that the anomaly exists.

The storage module 412 may store information. This information may include programs, software, algorithms, data, text, numbers, images, and some other information. For example, the information may include image data related to a medical procedure, a trained machine learning model for anomaly detection, and the like.

The description of the processing device 130 above is for illustrative purposes only and is not intended to limit the scope of the present application. Various changes and modifications will occur to those skilled in the art based on the description herein. For example, the components and/or functionality of the processing device 130 may be changed or varied depending on the particular implementation. For example only, the determination module 420 and the feedback module 430 may be integrated into a single module.

Fig. 4B is a block diagram of another exemplary processing device, shown in accordance with some embodiments of the present application. In some embodiments, processing device 130 may be implemented on computing device 200 (e.g., processor 210) shown in fig. 2 or CPU 340 shown in fig. 3. As shown in fig. 4B, processing device 130 may include an acquisition module 450, an extraction module 460, a training module 470, and a storage module 480. Each of the modules described above may be hardware circuitry designed to perform certain actions in accordance with a set of instructions stored in one or more storage media and any combination of one or more storage media and/or hardware circuitry.

The acquisition module 450 may be configured to acquire at least two training samples, each sample including image data (e.g., images, videos, etc.) related to a normal scene of a medical procedure to which the training sample relates. In some embodiments, the acquisition module 450 may be configured to acquire at least two training samples, a portion of which includes image data (e.g., images, videos, etc.) relating to an abnormal scene of a medical procedure. As used herein, training samples for an abnormal scenario may also be referred to as samples that include abnormal samples. If the training samples include anomalous samples, the training samples may include a label indicating that the samples are anomalous. Each of the at least two training samples may include historical image data collected by one or more visual sensors by monitoring historical medical procedures over a historical period of time (e.g., one or more years past, one or more months past). For example, the training sample may include one or more still images captured by one or more vision sensors. In some embodiments, the training samples may be obtained from the monitoring device 120 or from a storage device (e.g., storage device 150, an external data source), the terminal 140, or any other storage device.

In some embodiments, the label of each of the at least two training samples may be a negative label. If the training sample is a negative training sample with no sample anomalies, the training sample may be labeled with a negative label. In some embodiments, if the training sample is a positive training sample with sample anomalies, each of the portions of the at least two training samples may be labeled with a positive label. The training samples may be labeled with a binary label (e.g., 0 or 1, positive or negative, etc.). For example, negative training samples may be labeled with a negative label (e.g., "0"), while positive training samples may be labeled with a positive label (e.g., "1"). The use of a positive sample of the at least two training samples may improve the accuracy of a trained machine learning model for anomaly detection trained using the at least two training samples.

The extraction module 460 is configured to determine at least two regions in each of at least two training samples using an initial machine learning model. In some embodiments, the at least two regions may be determined based on a sliding window algorithm, a region suggestion algorithm, an image segmentation algorithm, etc., using an initial machine learning model. In some embodiments, the extraction module 460 may be further configured to extract image features from each of the at least two regions. The image features may refer to representations of particular structures in the training sample region, such as points, edges, objects, and the like. The extracted image features may be binary, numeric, categorical, ordinal, binomial, interval, text-based, or a combination thereof. In some embodiments, the image features may include low-level features (e.g., edge features, texture features), high-level features (e.g., semantic features), or complex features (e.g., deep-level features). The initial machine learning model may process the input training samples through multi-layer feature extraction (e.g., convolutional layers) to extract image features.

The training module 470 may be configured to train the initial machine learning model to obtain a trained machine learning model. In some embodiments, the trained machine learning model is obtained by training an initial machine learning model using a training algorithm based on image features extracted from each of at least two training samples. Exemplary training algorithms may include a gradient descent algorithm, a newton algorithm, a quasi-newton algorithm, a Levenberg-Marquardt algorithm, a conjugate gradient algorithm, the like, or combinations thereof.

The memory module 480 may store information. This information may include programs, software, algorithms, data, text, numbers, images, and some other information. For example, the information may include training samples, trained machine learning models for anomaly detection, initial machine learning models, training algorithms, and so forth.

Fig. 5 is a flow diagram of an exemplary process 500 of anomaly detection, shown in accordance with some embodiments of the present application. Process 500 may be performed by anomaly detection system 100. For example, the process 500 may be implemented as a set of instructions (e.g., an application) stored in the storage device 150 in the anomaly detection system 100. Processing device 130 may execute the set of instructions, and processing device 130 may thus be instructed to perform process 500 in anomaly detection system 100. The operations of the illustrated process 500 presented below are illustrative. In some embodiments, process 500 may include one or more additional operations not described, and/or may be accomplished without one or more of the operations discussed. Additionally, the order in which the operations of process 500 are illustrated in FIG. 5 and described below is not intended to be limiting.

In 510, the processing device 130 (e.g., the acquisition module 410) may obtain image data collected by one or more vision sensors by monitoring the medical procedure.

The image data may include a representation of a scene about the medical procedure. For example, the image data may include a display of one or more objects appearing in the scene that are relevant to the medical procedure. The one or more vision sensors may be configured to monitor a region of interest (AOI) or one or more objects within range of the one or more vision sensors performing the medical procedure. More description of medical procedures and/or visual sensors may be found in fig. 1 and its description. The image data collected by the one or more vision sensors may include a display of one or more objects that appear while the medical procedure is being performed. Objects that appear where a medical procedure is performed may include individuals (e.g., doctors, patients), medical devices, or any other item object, such as an accessory of an individual (e.g., a bracelet, necklace, or glasses), a patient wheelchair, and so forth. In some embodiments, the image data may include still images, video, or a combination thereof obtained by one or more vision sensors. For example, the one or more vision sensors may include an Infrared (IR) camera, a video camera, an RGB-D camera, and the like. The infrared camera is configured to collect IR images for recording one or more scenes during a medical procedure. The video camera is configured to capture video recording a medical procedure, and the RGB-D camera is configured to capture images of one or more scenes recorded during the medical procedure. In some embodiments, the image data may include multiple frames of video, one or more still images, and the like. Each of the plurality of frames or each of the one or more still images may have a time stamp recording the time at which the image data was captured by the one or more vision sensors. The image data may record the medical procedure over time according to a timestamp associated therewith. For example, changes in the position of an object (e.g., a sponge) over a period of time during a procedure may be recorded from the image data based on a timestamp associated with the image data. In some embodiments, the processing device 130 may obtain image data from the monitoring device 120, the storage device 150, the terminal 140, or any other storage device on an irregular or periodic basis. For example, the image data may be collected by the monitoring device 120 and sent to one or more components of the anomaly detection system 100. For example, image data collected by the monitoring device 120 may be sent in real-time directly to the processing device 130 for further processing. As another example, the image data collected by the monitoring device 120 may be sent to the storage device 150 or an external source for storage. Processing device 130 may retrieve at least a portion of the image data from storage device 150 or an external storage device. As another example, image data obtained by one or more vision sensors may be sent to terminal 140 for display. Processing device 130 may send at least a portion of the image data (e.g., after processing) to terminal 140 via network 160.

In 520, the processing device 130 (e.g., the acquisition module 410) may obtain a trained machine learning model for anomaly detection. In some embodiments, based on particular image data associated with a particular medical procedure, a trained machine learning model for anomaly detection may be configured to detect anomalies in the particular medical procedure. As used herein, an abnormality in a particular medical procedure may refer to the presence or existence of one or more objects of interest in the medical procedure, which may result in damage or abnormality to the medical device (e.g., medical device 110), an individual (e.g., a doctor, a patient, etc.), etc., associated with the medical procedure. In some embodiments, in response to determining that an abnormality exists with respect to a particular medical procedure, the trained machine learning model may be used to identify and/or determine positional information of one or more objects of interest in the input particular image data that are capable of causing an abnormality with respect to the particular medical procedure.

In some embodiments, processing device 130 may invoke the trained machine learning model from storage device 150 or any other storage device. For example, the trained machine learning model may be obtained by training the machine learning model offline using a processing device that is different from or the same as processing device 130. The processing device 130 may store the trained machine learning model in the storage device 150 or any other storage device. In response to receiving the request for anomaly detection, processing device 130 may invoke the trained machine learning model from storage device 150 or any other storage device. More description of the training of machine learning models for anomaly detection can be found elsewhere in this application. See, for example, fig. 6 and its associated description.

In 530, the processing device 130 (e.g., the determination module 420) may determine a detection result of the medical procedure based on the image data by using the trained machine learning model.

In some embodiments, the detection of the medical procedure may indicate whether an abnormality exists in the medical procedure. In some embodiments, an object that may cause damage or an abnormality to a medical device (e.g., medical device 110), an individual (e.g., a doctor, a patient, etc.), and the like is referred to as an abnormality. For example, an anomaly in an MR scan may include one or more magnetically active elements present in the MR room during the MR scan (e.g., a patient's metal ornaments (e.g., watches, jewelry, hair pins), the patient's wheelchair, etc.), which may pose a serious threat to the patient and/or damage the MR scanner. As another example, an abnormality during a surgical procedure may include one or more foreign objects (e.g., sponges) that may be accidentally left in the patient after the procedure, which may cause injury to the patient. As another example, an abnormality in a medical procedure may include one or more objects that are not in a predetermined location according to a prescription (e.g., a scanning protocol, a procedure, etc.). As yet another example, the abnormality in the medical procedure includes an obstacle on a trajectory of the medical device that moves during the medical procedure. In some embodiments, an abnormality in a medical procedure may include an event that may cause damage or an abnormality to a medical device (e.g., medical device 110), an individual (e.g., a doctor, a patient, etc.) associated with the medical procedure. For example, an abnormality in the medical procedure may include an abnormal setting of the medical device (e.g., a position of the scanning table) related to the medical procedure. As another example, an abnormality in a medical procedure may include abnormal behavior of an individual (e.g., a patient) in the medical procedure. As another example, abnormal behavior of an individual may include the patient being incorrectly positioned, the individual moving to or being at a dangerous location, etc.

In some embodiments, the detection result of the medical procedure may include a positive result or a negative result. A positive result may indicate the presence of an abnormality associated with the medical procedure. A negative result may indicate that no anomalies are present in the image data. The processing device 130 may input the image data into the trained machine learning model. The processing device 130 may generate a detection result based on the input image data. For example, the trained machine learning model may divide image data (e.g., an image or video) into one or more regions (or segments or instances). The processing device 130 may determine a prediction result for each of the one or more regions (or segments or instances). The prediction result of the region (or segment or instance) may indicate whether the region (or segment or instance) includes an object of interest that causes an abnormality in the image data. In other words, the predicted outcome of a region (or segment or instance) may indicate whether the region has an abnormality associated with a medical procedure. The predicted outcome for a region (or segment or instance) may include a predicted positive outcome or a predicted negative outcome. The positive outcome of the prediction of the region may indicate that the region (or segment or instance) includes an object of interest that causes an abnormality in the medical procedure. A predicted negative outcome for a region may indicate that the region (or segment or instance) includes an object of no interest that does not cause an abnormality in the medical procedure or lack of an object of interest. In some embodiments, a predicted positive result may be represented by a positive label, such as a "1". The predicted negative result may be represented by a negative label, such as "0". The processing device 130 may determine a detection result of the image data based on the prediction result of the one or more regions. For example, if all of the predicted outcomes for one or more regions are negative, i.e., the prediction signatures for one or more regions are negative signatures, then the processing device 130 may determine that there is no abnormality in the medical procedure and the detected outcome of the medical outcome is a negative outcome. If the at least one prediction outcome for the one or more regions is positive, i.e., the at least one prediction tag for the one or more regions is a positive tag, then processing device 130 may determine that an abnormality exists in the medical procedure and the detection outcome of the medical procedure may be a positive outcome.

In some embodiments, input image data (i.e., images, video) may be divided into a plurality of segments or regions, each segment including a display of an object. Image features may be extracted from each segment or instance. The one or more image features extracted and/or output may also be referred to as a feature map or vector. Exemplary image features may include low-level features (e.g., edge features, texture features), high-level features (e.g., semantic features), complex features (e.g., deep-level features), and the like. Based on image features extracted from a particular region in the image data, the processing device 130 may determine a prediction result for the particular region. For example, based on the extracted image features, the trained machine learning model may determine an anomaly score for a particular region and determine a prediction result based on the anomaly score for the particular region. For example, if the trained machine learning model determines that the anomaly score for a particular region is greater than the anomaly threshold, the trained machine learning model may determine that the detection result for the particular region is positive and/or specify a positive label "1" for the particular region; otherwise, the trained machine learning model may determine that the prediction result for the particular region is negative and/or assign a negative label "0" to the particular region.

In some embodiments, the trained machine learning model may determine an anomaly score based on the extracted image features of the plurality of segments or regions. The anomaly score may indicate a probability that the input image data includes an anomaly. The trained machine learning model may determine whether an abnormality exists in the medical procedure based on the abnormality score. For example, the trained machine learning model may compare the abnormality score to an abnormality threshold and if the abnormality score exceeds the abnormality threshold, the trained machine learning model may determine that the medical procedure is abnormal. In some embodiments, the trained machine learning model may determine an anomaly score based on image features extracted from each of a plurality of segments or regions. Each of the plurality of segments or regions may be assigned an anomaly score. The trained machine learning model may determine whether an abnormality exists for the medical procedure based on one or more abnormality scores corresponding to the plurality of segments or regions. For example, the trained machine learning model may compare a maximum abnormality score of the abnormality scores to an abnormality threshold, and if the maximum score exceeds the abnormality threshold, the trained machine learning model may determine that the medical procedure is abnormal. An anomaly score for a particular region may be determined based on a trained machine-learned probability generating function. The trained machine-learned probability generation functions may include logistic functions, sigmoid functions, and the like.

In some embodiments, based on the image data, the trained machine learning model may determine that an anomaly exists in the image data related to the medical procedure. In response to determining that the image data includes abnormalities, the trained machine learning model may be used to identify and/or determine location information for one or more objects of interest in the image data that cause abnormalities related to the medical procedure. In other words, the trained machine learning model may classify one or more objects present in the input image data into two classes, including a positive class and a negative class. Objects belonging to the negative category (also referred to as objects of no interest) do not cause abnormalities regarding the specific medical procedure. Objects belonging to the positive category (also referred to as objects of interest) may cause abnormalities regarding the medical procedure. In some embodiments, the trained machine learning model may use bounding box labeling and/or locating objects of interest in the input image data that can cause abnormalities with respect to the medical procedure. The bounding box may refer to a box enclosing at least a portion of the detected object of interest in the image data. The bounding box may have any shape and/or size. For example, the bounding box may have a square, rectangular, triangular, polygonal, circular, elliptical, irregular, etc. shape. In some embodiments, the bounding box may be a minimum box having a preset shape (e.g., rectangle, square, polygon, circle, ellipse) and completely enclosing the detected object of interest. As used herein, a minimum bounding box having a preset shape (e.g., rectangle, square, polygon, circle, ellipse) and completely enclosing a detected object of interest means that at least a portion of the detected object of interest is outside the minimum bounding box if the size of the minimum bounding box (e.g., the radius of the circular minimum bounding box, the length or width of the rectangular minimum bounding box, etc.) is reduced. The trained machine learning model may be configured to output a portion of the at least one processed image data with a bounding box labeling the detected object of interest. For example, the trained machine learning model may be configured to output a bounding box with the detected object of interest that caused the medical procedure abnormality.

In some embodiments, the trained machine learning model may be configured to track an object of interest in input image data (e.g., two adjacent frames of a video). For example, the trained machine learning model may determine a similarity of two objects of interest present in two adjacent frames of input image data (e.g., video). If the similarity of two objects of interest present in two adjacent frames of the video satisfies a condition, the trained machine learning model may designate the two objects of interest as one same object of interest.

In 540, in response to the detection of the presence of the anomaly, the processing device (e.g., feedback module 430) may provide feedback related to the anomaly.

In some embodiments, the feedback provided by the processing device 130 may include a detection result regarding the abnormal presence of the medical procedure. For example, to provide feedback regarding the exception, processing device 130 may generate a notification informing of the existence of the exception. A notification is sent to the device (e.g., terminal 140) informing of the existence of the anomaly. The device (e.g., terminal 140) may play and/or display a notification to the relevant individual (e.g., patient, doctor) to inform that an abnormality exists. Feedback or notifications related to abnormalities in the medical procedure may be in the form of images, text, voice, and the like. For example, a wheelchair may be left in the scanning room before the MR scan of the patient. The terminal 140 may receive the notification and issue an alarm to notify the operator of the MR scan of the existence of the anomaly with respect to the MR scan. As another example, the terminal 140 may display a message such as "foreign object! "or the like to inform an operator of the MR scan of the presence of an anomaly with respect to the MR scan.

In some embodiments, the detection result may include position information of at least one of the one or more objects causing the abnormality of the medical instrument. The feedback or notification provided by the processing device 130 may include location information of at least one of the one or more objects that caused the abnormality with respect to the medical device. For example, the processing device 130 may transmit to a device (e.g., the terminal 140) a portion of the image data that contains one or more objects of interest that have detected and/or marked an abnormality related to the medical procedure. Causing the device to display at least a portion of the received image data. The display includes a form of video or still image. The device may also display object of interest location information of at least one of the objects of interest. The position information of at least one of the objects of interest may be part of the received image data. For example, as described above, the position information of the at least one object of interest may be represented by a bounding box. In some embodiments, the device may highlight at least one object of interest of the one or more objects of interest. For example, the device may highlight the region of the object of interest enclosed by the bounding box using a different color than other regions around the object of interest.

It should be noted that the above description is merely for convenience and is not intended to limit the present application to the scope of the illustrated embodiments. Various changes and modifications will occur to those skilled in the art based on the description herein. However, such changes and modifications do not depart from the scope of the present application. For example, the processing device may pre-process the image data after it is obtained by the processing device 130. The pre-processing of the image data may include cropping, taking a snapshot, scaling, denoising, rotating, recoloring, subsampling, background removal, normalization, and the like, or any combination thereof. In some embodiments, the processing device 130 may obtain audio data obtained by one or more sound detectors. The audio data may be coupled with the image data. In some embodiments, speech recognition techniques may be used to convert audio data into textual data, such as one or more sentences, words, paragraphs, or the like. The trained machine learning model may determine whether an abnormality related to the medical procedure is present based on one or more images in the text data and/or the image data. In some embodiments, audio data may be input into the trained machine learning model along with image data. The trained machine learning model may determine whether an abnormality exists in the medical procedure based on the audio data and/or the image data. In some embodiments, operation 540 may be omitted.

Fig. 6 is a flow diagram of an exemplary process 600 of training a machine learning model, shown in accordance with some embodiments of the present application. In some embodiments, process 600 may be an offline process. Process 600 may be performed by anomaly detection system 100. For example, process 600 may be implemented as a set of instructions (e.g., an application program) stored in a storage device in processing device 130. The processing device 130 may execute the set of instructions and, thus, be instructed to perform the process 600 in the anomaly detection system 100. The operations of the illustrated process 600 presented below are illustrative. In some embodiments, process 600 may include one or more additional operations not described, and/or be accomplished with one or more operations discussed removed. Additionally, the order in which the operations of process 600 are illustrated in FIG. 6 and described below is not intended to be limiting.

In 610, the processing device 130 (e.g., the acquisition module 450) may acquire at least two training samples. In some embodiments, the at least two training samples may comprise negative samples. The training sample may be a negative sample or a normal sample if the training sample includes image data of the medical procedure associated with the training sample without abnormalities. In some embodiments, the at least two training samples may comprise positive samples. If the training sample comprises image data with abnormalities, e.g. a patient wearing a watch in an MR scan room, the training sample may be marked as a positive sample or an abnormal sample. Each of the at least two training samples may include historical image data collected by one or more vision sensors by monitoring historical medical procedures over a historical period of time (e.g., one or more years past, one or more months past, etc.). For example, the training sample may include one or more still images captured by one or more vision sensors. In some embodiments, the training samples may be obtained from the monitoring device 120 or from a storage device (e.g., storage device 150, an external data source), the terminal 140, or any other storage device.

In some embodiments, the at least two training samples include at least two negative training samples, each of the at least two negative training samples containing no sample anomalies. In some embodiments, the at least two training samples may all be negative training samples (or negative samples). All subjects present in the negative training sample do not cause abnormalities in the medical procedure associated with the negative training sample. Using negative training examples, the machine learning model may be trained to understand what the normal case or scenario may be, and thus configured to detect deviations from the normal case or scenario to identify anomalies. In some embodiments, the at least two training samples may include a first portion and a second portion. The first portion may include at least two negative training examples, each of the at least two negative training examples containing no sample anomalies. The second portion may include at least two positive training samples (or positive samples), each of the at least two positive training samples containing a sample anomaly. The ratio of the count or number of at least two negative training samples in the first portion to the count or number of at least two positive training samples in the second portion may be a constant. The constant may be a default setting for the anomaly detection system 100. The larger the ratio of the count or number of the at least two positive training samples to the count or number of the at least two negative samples, the higher the detection rate of the trained machine learning model generated based on the at least two training samples, and the higher the false alarm rate of the trained machine learning model may be. The relevance ratio of the trained machine learning model may also be referred to as the sensitivity of the trained machine learning model. The detection rate of the trained machine learning model can be improved by increasing the proportion of the positive training samples in the at least two training samples. The false alarm rate can be reduced by increasing the proportion of negative training samples in the at least two training samples. The trained machine learning model is expected to have high detection rate and low false alarm rate. In order to achieve a desired balance between the two performance criteria of detection rate and false positive rate, the ratio of the counts of the at least two positive training samples to the counts of the at least two negative training samples may be close to or equal to the actual incidence of abnormalities in the clinic. For example, the actual incidence of abnormalities in clinical applications may be determined based on historical medical procedures over a historical period (e.g., the past year). Further, the number or count of historical medical procedures that include an abnormality and the number or count of historical medical procedures that have not been abnormal over a historical period of time may be statistically determined. The ratio of the count of the at least two positive training samples to the count of the at least two negative training samples may be close to or equal to the ratio of the number or count of historical medical procedures that include an abnormality to the historical number or count without an abnormality.

In some embodiments, the trained machine learning model for anomaly detection is constructed based on a weakly supervised learning model. In some embodiments, training the initial machine learning model using at least two training samples results in a trained machine learning model based on a weakly supervised learning technique (e.g., the trained machine learning model determined in 640). Exemplary weakly supervised learning techniques may include incompletely supervised learning techniques (e.g., active learning techniques and semi-supervised learning techniques), inaccurately supervised learning techniques (e.g., multiple example learning techniques), inaccurately supervised learning techniques, and the like. Using a weakly supervised learning technique, each of the at least two training samples may be labeled with a label indicating whether each of the at least two training samples contains an abnormality related to a historical medical procedure. The training sample may be a positive training sample if the training sample includes anomalies related to historical medical procedures. The training sample may be a negative training sample if the training sample does not include anomalies related to historical medical procedures. The training labels of the sample may be at the image level or the video level. In other words, the training labels (abnormal or normal) of the training samples may be labeled or known, while the training labels (abnormal or normal) of one or more subjects present in the training samples may be unknown or unlabeled. The labels of the training samples may include positive labels or negative labels. If the training sample is a negative training sample with no sample anomalies, the training sample may be labeled with a negative label. If the training sample is a positive training sample with sample anomalies, the training sample may be positively labeled. The training samples may be labeled with a binary label (e.g., 0 or 1, positive or negative, etc.). For example, a negative training sample may be labeled with a negative label (e.g.,

"0") and a positive training sample may be labeled with a positive label (e.g., "1").

In 620, the processing device 130 (e.g., the extraction module 460) may determine at least two regions in each of at least two training samples using the initial machine learning model. In some embodiments, the initial machine learning model may comprise a machine learning model that has not been trained using any training data. For example, the initial machine learning model may include structural parameters such as the number of layers or total number of layers, the number of nodes or total number of nodes per layer, etc., as well as learning parameters such as connection weights, bias vectors, etc. The structural parameters of the initial machine learning model, which are not updated during the training of the initial machine learning model, may be set by an operator of the processing device 130. The learning parameters may be unknown because the initial machine learning model has not been trained using any training data and is updated during training of the initial machine learning model using the at least two training samples obtained in 610. In some embodiments, the initial machine learning model may comprise a pre-trained machine learning model trained using a training set. The training data in the training set may be partially or completely different from the at least two training samples obtained in 610. For example, the pre-trained machine learning model may be provided by a system of a vendor that provides and/or maintains the pre-trained machine learning model. The structural parameters of the initial machine learning model may be set by the vendor that provides and/or maintains such a pre-trained machine learning model. The learning parameters of the pre-trained machine learning model may be predetermined using the training set, and may be further updated based on the at least two training samples obtained in 610.

In some embodiments, the trained machine learning model is constructed based on a neural network model. In some embodiments, the initial machine learning model may be constructed based on a neural network model, a deep learning model, a regression model, or the like. Example neural network models may include Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs) (e.g., region-based convolutional networks (R-CNNs), Fast region-based convolutional networks (Fast R-CNNs), region-based Fast convolutional networks (Fast R-CNNs), etc.), spatial pyramid pool networks (SPP-nets), etc., or any combination thereof. Exemplary deep learning models may include one or more Deep Neural Networks (DNNs), one or more Deep Boltzmann Machines (DBMs), one or more stacked autoencoders, one or more Deep Stacked Networks (DSNs), and the like. Exemplary regression models may include a support vector machine, logistic regression model, and the like. In some embodiments, the initial machine learning model may comprise a multi-layered structure. For example, the initial machine learning model may include an input layer, an output layer, and one or more hidden layers between the input layer and the output layer.

In some embodiments, the at least two regions may be determined based on a sliding window algorithm, a region suggestion algorithm, an image segmentation algorithm, etc., using an initial machine learning model. For example, using a sliding window algorithm, the initial machine learning model may divide the image data into at least two regions by sliding a fixed-size window. For another example, a region suggestion algorithm may be used, with the initial machine learning model configured to designate each pixel in the input training sample as a group. The initial machine learning model may be configured to determine a texture feature for each group and determine a similarity between the two groups. The initial machine learning model may combine multiple groups, each group including a similarity satisfying a condition, such as exceeding a threshold. In some embodiments, the initial machine learning model may use an image segmentation algorithm (e.g., an edge detection algorithm) to extract a preliminary frame or contour of one or more objects to be identified in the training sample. The processing device 130 may partition the area to cover a preliminary frame or outline of each of the one or more objects. In some embodiments, the regions may be determined based on one or more feature points in the image data (e.g., corner points, boundary locations, or edge points of the object). As used herein, a feature point may refer to a point at which a gray value of an image changes sharply or a curvature of an edge is large (i.e., an intersection of two edges). Specifically, after one or more specific feature points are identified in the image data, a region of a predetermined shape or size may be determined, the specific feature points being located within the region. In some embodiments, the shape of each of the at least two regions may include a rectangle, a circle, an ellipse, a polygon, an irregular shape, and so forth. At least one parameter of the size, shape, number, or the like of the at least two regions may be a default value determined by the abnormality detection system 100 or a default value preset by a user or an operator via the terminal 140. In some embodiments, each of the one or more parameters may be assigned a value, and the one or more parameters may be determined based on the assigned values. For example, the size and shape of each of the at least two regions may be allocated, and the number of the at least two regions may be determined based on the size and shape of each of the at least two regions.

In 630, the processing device 130 (e.g., the extraction module 460) may extract image features from each of the at least two regions.

The image features may refer to representations of particular structures in the training sample region, such as points, edges, objects, and the like. The extracted image features may be binary, numeric, categorical, ordinal, binomial, interval, text-based, or a combination thereof. In some embodiments, the image features may include low-level features (e.g., edge features, texture features), high-level features (e.g., semantic features), or complex features (e.g., deep-level features). The initial machine learning model may process the input training samples through multi-layer feature extraction (e.g., convolutional layers) to extract image features.

In 640, the processing device 130 (e.g., the training module 470) may train the initial machine learning model using the extracted image features and the at least two annotated training samples.

In some embodiments, an initial machine learning model is trained using a training algorithm to obtain a trained machine learning model based on image features extracted for each of at least two training samples. Exemplary training algorithms may include a gradient descent algorithm, a newton algorithm, a quasi-newton algorithm, a Levenberg-Marquardt algorithm, a conjugate gradient algorithm, the like, or combinations thereof. In some embodiments, the initial machine learning model may be trained by performing at least two iterations. Parameters of the initial machine learning model may be initialized prior to at least two iterations. For example, the connection weights of the nodes and/or the bias vectors of the nodes of the initial machine learning model may be initialized by assigning random values in the range of-1 to 1. As another example, the weights of all connections of the initial machine learning model may be assigned a same value, ranging from-1 to 1, such as 0. Still by way of example, the bias vectors for the nodes in the initial machine learning model may be initialized by assigning random values ranging from 0 to 1.

In some embodiments, the parameters of the initial machine learning model may be initialized based on a gaussian random algorithm, an Xavier algorithm, or the like, and then at least two iterations may be performed to update the parameters of the initial machine learning model until a termination condition is satisfied. The termination condition may indicate whether the initial machine learning model is sufficiently trained. For example, the termination condition may be satisfied if the value of the cost function or error function associated with the initial machine learning model is minimal or less than a threshold (e.g., a constant). For another example, the termination condition may be satisfied if the values of the cost function or the error function converge. Convergence may be considered if a change in the value of the cost function or the error function in two or more successive iterations is less than a threshold value (e.g., a constant). Still by way of example, the termination condition may be satisfied when a specified number of times or iterations are performed during the training process. For each of the at least two iterations, the image features and the corresponding labels for each of the at least two regions of the training sample may be input into an initial machine learning model. The image features may be processed by one or more layers of an initial machine learning model to generate a prediction result for each of at least two regions in the input training sample. The prediction result of the particular region may indicate whether the particular region includes a sample anomaly. In other words, the prediction result of the specific region may indicate whether the specific region includes an object of interest that causes an abnormality in the training sample. In some embodiments, the predicted outcome for the particular region may include a positive outcome indicating that the particular region includes an anomaly and may also include a negative outcome indicating that the particular region does not have an anomaly. Based on the image features extracted from the particular region, the initial machine learning model may determine a prediction result for the particular region by determining an anomaly score for the particular region. For example, if the anomaly score for a particular region exceeds an anomaly threshold, the initial machine learning model may determine that the prediction result for the particular region is positive. For example, a positive result may be represented by a value of "1". If the anomaly score for the particular region is less than the anomaly threshold, the initial machine learning model may determine that the prediction result for the particular region is negative. For example, a negative result may be represented by a value of "0". In some embodiments, the prediction result of the particular region may include an anomaly score for the particular region. Based on the cost function or error function of the initial machine learning model, the predicted outcome for each of the at least two regions in the input training sample may be compared to the expected outcome (i.e., label) associated with the training sample. The cost function or error function of the initial machine learning model may be configured to evaluate the total difference (also referred to as global error) between the test values (e.g., the predicted results for each region) and the expected values (e.g., the labels of the training samples) of the initial machine learning. The total difference (also referred to as global error) between the test value (e.g., the predicted result for each region) and the expected value (e.g., the label of the training sample) of the initial machine learning model may be equal to the sum of the plurality of differences. Each difference of the plurality of differences refers to a difference between one of the predictors for the at least two regions and the label of the input training sample. If the value of the cost function or the error function exceeds a threshold in the current iteration, the parameters of the initial machine learning model may be adjusted and/or updated to reduce the value of the cost function or the error function to a value less than the threshold. Thus, in the next iteration, the image features of each region in another training sample may be input into the initial machine learning model to train the initial machine learning model as described above until a termination condition is satisfied.

In some embodiments, the termination condition may be that the value of the cost function or the error function in the current iteration is less than a threshold. In some embodiments, the termination condition may include that a maximum number of iterations have been performed, the approximation error is less than a certain threshold, a difference between a value of the cost function or error function obtained for a previous iteration and a value of the cost function or error function obtained for a current iteration (or between values of the cost function or error function within a count of a number of consecutive iterations) is less than a certain threshold, and a difference between the approximation error between the previous iteration and the current iteration (or among approximation errors within a count of a number of consecutive iterations) is less than a certain threshold. In response to determining that the termination condition is not satisfied, the processing device 130 may adjust parameters of the initial machine learning model and perform iterations. For example, the processing device 130 may update the values of the parameters by executing a back propagation machine learning training algorithm (e.g., a random gradient descent back propagation training algorithm). In response to determining that the termination condition is satisfied, the iterative process may terminate and the trained machine learning model may be stored and/or output. In some embodiments, after learning is complete, the validation set may be processed to validate the learning results.

In some embodiments, the trained machine learning model may include two parts: an abnormality detection component that detects whether an abnormality exists in the medical procedure and a classification component that determines and/or outputs a location of one or more objects of interest that caused the abnormality. The two components may be connected to each other. In some embodiments, the output of the anomaly detection component can be an input to a classification component. The classification component may determine one or more objects of interest that caused the anomaly to be detected by the anomaly detection component. In some embodiments, two components may share the same multiple layers to extract image features from the input image data. The extracted image features may be input into each of the two parts separately. Each of the two portions may generate an output based on the extracted image features.

It should be noted that the foregoing is provided for illustrative purposes only and is not intended to limit the scope of the present application. Various changes and modifications will occur to those skilled in the art based on the description herein. For example, the operation 620 of determining at least two regions and the operation 630 of extracting image features may be integrated into the operation 640 of training the initial to determine at least two regions and image features for training the initial machine model. For another example, an update process of the trained machine learning model may be added to update the trained machine learning model periodically or at different times. However, such changes and modifications do not depart from the scope of the present application.

Examples

The following examples are for illustrative purposes only and are not intended to limit the scope of the present application.

Example 1: exemplary test results for surgical procedure

FIG. 7 is a schematic illustration of test results shown in connection with an exemplary medical procedure according to some embodiments of the present application. As shown in FIG. 7, the wheelchair is detected by the trained machine learning model and marked with a bounding box 710 in the image representing the surgical procedure. The wheelchair is located on the trajectory of the medical device that is moved during the procedure, the wheelchair being identified as the cause of the abnormality during the procedure.

Example 2: exemplary detection results of imaging scans

FIG. 8 is a schematic illustration of a test result for another exemplary medical procedure, shown in accordance with some embodiments of the present application. As shown in FIG. 8, the wheelchair is detected by the trained machine learning model and marked with a bounding box 810 in the image associated with the imaging scan. The wheelchair may cause damage or abnormalities to the medical equipment (e.g., MR scanner) used to perform the imaging scan, which is determined to be the cause of the imaging scan abnormalities.

Example 3: exemplary test results for surgical procedure

Fig. 9 is a schematic illustration of anomaly detection for an exemplary surgical procedure, shown in accordance with some embodiments of the present application. As shown in fig. 9, images 1 and 2 are collected by a camera during a surgical procedure. In some embodiments, image 1 and image 2 may be two frames in a video collected by a camera. Each of the images 1 and 2 has a time stamp indicating a time point at which each of the images 1 and 2 was collected. The timestamp shows that image 2 was obtained later than image 1. According to process 500 described elsewhere in this application, sponges for the surgical procedure are detected in image 1 using a trained machine learning model and labeled using bounding box A. If the sponge is inadvertently left in the patient after surgery, it may cause injury or abnormality to the patient undergoing surgery. The sponges detected in image 1 are continuously tracked during the procedure. For example, the sponge used during the procedure is detected in image 2 and marked using bounding box B. Images (e.g., image 1 and image 2) of the marked sponge generated during the procedure can be displayed to a surgeon (e.g., a terminal device) on the device so that the surgeon can know the location of the sponge at different times during the procedure.

Having thus described the basic concepts, it will be apparent to those of ordinary skill in the art having read this application that the foregoing disclosure is to be construed as illustrative only and is not limiting of the application. Various modifications, improvements and adaptations to the present application may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the present application and thus fall within the spirit and scope of the exemplary embodiments of the present application.

Claims

1. A method for anomaly detection in a medical procedure, the method comprising:

obtaining image data collected by one or more vision sensors through monitoring a medical procedure;

determining, based on the image data, a detection result of the medical procedure using a trained machine learning model for anomaly detection, the detection result including whether the medical procedure is anomalous; and

providing feedback relating to the anomaly in response to the detection of the presence of the anomaly.

2. The method of claim 1, wherein the image data comprises an image of the one or more objects of interest causing the abnormality.

3. The method of claim 2, wherein the detection of the medical procedure includes location information of at least one of the one or more objects of interest.

4. The method of claim 3, wherein the detection of the medical procedure is determined using the trained machine learning model based on the image data, the method further comprising:

in response to a detection result that the medical procedure is abnormal, determining, based on the image data, location information of at least one object of interest of the one or more objects of interest using the trained machine learning model.

5. The method of claim 4, wherein to determine location information of at least one object of interest of the at least one or more objects of interest, the method further comprises:

extracting at least two regions of an image in the image data;

determining a score for each of the at least two regions, the score for each of the at least two regions representing a probability that the each of the at least two regions includes at least one of the one or more objects of interest; and

determining location information for at least one object of interest of the one or more objects of interest in the image data based on the score for each of the at least two regions.

6. A method according to claim 2, wherein to provide feedback relating to the anomaly, the method comprises:

generating a notification informing that the exception exists; alternatively, the first and second electrodes may be,

displaying at least a portion of image data of the image data on a device, and highlighting at least one object of interest of the one or more objects of interest.

7. The method of claim 1, wherein the trained machine learning model for anomaly detection is constructed based on a weakly supervised learning model; the trained machine learning model is obtained by operations comprising:

obtaining at least two training samples, each training sample including a label indicating whether the training sample includes a sample anomaly;

determining at least two regions in each of the at least two training samples, at least a portion of the at least two regions comprising an object;

extracting image features from each of the at least two regions; and

training an initial machine learning model using the extracted image features and the labels of the at least two training samples.

8. A system for anomaly detection in a medical procedure, comprising an acquisition module, a determination module, and a feedback module;

the acquisition module is for obtaining image data collected by one or more vision sensors through monitoring a medical procedure;

the determining module is configured to determine a detection result of the medical procedure based on the image data using a trained machine learning model for anomaly detection, the detection result including whether the medical procedure is anomalous;

the feedback module is configured to provide feedback relating to the anomaly in response to the detection of the presence of the anomaly.

9. An apparatus for anomaly detection in a medical procedure, the apparatus comprising a processor and a memory; the memory for storing instructions, wherein the instructions, when executed by the processor, cause the apparatus to implement the method for abnormality detection in a medical procedure of any one of claims 1-7.

10. A computer readable storage medium storing computer instructions which, when read by a computer, cause the computer to perform the method for abnormality detection in a medical procedure of any one of claims 1 to 7.