TW202242904A - Pressure injury identification and analysis system and method - Google Patents

Abstract

The present invention relates to a pressure injury identification and analysis method, comprising steps of using a mobile device to acquire an actual image pertaining to a pressure injury and uploading the actual image to a medical server; implementing a pressure injury identification and analysis deep learning model on the medical server in a remote end, so as to detect and acquire an injury image from the actual image and identify and analyses an injury information pertaining to the pressure injury shown by the injury image; and returning the injury information to the mobile device and providing the injury information for a user to read through the mobile device.

Description

Pressure ulcer identification and analysis system and method

The present invention relates to a pressure sore identification and analysis system and method, in particular to a system and method for detecting and analyzing pressure sores using image recognition technology and grading pressure sores.

Pressure injury, also known as bedsore, is mainly caused by a specific part of the patient's body being oppressed by body weight for a long time, resulting in obstruction of blood flow, resulting in tissue ulceration or necrosis. Pressure sores are common in the elderly Elderly, poor mobility, long-term bedridden patients, poor blood circulation, prone to bone protrusion, friction, long-term pressure, less fat and other parts of the patient, such as the head, shoulders, elbows, tail vertebrae, heels, ears, buttocks, inner and outer thighs and other parts that are prone to compression. According to statistics, most residents of residential long-term care institutions are at high risk of pressure injury, so the incidence of pressure injury is listed as one of the important indicators of quality of care.

After a pressure sore is formed, depending on the location, it will have different conditions such as size, shape, color, depth, taste, pus, exudate, pain, infection, and necrosis, and usually worsens The speed is very fast, and it is labor-intensive to deal with. Generally speaking, nurses’ care measures for pressure injury wounds include assessing wound grade, color and size, performing wound care regularly, changing dressings, and recording wound changes, etc. , not only to help the patient turn over every 1-2 hours, but also pay attention to the patient's posture and positioning, and thoroughly clean the patient before each medication. Wash the wound, but the wound is often delayed for several weeks and still difficult to heal. Other complications are likely to occur during the process, such as cellulitis, infection of bones and joints, sepsis, etc., and severe cases even require amputation.

Pressure sores are very common among patients in medical institutions and long-term care institutions, and they also cause a great burden on the medical care system. When a patient develops a pressure sore, the frontline nursing staff needs to intensively inspect and evaluate the wound. Care records should also be kept for pressure ulcer injuries. In order to provide nursing staff with a benchmark for assessment and care, and to integrate the definition and assessment of pressure ulcer injuries, the latest version of the staging system for pressure ulcers was proposed by the National Pressure Ulcer Advisory Committee (NPUAP) in 2016. According to different states, pressure ulcers are divided into four grades and non-gradable two grades, so as to facilitate the reference and use of nurses.

Although there is a grading system as a guideline, in conventional techniques, when nursing staff record a patient's pressure sore injury, they usually describe it in words and record it in the nursing record. However, due to the high complexity of the pressure sore wound status , Assessing and recording wounds requires professional knowledge, which is related to the experience and cognition of nurses. Clinically, it is often found that due to different cognitive standards of nurses, problems such as inconsistencies between assessment and records are found. Factors such as asynchronous cognition of wounds by personnel and caregivers make this kind of artificial description and evaluation of pressure sore injury characteristics still too general, often with many errors and unreliable, which not only increases the cost of care, but also increases the cost of medical care. system burden.

For this reason, in view of the shortcomings in the conventional technology, the inventor, after careful trial and research, and a persistent spirit, finally conceived the "pressure sore injury identification and analysis system and method", which can overcome the above shortcomings, The following is a brief description of the present invention.

The present invention proposes a deep learning model of pressure sore injury identification and analysis based on image processing technology, which can be integrated into the smart nursing platform as a program component module, and brought into the front-end interface program through the function keys, and the user can use it through operation Press the corresponding function keys in the operator interface to start the model program component module and execute it.

The deep learning model for pressure sore injury identification and analysis of the present invention includes object detection program components and object classification program components established by applying deep learning. The object classification program component is preferably a multi-category classifier, which can be performed according to predefined rules The training image set is composed of pre-processed images of pressure sores. The model learns how to detect pressure sores from real images of pressure sores by reading, detecting, identifying and analyzing a large number of images of pressure sores. According to the image of pressure sore injury, identify the wound characteristics of pressure sore injury according to the predefined rules, and give appropriate classification. The predefined rules are preferably based on the 2016 version of the NPUAP pressure injury grading system but mixed with the user's adaptation A set of pre-defined rules that can be corrected, the model of the present invention can feed back the corresponding pressure sore injury grade and healing prediction to the user, and assist the user to accurately track the pressure sore injury status.

Accordingly, the present invention proposes a pressure sore injury identification and analysis method, which includes using a mobile device to take a real image of a pressure sore injury and uploading it to a remote medical server; implementing deep learning object detection on the medical server The program component detects and extracts the pressure sore image of the pressure sore from the live image; implements the deep learning object classification program component on the medical server to identify and analyze the information about the pressure sore image displayed Wound information of the pressure sore; and returning the wound information and providing it to the user to read through the mobile device.

The present invention further proposes a pressure sore identification and analysis method, which includes using a mobile device to obtain real-shot images of pressure sores and uploading them to a medical server; implementing deep learning for pressure sore identification and analysis on the remote medical server model to detect and extract from the live image The pressure sore image of the pressure sore, and identifying and analyzing the wound information about the pressure sore displayed in the pressure sore image; and returning the wound information and providing it to the user for reading through the mobile device.

The present invention further proposes a pressure sore injury identification and analysis system, which includes a medical server, which includes a deep learning model for pressure sore injury identification and analysis; and a mobile device, which communicates with the medical server and provides caregivers to operate Obtain a real-shot image of the pressure sore injury of the subject, and upload the real-shot image to the deep learning model for pressure sore identification and analysis, wherein the deep learning model for pressure sore identification and analysis is executed from the real shot Detect and capture the pressure sore image of the pressure sore in the image, identify and analyze the wound information about the pressure sore displayed in the image of the pressure sore, return the wound information and provide it through the mobile device Read to the caregiver.

The above summary of the invention is intended to provide a simplified summary of the disclosure to enable readers to have a basic understanding of the disclosure. This summary of the invention is not intended to disclose a complete description of the invention, and is not intended to point out important/key elements or components of the embodiments of the invention. define the scope of the invention.

100: Nursing evaluation and diagnosis system of the present invention

111-116: Mobile devices

120: nursing server

131: Networked blood pressure monitor

132: Network forehead temperature gun

133: Connected Heartbeat Meter

134:Connected oximeter

141: Exposure subject

151: Caregiver

152-154: Caregivers

161-162: Family

200: Smart Nursing Cart

210:Mobile device connection point

220: Convenience

CE: clinical end

SE: nursing station

FE: family end

BB: border

300: user interface

301: Pressure injury module button

303: Add image window

305: Add button

307: Image Analysis Window of Pressure Injury

309: Wound description record window

311: confirm button

313:Pressure injury record list window

315: Preliminary Analysis Window of Pressure Injury Images

401-425: System operation steps

500: Identification and analysis method of pressure sore injury of the present invention

501-507: Implementation steps

Figure 1 is a schematic diagram showing the system architecture of the pressure sore injury identification and analysis system of the present invention;

Figure 2 and Figure 3 reveal the real-shot images of pressure sores after marking in the frame in the present invention;

Fig. 4 is a schematic diagram showing the format of the record auxiliary card of the present invention;

Fig. 5 is a schematic diagram showing the user interface of the pressure injury module of the present invention;

Fig. 6 is a schematic diagram showing the window of the pressure injury analysis result of the present invention;

Fig. 7 is a schematic diagram showing the image analysis window of the pressure injury wound of the present invention;

Figure 8 is a schematic diagram showing the results of the wound image analysis of the present invention;

Figure 9 is a schematic diagram showing the wound description record window of the present invention;

Fig. 10 is a schematic diagram showing the pressure injury record list window of the present invention;

Fig. 11 is a flowchart showing the operation steps of the pressure sore injury identification and analysis system of the present invention; and

Fig. 12 is a flowchart showing the implementation steps of the pressure sore identification and analysis method of the present invention.

The present invention can be fully understood by the following examples, so that those skilled in the art can complete it, but the implementation of the present invention can not be limited by the following examples of implementation; the drawings of the present invention are not limited No limitation on size, dimension and scale is included, and the size, dimension and scale of the present invention are not limited by the drawings of the present invention during the actual implementation.

The word "preferred" in this article is non-exclusive and should be understood as "preferably but not limited to". order, the scope of the present invention should be determined only by the appended claims and their equivalents, not by the examples illustrated in the implementation; when the term "comprising" and its variations appear in the specification and claims, it is An open-ended term without a restrictive meaning that does not exclude other features or steps.

Figure 1 is a schematic diagram showing the system architecture of the pressure sore injury identification and analysis system of the present invention; the pressure sore injury identification and analysis system 100 of the present invention is composed of a series of associated hardware dispersed in the clinical end CE, nursing station SE, and family end FE A cloud system consisting of a set of distributed smart nursing platform software programs executed on these associated hardware devices, and a set of deep learning models for pressure sore identification and analysis based on image processing technology running on the smart nursing platform ; Clinical end CE is Refers to the surrounding area centered on the care receiver 141; in some embodiments, the combination of the clinical end CE and the nursing station SE is regarded as the institutional end where the medical or nursing institution is located.

On the clinical end CE, the nursing station SE and the family end FE, there are different numbers of mobile devices 111-116 distributed respectively, such as but not limited to: smart phones or tablets, etc., and each mobile device 111-116 is installed with a Support the front-end program of the smart nursing platform, and establish a communication link (communication link) with the medical server 120 through the built-in radio frequency communication module to generate a communication link. The radio frequency communication module is preferably such as but not limited to : Commonly used Wi-Fi, Bluetooth or Bluetooth Low Energy (BLE) modules, Sub-1G modules, 4G or 5G mobile communication modules, etc., between mobile devices 111-116 and medical server 120 The communication link between them is preferably composed of a combination of multiple wired or wireless communication links.

The back-end management program of the smart nursing platform is installed on the medical server 120. Through the connection of the communication link, the front-end programs of the smart nursing platform of the mobile devices 111-116 can access the back-end management program of the smart nursing platform of the medical server 120, and send The back-end management program of the smart nursing platform uploads or downloads data, or accepts control instructions from the back-end management program of the smart nursing platform; the pressure sore injury identification and analysis system 100 of the present invention is based on software as a service (SaaS) and platform as a service (PaaS ) based cloud technology.

Multiple Internet of Things (IoT based) nursing devices with network connection capabilities, such as but not limited to: connected blood pressure monitor 131, connected forehead temperature gun 132, connected heart rate meter 133 and connected oximeter 134, etc., provide Used to measure the basic vital signs of the subject 141, these IoT nursing devices preferably have a built-in wireless communication module that can form a wireless local area network (WLAN) or the Internet of Things (IoT), such as but not limited to: commonly used Wi-Fi, Bluetooth or Bluetooth low-power modules or Sub-1G modules, etc., among which Sub-1G modules are preferably various RF communication modules using ISM frequency bands, such as but not limited to 868MHz module, 916MHz module, 926MHz module, NB-IOT module, Zigbee module, Xbee module, Z-Wave module or LoRa module, etc.

Through the built-in wireless communication module, these IoT nursing devices establish a communication link with a mobile device 111 located nearby and at the clinical CE, and transmit the measured vital sign readings to the mobile device in real time 111 The front-end program of the smart nursing platform, and then upload the back-end management program of the medical server 120 smart nursing platform through the front-end program of the smart nursing platform, and record it in the cloud database specified by the back-end management program of the smart nursing platform.

A smart nursing work vehicle 200 can provide a mobile device connection point 210 to fix the mobile device 111 at an appropriate height from the ground, and provide convenient parts 220 to centrally store and place multiple pieces of the aforementioned IoT nursing equipment, so that caregivers ( care provider) 151, in the process of moving from the nursing station SE to the clinical CE, or between the beds of multiple patients, use the smart nursing work vehicle 200 to easily move and manage multiple pieces of IoT nursing equipment The height from the ground of the mobile device connection point 210 and the convenience piece 220 can be adjusted according to the caregivers 151 of different heights, so that the caregivers 151 can operate and use the mobile device 111 and multiple pieces of IoT care equipment.

The intelligent nursing work vehicle 200 can preferably be used as a mobile bedside assistant (bedside assistant), a mobile clinical assistant (clinical assistant) or used to deploy a mobile nursing station, etc. Its detailed structure and related technologies involved have been disclosed in the applicant of this case In the Republic of China Invention Patent No. I687209, it is protected by the invention patent and enjoys the patent right.

Through the use of the smart nursing cart 200, the caregiver 151 can choose to hold the mobile device 111 or fix the mobile device 111 on the mobile device connection point 210 of the smart nursing cart 200. When the mobile device 111 establishes communication with the IoT nursing equipment After the connection, the vital sign readings measured by the IoT nursing equipment from the subject 141 can be passed through the front end of the mobile device 111 smart nursing platform The program is uploaded to the medical server 120 smart nursing platform back-end management program in real time, and then the smart nursing platform back-end management program pushes the vital sign readings back to the clinical end CE mobile device (the first mobile device) 111 on the smart The front-end program of the nursing platform, or pushed to the nursing station SE mobile device (second mobile device) 112-114, the front-end program of the smart nursing platform is simultaneously displayed to other caregivers 152-154, or further selectively pushed to the family terminal The front-end program of the smart nursing platform on the FE mobile device 115-116 is displayed to multiple family members 161-162 simultaneously.

A caregiver preferably refers to a person who provides care services, such as but not limited to: nurses, nurses, nursing staff, medical staff, housekeepers, caregivers, etc. A care recipient preferably refers to a person receiving care services, such as but not limited to : Residents, patients, elderly or disabled persons, etc.

It is common for subjects 141 who are elderly, inactive, and bedridden to have pressure sores on their bodies. For periodic nursing records, generally follow the latest version of the staging system for pressure ulcers proposed by the National Pressure Ulcer Advisory Committee (NPUAP) in 2016 as a guideline for writing records, but pressure sore wounds Different subjects will show completely different states. It is not easy to describe the pressure sore wound clearly and accurately even with the NPUAP guidelines. Recording requires professional knowledge, and it is related to the experience and cognition of nurses. In clinic, problems such as inconsistency between assessment and record content are often found due to different cognitive standards of nurses. In addition, the follow-up time for pressure sore injuries is long. Family members, exposed patients141, The unsynchronized cognition of wounds by caregivers151 and doctors makes the conventional manual description and evaluation of pressure sore injury characteristics still very general, with many errors and unreliable, which urgently needs to be improved.

The present invention proposes a deep identification and analysis of pressure sore injury based on image processing technology. The degree learning model can be integrated into the medical server 120 smart nursing platform as a program component module, and brought into the front-end interface program through the function keys, and the user activates the model program by operating the corresponding function keys in the user interface Component modules are executed. The pressure sore injury identification and analysis deep learning model of the present invention includes object detection program components and object classification program components established by applying deep learning. The object classification program component is preferably a multi-category classifier, which can be passed according to predefined rules The training image set is composed of pre-processed pressure sore injury images. The model learns how to detect pressure sores from real pressure sore injury images by reading, detecting, identifying and analyzing a large number of pressure sore injury images According to the image of pressure sore injury, identify the wound characteristics of pressure sore injury according to the predefined rules, and give an appropriate classification. The predefined rules are preferably based on the 2016 version of the NPUAP pressure injury grading system but mixed with the user's A set of predefined rules for adaptive modification, the model of the present invention can feed back the corresponding pressure sore injury level and healing prediction to the user, and assist the user to accurately track the pressure sore injury status.

The deep learning object detection program element and the deep learning object classification program element included in the model of the present invention are preferably selected from a CNN algorithm, an R-CNN algorithm, a Fast R-CNN algorithm, and a Faster R-CNN algorithm method, a Mask R-CNN algorithm, a multi-column convolutional neural network (MCNN) algorithm, a YOLO algorithm, an SSD algorithm, a recurrent neural network (RNN) algorithm, and a convolution Recurrent neural network (CRNN) algorithm, a bidirectional neural network (BRNN) algorithm, a deep recurrent neural network (DRNN) algorithm, a fully convolutional neural network (FCN) algorithm, a type of neural network (ANN) algorithm, a multi-layer perception (MLP) algorithm, a deep residual network (DRN) algorithm, a deep convolutional neural (DCNN) algorithm, an AlexNet algorithm, a VGG algorithm, a One of the GoogleLeNet algorithm, an Inception-ResNet algorithm and a combination thereof.

In order to establish a deep learning model for identification and analysis of pressure sore injuries with artificial intelligence, it is necessary to obtain a large amount of clear and positive real-time image data containing pressure sore injuries in advance, and to filter qualified images according to the following predefined rules (the first type of image) , and carry out the bounding box labeling step for the wound position of each qualified image, and establish the training image set and verification image set required for training and verifying the deep learning object detection program components:

(1) Remove images of wounds reproduced from the screens of other devices, images of wounds covered by gauze, and images of wounds whose body is covered.

(2) When there are several wounds scattered at the same time, if the skin of different colors around the wound is connected together, mark the entire range.

(3) If it is a completely independent wound, mark it separately.

(4) Mark the image where the body of the wound is not covered but the surrounding pink skin is covered.

(5) Pressure sores are located in the head area, and if the location and scope cannot be judged at all, such as being blocked by hair, it is directly marked as a first-level pressure sore, because the invisible head pressure sore usually belongs to the first-level pressure sore.

(6) The range of the border should exclude the melanin deposits around the wound.

For the scope definition of pressure sore wounds, the following predefined rules are established:

(1) Level 1: The color of the skin is different from other parts, such as redness and swelling.

(2) Levels 2-4: The skin area of the wound body and its surrounding areas with a different color from normal skin.

Then the images of pressure sores were cropped according to the marked wound range. Figure 2 and Figure 3 are real shots of pressure sores after marking in the frame in the present invention Image: The real-shot images of pressure sores in Figures 2 and 3 have been marked with a border BB according to the aforementioned predefined rules, so as to mark the object range of pressure sores from the real-shot images. After the deep learning object detection program component is trained, it can find the exact location of the pressure sore wound in any live image and mark the wound with a square border.

From the images that have been marked with borders, further select the second type of images that can clearly and clearly distinguish the severity level of the wound, the shape of the wound, and the color of the wound, and according to the following predefined rules, and for each second type of image The multi-category classification step of the wound, the establishment of the training and verification training image set and verification image set required for the deep learning object classification program component:

(1) Select a reference standard for injury grading, such as but not limited to: the 2016 edition of the NPUAP pressure injury grading system, the 2009 edition of the NPUAP pressure injury grading system, or other appropriate grading systems. In this example, the 2016 version of the NPUAP pressure injury grading system was selected, and the wounds were divided into grades 1 to 4.

(2) Color classification is to observe the deepest tissue exposed inside the wound and classify it according to its color. For example, if the deepest bone is exposed, it is defined as white, and if the deepest muscle is exposed, it is defined as red. However, if there are special circumstances, such as yellow carrion, it is defined as yellow.

(3) The shape is divided into round, oval and irregular. Among them, if the whole area of the wound appears to be a perfect circle in the image, it is defined as a circle.

(4) If the entire area of the wound is pebble-shaped in the image and there is no sharp angle at the edge, it is defined as an oval shape; other oval shapes that are not classified as round are irregular.

(5) According to the rules in points (2)-(4), the nurse will give appropriate grades to the grade, color, and shape of the pressure sores in each photo.

Split the pre-processed image, preferably at a ratio of 4:1, for training Image set and verification image set, the training image set is input into the deep learning object classification program component, and the training deep learning object classification program component automatically detects the pressure sore injury image contained in the real shot image, and automatically gives a professional wound classification. After the deep learning object classification program component is trained, it can automatically identify pressure sores in live images, and automatically classify at least the wound color, wound shape, wound grade, etc. according to predefined rules. The above procedures are preferably performed manually, especially by senior nurses, so as to endow the model with artificial intelligence.

Figure 4 is a schematic diagram showing the form of the recording aid card of the present invention; the method for identifying and analyzing pressure sore injuries of the present invention proposes to use the recording aid card as disclosed in Figure 4, and the recording aid card is printed with such as but not limited to 6×4 The color block matrix arranged in an array, the scale bar is printed under the color block matrix, the three primary color light mode (RGB) color mode values and the printing four color separation mode (CMYK) color mode values, color block sizes, and scale standards of the 24 color blocks etc., have been preset in the system, so the record auxiliary card can be provided to the system as a reference and basis for correcting the color and size of the pressure sore wound.

When users take pictures of pressure sore wounds, they can choose to use the recording auxiliary card. When the user chooses to use the auxiliary card, they only need to put the printed color block and scale on the front, and place the scale under the card Place the card next to the wound and take a real photo sample with the wound. The card should be parallel to the wound and on the same plane as possible. The card and the wound should be placed in the center of the photo as much as possible. The card should be presented as square as possible. In the photo, and all color blocks should be fully presented.

All the mobile devices 111-116 equipped with photography functions and camera lenses can be used to actually photograph the pressure sores on the body of the subject 141 to obtain real images of the pressure sores. The deep learning model for pressure sore identification and analysis of the present invention is integrated in the medical server in the form of a functional module 120 on the smart nursing platform, and displayed in the user interface 300 of the front-end program of the mobile device 111 in the form of function buttons, the caregiver 151 presses the corresponding function button from the user interface 300 to start the model and the included program Component modules are executed.

Figure 5 is a schematic diagram showing the user interface of the pressure injury module of the present invention; in actual operation, the caregiver 151 can use his own mobile device 111 such as a tablet before applying medicine to the subject 141, before and after cleaning the wound device to help the subject 141 take one or more real-time photos of the pressure sore wound. After the photos are taken, the caregiver 151 presses the pressure injury module button 301 from the user interface 300 of the smart nursing platform to enter Execute the pressure sore injury recording module of the deep learning model of pressure sore injury identification and analysis of the present invention, a newly added image window 303 appears in the user interface 300, and the user presses the add button 305 in the newly added image window 303, and the Photo A is uploaded to the system platform, as shown in Figure 5.

Figure 6 is a schematic diagram showing the window of the pressure injury analysis result of the present invention; after the model receives the newly added real image of the pressure sore injury, that is, photo A, it will execute the deep learning object detection program component to perform bounding prediction on the photo box prediction) program, which detects the location of the wound object from the photo and gives it a label, and extracts the close-up. The deep learning object classification program component then analyzes the close-up of the wound, and the analysis result is displayed in the pressure injury as shown in Figure 6 Image preliminary analysis window 315 .

For example, in the process of executing the frame prediction program, the deep learning object detection program component will look for pixels or partial images that meet the conditions in the photo, and identify whether it belongs to the wound area. If found, it will mark the frame and output For the border and its range, if no pixel or partial image that meets the conditions is found in the photo, no output will be made. When the deep learning object detection program component does not output the border range, it means that the photo does not contain a wound.

The preliminary analysis window 315 of the pressure injury image in FIG. 6 allows the caregiver 151 to modify the analysis results. For example, in this embodiment, the analysis results of the pressure sore injury included in photo A by the model are grade 3, pink, egg It is round and has a size of about 7.0×5.1×0.8cm, but if the caregiver 151 thinks that the pressure sore injury should be grade 2, which is more in line with the actual situation, the caregiver 151 can directly click on the pressure injury image preliminary analysis window 315 Displays level 2 buttons to modify the analysis results of the model. If the caregiver 151 corrects the model analysis results at this stage, the model will re-train, self-learn or self-adjust according to the corrected data to improve the accuracy of the model.

Fig. 7 is a schematic diagram showing the image analysis window of the pressure injury wound of the present invention; Fig. 8 is a schematic diagram revealing the result of the wound image analysis of the present invention; when the caregiver 151 confirms that the analysis result is correct, the final platform is revealed through Fig. 7 A pressure injury image analysis window 307 displays the final analysis results of the model, including but not limited to judging wound grade, size, shape, color, wound evaluation, wound image analysis, healing prediction, medication recommendations and other wound information. More wound image analysis results of different wounds are revealed in Figure 8.

Figure 9 is a schematic diagram revealing the wound description record window of the present invention; Figure 10 is a schematic diagram revealing the pressure injury record list window of the present invention; then the platform displays a wound description record window 309 as disclosed in Figure 9, and the caregiver 151 can Further add subjective observation information on the wound, including but not limited to: exudate, exudate volume, surrounding skin condition, surrounding skin temperature, wound infiltration, necrotic tissue, pain performance, dressing change, etc. After the caregiver 151 completes the record , press the confirmation button 311, the platform will display all the information of this and each pressure sore injury treatment in the pressure injury record list window 313 as shown in Figure 10, and assist the caregiver 151 to accurately track the pressure sore injury the nursing process.

Fig. 11 shows the flow of operation steps of the pressure sore injury identification and analysis system of the present invention Process diagram; the operation process of the pressure sore injury identification and analysis system of the present invention is roughly as disclosed in Figure 11. First, the system receives the pressure sore injury image uploaded by the user (step 401), and the system starts the deep learning object detection program component to carry out Wound detection (step 409), and determine whether there are pixels or partial images that meet the conditions in the image to identify the existence of the wound (step 411), if there is no wound in the image, then end the process (step 413), if the image If there is a wound in the object detection program component, the object detection program component starts to perform frame prediction to determine the position and range of the wound (step 415), and calculates the size of the wound (step 423), then crops the close-up of the wound (step 417), and then normalizes according to the requirements ( formalize) or standardize (standardize) to process wound close-up images (step 419), the system starts the deep learning object classification program component for wound identification (step 421) to analyze and obtain wound information, including but not limited to pressure sore injury grade, shape, Color, size, wound assessment, wound image analysis, healing prediction, medication suggestion, etc. (step 425).

After the system receives the image of the pressure sore injury uploaded by the user (step 401), it will simultaneously detect whether there is an image of the auxiliary recording card (step 403), and check whether there is an auxiliary recording card next to the wound (step 405). card then returns to wound detection (step 409), if there is a recording auxiliary card next to the wound, the color and scale indication on the recording auxiliary card are used as calculation reference values for wound length, width and color (step 407), and then the system calculates the wound size (step 423).

Fig. 12 is a flow chart showing the implementation steps of the pressure sore identification and analysis method of the present invention; in summary, the pressure sore identification and analysis method 500 of the present invention preferably includes the following steps: using a mobile device to obtain the pressure of the pressure sore injury Real shot images of sores and uploaded to the medical server (step 501); implement deep learning object detection program components on the remote medical server to detect and extract the pressure sores from the real shot images of pressure sores Image of pressure sore injury (step 503); implement deep learning object classification program component on the medical server to identify and analyze the pressure displayed by the image of pressure sore injury The wound information of the sore (step 505); and the wound information is sent back and provided for the user to read through the mobile device (step 507).

The deep learning model for identification and analysis of pressure sore injury proposed by the present invention analyzes the wound healing status and provides nursing suggestions based on the patient and its wound development data for reference by nurses and assists nurses to track the wound. The pressure sore proposed by the present invention Injury identification and analysis deep learning model, using artificial intelligence (AI) technology to assist nurses in their care work, automatically recommend wound record items, and reduce the burden of care.

In summary, the deep learning model for identification and analysis of pressure sore injuries proposed by the present invention is an AI image analysis tool for pressure sore injuries, combining two technologies of image object detection and image object classification, and supplemented by a reference color card to enhance the accuracy of judgment Accurate, automated and standard-consistent assessment of pressure ulcer imaging. Evaluation items include: wound severity grading, wound size, wound shape, and wound color. The purpose is to help nursing staff in nursing institutions save effort and time spent on wound records, and also give suggested values for wound evaluation items to avoid different nurses. The problem of inconsistency in the recording results. After the nursing staff finishes cleaning the patient’s wound, they place the recording assistant card next to the wound and co-planar with it with one hand, and take a clear recording photo of the wound with a smartphone in the other hand, and then the AI judgment tool will be used. Analysis, and automatically fill in the analysis results in the record page.

The present invention also has the following features: (1) Automatically judge the actual wound range. Wounds often only occupy part of the image, making subsequent analysis difficult. We use object detection AI to cut close-ups of wounds. (2) Capture close-ups of wounds for AI analysis. After the wound image is cut, a multi-output CNN model is used to identify the key points of each record. (3) Provide care advice and assist caregivers to track. Based on the patient and wound development data, analyze the wound healing status and care suggestions for the reference of caregivers, help caregivers in care institutions save effort and time spent on wound records, and give suggested values for wound assessment items In order to avoid the problem of inconsistent results recorded by different nurses.

The above embodiments of the present invention can be arbitrarily combined or replaced with each other, thereby deriving more implementation forms, but none of them depart from the scope of protection intended by the present invention. More embodiments of the present invention are further provided as follows:

Embodiment 1: A pressure sore injury identification and analysis method, which includes: using a mobile device to take a real shot image of a pressure sore injury and uploading it to a remote medical server; implementing deep learning object detection on the medical server The program component detects and extracts the pressure sore image of the pressure sore from the live image; implements the deep learning object classification program component on the medical server to identify and analyze the information about the pressure sore image displayed Wound information of the pressure sore; and returning the wound information and providing it to the user to read through the mobile device.

Embodiment 2: The pressure sore injury identification and analysis method as described in embodiment 1, wherein the deep learning object detection program component is constructed by implementing the following steps: providing a large number of complex modeling images including pressure sore wounds; Implement image pre-processing procedures to select and retain clear, orthographic and clearly identifiable wounds from these modeling images. Marking the position of the wound with a rectangular frame; and inputting the marked first-type images into the deep learning object detection program component to train the deep learning object detection program component to be an image containing a pressure sore wound, marked with a rectangular frame The location of the wound.

Embodiment 3: The pressure sore injury identification and analysis method as described in Embodiment 2, wherein the deep learning object classification program component is constructed by implementing the following steps: implementing an image screening program, and retaining from the qualified images that have been marked A plurality of second-class images that can clearly and clearly distinguish the severity level of the wound, the shape of the wound, and the color of the wound; Classify the images; gather the classified images of the second type and establish a training image set and a verification image set; input the training image set into the deep learning object classification program component to train the deep learning object classification program component to learn according to The predefined rules classify and analyze images; and input the verification image set into the deep learning object classification program component to verify and adjust the deep learning object classification program component.

Embodiment 4: The method for identifying and analyzing pressure sore injuries as described in Embodiment 3, wherein the predefined rules are based on the 2016 version of the pressure injury classification of the US National Pressure Ulcer Advisory Committee and the 2009 version of the pressure injury of the US National Pressure Ulcer Advisory Committee Grade one of them and mix in adaptive rules defined by that user.

Embodiment 5: The method for identifying and analyzing pressure sore injuries as described in Embodiment 1, wherein the wound information includes wound location, wound grade, wound size, wound color, wound shape, wound depth, wound assessment, wound image analysis, medication One of recommendations, estimates of healing, and surrounding skin conditions.

Embodiment 6: The method for identifying and analyzing pressure sore injuries as described in Embodiment 1, further comprising: placing a supplementary record card next to the pressure sore injury; and using the mobile device to photograph the pressure sore injury and the supplementary record card The real shot image is uploaded to the remote medical server.

Embodiment 7: The pressure sore injury identification and analysis method as described in Embodiment 6, wherein the content of the auxiliary recording card includes a matrix of color blocks and a scale arranged in an array.

Embodiment 8: A method for identifying and analyzing pressure sore injuries, which includes: using a mobile device to obtain real-shot images of pressure sore injuries and uploading them to a medical server; implementing pressure sore injury identification and analysis depth on the remote medical server Learning a model to detect and extract a pressure sore image of the pressure sore from the live image, and identify and analyze information about the pressure sore shown in the image of the pressure sore Wound information; and returning the wound information and providing it to the user to read through the mobile device.

Embodiment 9: The pressure sore injury identification and analysis method as described in embodiment 8, wherein the pressure sore injury identification and analysis deep learning model is constructed by implementing the following steps: providing a large number of complex models including pressure sore injuries images; implement image pre-processing procedures to select and retain a plurality of first-class images that are clear, orthographic, and clearly identifiable wounds from these modeling images; The location of the pressure sore wound is marked with a rectangular frame; the first type of images after the annotation are input into the deep learning model for pressure sore identification and analysis, so as to train the deep learning model for pressure sore identification and analysis to include images of pressure sore wounds , mark the location of the wound with a rectangular border; implement an image screening procedure to retain multiple second-class images that can clearly and clearly distinguish the severity level of the wound, the shape of the wound, and the color of the wound from the qualified images that have been marked; according to the predefined rules Classify the second-type images; collect the second-type images after classification and establish a training image set and a verification image set; input the training image set into the pressure sore injury identification and analysis deep learning model to train The pressure sore identification and analysis deep learning model learns to classify and analyze images according to the predefined rules; and input the verification image set into the pressure sore identification and analysis deep learning model to verify and adjust the pressure sore identification and analyze deep learning models.

Embodiment 10: A pressure sore injury identification and analysis system, which includes: a medical server, which includes a deep learning model for pressure sore injury identification and analysis; and a mobile device, which communicates with the medical server and provides caregivers Operation to obtain real-shot images of the pressure sore injuries of the subject, and upload the real-shot images to the pressure sore injury identification and analysis deep learning model, wherein the pressure sore injury identification and analysis deep learning model is executed from the actual Detect and capture the pressure sore image of the pressure sore in the shooting image, and identify and analyze the wound about the pressure sore displayed in the image of the pressure sore Information, the wound information is returned and provided to the caregiver to read through the mobile device.

The various embodiments of the present invention can be combined or replaced arbitrarily with each other, thereby deriving more implementation forms, but none of them depart from the intended protection scope of the present invention, and the definition of the protection scope of the present invention is fully defined by the patent scope of the present invention application The recorder shall prevail.

500: Identification and analysis method of pressure sore injury of the present invention

501-507: Implementation steps

Claims (10)

A pressure sore injury identification and analysis method, comprising: Use a mobile device to take a real-time image of a pressure sore injury and upload it to a remote medical server; implementing a deep learning object detection (object recognition) program component on the medical server to detect and capture a pressure sore image of the pressure sore from the live image; implementing a deep learning object classification program component on the medical server to identify and analyze wound information about the pressure sore injury displayed by the image of the pressure sore injury; and The wound information is returned and provided for a user to read through the mobile device. The pressure sore injury identification and analysis method as described in Claim 1, wherein the deep learning object detection program component is constructed by implementing the following steps: Provide a large number of complex modeling images including pressure sore wounds; Implementing an image pre-processing procedure to select and retain a plurality of first-class images that are clear, orthographic, and clearly identifiable wounds from the modeling images; implementing a bounding box labeling procedure to mark rectangular bounding boxes for pressure sore wound locations included in the Type 1 images; and The annotated images of the first type are input into the deep learning object detection program component, so as to train the deep learning object detection program component to be an image containing a pressure sore wound, and the wound position is marked with a rectangular frame. The pressure sore injury identification and analysis method as described in Claim 2, wherein the deep learning object classification program component is constructed by implementing the following steps: Implement an image screening process to retain multiple second-class images that can clearly and clearly distinguish the severity level of the wound, the shape of the wound, and the color of the wound from the qualified images that have been marked; classifying the images of the second type according to a predefined rule; Aggregating the classified images of the second type and establishing a training image set and a verification image set; inputting the training image set into the deep learning object classifier component to train the deep learning object classifier component to learn to classify and analyze images according to the predefined rules; and The verification image set is input into the deep learning object classifier component to verify and adjust the deep learning object classifier component. The pressure sore injury identification and analysis method as described in claim 3, wherein the predefined rule is based on a 2016 version of the pressure injury classification of the U.S. National Pressure Ulcer Advisory Committee and a 2009 version of the pressure injury classification of the U.S. National Pressure Ulcer Advisory Committee. , mixed with adaptive rules defined by that consumer. The pressure sore injury identification and analysis method as described in Claim 1, wherein the wound information includes a wound location, a wound grade, a wound size, a wound color, a wound shape, a wound depth, a wound assessment, a wound One of image analysis, a medication recommendation, a healing estimate, and a surrounding skin condition. The pressure sore injury identification and analysis method described in Claim 1 also includes: place a recording aid next to the pressure sore; and Use the mobile device to shoot the live image including the pressure sore injury and the record auxiliary card And upload the remote medical server. The pressure sore injury identification and analysis method as described in Claim 6, wherein the content of the record auxiliary card includes a matrix of color blocks arranged in an array and a scale. A pressure sore injury identification and analysis method, comprising: Use a mobile device to obtain a live image of a pressure sore injury and upload it to a medical server; Implement a pressure sore injury identification and analysis deep learning model on the remote medical server to detect and extract a pressure sore injury image from the real shot image, and identify and analyze the pressure sore injury Wound information about one of the pressure sores shown on the sore image; and The wound information is returned and provided for a user to read through the mobile device. The pressure sore injury identification and analysis method as described in Claim 8, wherein the pressure sore injury identification and analysis deep learning model is constructed by implementing the following steps: Provide a large number of complex modeling images including pressure sore wounds; Implementing an image pre-processing procedure to select and retain a plurality of first-class images that are clear, orthographic, and clearly identifiable wounds from the modeling images; implementing a bounding box labeling procedure to mark rectangular bounding boxes for pressure sore wound locations included in the Type 1 images; Inputting the annotated images of the first type into the deep learning model for identification and analysis of pressure sores to train the deep learning model for identification and analysis of pressure sores as images containing wounds of pressure sores, marking the location of the wound with a rectangular border; Implement an image screening process to retain multiple second-class images that can clearly and clearly distinguish the severity level of the wound, the shape of the wound, and the color of the wound from the qualified images that have been marked; classifying the images of the second type according to a predefined rule; Aggregating the classified images of the second type and establishing a training image set and a verification image set; inputting the training image set into the deep learning model for pressure sore identification and analysis, so as to train the deep learning model for pressure sore identification and analysis to learn to classify and analyze images according to the predefined rules; and The verification image set is input into the deep learning model for identification and analysis of pressure sore injuries, so as to verify and adjust the deep learning model for identification and analysis of pressure sore injuries. A pressure sore injury identification and analysis system, comprising: A medical server including a deep learning model for pressure sore identification and analysis; and A mobile device, which communicates with the medical server, and provides a caregiver to operate to obtain a real-shot image of a pressure sore injury of a subject, and upload the real-shot image to the pressure sore identification and Analyzing deep learning models, Wherein the pressure sore injury recognition and analysis deep learning model is executed to detect and extract one of the pressure sore injury images from the real shot image, and identify and analyze the information about the pressure sore injury shown in the pressure sore injury image One of the wound information of the pressure sore injury, the wound information is returned and provided to the caregiver to read through the mobile device.

Images