KR20240059918A - Device and method for evaluating airway using picture of face and neck - Google Patents

Abstract

An electronic device according to an embodiment of the present invention includes a memory that stores an artificial intelligence model learned to perform an evaluation of the airway and a processor connected to the memory, wherein the processor is configured to include the subject's face and neck. By inputting a plurality of captured images into the artificial intelligence model, items related to the ease of airway intubation for the subject can be extracted, and values for the extracted items can be calculated to classify the difficulty level of the subject's airway. .

Description

Device and method for evaluating airway using picture of face and neck}

The present invention relates to an apparatus and method for evaluating a subject's airway using a photograph including the subject's face and neck.

Conventionally, a variety of general methods have been used to evaluate a patient's airway, but the existing methods were often based on visual observation by an expert.

Figure 1 shows a conventional airway evaluation method. Mallampati grade (10), which classifies the difficulty of tracheal intubation according to the degree to which the patient's oral cavity is visually observed and the uvula and throat are visible, as well as Thyromental height (20), Thyromental distance (30), and Sternomental distance (40) , the neck circumference measurement method (50) has been mainly used. These conventional airway evaluation methods are relatively simple and some methods may be objective, but differences may occur depending on the person measuring, it is difficult to represent the body parts related to the airway with one method, and the performance as an evaluation method is not excellent. There is.

Republic of Korea Patent Publication No. 10-2457342 (System and control method for tracheal intubation training)

Embodiments of the present invention were designed to enable the airway evaluation method to evaluate the degree of difficulty in intubating a patient's airway to be performed more objectively through deep learning.

In addition, embodiments of the present invention are designed to support airway evaluation based on photos taken using a smartphone without the need for a specific location or equipment.

The objects of the present disclosure are not limited to the purposes mentioned above, and other objects and advantages of the present disclosure that are not mentioned can be understood by the following description and will be more clearly understood by the examples of the present disclosure. Additionally, it will be readily apparent that the objects and advantages of the present disclosure can be realized by the means and combinations thereof indicated in the patent claims.

An electronic device according to an embodiment of the present invention includes a memory that stores an artificial intelligence model learned to perform an evaluation of the airway and a processor connected to the memory, wherein the processor is configured to include the subject's face and neck. By inputting a plurality of captured images into the artificial intelligence model, items related to the ease of airway intubation for the subject can be extracted, and values for the extracted items can be calculated to classify the difficulty level of the subject's airway. .

At this time, the plurality of images include a first image corresponding to a frontal photo taken so as to include the face and neck, a second image corresponding to a frontal photo taken with the neck extended, and a side photo taken with the neck extended. It may include a third image and a fourth image corresponding to a frontal photo taken with the mouth open and the tongue sticking out so that the entire mouth and tongue are visible.

Additionally, the processor may include a preprocessor that performs a normalization operation on the plurality of images, and the normalization operation may include changing brightness, changing contrast, and white balancing between color channels for the subject's photo. In addition, the preprocessor performs a preprocessing operation on a plurality of images already secured to increase the amount of learning data input to the learning of the artificial intelligence model, and uses different preprocessing techniques in response to the individual shooting characteristics of the plurality of images. It can be set as follows.

In addition, the processor inputs each of the plurality of images into an individual CNN (Convolutional Neural Networks)-based image encoder and performs a voting operation on the plurality of labels calculated to produce the artificial intelligence model to derive the final result. It may include a learning unit that learns. In addition, the processor may include a determination unit that derives a final result by applying voting weights to a plurality of labels calculated as a result of computing a plurality of images by inputting them into an artificial intelligence model. And at this time, the voting weight may be specified differently depending on at least one of the shooting characteristics of the plurality of images and the degree of freedom of neck movement of the subject.

Additionally, the processor may include a data acquisition support unit that provides guide information related to the shooting posture for the plurality of images to the subject's terminal and acquires the plurality of images from the subject's terminal.

In addition, the method of controlling an electronic device according to an embodiment of the present invention includes the steps of training an artificial intelligence model to enable the electronic device to perform an evaluation of the airway, and allowing the electronic device to include the subject's face and neck. Obtaining a plurality of captured images and inputting them into a previously learned artificial intelligence model, and the electronic device drives the artificial intelligence model to extract and extract items related to the ease of airway intubation of the subject from the images. It may include a step of classifying the difficulty of the subject's prayer by calculating the values for the items.

And, in a system consisting of an electronic device and a subject's terminal, the electronic device includes a memory that stores an artificial intelligence model learned to perform an evaluation of the airway and a processor connected to the memory, and the processor includes a subject's terminal. A plurality of images taken including the face and neck of the subject are input into the artificial intelligence model to extract items related to the ease of airway intubation for the subject, and the values for the extracted items are calculated to determine the subject's airway. After classifying the level of difficulty, the subject terminal may receive guide information related to the shooting posture from the electronic device, display it on the shooting screen, and transmit the image of the subject captured according to the guide information to the electronic device.

And the control method of the system composed of the electronic device and the subject's terminal includes the steps of training an artificial intelligence model so that the electronic device performs an evaluation of the airway, and the subject's terminal provides guidance related to the shooting posture from the electronic device. Receiving information and displaying it on a capturing screen, transmitting, by the subject terminal, an image of the subject captured according to the guide information to the electronic device, the electronic device capturing the image to include the subject's face and neck Obtaining a plurality of images and inputting them into a previously learned artificial intelligence model; and the electronic device drives the artificial intelligence model to extract items related to the ease of intubation of the airway for the subject from the images, and the extracted items It may include the step of classifying the difficulty level of the subject's prayer by calculating the value for .

Embodiments of the present invention can perform airway evaluation through a relatively simple method of taking pictures including the patient's face and neck with a simple device such as a smartphone at the patient's bedside.

As airway evaluation of patients requiring airway management can be easily performed, equipment and personnel to prepare for difficult airways can be prepared in advance.

Figure 1 is a diagram showing an example of a conventional airway evaluation method.
Figure 2 is a diagram showing the configuration of an electronic device according to an embodiment of the present invention.
Figure 3 is a diagram illustrating a conventional airway evaluation method to explain label setting standards according to an embodiment of the present invention.
Figure 4 is a diagram showing an example of a photograph of a subject according to an embodiment of the present invention.
Figure 5 is a diagram illustrating an artificial intelligence calculation process according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless clearly specifically defined.

As used in the specification, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and the “unit” or “module” performs certain roles. However, “part” or “module” is not limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, a “part” or “module” refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within components and “parts” or “modules” can be combined into smaller components and “parts” or “modules” or into additional components and “parts” or “modules”. Could be further separated.

Spatially relative terms such as “below”, “beneath”, “lower”, “above”, “upper”, etc. are used as a single term as shown in the drawing. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms that include different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is flipped over, a component described as "below" or "beneath" another component will be placed "above" the other component. You can. Accordingly, the illustrative term “down” may include both downward and upward directions. Components can also be oriented in other directions, so spatially relative terms can be interpreted according to orientation.

The present invention relates to a method for evaluating the airway of a patient attempting endotracheal intubation, etc., and specifically, to evaluate the difficulty related to the intubation procedure for the patient's (subject's) airway, and to determine whether the subject's airway is intubated. An operation is initiated to evaluate whether it corresponds to a 'difficult airway' that is not easy for the procedure. ‘Difficult airway’ refers to an airway that is expected to be difficult when performing airway-related procedures such as endotracheal intubation, and is based on various existing visual observation and actual measurement methods (e.g., Mallampati grade, Thyromental height, Thyronmental distance). , sternomental distance, neck circumference, etc.) can be used to evaluate whether the patient has a difficult airway.

Meanwhile, the electronic device 100 according to an embodiment of the present invention inputs photographic data including the face and neck of the subject (patient) into an artificial intelligence model, and determines the difficulty level related to the procedure for the subject's airway according to the results of calculating it. and whether it corresponds to 'difficult prayer' can be determined.

The electronic device 100 according to an embodiment of the present invention can perform various functions to evaluate a subject's airway using an artificial intelligence model.

Hereinafter, the electronic device 100 that performs an airway evaluation operation according to an embodiment of the present invention will be described with reference to FIG. 2 and below.

Figure 2 is a diagram showing the configuration of an electronic device according to an embodiment of the present invention.

As shown in FIG. 2, the processor 101 of the electronic device 100 may be configured to include a preprocessor 110, a learning unit 120, a determination unit 130, and a data acquisition support unit 140. there is.

The pre-processing unit 110 may perform a processing operation of images required to perform a learning operation and an airway evaluation operation of an artificial intelligence model according to an embodiment of the present invention.

First, the preprocessor 110 may perform a preprocessing operation to normalize learning data.

Specifically, the preprocessor 110 may perform a preprocessing operation on images set as a learning dataset for the purpose of normalizing the distribution of images in terms of the stability of the artificial intelligence model being learned. This preprocessing operation for normalization may include a process of uniformly matching brightness information through white balancing between the brightness and contrast values of the image and color channels.

And the preprocessing unit 110 may perform a preprocessing operation for the purpose of increasing the amount of learning data.

In this way, the pre-processing unit 110 may perform a processing process of applying various transformations to the subject's photo, which is already secured image data, in order to increase the limited amount of learning data.

Such preprocessing operations for data augmentation may include, for example, geometric transformation operations such as rotation and resizing of images. Preferably, the preprocessing operation for data enhancement may include arbitrary color correction, random cutting, and elastic transformation. In addition, the preprocessor 110 can perform rotation, symmetry, and pixel transformation (bright, grayscale, blur, etc.) as preprocessing operations for data augmentation.

The purpose of this data augmentation is to implement a more accurate artificial intelligence model by increasing the amount of data input for learning. Accordingly, when performing a preprocessing operation for data enhancement, the preprocessing unit 110 can perform a processing operation so that the core characteristics of the image are maintained. For example, when performing random cropping on an image for data augmentation, the preprocessor 110 may set a limit on the cropping range so that key features included in the image can be maintained.

According to various embodiments of the present invention, in the process of learning the artificial intelligence model and analyzing the results through the artificial intelligence model, images input to the artificial intelligence model may be composed of a total of four types. Accordingly, the preprocessing unit 110 may apply different types of preprocessing techniques to each of the four types of images based on different standards.

First, refer to FIG. 4 for an explanation of the four types of images input into the artificial intelligence model.

Figure 4 is a diagram showing an example of a photograph of a subject according to an embodiment of the present invention.

As shown in FIG. 4, photos of subjects to be input into artificial intelligence according to an embodiment of the present invention may be configured to include four types. According to various embodiments, types of photos of the subject may be added.

Among the four types of images (photos of the subject) to be input into the artificial intelligence model according to an embodiment of the present invention, the first image may be designated as a frontal photo taken to include the subject's face and neck, as shown in 401. More specifically, the first image may be set to include from below the subject's nose to the top of the sternum.

Among the photos of the subject to be input into the artificial intelligence model according to an embodiment of the present invention, the second image may be designated as a frontal photo with the patient's neck extended, as shown in 402. More specifically, the second image may be set to include the subject's chin to the upper sternum, which is the subject of airway determination.

Among the photos of the subject to be input into the artificial intelligence model according to an embodiment of the present invention, the third image may be designated as a side photo with the subject's neck extended, as shown in 403. More specifically, the third image may refer to a side photo taken with the subject's neck tilted back and including the subject's chin and upper sternum.

Among the photos of the subject to be input into the artificial intelligence model according to an embodiment of the present invention, the fourth image may be set as a photo with the mouth open and the tongue sticking out of the mouth, as shown in 404. At this time, the fourth image may be a photo taken with the subject's mouth open enough to allow the inside of the oral cavity to be seen, and may be set as a frontal photo taken to include the subject's entire mouth and tongue.

In this way, image data input to artificial intelligence according to an embodiment of the present invention may be composed of four types, and as mentioned above, the preprocessing unit 110 uses preprocessing techniques based on different standards corresponding to each of the four types of images. Types can be applied.

For example, the pre-processing unit 110 is a data enhancement pre-processing technique applicable to the first image (corresponding to 401 in FIG. 4, an image taken from below the subject's nose to the top of the sternum) among four types of images. Three types of cutting, left-right conversion, and color correction are applied, and left-right conversion and elastic are applied as preprocessing techniques to the fourth image (corresponding to 404 in Figure 4, an image taken from the front showing the subject's mouth open). ) Only conversion can be performed.

Furthermore, the pre-processing unit 110 may set different standards for the pre-processing techniques performed by considering the characteristics of the situation in which the four types of images are captured (e.g., degree of proximity at the time of shooting, etc.).

Specifically, for example, the preprocessor 110 processes the fourth image (corresponding to 404 in FIG. 4) taken at a relatively close location compared to the first image (corresponding to 401 in FIG. 4) taken at a relatively long distance. In this case, as the frequency of occurrence of out-of-focus data may be high, to prepare for this, a blur (processing operation to blur the focus) technique may be additionally included during the pre-processing operation for the fourth image. .

According to various embodiments, the preprocessing unit 110 may set different types of preprocessing techniques to correspond to the characteristics of each object (e.g., area ratio of the object in the image) for each of the four types of images. At this time, the object is a concept relative to the background, such as the subject's face, the subject's neck, or an image of the subject's inside the oral cavity, and may mean an image that is the subject of airway judgment.

In addition, according to various embodiments, even when the preprocessor 110 performs an operation to evaluate the subject's airway by inputting data into an artificial intelligence model on which learning has been completed, the preprocessor 110 provides information on image data for the stability of the analysis operation. Preprocessing can be performed. Even in this case, the preprocessor 110 can perform processing operations (e.g., brightness adjustment, contrast adjustment, white balancing, etc.) to normalize the distribution of the image, like the preprocessing operation performed to construct the learning dataset. there is.

The processor 101 may be configured to include a learning unit 120 that trains an artificial intelligence model using image data processed by the preprocessor 110.

The learning unit 120 sets the image data processed in the pre-processing unit 110 and labels corresponding to each image data as learning data, and uses the learning data as training data based on CNN (Convolutional Neural Networks). Learning to calculate weights can be performed by inputting it into an artificial intelligence model.

In the present invention, an image encoder and classifier method based on CNN (Convolutional Neural Networks), which shows the most stable performance in extracting image features, can be preferably used. In order to receive images from various angles as input, the learning unit 120 sets the four types of images (photos of the subject) described above as learning data, and sets each image (first image, second image, third image) as learning data. image, fourth image) can be independently input to an individual image encoder to perform individual learning corresponding to each image.

At this time, the learning data can be divided into input data and output data. The input data can correspond to four types of images, and the output data can correspond to labels corresponding to each of the four types of images. The label may be information set based on classification information according to the conventional airway evaluation method shown in FIG. 3.

Figure 3 is a diagram illustrating a conventional airway evaluation method to explain label setting standards according to an embodiment of the present invention.

Figure 3 is a photograph of the epiglottis and vocal cords as seen through laryngoscope. Conventionally, the difficulty of the patient's airway procedure could be evaluated according to the condition of the epiglottis and vocal cords as confirmed by laryngoscopy. According to this conventional evaluation method, 301 in FIG. 3 corresponds to grade 1 with the vocal cords fully visible. 302 corresponds to grade 2, with part of the vocal cords visible. And 303 corresponds to grade 3, with only the epiglottis visible and the vocal cords not visible. 304 corresponds to grade 4, with both the epiglottis and vocal cords not visible. According to the conventional evaluation method, grades 3 and 4 shown in FIG. 3 are evaluated as 'difficult airways' according to medical definitions.

Among the learning data according to an embodiment of the present invention, output data may also be set from level 1 to level 4 according to the dividing criteria in FIG. 3. However, at this time, the output data is the airway evaluation grade corresponding to the four types of images of the subject as input data, and the artificial intelligence model learned accordingly creates a primary label according to the classification results for the images input to each individual encoder. can be calculated separately. According to one embodiment, the primary label may be calculated in the form of grades 1 to 4.

And at this time, after the learning data is input into the artificial intelligence model, the learning unit 120 extracts characteristic information (feaure) required in the CNN operation process of deriving a label from an input image in the learning process of the artificial intelligence model. You can. The feature information refers to items related to the ease of airway intubation for the subject. And the items related to the ease of airway intubation are, in other words, characteristics related to assessing whether the subject's airway corresponds to an airway medically defined as a 'difficult airway' (e.g., upper shield of the thyroid cartilage in the jaw) It can mean the length from the thyroid notch, the length from the chin to the top of the sternum, etc.) Once learning is completed, the feature information extracted later is used to derive the primary label.

Additionally, according to various embodiments, the learning unit 120 may train an artificial intelligence model to calculate a final airway evaluation result through a voting process after the primary label is calculated. The voting is a type of ensemble technique for calculating the final result by combining classification results calculated through a plurality of individual encoders. Therefore, the learning unit 120 can learn to calculate the final result by combining the values corresponding to the primary labels given to each of the four types of images input through individual encoders. The final result may be whether the subject's airway falls under the medically defined 'difficult airway'. Alternatively, the final result may be calculated as information on the degree of difficulty related to the procedure of the subject's airway.

The determination unit 130 may evaluate the airway of a patient (subject) in need of airway management by inputting data into the artificial intelligence model generated by the learning unit 120.

The determination unit 130 inputs a photo of the subject that has undergone preprocessing such as normalization and data augmentation through the preprocessor 110 into the artificial intelligence model, and selects items related to the ease of endotracheal intubation from the photo of the subject. By extracting and calculating the values for the extracted items, it is possible to determine whether the subject's airway corresponds to a 'difficult airway'.

More specifically, the artificial intelligence calculation operation performed by the determination unit 130 can be described with reference to FIG. 5.

Figure 5 is a diagram illustrating an artificial intelligence calculation process according to an embodiment of the present invention.

The pre-processing unit 110 according to an embodiment of the present invention may input four types of image data 501 into a model 502 that performs a pre-processing operation and perform processing.

At this time, the four types of image data consist of photos taken including the subject's face and neck, and may consist of photos taken through the subject's terminal, such as 401, 402, 403, and 404 in FIG. 4, respectively. .

And the preprocessing operation may refer to the operation of normalizing image data before inputting it to the CNN encoder.

After the preprocessing operation for the four types of image data is performed, the determination unit 130 may input the image data that has undergone the preprocessing process into the previously learned CNN encoder (CNN-based image encoder) 503. At this time, the determination unit 130 can input each of the four types of images to an individual CNN encoder.

And as the determination unit 130 inputs image data to the CNN encoder 503, the classifier 504 calculates a primary label based on the feature information and the weight calculated during the learning process. You can proceed with the calculation process to do so.

Thereafter, the determination unit 130 can obtain a final prediction result through voting 505 on the label value primarily derived through a classification operation through the classifier 504.

When performing voting, artificial intelligence models according to various embodiments can derive results by applying different weights (hereinafter referred to as voting weights) to the classification results calculated through individual encoders, and the artificial intelligence model Voting weights are determined according to the individual characteristics of the four types of images input (e.g., difficulty of shooting, measurement deviation for each subject for the objects included in the image, etc.) or the subject's situation (degree of freedom of neck movement, emergency situation, etc.) It may be specified differently.

In a typical situation, when the artificial intelligence model derives the final result by combining the four primary labels obtained as a result of four types of images being input to individual encoders, the ratio of voting weights applied to the four labels can be specified equally. there is.

Meanwhile, according to various embodiments, the artificial intelligence model is used for the primary label calculated from a specific image (e.g., a second image) that is judged to have a significant deviation in measurement values for each subject due to the characteristics of the shooting among the four types of images. It can be set by increasing the voting weight compared to the primary label of other images (first image, third, and fourth image). Accordingly, the artificial intelligence model according to the embodiment of the present invention can prevent the problem of not properly reflecting the airway characteristics of each subject due to images in which differences between subjects are not noticeable due to imaging problems.

Additionally, according to various embodiments, the artificial intelligence model may be designed by setting the voting weight for the primary label of each image differently depending on the subject's situation. Specifically, the artificial intelligence model can be designed under the assumption that the reliability is different for each type of image depending on the degree to which the subject can freely move his or her neck. Accordingly, the electronic device can first receive input from the subject's terminal about the degree of movement of the subject's neck, and request that the degree of freedom of neck movement be specified as a number, for example, 1 to 5 (the larger the number, the more freedom).

As the subject's neck moves more freely, the voting weight applied to the artificial intelligence model can be set to the same value. On the other hand, as the subject's neck movement becomes more uncomfortable, the voting weight applied to the artificial intelligence model may increase to different degrees. Specifically, in the artificial intelligence model, as the subject's neck movement becomes more uncomfortable, the voting weight applied to the label calculated from the second and third images that require neck extension is reduced, and the The voting weight applied to the label calculated from the first image and the fourth image that can be photographed may be set to increase.

Meanwhile, the processor 101 may perform operations to obtain various image data from the subject's terminal in conjunction with the subject's terminal.

In this regard, the processor 101 according to various embodiments of the present invention may include a data acquisition support unit 140.

The data acquisition support unit 140 is connected to the subject terminal (not shown) and the electronic device 100, and in the process of acquiring four types of images to be input into the artificial intelligence model from the subject terminal, the photo is taken in what posture. Guidance on whether to use the device can be provided to the target terminal.

Specifically, the data acquisition support unit 140 can provide a shooting guide for each of the four types of images, and sequentially guides taking photos corresponding to the first image, followed by the second image, the third image, and the third image. 4 You can be guided to take images.

In addition, the data acquisition support unit 140 can determine whether the subject's neck is properly extended and the degree of neck extension based on a photo taken through the subject's terminal or a video of the subject's neck movement confirmed during the shooting operation stage. there is.

If the data acquisition support unit 140 determines that the subject's shooting posture, including the degree of neck extension, does not meet the standards for images to be input into the artificial intelligence model, it may request re-photography.

In addition, if the data acquisition support unit 140 determines that the subject's posture does not meet the standard even though the image has been taken more than a preset number of times, the data acquisition support unit 140 receives the photo taken by the subject, determines that the movement of the subject's neck is uncomfortable, and determines the degree of discomfort. can be quantified. The degree of discomfort in the neck movement specified by the data acquisition support unit 140 may be determined based on the photo transmitted from the subject's terminal and the video of the subject's neck movement confirmed during the shooting stage.

The data acquisition support unit 140 sets different artificial intelligence models (models with different voting weights) in response to numerical information indicating the degree of discomfort in the subject's neck movement, and four types of images of the subject acquired in the set artificial intelligence model. You can request to put in . For example, when the data acquisition support unit 140 determines that the subject's neck movement is uncomfortable, the voting weight to be applied to the label value corresponding to the second image and the third image, which are photos requiring neck extension, determines the neck movement. The artificial intelligence model with the reduction set in response to the discomfort value may be selected and the judgment unit 130 may be requested to input the four types of images obtained into the selected artificial intelligence model.

In addition, the processor 101 according to an embodiment of the present invention can perform various airway evaluation-related operations in conjunction with other terminals.

For example, the processor 101 may perform an operation of transmitting the result of an artificial intelligence operation on an image acquired from the patient's terminal to the patient's terminal and an operation of guiding a response plan corresponding to the result to the medical staff terminal. .

The electronic device 100 according to an embodiment of the present invention may include a memory (not shown), a communication unit (not shown), and a processor 101.

Memory can store various programs and data necessary for the operation of electronic devices. Memory can be implemented as non-volatile memory, volatile memory, flash-memory, hard disk drive (HDD), or solid state drive (SSD).

The communication unit can perform communication with external devices. In particular, the communication unit may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a low-power Bluetooth chip (BLE chip). At this time, the Wi-Fi chip, Bluetooth chip, and NFC chip communicate in the LAN method, WiFi method, Bluetooth method, and NFC method, respectively. When using a Wi-Fi chip or Bluetooth chip, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. A wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), and LTE (Long Term Evolution).

The processor 101 can control the overall operation of the user device using various programs stored in memory. The processor may be composed of RAM, ROM, graphics processing unit, main CPU, first to n interfaces, and bus. At this time, RAM, ROM, graphics processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the electronic device is booted, the O/S is stored in RAM, and various application data selected by the user may be stored in RAM.

ROM stores a set of instructions for booting the system. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 200 to RAM according to the command stored in the ROM, executes the O/S, and boots the system. When booting is complete, the main CPU copies various application programs stored in memory to RAM and executes the application programs copied to RAM to perform various operations.

The main CPU accesses memory and performs booting using the OS stored in the memory. And, the main CPU performs various operations using various programs, content, data, etc. stored in memory.

The first to n interfaces are connected to the various components described above. One of the first to n interfaces may be a network interface connected to an external device through a network.

Meanwhile, furthermore, the processor can control the artificial intelligence model. In this case, of course, the control unit may include a graphics processor (eg, GPU) for controlling the artificial intelligence model.

The processor may include one or more cores (not shown), a graphics processing unit (not shown), and/or a connection passage (for example, a bus, etc.) for transmitting and receiving signals to and from other components.

A processor according to one embodiment performs the method described in connection with the present invention by executing one or more instructions stored in memory.

Meanwhile, the processor further includes RAM (Random Access Memory, not shown) and ROM (Read-Only Memory, not shown) that temporarily and/or permanently store signals (or data) processed inside the processor. It can be included. Additionally, the processor 130 may be implemented in the form of a system on chip (SoC) that includes at least one of a graphics processing unit, RAM, and ROM.

Programs (one or more instructions) for processing and controlling the processor can be stored in the memory. Programs stored in the storage unit may be divided into a plurality of modules according to their functions.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) and stored in a medium in order to be executed in conjunction with a hardware computer. Components of the invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as combinations of data structures, processes, routines or other programming constructs, such as C, C++, , may be implemented in a programming or scripting language such as Java, assembler, etc. Functional aspects may be implemented as algorithms running on one or more processors.

Although the present invention has been described in detail with reference to the above-described examples, those skilled in the art may make modifications, changes, and variations to the examples without departing from the scope of the present invention. In short, in order to achieve the intended effect of the present invention, it is not necessary to separately include all functional blocks shown in the drawings or to follow all the orders shown in the drawings, and even if not, the technical aspects of the present invention described in the claims may be used. Please note that it may fall within the range.

100: electronic device
110: preprocessing unit
120: Learning Department
130: Judgment unit
140: Data acquisition support department

Claims (11)

In electronic devices,
Memory to store an artificial intelligence model trained to perform an evaluation of the airway; and
Including a processor connected to the memory,
The processor,
A plurality of images captured including the subject's face and neck are input into the artificial intelligence model, items related to the ease of airway intubation for the subject are extracted, and values for the extracted items are calculated to determine the subject's airway. An electronic device characterized by classifying the level of difficulty. According to paragraph 1,
The plurality of images are
The first image corresponds to a frontal photograph taken to include the face and neck, the second image corresponds to a frontal photograph taken with the neck extended, the third image corresponds to a side photograph with the neck extended, and the mouth is open and the tongue is held. An electronic device, characterized in that it includes a fourth image corresponding to a frontal photograph taken so that the entire mouth and tongue are visible by sticking out. According to paragraph 1,
The processor is
It includes a preprocessor that performs a normalization operation on the plurality of images,
The normalization operation includes changing the brightness of the subject's photo, changing the contrast, and white balancing between color channels. According to paragraph 3,
The preprocessing unit
In order to increase the amount of learning data input to the learning of the artificial intelligence model, a preprocessing operation is performed on a plurality of images already secured,
An electronic device, characterized in that pre-processing techniques are set differently in response to individual shooting characteristics of the plurality of images. According to paragraph 1,
The processor is
A learning unit that trains the artificial intelligence model to derive the final result by performing a voting operation on the plurality of labels calculated by inputting each of the plurality of images into an individual CNN (Convolutional Neural Networks)-based image encoder. An electronic device, characterized in that it includes ; According to paragraph 1,
The processor is
An electronic device comprising: a determination unit that derives a final result by applying voting weights to a plurality of labels calculated as a result of inputting a plurality of images into an artificial intelligence model. According to clause 6,
The voting weight is
An electronic device, characterized in that the plurality of images are differently specified by at least one of the capturing characteristics and the degree of freedom of neck movement of the subject. According to paragraph 1,
The processor is
An electronic device comprising a data acquisition support unit that provides guide information related to the shooting posture for the plurality of images to the subject terminal and acquires the plurality of images from the subject terminal. In a method of controlling an electronic device,
Learning an artificial intelligence model to enable the electronic device to evaluate airways;
The electronic device acquires a plurality of images captured to include the subject's face and neck and inputs them into a previously learned artificial intelligence model; And
The electronic device, driving the artificial intelligence model to extract items related to the ease of intubation of the airway of the subject from the image, calculating values for the extracted items to classify the difficulty level of the subject's airway; A method comprising: In a system consisting of an electronic device and a subject terminal,
The electronic device is
Memory to store an artificial intelligence model trained to perform airway evaluation; and
Including a processor connected to the memory,
The processor,
A plurality of images captured including the subject's face and neck are input into the artificial intelligence model, items related to the ease of airway intubation for the subject are extracted, and values for the extracted items are calculated to determine the subject's airway. Classify the level of difficulty,
The target terminal is
The system is characterized in that it receives guide information related to the shooting posture from the electronic device, displays it on the shooting screen, and transmits the image of the subject captured according to the guide information to the electronic device. In a method of controlling a system consisting of an electronic device and a subject terminal,
Learning an artificial intelligence model to enable the electronic device to evaluate airways;
Receiving, by the subject terminal, guide information related to the shooting posture from the electronic device and displaying it on the shooting screen;
transmitting, by the subject terminal, an image of the subject captured according to the guide information to the electronic device;
The electronic device acquires a plurality of images captured to include the subject's face and neck and inputs them into a previously learned artificial intelligence model; And
The electronic device, driving the artificial intelligence model to extract items related to the ease of intubation of the airway of the subject from the image, calculating values for the extracted items to classify the difficulty level of the subject's airway; A method comprising: