WO2024019220A1

WO2024019220A1 - Object behavior detection system using thermal imaging

Info

Publication number: WO2024019220A1
Application number: PCT/KR2022/013087
Authority: WO
Inventors: 이용권; 강연준
Original assignee: 주식회사 럭스로보
Priority date: 2022-07-19
Filing date: 2022-09-01
Publication date: 2024-01-25
Also published as: KR102636116B1; KR20240011512A

Abstract

The present invention relates to an object behavior detection system for detecting behavior of an object by analyzing a thermal image of the object. The object behavior detection system of the present invention may comprise: a thermal imaging camera that includes a communication module, is installed in each of a plurality of spaces, and acquires a thermal image of an object in each of the installed spaces; and a server that receives a thermal image from the thermal image camera and analyzes the received thermal image to determine whether the object is in a dangerous situation.

Description

Object behavior detection system using thermal imaging

The present invention relates to a system for detecting the behavior of an object using thermal imaging, and in particular, to a system for detecting the behavior of an object by analyzing thermal imaging of the object to detect whether the object is in danger.

Recently, as human lifespan has increased, interest in individual health management is increasing along with the desire to live a long and healthy life.

Due to the nuclear family, people are spending more time alone at home, but it is difficult to respond proactively in the event of an emergency or medical emergency.

If an elderly or sick person is alone at home and a dangerous situation such as a fall occurs, it is important to quickly detect and take action.

Recently, a system is being used to install general cameras or CCTV in apartments or other residential spaces to send emergency calls to guardians or emergency centers when an event such as a dangerous situation occurs.

However, in this case, there is a problem that there is a risk of invasion of privacy, and emergency measures are limited because no one except the guardian can monitor the dangerous situation.

To solve these problems, a detection system using a thermal imaging camera is being introduced.

Registered Patent Publication No. 10-2052883 (Prior Document 1) discloses a fall prediction system control method using a thermal imaging camera that determines the risk of falling based on the x-y axis histogram data of the captured thermal image and generates a notification.

Registered Patent Publication No. 10-1916631 (Prior Document 2) discloses a fall detection device including a thermal imaging camera that determines whether a patient has fallen through imaging data captured by the thermal imaging camera.

Registered Patent Publication No. 10-1927220 (Prior Document 3) describes an object of interest that detects the temperature of an object of interest located within the target area using thermal imaging images and determines the state and situation of the object of interest using the detected temperature. A sensing method and device are disclosed.

However, conventional detection systems using thermal imaging, including those in prior literature 1-3, have limitations in detecting minute movements, which reduces the accuracy of determining dangerous situations.

In addition, in the prior art, when multiple thermal imaging cameras are installed in various spaces within a home, each thermal imaging camera is distributed and installed in each space, so a time delay occurs in collecting detection information through communication from each camera. There is a problem with the analysis of thermal imaging images being slow, which can have bad results for people who need quick emergency treatment.

Accordingly, in this technology field, accurate detection performance is required when detecting behavior using thermal imaging images, and the development of technology that enables rapid processing of detection information even when thermal imaging cameras are distributed and installed in multiple spaces is required. It is becoming.

The purpose of the present invention is to provide an object behavior detection system using thermal imaging images that can accurately detect object behavior from thermal imaging images.

The present invention is an object behavior detection method using thermal imaging that enables communication between a plurality of thermal imaging cameras distributed and installed in various spaces, allowing rapid collection of detection information from thermal imaging cameras through communication between thermal imaging cameras. The purpose is to provide a system.

The present invention provides an object behavior detection system using thermal imaging that enables quick and accurate detection of a dangerous situation of an object without infringing on individual privacy by acquiring and analyzing thermal imaging of the object through a thermal imaging camera. There is a purpose to doing this.

An object behavior detection system using thermal imaging according to an embodiment of the present invention includes a thermal imaging camera that includes a communication module and is installed in a plurality of spaces and acquires thermal imaging images of objects in the installed spaces; It includes a server that receives a thermal image from the thermal image camera and analyzes the received thermal image to determine a risk situation of the object, wherein the server extracts a region of interest from the thermal image and collects the thermal image. The representative brightness value in the region of interest is calculated by setting the brightness value corresponding to the color according to the temperature of the object in the plurality of cells constituting the image, (1) the moving speed of the region of interest, (2) the It is possible to recognize whether the object is in danger using the moving distance of the area, (3) the change rate of the representative brightness value of the immediately previous area of interest, and (4) the duration of the representative brightness value of the currently moved area of interest.

In this embodiment, the region of interest may include a facial area of the object.

In this embodiment, when the region of interest corresponds to one cell, the brightness value of the one cell can be calculated as the representative brightness value.

In this embodiment, when the region of interest corresponds to two or more cells, the average of the brightness values of the two or more cells can be calculated as the representative brightness value.

In this embodiment, when the region of interest corresponds to two or more cells, the largest brightness value among the brightness values of the two or more cells can be calculated as the representative brightness value.

In this embodiment, the communication modules of the plurality of thermal imaging cameras can transmit and receive the thermal images to each other through mesh Wi-Fi communication.

In this embodiment, at least one of the plurality of thermal imaging cameras is connected to a gateway and can transmit thermal imaging images to the server through the gateway.

In this embodiment, if the moving speed of the area of interest is greater than or equal to a preset first standard value (first condition), the server may recognize that a dangerous situation has occurred in the object.

In this embodiment, the server may recognize that a dangerous situation has occurred in the object if the moving distance between the areas of interest is greater than a preset second standard value (second condition).

In this embodiment, the server may recognize that a dangerous situation has occurred in the object if the rate of change of the representative brightness value in the immediately preceding region of interest is greater than or equal to a preset third standard value (third condition).

In this embodiment, if the duration of the representative brightness value in the moved region of interest is greater than or equal to a preset fourth standard value (fourth condition), the server may recognize that a dangerous situation has occurred in the object.

In this embodiment, the server determines that the moving speed of the area of interest is greater than or equal to a preset first reference value (first condition), the moving distance between the areas of interest is greater than or equal to a preset second reference value (second condition), and The rate of change of the representative brightness value in the area of interest satisfies at least one of the preset third reference values or more (third condition), and the duration of the representative brightness value in the moved area of interest is more than the preset fourth reference value. If this is the case (condition 4), it can be recognized that a dangerous situation has occurred in the object.

The object behavior detection system using thermal imaging according to an embodiment of the present invention has one or more of the following effects.

According to the present invention, the temperature in a thermal image is subdivided into numerical values and the behavior of an object is detected according to changes in the numerical values, so that the behavior can be accurately detected in the thermal image.

According to the present invention, a plurality of thermal imaging cameras can be distributed and placed in multiple spaces, making it possible to detect object behavior occurring in multiple spaces.

According to the present invention, even if thermal imaging cameras are distributed and deployed in various spaces, communication between each thermal imaging camera can be performed, and detection information from a plurality of thermal imaging cameras can be quickly collected and processed, enabling rapid detection.

According to the present invention, by acquiring and analyzing a thermal image of an object through a thermal imaging camera, it is possible to quickly and accurately detect a dangerous situation of an object without infringing on privacy.

Figure 1 is a configuration diagram of an object behavior detection system using thermal imaging images according to an embodiment of the present invention.

Figure 2 is a diagram for explaining the mesh Wi-Fi connection of a thermal imaging camera according to an embodiment of the present invention.

Figure 3 is a detailed configuration diagram of a thermal imaging camera constituting an object behavior detection system using thermal imaging images according to an embodiment of the present invention.

Figure 4 is a detailed configuration diagram of a server that configures an object behavior detection system using thermal imaging images according to an embodiment of the present invention.

Figure 5 is an example diagram of a thermal image acquired by a thermal imaging camera according to an embodiment of the present invention.

Figures 6a and 6b are examples of converting the temperature of each part of an object into a color value in a thermal image according to an embodiment of the present invention.

Figure 7 is a flowchart showing the process of detecting the behavior of an object using a thermal image according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

The present invention provides a system and method for detecting the behavior of an object by analyzing a thermal image of the object. Thermal imaging cameras are distributed and placed in various spaces to acquire thermal images from each thermal imaging camera. By transmitting and receiving thermal imaging images between thermal imaging cameras through mesh Wi-Fi communication, thermal imaging images from each thermal imaging camera can be quickly collected. Let it happen. In addition, the change in color brightness value according to the temperature of the cells that make up the thermal image is used to detect the behavior of the object and recognize the dangerous situation of the object. The present invention enables fast and accurate object behavior detection.

Hereinafter, an object behavior detection system using thermal imaging according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a schematic overall configuration diagram of an object behavior detection system using thermal imaging images according to an embodiment of the present invention.

Referring to FIG. 1, an object behavior detection system using thermal imaging according to an embodiment of the present invention may include at least one thermal imaging camera 100.

The thermal imaging camera 100 may be distributed and installed in a plurality of separate spaces, and may acquire thermal imaging images of each space, particularly thermal imaging images of objects.

The thermal imaging camera 100 may include an infrared (IR) sensor capable of measuring the temperature of an object, and the IR sensor may measure the temperature of the object by detecting radiant heat (mainly infrared) emitted from the object.

The temperature of the object measured in this way may be displayed in a color corresponding to the temperature through a display connected to the thermal imaging camera 100.

The thermal imaging camera 100 has a communication module and can communicate with other thermal imaging cameras, or connect to the AP device 10 and/or the network 20.

The thermal imaging camera 100 may be equipped with different types of communication modules or may be equipped with a plurality of communication modules. For example, the thermal imaging camera 100 may include a Wi-Fi communication module, NFC communication module, Zigbee communication module, Bluetooth™ communication module, etc.

In this embodiment, for example, the thermal imaging cameras 100 are connected to each other through mesh Wi-Fi and mesh Wi-Fi communication is possible between the thermal imaging cameras 100, so that data can be transmitted and received between the thermal imaging cameras 100.

The thermal imaging camera 100 may be connected to the gateway 200 through a communication module.

The gateway 200 is a device that connects the network 300 and the home network (1), and transmits thermal image images from the thermal imaging camera 100 to a server 400, which will be described later, through the network 300. It can be sent to .

The network 300 may be a cellular communication network such as LTE, 5G, etc., or a typical Internet network.

The server 400 may receive a thermal image of an object transmitted from the thermal imaging camera 100 and store it in an internal storage unit.

The server 400 can analyze the behavior of objects from thermal imaging images.

The server 400 can transmit the thermal image image and the analysis results of the object's behavior to the user terminal 500.

The user terminal 500 can display the thermal image and analysis results received from the server 400 on the screen. As a result, the user can check this on the user terminal 500.

The user terminal 200 may be, for example, a portable mobile terminal such as a smart phone or tablet PC.

The server 400 may transmit an alarm to a related agency server (not shown) according to the analysis results. In other words, if a dangerous or emergency situation is confirmed as a result of the analysis of the thermal image, a notification can be sent to the server of related organizations such as medical institutions, police stations, and public health centers.

Figure 2 is a diagram for explaining mesh Wi-Fi connection of a plurality of thermal imaging cameras according to an embodiment of the present invention.

Referring to FIG. 2, each thermal imaging camera 100 includes a communication module 110 for mesh Wi-Fi communication.

Thermal imaging cameras 100 can be installed in a plurality of

spaces

10, 20, and 30, respectively, and each thermal imaging camera 100 can perform wireless communication with each other using the mesh Wi-Fi communication module 110. there is.

At least one of the plurality of thermal imaging cameras 100 may be connected to the gateway 200. In the drawing, for example, the communication module 110 of the thermal imaging camera 100 in the second space is connected to the gateway 200.

The thermal imaging cameras 100 of the first, second, and third spaces can acquire thermal image images of each space. If an object exists in that space, you can naturally obtain a thermal image of the object.

The thermal imaging camera 100 in the first and third spaces can acquire thermal image images for the space and transmit them to the communication module 110 of the thermal imaging camera 100 in the second space through the communication module 110. there is.

The thermal imaging camera 100 in the second space can transmit thermal imaging images of its own space and thermal imaging images of the first and third spaces to the gateway 200 through the communication module 110.

The gateway 200 can transmit all thermal image images of

spaces

1, 2, and 3 to the server 400 through the network 300.

In the mesh Wi-Fi communication method according to this embodiment, two-way communication is possible between the communication modules 110 of the thermal imaging camera 100. Compared to the existing expandable Wi-Fi communication method, the data transmission speed becomes faster as the number of communication modules increases.

In other words, by constructing a plurality of communication modules 110 into one communication network, two-way communication occurs between each communication module 110, thereby increasing the communication speed.

The communication module 110 can also communicate with a mobile communication terminal in real life. Even in this case, mesh Wi-Fi communication is performed between the communication modules 110 through a separate communication route, so communication of the mobile communication terminal is not affected.

In other words, mesh Wi-Fi communication in this embodiment communicates through a communication route different from that of a mobile communication terminal, so communication speed can be optimized even when communicating with a mobile communication terminal.

In mesh Wi-Fi communication, a plurality of communication modules 110 are set to one SSID (service set identifier), so uninterrupted Wi-Fi communication is possible even when the thermal imaging camera 100 is moved.

Figure 3 is a detailed configuration diagram of a thermal imaging camera constituting a behavior monitoring system according to an embodiment of the present invention.

Referring to FIG. 3, the thermal imaging camera 100 includes a communication module 110 that communicates with an external device, a thermal imaging sensor 120 that acquires a thermal image of an object, and a power supply unit 130 that supplies power. , a storage unit 140 that stores data and programs, and a camera control unit 150 that controls the overall operation of the thermal imaging camera 100.

The communication module 110 can communicate with the communication module 110 of another thermal imaging camera 100. As described above, the communication module 110 can perform mesh Wi-Fi communication.

The communication module 110 can transmit and receive thermal images.

The thermal imaging image may include metadata information that can be used to determine which thermal imaging camera 100 and when the image was acquired.

Metadata may include information such as identification information of the thermal imaging camera 100 that captured the thermal imaging image, the time the thermal imaging image was captured, and the location where the thermal imaging image was captured.

The communication module 110 may communicate with the gateway 200. The gateway 200 may transmit the thermal image received from the communication module 110 to the server 400.

The thermal image sensor 120 may be configured as, for example, an infrared (IR) sensor.

The thermal image sensor 120 can acquire thermal image images of the surroundings. In particular, in this embodiment, thermal image images of the space where the thermal imaging camera 100 is placed and objects (eg, people) in the space can be acquired.

The power supply unit 130 may include a battery and may supply power to the thermal imaging camera 100. The power supply unit 130 may include a charging device (not shown) that can charge the battery.

Alternatively, the power supply unit 130 may receive commercial power, convert it to a voltage suitable for the thermal imaging camera 100, and supply power.

Various data and programs necessary for the operation of the thermal imaging camera 100 may be stored in the storage unit 140. In particular, a thermal image acquired by the thermal image sensor 120 may be stored in the storage unit 140.

The camera control unit 150 can control the operations of the communication module 110, the thermal image sensor 120, and the power supply unit 130.

The camera control unit 150 can control the thermal imaging sensor 120 to acquire thermal image images of the surroundings and store them in the storage unit 140, and can control other thermal imaging cameras 100 and other thermal imaging cameras 100 through the communication module 110. / Alternatively, it can be controlled to transmit the thermal image to the gateway 200, and can be controlled to supply power from the power supply unit 130.

Figure 4 is a detailed configuration diagram of a server constituting a behavior detection system according to an embodiment of the present invention.

Referring to FIG. 4, the server 400 according to an embodiment of the present invention may be configured to include a server communication unit 410, a database (DB) 420, and a server control unit 430.

The server communication unit 410 can connect to the network 300 and communicate with the thermal imaging camera 100 and/or the user terminal 500. Optionally, the server communication unit 410 may communicate with servers (not shown) of related organizations such as medical institutions, police stations, emergency rooms, and administrative agencies.

The database (DB) 420 can store various programs and data necessary for the operation of the server 400. In particular, the DB 420 can store thermal image images transmitted from a plurality of thermal imaging cameras 100.

In the drawing, for example, the DB 420 is shown as installed inside the server 400, but alternatively, the DB 420 may be built separately outside the server 400 and operate in conjunction with the server 400. .

The server control unit 430 can control the overall operation of the server 400.

The server control unit 430 can analyze the behavior of objects using thermal imaging images.

In this embodiment, the server control unit 430 sets the color brightness value according to the temperature of the object for each of the plurality of cells constituting the thermal image image and analyzes the object's behavior using the change in brightness value in a specific cell. .

For example, through changes in brightness values in a plurality of cells that make up a thermal image, it is possible to analyze behavior such as whether an object is sitting, standing, or lying down, whether the object is sitting and then standing up, or standing and lying down, etc. .

Additionally, the server control unit 430 can recognize whether an object is in a dangerous situation by analyzing the thermal image.

In this embodiment, the server control unit 430 sets the color brightness value according to the temperature of the object for each of the plurality of cells constituting the thermal image and uses the change in brightness value in a specific cell to determine whether the object performs routine actions. You can recognize whether you are making a dangerous move or are in a dangerous situation.

For example, the server control unit 430 can determine whether an object is in a dangerous situation by checking the change in brightness value and the speed of change in brightness value in a plurality of cells.

In this embodiment, the server control unit 430 is equipped with an artificial neural network (ANN) pre-trained through machine learning and can detect the behavior of objects and recognize dangerous situations based on thermal images.

Artificial neural networks (ANNs) can be implemented in software form or in hardware form such as chips. For example, the server control unit 430 may include a deep neural network (DNN) such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and DBN (Deep Belief Network) learned through deep learning. You can.

Deep learning can represent a set of machine learning algorithms that extract key data from a plurality of data by sequentially going through hidden layers.

The deep learning structure can be composed of deep neural networks (DNN) such as CNN, RNN, and DBN.

A deep neural network (DNN) may include an input layer, a hidden layer, and an output layer.

One with multiple hidden layers can be called a Deep Neural Network (DNN). Each layer contains multiple nodes, and each layer is related to the next layer. Nodes can be connected to each other with weights.

The output from any node belonging to the first hidden layer (Hidden Layer 1) becomes the input to at least one node belonging to the second hidden layer (Hidden Layer 2). At this time, the input of each node may be a weight applied to the output of the node of the previous layer.

Weight may refer to the strength of the connection between nodes. The deep learning process can also be seen as a process of finding appropriate weights.

Meanwhile, learning of an artificial neural network can be accomplished by adjusting the weight of the connection lines between nodes to produce a desired output for a given input. Additionally, artificial neural networks can continuously update weight values through learning.

If the server control unit 430 recognizes that an object is in a dangerous situation, it may transmit a notification to the user terminal 500 and/or a related organization server (not shown) through the server communication unit 410.

Figure 5 is an example diagram showing a thermal image obtained from a thermal imaging camera according to an embodiment of the present invention.

Referring to FIG. 5 , a thermal image of an object acquired by the thermal imaging camera 100 may be displayed on a display connected to the thermal imaging camera 100 by wire or wirelessly.

When a thermal image is displayed on a display, colors depending on the temperature of each part of the object may be displayed. Areas with high and low temperatures may be displayed in different colors.

Among the thermal image images illustrated in Figure 5, (a) represents a person raising a hand, (b) represents a person sitting on a chair, and (c) represents a person sitting on the floor.

The server control unit 430 can detect the behavior of an object using values corresponding to colors according to the temperature of each part of the object in the thermal image.

FIG. 6A is an example of converting the temperature of a plurality of cells constituting the thermal image of the object shown in (b) of FIG. 5 into color values.

FIG. 6B is an example of converting the temperature of a plurality of cells constituting the thermal image of the object shown in (c) of FIG. 5 into color values.

A thermal image may consist of a plurality of cells. Additionally, a color corresponding to the temperature of the object may be displayed in each cell. Different temperatures are displayed in different colors. The higher the temperature in the cell, the relatively brighter the color may be displayed.

The color value converted to the drawing may be the relative brightness value of the color corresponding to the temperature of the object. That is, the higher the temperature, the brighter the color, and the brighter the color, the higher the brightness value can be set.

The color of the cell can be displayed as a combination of red (R), green (G), and blue (B). For example, R, G, and B can each have color values between 0 and 254, and these color values can be the brightness value of each cell. The brighter it is, the closer it is to 254, and the darker it is, the closer it is to 0.

The brightness value of the color can be set to an arbitrary range. In this embodiment, the color brightness value for each cell is classified into levels 0 to 100.

The color in the cell corresponding to the area where the temperature of the object is relatively high is set to a relatively large brightness value, and conversely, the color in the cell corresponding to the area where the temperature of the object is relatively low is set to a relatively small brightness value. will be.

Looking at the example of FIGS. 6A and 6B, the thermal image consists of 10×15 cells.

Of course, this is only an example, and may be composed of M×N (M, N ≥ 2 integer) cells depending on the performance of the thermal imaging camera 100 and the resolution of the thermal imaging image.

On the left, the thermal image displays colors with different brightness depending on the temperature of the object, and on the right, in the 10×15 cells corresponding to the thermal image on the left, the temperature corresponds to the brightness of the color for each cell corresponding to the object part. An example of conversion to a brightness value is shown.

In Figure 6a, the brightness value of the darkest part is 0 and the brightness value of the brightest part is 81. In Figure 6b, the brightness value of the darkest part is 0 and the brightness value of the brightest part is 79.

Below, the operation of recognizing object behavior and dangerous situations in the server according to the present invention will be described with reference to FIGS. 6A and 6B.

In the server 400, the server control unit 430 may extract a region of interest from the thermal image. A region of interest may be an area set to determine a specific part of an object.

In this embodiment, the region of interest may be the upper body of the object, preferably the face. In another embodiment, it may be the brightest part of the thermal image, or an area including the brightest part and surrounding areas.

The server control unit 430 can calculate a representative brightness value in the region of interest. In one embodiment, when the region of interest corresponds to one cell, the brightness value of that cell can be calculated as the representative brightness value. In another embodiment, when the region of interest corresponds to two or more cells, the average of the brightness values of each cell can be calculated as the representative brightness value, or the highest brightness value can be calculated as the representative brightness value.

In FIG. 6A, the region of interest in the thermal image may be the brightest M region, and the region of interest in the corresponding cell may be the N region corresponding to the M region.

Since there are 4 cells in the N area, the representative brightness value of the N area can be 70.5 (=(55+79+67+81)/4), which is the average value of the brightness values of the 4 cells, or alternatively, the 4 cells The largest brightness value, 81, may be a representative brightness value.

If the object does not move, the brightness value in area N will not change, and even if it changes, the amount of change will be small.

When the object moves from the posture in FIG. 6A to the posture in FIG. 6B, the region of interest may change from area N to area N'. In FIG. 6B, the region of interest in the thermal image may be the brightest M' region, and the region of interest in the cell corresponding to this may be the N' region corresponding to the M' region.

When the representative brightness value is set as the average value, the representative brightness value in the N' area is 46, and when the representative brightness value is set as the largest brightness value, the representative brightness value in the N' area is 71.

The server control unit 430 can detect the behavior of an object by detecting a change in the area of interest from the N area to the N' area, and can also detect a dangerous situation of the object.

When the area of interest moves, the object's action can be detected. In other words, when the object's face moves from area N to area N', it can be detected that movement of the object has occurred.

Meanwhile, the server control unit 430 can determine whether an object is in danger using characteristics according to the movement of the area of interest.

In this embodiment, the server control unit 430 determines (1) the moving speed of the area of interest, (2) the moving distance of the area of interest, (3) the change speed of the representative brightness value of the previous area of interest, and (4) the moved interest. The danger situation of an object can be determined using the duration of the representative brightness value of the area.

(1) Movement speed of area of interest

When a dangerous situation occurs in an object, the speed at which the area of interest moves can be relatively fast compared to normal, everyday movement. For example, if an object suddenly falls or suddenly sits down, the region of interest will move quickly.

The server control unit 430 may recognize a dangerous situation when the area of interest moves faster than a certain speed. That is, if the speed at which the area of interest moves from area N to area N' is greater than or equal to the preset first reference value (first condition), it can be recognized that a dangerous situation has occurred in the object.

(2) Movement distance of area of interest

Additionally, when a dangerous situation occurs in an object, the moving distance of the area of interest may be relatively long compared to normal, everyday movement. For example, when an object suddenly falls while standing, the distance between the initial area of interest (e.g. area N) and the moved area of interest (e.g. area N') increases.

The server control unit 430 may recognize a dangerous situation when the moving distance between areas of interest is more than a certain distance. In other words, if the distance that the area of interest moves from area N to area N' is more than the preset second standard value (second condition), it can be recognized that a dangerous situation has occurred in the object.

(3) Speed of change of representative brightness value of previous area of interest

Additionally, if a dangerous situation occurs in an object, the area of interest can move quickly. As in the example of FIGS. 6A and 6B, when an object stands and then falls down, the region of interest moves from area N to area N'.

When the representative brightness value is set as the value with the largest brightness value, the representative brightness value of area N, which is the area of interest, in Figure 6a where the object is standing is 81. In Figure 6b, where the object is sitting down, the area of interest moves to area N', and the representative brightness value of area N, which was the area of interest just before, becomes 8.

When area N was an area of interest, the representative brightness value was 81, but suddenly changed to 8. This means that there was movement of the object, and when a dangerous situation occurs in the object, the movement becomes faster.

The server control unit 430 can recognize a dangerous situation of an object by detecting that the representative brightness value in the area of interest increases beyond a certain speed. In other words, if the rate of change of the representative brightness value in area N, which was the area of interest, is more than the preset third standard value (third condition), it can be recognized that a dangerous situation has occurred in the object.

Preferably, if the deceleration speed of the representative brightness value in the area of interest is greater than or equal to the third standard value, it can be recognized that a dangerous situation has occurred in the object.

(4) Duration of representative brightness value of the moved region of interest

Meanwhile,

conditions

1, 2, and 3 can occur not only when a dangerous situation occurs in an object, but also when the object moves quickly for another purpose. For example, even when an object quickly moves from sitting to standing, any one of the first, second, and third conditions may be satisfied. In this case, there is a risk that the reliability of the object's recognition of dangerous situations may decrease.

Accordingly, in the present invention, reliability can be increased in determining whether a dangerous situation has occurred in an object by using the duration of the representative brightness value in the changed area of interest.

In other words, the fact that the representative brightness value of the moved region of interest persists for more than a certain period of time may mean that there is no movement for more than a certain period of time after the object moves.

For example, if a person falls from standing, the area of interest may move from above to below. After the region of interest moves downward, if there is no change in the representative brightness value in the moved region of interest or if the representative brightness value persists for more than a certain period of time, it may mean that there is no movement in the object. This can be a dangerous situation for the object.

Accordingly, the server control unit 430 can determine whether the representative brightness value in the moved area of interest continues for more than a certain period of time and recognize the dangerous situation of the object. In other words, if the time for which the representative brightness value lasts in area N', which is the moved area of interest, is more than the preset fourth standard value (fourth condition), it can be recognized that a dangerous situation has occurred in the object.

Meanwhile, in the present invention, the server 400 can recognize that a dangerous situation has occurred in an object when at least one of the first, second, third, and fourth conditions is satisfied.

Alternatively, the server 400 may recognize that a dangerous situation has occurred in an object when at least one of the first, second, and third conditions and the fourth condition are simultaneously satisfied.

For example, if an object falls and cannot move, the fourth condition is satisfied, so if it is determined that a dangerous situation has occurred in the object in this case, accuracy will be improved.

However, if at least one of the first, second, and third conditions is satisfied, a quick decision can be made if the dangerous situation of the object is determined.

Therefore, it is possible to determine whether the object is in a dangerous situation by appropriately combining the first to fourth conditions according to the location where the thermal imaging camera 100 is installed and the behavioral characteristics of the object.

Referring to FIG. 7, the object behavior detection method according to the present invention acquires a thermal image of the object by thermal imaging cameras 100 installed in a plurality of spaces (S101). Acquisition of thermal imaging images can be done in real time or at set intervals.

The thermal imaging camera 100 transmits each acquired thermal imaging image to a specific thermal imaging camera 100 connected to the gateway 200 through mesh Wi-Fi communication (S102).

A specific thermal imaging camera 100 transmits a thermal imaging image acquired by itself and a thermal imaging image transmitted from another thermal imaging camera 100 to the server 400 through the gateway 200 and the network 300 (S103) ).

The server 400 analyzes the received thermal image (S104) and determines whether at least one of the first, second, third, and fourth conditions occurs (S105).

If at least one of the first to fourth conditions occurs, the server 400 recognizes that a dangerous situation has occurred in the object (S106). If the above condition does not occur, the server 400 proceeds to step S101 again and repeats the subsequent process.

The server 400 may transmit the occurrence of a dangerous situation of an object to the user terminal 500 and/or a related agency server (S107).

As described above, in the present invention, it is possible to determine whether an object is in danger by capturing a thermal image of an object with a thermal imaging camera and receiving and analyzing the thermal image from a server. In addition, in the present invention, since a plurality of thermal imaging cameras can communicate with each other through mesh Wi-Fi communication, all thermal imaging images can be quickly collected and transmitted to the server, which has the advantage of faster processing speed.

Although embodiments of the present invention have been described above with reference to the attached drawings, the present invention is not limited to the above embodiments and can be manufactured in various different forms, and can be manufactured in various different forms by those skilled in the art. It will be understood by those who understand that the present invention can be implemented in other specific forms without changing its technical spirit or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims

A thermal imaging camera that includes a communication module and is installed in each of a plurality of spaces and acquires thermal imaging images of objects in the installed spaces;

It includes a server that receives a thermal image from the thermal image camera and analyzes the received thermal image to determine a dangerous situation of the object,

The server is,

Extracting a region of interest from the thermal image, calculating a representative brightness value in the region of interest by setting a brightness value corresponding to the color according to the temperature of the object in a plurality of cells constituting the thermal image, ( At least one of the following: 1) the moving speed of the region of interest, (2) the moving distance of the region of interest, (3) the change rate of the representative brightness value of the immediately moved region of interest, and (4) the duration of the representative brightness value of the currently moved region of interest. An object behavior detection system that determines whether the object is in danger or not.
The object behavior detection system according to claim 1, wherein the region of interest includes a facial area of the object.
The object behavior detection system according to claim 1, wherein when the region of interest corresponds to one cell, the brightness value of the one cell is calculated as the representative brightness value.
The object behavior detection system according to claim 1, wherein, when the region of interest corresponds to two or more cells, an average of the brightness values of the two or more cells is calculated as the representative brightness value.
The object behavior detection system according to claim 1, wherein when the region of interest corresponds to two or more cells, the representative brightness value is calculated as the largest brightness value among the brightness values of the two or more cells.
The object behavior detection system according to claim 1, wherein the communication modules of the plurality of thermal imaging cameras transmit and receive the thermal images through mesh Wi-Fi communication.
The object detection system according to claim 1, wherein at least one of the plurality of thermal imaging cameras is connected to a gateway and transmits the thermal imaging image to the server through the gateway.
The object behavior detection system according to claim 1, wherein the server recognizes that a dangerous situation has occurred in the object when the moving speed of the area of interest is greater than or equal to a preset first standard (first condition).
The object behavior detection system according to claim 1, wherein the server recognizes that a dangerous situation has occurred in the object when the moving distance between the areas of interest is greater than a preset second standard (second condition).
The object behavior detection system according to claim 1, wherein the server recognizes that a dangerous situation has occurred in the object when the rate of change of the representative brightness value in the immediately preceding region of interest is greater than or equal to a preset third standard (third condition).
The object behavior detection system according to claim 1, wherein the server recognizes that a dangerous situation has occurred in the object when the duration of the representative brightness value in the moved region of interest is greater than or equal to a preset fourth standard value (fourth condition).
The method of claim 1, wherein the server determines whether the speed at which the region of interest moves is greater than or equal to a preset first reference value (first condition), the moving distance between the regions of interest is greater than or equal to a preset second standard value (second condition), and the immediately preceding The rate of change of the representative brightness value in the area of interest satisfies at least one of the preset third reference values or more (third condition), and the duration of the representative brightness value in the moved area of interest is more than the preset fourth reference value. In this case (fourth condition), an object behavior detection system recognizes that a dangerous situation has occurred in the object.