WO2023129098A1

WO2023129098A1 - Post-earthquake disaster management system and method facilitating search and rescue activities

Info

Publication number: WO2023129098A1
Application number: PCT/TR2022/051701
Authority: WO
Inventors: Deniz DAL; Mert AKSOY; Sema BAYRAK; İlhami BİLİCİ
Original assignee: Atatürk Üni̇versi̇tesi̇ Rektörlüğü Bi̇li̇msel Araştirma Projeleri̇ ( Bap ) Koordi̇nasyon Bi̇ri̇mi̇
Priority date: 2021-12-30
Filing date: 2022-12-29
Publication date: 2023-07-06
Also published as: TR2021021934A2

Abstract

A system that facilitates/accelerates post-earthquake disaster management and search and rescue activities, is based on artificial intelligence (machine learning), characterized in that, it comprises of the following; single-card computers (1) that are placed at appropriate points at the entrance and exit of settlements/buildings and that enable the processing of images from cameras (2) and the identification of individuals, motion detection sensors (4) that detect the presence of people entering/exiting the residential building, cameras (2) that are integrated into single-card computers (1) and that enable to obtain the images of people entering/leaving the relevant residential building, cloud database (5) that stores information about people whose faces are detected and recognized with single-card computers (1), as well as the floor and flat information of these people, web application (6) that runs the program that enables remote access to single-board computers (1) and cloud database (5) and attaches the data from the cloud database (5) to the 3D residential building model (8), mobile application (7) that delivers the 3D residential building model (8) to the disaster management authorities and search and rescue teams after the earthquake, 3D residential building model (8) that contains the floor and flat plans in the residential building where the system is active, visualizes information on how many people are on which floors and which apartments of the building, as well as how many people are in the building in total.

Description

Post-Earthquake Disaster Management System and Method Facilitating Search and Rescue Activities

Field of the Invention

The present invention relates to a system that facilitates/accelerates post-earthquake disaster management and search and rescue activities, is based on artificial intelligence (machine learning) and benefiting from image processing, edge computing, cloud computing, embedded system, data visualization, web and mobile application technologies.

State of the Art

It is reported according to the Turkey Earthquake Map that 92% of our country is in earthquake zones, 95% of our population lives under earthquake risk, and 98% of large industrial centers and 93% of our dams are also located in earthquake zones. Therefore, earthquakes are an inevitable reality of our country. On the other hand, it is known that many countries of the world also have earthquake zones.

As well as trying to reduce the damage caused by an earthquake before it strikes, it is also very important to maintain the disaster management, search and rescue activities in an organized manner as much as possible in the early post-earthquake period in order to reduce the loss of life. Likewise, past experiences indicate that the first 48 hours after the earthquake is very critical for human life.

It is essential that people who are get trapped under wreckage after an earthquake are rescued as soon as possible because the information obtained from the past earthquakes indicates that the rate of disaster victims who were taken alive from the wreckage after 24-48 hours after the earthquake disaster is gradually decreasing. For example, only 2 of the 235 disaster victims who were pulled out of the wreckage after the second day survived in the 1992 Luzon Earthquake in the Philippines. No living things were rescued from the wreckage 4 days after the earthquake in the Southern Italy Earthquake in 1983. In addition, it is also reported that there is an increase in the rate of loss of life, injury and debris in areas where adequate numbers of crews are not dispatched or cannot be deployed due to the insufficient organization of search and rescue teams. The knowledge of who is under the wreckage of a building after the earthquake is of vital importance in terms of using search and rescue teams in an optimum way and requesting auxiliary teams from neighbouring provinces when necessary. In the current situation, the authorities of the disaster management and coordination unit can access this information by listening to the voices coming from under the wreckage, the residents of the building who can go out of the building, and if any, the apartment officials, the residents of the neighbouring buildings and the neighbourhood headman, as well as the address-based population registration system inquiries. On the other hand, this information alone is not sufficient because it must be clarified whether a resident of the building was in the building at the time of the earthquake.

The first and simplest method that can be considered so as to solve the problem detailed above is to determine the number of individuals entering and exiting the building with the obstacle detection sensors to be placed at the building entrances and storing this information for use after a possible earthquake. This simple method has two major drawbacks: (1 ) this method has no function for identifying the people in the building. (2) This method does not contain any clues about the distribution of human density to the floors. However, search and rescue operations differ according to the floors in terms of entry to the wreckage and cleaning of the debris.

As a result of the research on the subject, the application numbered JP2020154574A and titled SAFETY INFORMATION MANAGEMENT SYSTEM was encountered. In this application, a structure where the images of people entering and leaving a residential building are taken with the help of a camera, these images are processed, and people are identified and recorded, is disclosed. Another document found as a result of the research on the subject is the application numbered KR1020160109516A and titled RESCUE OBJECT GRASP SYSTEM AND METHOD. In this application, a camera which allows to take the images of people entering and leaving the building is disclosed. The images taken with this camera are processed and if there is an entrance to the building, a counter is increased. When exiting the building, the relevant counter is reduced, and thus, in case of a disaster, it is determined how many people would have left the building by looking at the number of people the counter stores. On the other hand, there is no mention of a feature that visualizes the people in the building and their numbers on a 3D model specific to floors and apartments in both documents. In addition, there is no mention of a mathematical model which will take the data of the number of individuals on different floors and flats of the building obtained from the 3D model after the earthquake as an input to itself and will calculate the number of teams required for search and rescue with a value called the disaster score as an output in these applications. Likewise, since there is no 3-dimensional residential building model in these applications, there are no floor and flat plans integrated into this model, which will accelerate search and rescue activities.

As a result, due to the abovementioned disadvantages and the insufficiency of the current solutions regarding the subject matter, a development is required to be made in the relevant technical field.

Aim of the Invention

The invention aims to provide a structure having different technical features which brings a new development in this field different from the embodiments used in the state of the art.

The main aim of the present invention is to reveal a system which is based on artificial intelligence (machine learning) and benefiting from image processing, edge computing, cloud computing, embedded system, data visualization, web and mobile application technologies which will accelerate/facilitate post-earthquake disaster management and search and rescue activities based on the fact that search and rescue is a war against time, and with the motivation detailed in the paragraphs above.

The aim of the invention is to visualize in 3D how many residents are in a residential building at any given time, in terms of floors and flats on floors. Thus, very valuable information that will guide search and rescue activities is produced instantly and automatically. In addition, the number of teams that need to participate in search and rescue activities in the building related to the disaster score of a collapsed building can be determined by inputting the data stored by the invention into a mathematical model.

It is possible to save the people who are under the wreckage in the earthquake more quickly and thus to reduce the loss of life with the help of the invention.

Project output is minimal cost and easy to use. There is no extra equipment to be placed in residential buildings, except for the edge devices that are placed at the entrances and exits of the buildings and contain cameras. Likewise, the residents living in the residential building do not participate in the operation of the system. For example, residents do not have to constantly carry a smart watch or a similar special device with them. In addition, there is no need to use an RFID card or a special key to detect entrance or exit to the building.

All information is stored in the cloud so as to prevent damage to single-card computers and therefore the data on these cards in case of an earthquake and thus system security is kept at the highest level.

The present invention, besides facilitating post-earthquake search and rescue activities, also has the potential to provide great benefits in prioritizing and coordinating evacuation and fire response efforts in the event of a fire.

Personal data refers to any information relating to an identified or identifiable natural person. Some sensitive data, such as biometric and genetic data, are called special personal data. The Law on the Protection of Personal Data No. 6698 (KVKK) regulates the fundamental rights and freedoms of individuals, especially the privacy of private life, in the processing of personal data, and the obligations of natural and legal persons who process personal data, as well as the procedures and principles to be followed. One of the research areas that will be focused on within the scope of the invention is face detection and recognition. Since the face is a biometric data, it is possible to record and process personal data within the scope of the invention and in the meanwhile, it is necessary to act with care.

Personal data can be processed under different conditions. One of them is the acquisition and processing of data with explicit consent. Explicit consent refers to consent on a specific subject, based on information and expressed with free will. Active consent is used within the scope of this invention and the active consent of all residents of the residential buildings where the invention will be activated is obtained through an informative text and a signed consent form. Only the data of the residents whose active consent is obtained are used within the scope of the invention. Therefore, residents who do not express their consent are evaluated in the category of guests pointing to people whose location is unknown in the building.

In order to fulfil the above-mentioned aims, the present invention is a system that facilitates/accelerates post-earthquake disaster management and search and rescue activities, is based on artificial intelligence (machine learning), characterized in that, it comprises of the following; • Single-card computers, which are placed at appropriate points at the entrance and exit of settlements/buildings, and which enable the processing of images from cameras and the identification of individuals,

• Motion detection sensors that detect the presence of people entering/exiting the residential building,

• Cameras that are integrated into single-card computers and that enable to obtain the images of people entering/leaving the relevant residential building,

• Wide-angle lenses that deepen the viewing angles of the cameras,

• Cloud database that stores information about people whose faces are detected and recognized with single-card computers, as well as the floor and flat information of these people,

• Web application that runs the program that enables remote access to single-board computers and cloud database and attaches the data from the cloud database to the 3D residential building model,

• Mobile application that delivers the 3D residential building model to the disaster management authorities and search and rescue teams after the earthquake,

• 3D residential building model that contains the floor and flat plans in the residential building where the system is active, visualizes information on how many people are on which floors and which apartments of the building, as well as how many people are in the building in total.

The structural and characteristic features of the present invention will be understood clearly by the following drawings and the detailed description made with reference to these drawings and therefore the evaluation shall be made by taking these figures and the detailed description into consideration.

Figures Clarifying the Invention

Figure 1 , is the general view of the inventive system.

Figure 2, is an illustrative view of the inventive 3D residential building model.

The figures are not required to be scaled and the details which are not necessary for understanding the present invention may be neglected. Moreover, the elements that are at least substantially identical or have at least substantially identical functions are shown by the same number. Description of the Part References

1 . Single-Board Computer

2. Camera

3. Wide-Angle Lens

4. Motion Detection Sensor

5. Cloud Database

6. Web Application

7. Mobile application

8. 3D Residential Building Model

8.1 Number of People in the Residential Building

8.2 Flats with At Least One Person in the Residential Building

Detailed Description of the Invention

In this detailed description, the preferred embodiments of the invention are described only for clarifying the subject matter in a manner such that no limiting effect is created.

In the inventive system, two palm-sized single-board computers (1) to be placed at appropriate points at the entrance and exit of the residential buildings are used as edge computing devices (the device that processes data at the edge without sending it to a server). These single-board computers (1), which also have internet access, run a Linuxbased operating system. Motion is detected, faces are detected, faces are recognized and data is transferred to the cloud database (5) with the help of these computers.

Images of people entering/leaving the relevant residential building are obtained by means of high-resolution cameras (2) integrated into single-board computers (1 ). The viewing angle of the cameras (2) can be deepened with the help of the wide-angle lenses (3) integrated into the high-resolution cameras (2). The presence of people entering/leaving the residential building is primarily detected by motion detection sensors (4) and the image acquisition and processing with the cameras (2) is then carried out.

Information of the people whose faces are detected and recognized by means of single-card computers (1 ) are stored in an encrypted form in the cloud database (5), specific to the floors they live in and the flats on the floors.

The web application (6) is used to remotely access the single-board computers (1), the cloud database (5) and to run the program that will attach the data in the cloud database (5) to the 3D residential building model (8).

The mobile application (7) is used to deliver the 3D residential building model (8) to the disaster management authorities and search and rescue teams after the earthquake. The mobile application (7) works on mobile devices such as smartphones and/or tablets.

3D residential building model (8) is a 3D model that contains the floor and flat plans in the residential building where the system is active, visualizes information on how many people are on which floors and which apartments of the building, as well as how many people are in the building in total.

The operating principle of the inventive system is as follows:

The present invention includes two single-board computers (1) to be placed at the main entrance and exit points in residential buildings. High-resolution cameras (2) and motion detection sensors (4) are integrated into these single-board computers (1). Wide-angle lenses (3) are attached to high-definition cameras (2). The faces of the people living on each floor and in each flat on each floor of the relevant residential building are stored on singlecard computers (1) and their information is encrypted in the cloud database (5). A machine learning model is trained with face images and this model runs on single-board computers (1). The presence of a person entering the building is primarily detected by the motion detection sensor (4). Afterwards, the single-board computer (1) takes an image of the person entering the viewing angle of the high-resolution camera (2). First of all, the face of the person in the image is detected, and then the face is recognized by running a pretrained machine learning model. After this process, the value of the counter that stores the number of individuals in the flat where the relevant resident lives is increased by 1 . This data is transferred to the cloud database (5) by a single-board computer (1 ) and the necessary update is made in the relevant database table in encrypted form. The opposite of this process is realized with the help of the second single-board computer (1 ), wide-angle camera (2) and motion detection sensor (4) at the exit from the building. Therefore, the information of how many people are in each floor and flat of the residential building where the system is active at any time can be accessed via the cloud database (5). The web application (6) and mobile application (7) developed within the scope of the invention can access the data stored in the cloud database (5). Both applications (5, 6) visualize the data they have taken from the cloud database (5) by attaching them to the 3D residential building model (8) of the building whenever requested and present the same to the user. The windows of the flats (8.2) with people in the residential building are painted in red and the number of people is written thereon on the 3D residential building model (8). At the same time, the total number of people in the building (in the category of people whose location is known and guests whose location is unknown) is reflected on the top (8.1 ) of the 3D residential building model (8). Faces that cannot be recognized with the face recognition algorithm are included in the category of guests pointing to people whose location is unknown in the building. For example, in the building at the bottom left in Figure 2, there are a total of 9 people, including 8 residents whose locations are known, and 1 guest whose location is unknown. This building has flats with windows painted red (8.2) or unpainted. Flats with red windows (8.2) indicate flats containing at least one person. The numbers on the window indicate how many people are in the relevant flat.

A score, which can also be called the disaster score of a building, is calculated within the scope of the invention, and this score is implemented as a mathematical function. This function takes the floor and location information of the flats marked in red (8.2) in the 3D residential building model (8) and the number of people in these flats as input to itself and returns the disaster score and the number of experts required for search and rescue as output. Therefore, it will be possible to instantly determine how many people in a search and rescue team is needed for all buildings in which the invention is active in the event of an earthquake, if necessary, assistance can be requested from the surrounding provinces.

The 3D residential building model (8) also includes the floor and flat plans of the building. Thus, for example, in an earthquake that occurs at an hour when people are asleep, in the apartments where the output of the invention indicates that there are people, search and rescue operations can be directed to the areas where the bedrooms are located. Therefore, disaster management operations can be customized on a location-based basis in floors, flats and apartments.

If the output of the invention, for example, indicates that only the 7th floor of an 8-storey residential building collapses during the earthquake, search and rescue operations will continue until they reach the relevant floor and will be terminated afterwards. Thus, a very effective and organized disaster management can be implemented by directing the team to another collapsed building.

The processing steps carried out with the inventive system is as follows;

• Activating the relevant single-board computer (1 ) when motion is detected by the motion detection sensor (4) at the entrance of the settlement/building,

• Taking the image of the person entering the view of the high-resolution camera (2) and storing the same encrypted on a single-card computer (1 ) for processing,

• Detecting the face of the person whose image is taken with the running face detection algorithm on the single-board computer (1),

• Recognizing the face of the person whose image is taken with a machine learning model running on the single-board computer (1),

• Detecting the person whose face is recognized, on which floor and in which flat of the residential building he/she lives, by the single-card computer (1),

• Increasing or decreasing the number of people by 1 in the apartment where the person whose face is recognized lives, according to the entrance or exit status of the building, then transferring this information to the cloud database (5),

• Remotely connecting and interfering with single-board computers (1 ) with the web application (6),

• Remotely connecting and interfering with the cloud database (5) with the web application (6),

• T ransferring data from the cloud database (5) to the web application (6),

• Visualizing the data transferred from the cloud database (5) to the web application (6) by attaching the same to the 3D residential building model (8),

• Visualizing the 3D residential building model (8) on the mobile application (7),

• Calculating the disaster score of the building with a mathematical model that uses the human density data specific to floors and flats obtained from the 3D residential building model (8) as input, • Calculating the number of teams that will participate in the search and rescue activities in the building with a mathematical model that uses the human density data specific to floors and flats obtained from the 3D residential building model (8) as an input, • Facilitating the work of teams performing search and rescue activities with the help of the floor and flat plans integrated into the 3D residential building model (8).

Claims

CLAIMS A system that facilitates/accelerates post-earthquake disaster management and search and rescue activities, is based on artificial intelligence (machine learning), characterized in that, it comprises of the following;

• Single-card computers (1 ), which are placed at appropriate points at the entrance and exit of settlements/buildings, and which enable the processing of images from cameras (2) and the identification of individuals,

• Motion detection sensors (4) that detect the presence of people entering/exiting the residential building,

• Cameras (2) that are integrated into single-card computers (1 ) and that enable to obtain the images of people entering/leaving the relevant residential building,

• Wide-angle lenses (3), which are integrated into the cameras (2) and allow the cameras (2) to be deepened,

• Cloud database (5) that stores information about people whose faces are detected and recognized with the single-card computers (1 ), as well as the floor and flat information of these people,

• Web application (6) that runs the program that enables remote access to single-board computers (1 ) and cloud database (5) and attaches the data from the cloud database (5) to the 3D residential building model (8),

• Mobile application (7) that delivers the 3D residential building model (8) to the disaster management authorities and search and rescue teams after the earthquake,

• 3D residential building model (8) that contains the floor and flat plans in the residential building where the system is active, visualizes information on how many people are on which floors and which apartments of the building, as well as how many people are in the building in total. A method that facilitates/accelerates post-earthquake disaster management and search and rescue activities, is based on artificial intelligence (machine learning), characterized in that, it comprises of the following processing steps;

• Taking the image of the person entering the view of the camera

(2) and storing the same encrypted on the single-card computer (1 ) for processing,

• Detecting the face of the person whose image is taken with the running face detection algorithm on the single-board computer (1 ), • Recognizing the face of the person whose image is taken with a machine learning model running on the single-board computer (1 ),

• Detecting the person whose face is recognized, on which floor and in which flat of the residential building he/she lives, by the single-card computer (1 ),

• Remotely connecting and interfering with the single-board computers (1 ) with the web application (6),

• T ransferring data from the cloud database (5) to the web application (6),

• Visualizing the 3D residential building model (8) on the mobile application (7).

3. Method according to claim 2, characterized in that; it shows the total number of people in the building, including those whose location is known and those in the guest category whose location is unknown at the top (8.1 ) of the 3D residential building model (8).

4. Method according to claim 2, characterized in that; it colors the flats (8.2) in the 3D residential building model (8) with red if there is at least one person in the flat and reflecting the number of people thereon.

5. Method according to claim 2, characterized in that; it comprises the processing step of calculating the disaster score of the building with a mathematical model that uses the human density data specific to floors and flats obtained from the 3D residential building model (8) as input.

6. Method according to claim 2, characterized in that; it comprises the processing step of calculating the number of teams to participate in the search and rescue activities in the building with a mathematical model that uses the human density data specific to floors and flats obtained from the 3D residential building model (8) as an input.

7. Method according to claim 2, characterized in that; it comprises the processing step of guiding the teams performing search and rescue activities with the floor and flat plans integrated into the 3D residential building model (8).

14