KR20230035814A - Apparatus and method for detecting disaster based on artificial intelligence model - Google Patents

Abstract

The present disclosure relates to a method for detecting a disaster and an electronic device for performing the same. According to an embodiment, a method for detecting a disaster by an electronic device comprises the steps of: obtaining an image by photographing an area to be monitored; obtaining seismic wave information by sensing seismic waves at a location in which the electronic device is installed using at least one sensor; determining at least one fire index based on the obtained image analysis result; obtaining a fire occurrence probability value for the image from an artificial intelligence model by inputting the obtained image into the artificial intelligence model which outputs a probability value for whether or not a fire occurs when the image is input; determining whether the fire has occurred based on the determined fire index and the fire occurrence probability value; determining at least one seismic index based on the seismic wave information and determining whether an earthquake will occur based on the determined seismic index; and outputting notification information depending on whether the fire occurs or the earthquake occurs. According to the present invention, notification information can be effectively output when the disaster occurs.

Description

Disaster detection method based on artificial intelligence model and electronic device performing it {APPARATUS AND METHOD FOR DETECTING DISASTER BASED ON ARTIFICIAL INTELLIGENCE MODEL}

The present disclosure relates to a method for detecting a disaster and an electronic device for performing the same. More specifically, it relates to a method for detecting fire and earthquake and an electronic device for performing the same.

Recently, various disasters have occurred due to environmental issues such as climate change. In order to reduce the damage caused by these disasters, various disaster monitoring technologies are being developed. In particular, in modern society, the density of life-related facilities such as houses, companies, buildings, and factories increases as the population increases, and as the density increases, the scale of damage caused by disasters also increases.

Therefore, there is a demand for the development of technology that can be easily installed anywhere and can easily and accurately detect disasters such as earthquakes and fires.

Korean Patent Publication No. 2019-0142843

According to an embodiment, a method for detecting a disaster and an electronic device performing the same may be provided.

More specifically, an electronic device capable of detecting a fire or an earthquake based on an image analysis result and seismic wave information may be provided.

As a technical means for achieving the above-described technical problem, according to an embodiment, a method for detecting a disaster by an electronic device includes acquiring an image by photographing an area to be monitored; obtaining seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor; determining at least one fire index based on the obtained image analysis result;

obtaining a fire occurrence probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether a fire occurs when an image is input;

determining whether a fire occurs based on the determined fire index and the fire occurrence probability value; determining at least one earthquake index based on the seismic wave information, and determining whether an earthquake has occurred based on the determined earthquake index; and outputting notification information according to whether the fire or the earthquake has occurred. can include

As a technical means for achieving the above technical problem, according to another embodiment, a camera; network interface; a memory that stores one or more instructions; and at least one processor executing the one or more instructions; The at least one processor obtains an image by photographing an area to be monitored by executing the one or more instructions, and obtains seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor. By inputting the obtained image to an artificial intelligence model that acquires, determines at least one fire index based on the obtained image analysis result, and outputs a probability value for whether a fire occurs when the image is input, the artificial intelligence obtaining a fire occurrence probability value for the image from a model, and determining whether a fire occurs based on the determined fire index and the fire occurrence probability value; An electronic device that determines at least one earthquake index based on the seismic wave information, determines whether an earthquake has occurred based on the determined earthquake index, and outputs notification information according to whether the fire or the earthquake has occurred It can be.

According to another embodiment for achieving the above technical problem, a method for detecting a disaster by an electronic device includes acquiring an image by photographing an area to be monitored; obtaining seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor; determining at least one fire index based on the obtained image analysis result; obtaining a fire occurrence probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether a fire occurs when an image is input; determining whether a fire occurs based on the determined fire index and the fire occurrence probability value; determining at least one earthquake index based on the seismic wave information, and determining whether an earthquake has occurred based on the determined earthquake index; and outputting notification information according to whether the fire or the earthquake has occurred. A computer-readable recording medium in which a program for performing the method including a may be provided.

According to an embodiment, it is possible to effectively predict whether an earthquake or a fire will occur by using a camera device.

According to an embodiment, notification information can be effectively output when a disaster occurs.

1 is a diagram schematically illustrating an operation of detecting a disaster by an electronic device according to an exemplary embodiment.
2 is a diagram for explaining an operation of monitoring a predetermined surveillance target area by an electronic device according to an exemplary embodiment.
3 is a flowchart of a method for detecting a disaster according to an embodiment.
4 is a diagram for explaining in detail a process of determining whether an earthquake has occurred by an electronic device according to an exemplary embodiment.
5 is a diagram for explaining a process of processing 3-axis acceleration information by an electronic device according to an embodiment.
6 is a diagram for explaining a process of determining a progress based on a maximum ground acceleration by an electronic device according to an exemplary embodiment.
7 is a diagram for explaining a process for identifying an earthquake occurrence location by an electronic device according to an exemplary embodiment.
8 is a diagram for explaining a process in which an electronic device determines whether an earthquake has occurred based on an image analysis result according to another embodiment.
9 is a diagram for explaining in detail a process of determining whether an electronic device generates a fire according to an embodiment.
10 is a diagram for explaining a process of determining a fire candidate area by an electronic device according to an exemplary embodiment.
11 is a diagram for explaining a process of determining at least one fire index by an electronic device according to an exemplary embodiment.
12 is a diagram for explaining a process of determining a zero cross count index by an electronic device according to an exemplary embodiment.
13 is a diagram for explaining a process of determining a high pass filter area index by an electronic device according to an exemplary embodiment.
14 is a diagram for explaining a process of determining a periodicity index by an electronic device according to an exemplary embodiment.
15 is a diagram for explaining a process of determining a motion region index by an electronic device according to an embodiment.
16 is a diagram for explaining a process of determining a moving object index by an electronic device according to an embodiment.
17 is a flowchart of a method of detecting a disaster by an electronic device according to another embodiment.
18 is a block diagram of an electronic device according to an embodiment.
19 is a block diagram of an electronic device according to another embodiment.
20 is a block diagram of a server connected to an electronic device according to an embodiment.
21 is a diagram for explaining a process of performing a method of detecting a disaster by interworking an electronic device and a server according to an embodiment.
22 is a diagram for explaining a process of performing a method of detecting a disaster by interworking an electronic device and a server according to another embodiment.
23 is a diagram for explaining a process of generating learning data for learning an artificial intelligence model by an electronic device according to an embodiment.
24 is a diagram for explaining a process of generating an alarm event by using an artificial intelligence model by an electronic device according to an embodiment.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary according to the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly describe the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram schematically illustrating an operation of detecting a disaster by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may obtain at least one of at least one frame image 102 or seismic wave information 104 . The electronic device 1000 may analyze at least one frame image 102 and determine whether or not a fire has occurred based on the image analysis result. In addition, the electronic device 1000 may acquire seismic wave information 104 and determine whether an earthquake has occurred based on an analysis result of the seismic wave information. When at least one of a fire or an earthquake occurs, the electronic device 1000 may output information about the disaster determined to have occurred as disaster notification information 144 .

According to an embodiment, the electronic device 1000 may output disaster notification information 144 by interworking with the server 2000 . According to an embodiment, the electronic device 1000 may obtain at least one frame image or seismic wave information through the server 2000 . According to an embodiment, the server 2000 may include other computing devices that are communicatively connected to the electronic device 1000 through a network and capable of transmitting and receiving data. According to an embodiment, the network includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, a satellite communication network, and It includes a mutual combination of these, and is a comprehensive data communication network that enables each network constituent entity shown in FIG. 1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile wireless communication network.

According to an embodiment, the electronic device 1000 may be implemented in various forms. For example, the electronic device 1000 described in this specification includes a digital camera, a mobile terminal, a smart phone, a CCTV, a camera device, a laptop computer, a tablet PC, an electronic book terminal, and a digital broadcasting device. There may be terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, MP3 players, etc., but are not limited thereto. According to an embodiment, for convenience, the electronic device 1000 according to the present disclosure may be a computing device including a processor for controlling a CCTV device or a CCT device.

According to an embodiment, the electronic device 1000 may include a camera 110, at least one sensor 112, a network interface 114, a processor 120, and a memory 122. However, the electronic device 1000 is not limited to the above-described configuration, and the electronic device 1000 may include fewer components or may include more configurations in addition to the above-described configurations.

According to an embodiment, the processor 120 may control the overall operation of the electronic device 1000 by executing one or more instructions stored in the memory 122 . According to an embodiment, the camera 110 may acquire a plurality of frame images by capturing a predetermined area to be monitored under the control of the processor 120 . According to an embodiment, the camera 110 may acquire a surveillance image by monitoring a predetermined surveillance target area. An image to be monitored may include a plurality of frame images captured at a preset period.

According to an embodiment, the at least one sensor 112 may include at least one of an image sensor, a 3-axis acceleration sensor, and a geomagnetic sensor. At least one sensor 112 may obtain an image sensor value, 3-axis acceleration information, or geomagnetic information under the control of the processor 120, and transmit the acquired sensor values to the processor 120. there is.

According to an embodiment, the network interface 114 may transmit sensor values acquired by at least one sensor 112 to another electronic device connected to the electronic device. According to another embodiment, the network interface 114 may receive image or seismic wave information from the server 2000 and transmit disaster notification information analyzed by the electronic device 1000 to the server 2000 . Also, the network interface 114 may transmit disaster notification information analyzed by the electronic device to another electronic device connected to the electronic device. In addition, the network interface 114 may further receive seismic wave information and earthquake location information (eg, epicenter information and PS time information) from other electronic devices connected to the electronic device.

According to an embodiment, the disaster notification information 144 output by the electronic device 1000 may include at least one of fire occurrence information and earthquake occurrence information. According to an embodiment, the fire occurrence information may include at least one of fire occurrence location information, fire occurrence scale, emergency relief information according to fire occurrence, and real-time news information regarding fire occurrence. Also, according to an embodiment, the earthquake occurrence information may include at least one of an earthquake occurrence location, an earthquake magnitude, an earthquake intensity, emergency relief information according to an earthquake occurrence, and real-time news information related to an earthquake occurrence.

2 is a diagram for explaining an operation of monitoring a predetermined surveillance target area by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may be a CCTV device installed on a building or a predetermined outer wall 220 . According to an embodiment, the electronic device 1000 may be attached to an outer wall 220 inside a building or a predetermined space to monitor the surveillance target area 230 . The electronic device 1000 may obtain an image including a plurality of frame images at preset intervals by monitoring the surveillance target region.

In addition, the electronic device 1000 may sense seismic wave information according to the occurrence of an earthquake after being installed on a building or a predetermined outer wall 220 . According to an embodiment, the electronic device 1000 may sense vibration of a building or an exterior wall according to a predetermined cycle even when an earthquake does not occur. According to an embodiment, the electronic device 1000 determines the resonance frequency of the building or exterior wall in which the electronic device 1000 is installed by sensing the vibration of the building or exterior wall according to a preset period, and transmits information on the determined resonance frequency. It may be transmitted in advance to another electronic device connected to the electronic device. According to an embodiment, after the electronic device 1000 stores the resonance frequency of the building in which the electronic device 1000 is installed in advance, when seismic wave information corresponding to the resonance frequency is sensed, notification information may be output stronger. .

3 is a flowchart of a method for detecting a disaster according to an embodiment.

In S310, the electronic device 1000 may obtain an image by photographing the area to be monitored. For example, the electronic device 1000 may include a camera device and acquire images at predetermined cycles using the camera. Images obtained by the electronic device 1000 may be one frame image among images composed of a plurality of frames.

In S320, the electronic device 1000 may acquire seismic wave information by sensing seismic waves at a location where the electronic device 1000 is installed using at least one sensor. According to an embodiment, the electronic device 1000 may obtain 3-axis acceleration information and geomagnetism information using at least one of a 3-axis acceleration sensor and a geomagnetism sensor.

In S330, the electronic device 1000 may determine at least one fire index based on the image analysis result. The fire index may be a parameter for an image or video used by the electronic device 1000 to determine whether a fire has occurred.

In S340, when the image is input, the electronic device 1000 obtains a fire probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether or not a fire occurs. can do.

In S350, the electronic device 1000 may determine whether a fire occurs based on at least one of the determined fire index and the fire occurrence probability value. According to an embodiment, the electronic device 1000 determines at least one fire index, identifies whether the determined at least one fire index satisfies each index condition, and then determines whether all index conditions are satisfied. Thus, it is possible to determine whether a fire has occurred. However, according to another embodiment, the electronic device 1000 may determine that a fire has occurred only when all index conditions are satisfied and a fire occurrence probability value output from the artificial intelligence model is greater than or equal to a preset threshold probability value.

In S360, the electronic device 1000 may determine at least one earthquake index based on the seismic wave information, and determine whether an earthquake has occurred based on the determined earthquake index. In S370, the electronic device 1000 may output notification information according to whether a fire or an earthquake has occurred.

4 is a diagram for explaining in detail a process of determining whether an earthquake has occurred by an electronic device according to an exemplary embodiment.

In operation S410, the electronic device 1000 may identify a direction of the geomagnetism at a location where the electronic device is installed based on the geomagnetism information. According to an embodiment, the geomagnetic information may include information about north, south, east, west, and west directions. In S420, the electronic device 1000 may identify the direction of the 3-axis acceleration based on the 3-axis acceleration information. According to an embodiment, the electronic device 1000 may include a 3-axis acceleration sensor and identify a 3-axis acceleration direction based on 3-axis acceleration information obtained from the 3-axis acceleration sensor. According to an embodiment, the 3-axis acceleration direction may be determined based on acceleration values for each axis in 3-axis acceleration information. According to an embodiment, the direction of the 3-axis acceleration may be in a state that does not match the direction of the geomagnetic field.

According to an embodiment, although not shown in FIG. 4 , according to an embodiment, the electronic device 1000 may correct acceleration information of three-axis acceleration installed in the electronic device. According to an embodiment, the 3-axis acceleration sensor in the electronic device may not be installed parallel to the ground, but may be installed inclined from a reference axis. The electronic device 1000 may further perform a process of obtaining 3-axis acceleration information and correcting acceleration values for each axis in the obtained 3-axis acceleration information.

In S430, the electronic device 1000 may match the 3-axis acceleration direction to the identified geomagnetic direction. For example, the direction of the three-axis acceleration and the direction of the geomagnetic field may not match. The electronic device 1000 may identify which direction the 3-axis acceleration direction corresponds to with respect to the actual geomagnetic direction by matching the direction of the 3-axis acceleration with the direction of the earth's magnetic field.

In S440, the electronic device 1000 may determine the maximum ground acceleration by monitoring the magnitude of the 3-axis acceleration based on the 3-axis acceleration direction matched to the direction of the geomagnetic field. According to an embodiment, the maximum ground acceleration may be a maximum acceleration value measured from pure seismic waves. According to an embodiment, based on the maximum ground acceleration, the electronic device 1000 may determine the magnitude of the currently measured earthquake. In S450, the electronic device 1000 may determine whether an earthquake has occurred based on the 3-axis acceleration direction matched to the geomagnetic direction and the determined maximum ground acceleration.

5 is a diagram for explaining a process of processing 3-axis acceleration information by an electronic device according to an embodiment.

Referring to FIG. 5 , a process of correcting 3-axis acceleration information of a 3-axis acceleration sensor disposed inclined in the electronic device 1000 will be described. According to an embodiment, the 3-axis acceleration sensor in the electronic device 1000 may be disposed inclined from a horizontal or vertical plane of the electronic device. For example, the Y-axis 530 of the acceleration sensor and the Z-axis 540 of the acceleration sensor based on the axis 510 that is horizontal to the ground are the Y reference axis 510 and the Z reference axis parallel to the horizontal and vertical axes, respectively. It may be inclined by a predetermined angle from (520).

와 같은 식을 이용하여 현재 가속도 센서의 z축 출력 값을 결정할 수 있다. 전자 장치(1000)는 가속도 센서의 y축 출력 값 및 z축 출력 값을

와 같은 수식에 적용함으로써 x축이 지면과 이루는 각도를 측정할 수 있다. 또한, 전자 장치(1000)는 가속도 센서 x축 출력 값 및 z축 출력 값을

와 같은 수식에 적용함으로써, y축에 대한 각도를 결정할 수 있다. 일 실시 예에 의하면 z축에 대한 각도를 결정되지 않을 수 있다. 일 실시 예에 의하면, 재난감지 카메라 내 3축 가속도 센서의 z축은 실제 z축 방향과 일치하도록 마련될 수 있다. 전자 장치(1000)는 가속도 센서의 각 축에 대한 각도 값을 상술한 바와 같이 결정한 다음, 결정된 가속도 센서의 각축에 대한 각도 값을 수평인 각도로 치환하여 보정된 3축 가속도 센서의 출력 값을 획득할 수 있다.The electronic device 1000 may obtain an x-axis output value and a y-axis output value from the 3-axis acceleration sensor. The electronic device 1000

The z-axis output value of the current acceleration sensor can be determined using the same equation. The electronic device 1000 determines the y-axis output value and the z-axis output value of the acceleration sensor.

By applying the same formula, the angle formed by the x-axis with the ground can be measured. In addition, the electronic device 1000 calculates an x-axis output value and a z-axis output value of the acceleration sensor.

By applying an equation such as , the angle with respect to the y-axis can be determined. According to an embodiment, an angle with respect to the z-axis may not be determined. According to an embodiment, the z-axis of the 3-axis acceleration sensor in the disaster detection camera may be provided to coincide with the actual z-axis direction. The electronic device 1000 determines the angular value of each axis of the acceleration sensor as described above, and then obtains a calibrated output value of the 3-axis acceleration sensor by replacing the determined angular value of each axis of the acceleration sensor with a horizontal angle. can do.

6 is a diagram for explaining a process of determining a progress based on a maximum ground acceleration by an electronic device according to an exemplary embodiment.

Referring to FIG. 6 , a table in which earthquake magnitudes 610 are matched for each maximum ground acceleration 620 is shown. According to an embodiment, the electronic device 1000 may determine the maximum ground acceleration 620 using output values of the 3-axis acceleration sensor. According to an embodiment, the electronic device 1000 may measure the maximum ground acceleration 620 based on a horizontal direction on the ground. The electronic device 1000 may determine the progress based on the determined maximum ground acceleration based on the chart shown in FIG. 6 . The electronic device 1000 may determine that an earthquake has occurred based on the three-axis acceleration direction matched to the geomagnetic direction when the determined magnitude is equal to or greater than a preset threshold magnitude. Also, according to an embodiment, the electronic device 1000 may image seismic waves in real time using 3-axis acceleration information. The electronic device 1000 may measure the maximum ground acceleration by directly measuring the amplitude of the imaged seismic wave in real time.

7 is a diagram for explaining a process for identifying an earthquake occurrence location by an electronic device according to an exemplary embodiment. According to an embodiment, the electronic device 1000 may measure the PS time, which is the difference between the arrival times of the P wave 710 and the S wave 720, by using a 3-axis acceleration sensor. According to an embodiment, a point where the electronic device 1000 is installed in FIG. 7 is a point 732, and a point where two other electronic devices connected to the electronic device 1000 are installed are points 734 and 736, respectively. Assume.

When the PS time is measured, the electronic device 1000 may determine the distance from the location where the electronic device is currently installed to the epicenter by using the measured PS time. The electronic device 1000 may determine the first epicenter 838 whose radius is the distance to the epicenter. In addition, the electronic device 1000 may obtain, from two or more other electronic devices connected to the electronic device, location information of the other electronic device and epicenter information about the distance from the other electronic device to the epicenter determined by the other electronic device. The electronic device 1000 may determine the second epicenter 740 and the third epicenter 736 based on epicenter information obtained from other electronic devices.

According to an embodiment, the electronic device 1000 may identify an intersection point of a common chord of each epicenter having a radius of three epicenter distances as an actual epicenter. Also, the electronic device 1000 may determine a value of 1/2 of the length of the shortest common chord passing through the epicenter as the depth from the epicenter to the epicenter. The electronic device 1000 may determine the location of the epicenter and depth information from the epicenter to the epicenter as the epicenter information. The electronic device 1000 may output epicenter information, epicenter information, and information about the magnitude of an earthquake determined based on the maximum ground acceleration as notification information.

8 is a diagram for explaining a process in which an electronic device determines whether an earthquake has occurred based on an image analysis result according to another embodiment.

According to an embodiment, the electronic device 1000 may determine whether an earthquake has occurred by analyzing an image including a plurality of frame images or a predetermined frame image within the image. According to an embodiment, the electronic device 1000 acquires a plurality of frame images 802 .

According to an embodiment, the electronic device 1000 may determine a pixel for a reference object on a reference frame 822 among a plurality of frame images. The electronic device 1000 may identify a position of a pixel corresponding to the reference object in a frame 824 following the reference frame or a frame 826 following the reference frame 824 by tracking the motion of the pixel with respect to the reference object. . The electronic device may determine at least one vector by tracking a motion of a pixel corresponding to an object between frames.

According to an embodiment, the electronic device 1000 may identify a change in direction and magnitude of a vector determined based on a motion of a pixel with respect to a reference object. The electronic device 1000 may determine whether an earthquake has occurred based on the direction and magnitude change of the identified vector and the determined maximum ground acceleration. That is, the electronic device 1000 according to the present disclosure may more accurately determine whether an earthquake has occurred by further using an image analysis result in addition to the maximum ground acceleration.

9 is a diagram for explaining in detail a process of determining whether an electronic device generates a fire according to an embodiment.

In S910, the electronic device 1000 may convert RGB data of pixels in the obtained image into HSL data. According to an embodiment, HSL data may include hue, saturation, and lightness values. In S920, the electronic device 1000 may determine a fire candidate region in the image based on the HSL data. According to an embodiment, the electronic device 1000 may more accurately determine a fire candidate region by using thresholds of different levels in determining a fire candidate region in an image.

In S930, the electronic device 1000 may determine at least one fire index for the fire candidate region. In S940, the electronic device 1000 may identify the number of times that the fire candidate area is identified as a suspected fire area based on at least one fire index. For example, when at least one fire index satisfies all of the fire index conditions, the electronic device 1000 may identify the corresponding fire candidate area as a fire suspect area and increase the number of suspected fire counts by 1. .

In S950, the electronic device 1000 may determine that a fire has occurred when the number of times the identification is equal to or greater than a preset threshold and the fire occurrence probability value output from the artificial intelligence model is equal to or greater than a preset threshold probability value. According to an embodiment, the electronic device 1000 determines that the number of fire suspect counts increased as the fire candidate area is identified as a fire suspect area is 6 or more and the fire occurrence probability value output from the artificial intelligence model is greater than or equal to the threshold probability value. It may be determined that a fire has occurred.

Although not shown in FIG. 9 , according to another embodiment, when the electronic device 1000 determines that a fire has occurred in S950, the output value of the artificial intelligence model previously stored in the electronic device or the server 2000 connected to the electronic device The output value of the artificial intelligence model transmitted from may be further used.

For example, the electronic device 1000 may obtain an output value of the artificial intelligence model from the server 2000 after step S950. According to an embodiment, the artificial intelligence model may be a neural network model pre-learned to output a fire occurrence probability value when an image or video of an area to be monitored is input. In the electronic device 1000, the number of times that the fire candidate area is identified as a fire suspect area based on the fire index is greater than or equal to a threshold value, and the probability of fire occurrence included in the output value of the artificial intelligence model obtained from the server 2000 is a predetermined threshold value. It may be determined that a fire has occurred if it is greater than the probability value.

10 is a diagram for explaining a process of determining a fire candidate area by an electronic device according to an exemplary embodiment.

In operation S1010, the electronic device 1000 may identify color, saturation, and brightness values in the HSL data, and determine a first candidate region including pixels having all of the identified color, saturation, and brightness values equal to or greater than a predetermined threshold value.

In operation S1040, the electronic device 1000 may identify pixels having at least one of the color, saturation, and brightness within the HSL data equal to or greater than the predetermined threshold. According to an embodiment, the number of pixels identified in S1040 may be greater than the number of pixels included in the first candidate region. In S1060, the electronic device 1000 determines that, when all of the hue, saturation, and brightness values of pixels around the pixels identified in S1040 are equal to or greater than the predetermined threshold, the electronic device 1000 selects pixels having at least one of the hue, saturation, or brightness equal to or greater than the predetermined threshold. and a second candidate region including pixels included in the first candidate region may be determined. In S1080, the electronic device 1000 may determine the second candidate area as a fire candidate area.

That is, the electronic device 1000 according to the present disclosure determines a first candidate region from an image using a first threshold criterion (eg, all of the color, saturation, and brightness of each pixel in the image must be greater than or equal to the threshold value), and then determines the first candidate region. After determining the second candidate region using a second threshold criterion lower than the threshold criterion (eg, at least one of color, saturation, or brightness of each pixel in the image must be greater than or equal to the threshold value), the second candidate region may be determined as a fire candidate region. there is. According to an embodiment, the second candidate region may include pixels not included in the first candidate region.

Preferably, pixels not included in the first candidate region may be included as candidate regions in the second candidate region. The electronic device 1000 according to the present disclosure may more accurately determine a fire candidate region by setting a threshold criterion by maintaining a lightness criterion among color, saturation, and brightness, and relaxing a color criterion.

11 is a diagram for explaining a process of determining at least one fire index by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may determine whether a fire occurs by using at least one fire index. For example, in S1110, the electronic device 1000 may determine a zero cross count index by performing high-pass filtering on a fire candidate region in three adjacent frames among a plurality of frame images. According to an embodiment, the electronic device 1000 may initialize the counted number of times of identifying a suspected fire area when the zero cross count index is equal to or greater than a predetermined threshold value.

According to an embodiment, in S1120, the electronic device 1000 may determine a high-pass filter area index related to a variation amount for each frame image of the number of pixels in which the high-pass filter parameter is not 0. According to an embodiment, the high-pass filter area index may correspond to the area of a non-zero region after applying the high-pass filter.

According to an embodiment, a change in the area of a region belonging to a fire candidate region and having a value other than 0 may not be large in the resulting image after filtering is performed. The electronic device 1000 may initialize the number of suspected fire counts when the high-pass filter area index is greater than or equal to a predetermined threshold value.

In operation S1130, the electronic device 1000 may determine a periodicity index related to periodicity of a fire candidate region in a plurality of frame images. According to an embodiment, in the case of a rapidly repeated image, it may be erroneously detected as a suspected fire area. Accordingly, the electronic device 1000 may initialize the suspected fire count when the amount of change in the periodicity index determined to reduce the false detection rate is equal to or less than a predetermined threshold value.

In operation S1140, the electronic device 1000 may determine a motion region index regarding a degree of change in the number of pixels having a large pixel value variation among pixels in a plurality of frame images. For example, in the case of an actual fire area, a large variation of RGB data on the time axis can be measured. The electronic device 1000 may extract a pixel having a large variation in pixel value from among the plurality of frame images as a motion pixel, and determine a motion region index using an average and a standard deviation of the number of motion pixels for each frame.

In operation S1150, the electronic device 1000 may determine moving object indices related to positions of pixels whose disparity values of a specific pixel are greater than or equal to a preset number of times within a plurality of frame images. For example, the electronic device 1000 may monitor the change in the number of moving pixels in each frame on a frame-by-frame basis, but by monitoring the corresponding pixel value change on the basis of a pixel at a specific position in each frame, a fire occurs. may be more accurately determined.

In the electronic device 1000 according to the present disclosure, when the above-described zero cross count index, high-pass filter area index, earlyness index, motion region index, and moving object index satisfy respective index conditions, a fire suspicion for a fire candidate region is determined. The count number can be increased by 1. However, according to another embodiment, the electronic device 1000 may initialize the number of suspected fire counts when even one of the fire indices described above does not satisfy the index condition. Since the electronic device 1000 identifies that a fire has occurred only when the number of suspected fire counts is greater than or equal to a preset threshold count, the electronic device 1000 can determine whether a fire has occurred with high accuracy.

12 is a diagram for explaining a process of determining a zero cross count index by an electronic device according to an exemplary embodiment.

A process of determining the zero cross count index by the electronic device 1000 will be described in detail with reference to an example of an instruction shown in drawing 1202 of FIG. 12 . According to one embodiment, one of the characteristics of fire is flicker. For example, if high-pass filtering is temporally performed on a pixel with a flicker, frames repeatedly crossing a value of 0 may be obtained. The electronic device 1000 may determine the number of frames crossing 0 in a predetermined frame period unit as a zero cross count index, and increase the number of suspected fire counts when the determined zero cross count index is greater than a predetermined threshold. . However, according to another embodiment, when the determined zero cross count index is smaller than a predetermined threshold value, the electronic device 1000 may initialize the number of suspected fire counts.

According to an embodiment, the electronic device 1000 may determine a zero cross count index by performing high-pass filtering on a fire candidate region in three adjacent frames among a plurality of frame images.

In more detail, the electronic device 1000 determines each pixel for each frame based on -0.25, 0.5, and -0.25 weights determined for successive N-1 frames, N frames, and N+1 frames among a plurality of frame images, respectively. High-pass filtering may be performed by weighting the values, and a flicker variable according to a result of performing the high-pass filtering may be determined for each pixel. Referring to FIG. 12, the flicker variable (Flick_var) multiplies the pixel value (pre_pre) in frame N-1 of the corresponding position by -0.25, multiplies the pixel value (pre) in frame N by 0.5, and in frame N+1 It can be determined by multiplying the pixel value (cur) of the corresponding position by -0.25 and then adding all of them. The electronic device 1000 may perform high-pass filtering by applying a predetermined weight to each pixel in three adjacent frames at a corresponding position and weighting the pixels. The electronic device 1000 may determine a flicker variable for each pixel within a frame.

The electronic device 1000 may determine a high-pass filter variable for each pixel by comparing the flicker variable determined for each pixel with a predetermined threshold value. The electronic device 1000 may increase the zero cross count index based on whether the determined high pass filter variable for each pixel satisfies a predetermined condition.

In more detail, the electronic device 1000 identifies even one of the pixels in three adjacent frames as a pixel in the fire candidate region, and if the flicker variable of the corresponding pixel is smaller than ??threshold(5), a high Set the pass filtering parameter (HPFn) to -127, and if the flicker parameter is greater than threshold(5), set the high pass filtering parameter (HPFn) for the corresponding pixel to 128, and set the flicker parameter to ??threshold(5) and threshold (5), the high pass filtering variable (HPFn) can be set to 0. In addition, the electronic device 1000 may set the high-pass filtering variable HPFn to 0 when all pixels in three adjacent frames are not identified as fire candidate regions.

Also, when the high-pass filtering variable HPFn is not 0, the electronic device 1000 may increase HPF_count, which is a variable related to the number of pixels for which the high-pass filtering variable HPFn is not 0, for each frame. In addition, the electronic device 1000 determines whether a high-pass filtering parameter (HPFn) of a pixel at a specific location in the current frame is 128 and a high-pass filtering parameter (HPFn-1) of a pixel at a corresponding location in the previous frame is -127. The high-pass filtering factor (HPFn) of a pixel at a specific position in the current frame is -127 and the high-pass filtering factor (HPFn-1) of a pixel at a corresponding position in the previous frame is 128, or the high-pass filtering factor (HPFn-1) of a pixel at a specific position in the current frame is high-pass filtering When the filtering variable (HPFn) is 0 and the high-pass filtering variable (HPFn-1) of the corresponding pixel in the previous frame is 0, the zero cross count index may be increased.

According to the above process, the electronic device 1000 may identify a fire candidate area as a fire suspected area and increase the number of suspected fire counts by one when the zero cross count index is greater than a predetermined threshold. However, the electronic device 1000 may initialize a fire suspicion count when the zero cross count index is smaller than a predetermined threshold value.

13 is a diagram for explaining a process of determining a high pass filter area index by an electronic device according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may determine the high pass filter parameter HPFn for each pixel according to the process described above with reference to FIG. 12 . The electronic device 1000 may store the number of pixels for which the high-pass filter variable is not 0 for each frame in the HPF_Count variable. Referring to drawing 1204 of FIG. 13 , the electronic device 1000 may monitor a change in the number of pixels whose high-pass filter parameter HPFn is not 0 for each frame. According to an embodiment, the electronic device 1000 identifies the number of pixels in which the high pass filter variable HPFn is not 0 for each frame during frames 0 to 30, and the identified high pass filter variable HPFn is 0. It is possible to determine the minimum value, standard deviation and average of the number of pixels that are not

According to an embodiment, the electronic device 1000 may not determine the high-pass filter area index when the minimum number of pixels in which the high-pass filter parameter is not 0 for each frame is smaller than a predetermined threshold value. However, when the minimum value of the smallest number of pixels with a high-pass filter variable for each frame is greater than a predetermined threshold value, the electronic device 1000 calculates the standard deviation of the number of pixels with a high-pass filter variable for each frame that is not 0. Dividing by the average can determine the high-pass filter area index. The electronic device 1000 may increase the number of suspected fire counts when the high-pass filter area index is less than a predetermined threshold value. However, the electronic device 1000 may initialize the count of suspected fire when the high-pass filter area index is greater than a predetermined threshold value.

14 is a diagram for explaining a process of determining a periodicity index by an electronic device according to an exemplary embodiment.

Referring to the drawing 1206 of FIG. 14 , the electronic device 1000 may determine one frame among a plurality of frame images as a reference frame. According to an embodiment, the electronic device 1000 may determine a reference frame as a backup frame. The electronic device 1000 may perform period check inter-frame difference between a reference frame and a plurality of frame images (eg, 1 to 30 frames or 1 to 59 frames) including the reference frame.

For example, the electronic device 1000 may determine a difference variable (diff) for each pixel in a plurality of frame images by performing different mask operations on a reference frame and one frame among the plurality of frame images. . More specifically, it is assumed that the periodicity index is determined based on 30 frames and the reference frame is set to frame number 1.

The electronic device 1000 may perform period check inter-frame difference between frame 1, which is a reference frame, and frame 1 in a plurality of frame images (including the reference frame as frame 1). In addition, the electronic device 1000 performs period check inter-frame difference for frame 1 and frame 2, which are reference frames, and then repeats the above-described process until frame 1 and frame 30, period check inter-frame. Frame difference can be performed.

The electronic device 1000 performs period check inter-frame difference on two arbitrary frames including a reference frame among 30 frames, and applies different mask operations to the two frames. For example, the electronic device 1000 applies a 2 by 2 mask having -1 as an element and a 2 by 2 mask having 1 as an element to two frames including a reference frame within 30 frames. By applying each, period check inter-frame difference can be performed.

The electronic device 1000 determines result 1 by applying a 2 by 2 mask having -1 as an element to the reference frame, and sets 1 as an element to one frame other than the reference frame among 30 frames. Result 2 can be determined by applying a 2 by 2 mask with (element). The electronic device 1000 may determine the diff value by adding 128 to the average values of result 1 and result 2.

Also, according to an embodiment, the electronic device 1000 may quantize the difference variable (diff) by comparing the value of the difference variable (diff) determined for each pixel with a predetermined threshold value. For example, when the difference variable value is smaller than 128-threshold(5), the electronic device 1000 sets the corresponding difference variable to 0, and when the difference variable value is greater than 128+threshold(5), the corresponding difference variable is set to 255, and in other cases, the difference variable can be quantized to a predetermined quantum value by setting the difference variable to 128.

The electronic device 1000 may determine the periodicity index based on the standard deviation and average of the number of pixels of each frame image in which the quantized difference variable value does not match the predetermined quantum value. For example, the electronic device 1000 counts the number of pixels for which the quantized difference variable value (diff) is not 128 per frame, and counts the number of quantized difference variable (diff) values other than 128 counted for each frame. Standard deviations and averages are available.

The electronic device 1000 may determine the periodicity index by dividing the standard deviation of the number of pixels whose quantized difference variable value is not 128 for each frame by the average. If the periodicity index is less than a predetermined threshold, the electronic device 1000 may identify the fire candidate area as a fire suspect area and increase the number of suspected fire counts by 1. However, when the periodicity index is greater than a predetermined threshold value, the electronic device 1000 may initialize the number of suspected fire counts.

15 is a diagram for explaining a process of determining a motion region index by an electronic device according to an embodiment.

Referring to a drawing 1208 of FIG. 15 , the electronic device 1000 determines the number of motion pixels for each frame, determines the average and standard deviation of the number of motion pixels for each frame, and determines the standard number of motion pixels for each frame. The process of determining the motion range index by dividing the deviation by the mean is shown. When the motion area index is less than a predetermined threshold, the electronic device 1000 identifies the fire candidate area as a suspected fire area and increases the number of suspected fire counts by 1. The count number can be initialized.

More specifically, the electronic device 1000 may determine the average of R, G, and B values of pixel values in the frame image as the pixel value (Pixel_value). The electronic device 1000 may determine a difference between a pixel value and a pixel value at a corresponding location in a reference frame as a pixel value variance.

The electronic device 1000 may identify pixels whose pixel value variation per frame is greater than or equal to a predetermined threshold value as motion pixels. The electronic device 1000 may determine the motion region index by dividing the standard deviation of the number of motion pixels identified in each of the plurality of frame images by an average value.

Meanwhile, the electronic device 1000 may increase a moving object index (moving object) for a corresponding pixel by 1 when the variation of a pixel value for each frame is greater than or equal to a predetermined threshold value. After increasing the moving object index by 1, the electronic device 1000 may update a pixel value (Background) of a corresponding position within a reference frame.

16 is a diagram for explaining a process of determining a moving object index by an electronic device according to an embodiment.

Referring to figure 1212 of FIG. 16 , a process of determining a moving object index for each pixel of the electronic device is illustrated. For example, the electronic device 1000 may identify the number of times that variance values of specific pixels at corresponding positions within a plurality of frame images are identified as being equal to or greater than a predetermined threshold value. For example, a case in which the electronic device 1000 determines a moving object index in units of 30 frames will be described.

For example, the electronic device 1000 may track the disparity value of a pixel at a position (1,1) for 1 to 30 frames. For example, the electronic device 1000 may determine the number of times that a disparity value of a pixel at position (1,1) is identified as being greater than or equal to a threshold value during frames 1 to 30. The electronic device 1000 determines Moving object(1,1) as high when the number of cases in which the disparity value of the pixel at position (1,1) is identified as being higher than or equal to a threshold value for frames 1 to 30 is equal to or greater than a predetermined threshold value. can However, the electronic device 1000 may determine Moving object(1,1) as low when the number of times that the disparity value of the pixel at position (1,1) is identified as being equal to or higher than a threshold value for frames 1 to 30 is less than a predetermined threshold value. there is.

According to the method described above, the electronic device 1000 may determine the number of times that the disparity value per pixel is greater than or equal to a threshold value for all pixels within a frame, and then determine the moving object index for pixels per frame. According to an embodiment, the electronic device 1000 may further include, in the moving object index, location information of pixels for which the number of disparity values per pixel of the corresponding location is equal to or greater than a predetermined threshold number during 30 frames.

According to an embodiment, when the moving object index is determined, the electronic device 1000 determines the moving object index in the last frame used to determine the moving object index (for example, if the determination is made in units of 30 frames, the last frame is the 30th frame). It is possible to identify the positions of pixels determined to be high. The electronic device 1000 may determine an average value of RGB data of pixels for which the moving object index is determined to be high. The electronic device 1000 may increase the number of suspected fire counts when each average value of RGB data of pixels for which the moving object index is determined to be high is greater than a predetermined threshold. However, the electronic device 1000 may initialize the fire suspicion count when at least one of RGB data average values of pixels for which the moving object index is determined to be high is not greater than a predetermined threshold.

In addition, according to an embodiment, the electronic device 1000 determines the minimum distance between the positions of pixels having a high moving object index and the positions of pixels having a saturation value greater than or equal to a predetermined threshold (eg, coordinate value of each pixel of the frame). difference) and its average value can be determined. The electronic device 1000 determines the number of suspected fire counts when the minimum distance between the positions of pixels having a high moving object index and the positions of pixels having a saturation value of each pixel of each frame is less than a predetermined threshold. can be initialized. However, the electronic device 1000 suspects a fire when the minimum distance between the positions of pixels having a high moving object index and the positions of pixels having a saturation value of each pixel in the frame is greater than a predetermined threshold. The number of counts can be increased.

Also, according to an embodiment, the electronic device 1000 determines the minimum distance between the positions of pixels having a high moving object index in the current frame and the positions of pixels having a high moving object index in a frame prior to a predetermined period (eg, 30 frames). The sum of the distances and their average can be determined. The electronic device 1000 fires when the minimum distance between the positions of pixels with a high moving object index in the current frame and the positions of pixels with a high moving object index in a frame prior to a predetermined period (eg, 30 frames) is less than a predetermined threshold. The number of suspect counts can be reset. However, the electronic device 1000 determines that the minimum distance between the positions of pixels with a high moving object index in the current frame and the positions of pixels with a high moving object index in a frame prior to a predetermined period (eg, 30 frames) is greater than a predetermined threshold. If it is large, the number of suspected fire counts can be increased.

17 is a flowchart of a method of detecting a disaster by an electronic device according to another embodiment.

Referring to FIG. 17 , a flowchart of a method of detecting a disaster by an electronic device according to another embodiment is shown.

A mark 1701 may indicate an image input standby state. In S1702, the electronic device 1000 converts RGB data in the image into HSL data. In S1706, the electronic device 1000 extracts a fire candidate region (eg, a fire color region) from the image. In S1708, the electronic device 1000 may determine a periodicity index from the fire candidate region.

In S1710, the electronic device 1000 determines a motion area of the fire candidate area. In S1712, the electronic device 1000 determines the width of the motion area in the fire candidate area for each frame and determines the degree of change in the area of the motion area. The electronic device 1000 may determine the motion region index through S1710 and S1712. Also, according to an embodiment, the electronic device 1000 may further determine a moving object index for each pixel.

In operation S1714, the electronic device 1000 determines a zero cross count exponent by counting the number of pixels crossing 0 for each pixel after performing the temporal wavelet in the candidate region. In S1716, the electronic device 1000 determines a high-pass filter area index by performing temporal wavelet transform on the fire candidate area for each frame and checking the width of a non-zero area.

In S1718, the electronic device 1000 identifies whether or not the number of frames has reached 30. For example, when the number of frames does not reach 30, the electronic device 1000 may enter a fire suspicion count initialization state 1720 . However, when the number of frames reaches 30, the electronic device 1000 identifies whether or not the number of times that the disparity value for each pixel is identified as greater than or equal to a predetermined threshold is greater than or equal to a predetermined threshold during 30 frames. In operation S1724, the electronic device 1000 determines the moving object index of the corresponding pixel as HIGH when the number of times that the disparity value of each pixel is identified as being equal to or greater than the predetermined threshold value is equal to or greater than the predetermined threshold value for 30 frames. However, in S1726, the electronic device 1000 determines the moving object index of the corresponding pixel as LOW when the number of times that the disparity value of each pixel is identified as greater than or equal to a predetermined threshold value is less than the predetermined threshold value during 30 frames.

In S1728, when the moving object index is LOW, the electronic device 1000 enters S1736, initializes a suspected fire count, and enters a suspected fire count initialization state 1720. However, in S1730, when the moving object index is identified as HIGH, the electronic device 1000 may obtain an average of RGB data of pixels identified as HIGH. In S1732, the electronic device 1000 enters the connection step 1734 when each average value of the RGB data is greater than a predetermined threshold value, but if even one of the average values of the RGB data is not greater than a predetermined threshold value, Step S1736 is entered to initialize the suspected fire count.

In operation S1704, the electronic device 1000 may determine a minimum distance between the positions of pixels whose moving object index is determined to be HIGH and the positions of pixels whose saturation values are greater than or equal to a predetermined threshold. In S1742, when the minimum distance determined in S1704 is smaller than a predetermined threshold value, the electronic device 1000 initializes a fire suspicion count. However, if the minimum distance is greater than a predetermined threshold value, the electronic device 1000 may proceed to step S1748.

In S1748, when the zero cross count index is greater than a predetermined threshold, the electronic device 1000 enters step S1750, but when the zero cross count index is less than a predetermined threshold, the electronic device 1000 enters step S1736 to initialize the suspected fire count. there is.

In S1750, the electronic device 1000 may determine the high pass filter area index by performing the process described above with reference to FIG. 13 . In S1752, when the determined high-pass filter area index is greater than a predetermined threshold, the electronic device 1000 enters step S1736 to initialize the number of suspected fire counts. However, when the determined high-pass filter area index is smaller than a predetermined threshold value, the electronic device 1000 may enter step S1754.

In S1754, the electronic device 1000 may determine a periodicity index by checking periodicity for each frame. In S1756, when the periodicity index is less than a predetermined threshold value, the electronic device 1000 may initialize the number of suspected fire counts by entering step S1736. However, when the periodicity index is greater than a predetermined threshold value, the electronic device 1000 may enter step S1758.

In S1758, the electronic device 1000 determines a motion region index by calculating an area average and an area change amount of the motion region for each frame. In S1760, when the motion region index is greater than a predetermined threshold value, the electronic device 1000 may increase the movement region variation count in step S1764. In S1750, when the motion area index is less than a predetermined threshold value, the electronic device 1000 may increase the number of suspected fire counts by proceeding to S1766.

In S1768, when the motion area variation count is greater than a predetermined threshold, the electronic device 1000 enters step S1736 to initialize a fire suspicion count. However, in S1768, the electronic device 1000 may enter the suspected fire count initialization state 1720 when the motion region variation count is less than a predetermined threshold or in S1772, when the increased number of suspected fire counts is not greater than 6. . However, in S1772, when the number of suspected fire counts is greater than 6, the electronic device 1000 may output fire alarm information by entering S1174. After outputting the fire alarm information, the electronic device 1000 may enter the fire suspicion count initialization state 1720 again.

That is, the electronic device 1000 according to the present disclosure may determine at least one of the determined zero cross count index, the high-pass filter area index, the periodicity index, the motion region index, or the moving object index as at least one fire index. It identifies whether an index condition is satisfied for the at least one fire index, and if even one of the at least one fire index does not satisfy the index condition, the number of times (eg, the number of suspected fire counts) in which a fire candidate region is identified as a fire suspect region may be initialized. there is. However, the electronic device 1000 increases the number of times a fire candidate area is identified as a fire suspect area when all predetermined index conditions for the at least one fire index are satisfied, and the increased number is equal to or greater than a preset threshold. (For example, 6), it can be determined that a fire has occurred.

Also, according to an embodiment, the electronic device 1000 may determine a fire occurrence index or determine whether a fire occurs according to a predetermined cycle. For example, the electronic device 1000 may determine the at least one fire index for each of a plurality of frame images or for a frame according to a predetermined frame period among a plurality of frame images.

18 is a block diagram of an electronic device according to an embodiment.

19 is a block diagram of an electronic device according to another embodiment.

As shown in FIG. 18 , an electronic device 1000 according to an embodiment may include a processor 1300 and a memory 1700. However, not all illustrated components are essential components. The electronic device 1000 may be implemented with more components than those illustrated, or the electronic device 1000 may be implemented with fewer components.

For example, as shown in FIG. 19 , the electronic device 1000 includes a processor 1300 and a memory 1700, a user input interface 1100, an output unit 1200, a sensing unit 1400, and a network interface. 1500, an A/V input unit 1600, and a memory 1700 may be further included.

The user input unit 1100 means a means through which a user inputs data for controlling the electronic device 1000 . For example, the user input unit 1100 includes a key pad, a dome switch, a touch pad (contact capacitance method, pressure resistive film method, infrared sensing method, surface ultrasonic conduction method, integral type tension measuring method, piezo effect method, etc.), a jog wheel, a jog switch, and the like, but are not limited thereto.

The user input unit 1100 may obtain other user inputs for controlling the electronic device 1000 . Also, the user input unit 1100 may receive a user input of touching a predetermined audio control panel.

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, a sound output unit 1220, and a vibration motor 1230. there is.

The display unit 1210 includes a screen for displaying and outputting information processed by the electronic device 1000 . Also, the screen may display an image. For example, the display unit 1210 may display at least one of disaster notification information, earthquake notification information, fire notification information, and a news screen.

The audio output unit 1220 outputs audio data received from the network interface 1500 or stored in the memory 1700 . In addition, the sound output unit 1220 may output a predetermined warning sound or the like along with outputting disaster notification information. Also, the sound output unit 1220 outputs sound signals related to functions performed by the electronic device 1000 (eg, a call signal reception sound, a message reception sound, and a notification sound).

The processor 1300 typically controls overall operations of the electronic device 1000 . For example, the processor 1300 executes programs stored in the memory 1700, so that the user input unit 1100, the output unit 1200, the sensing unit 1400, the network interface 1500, and the A/V input unit ( 1600) can be controlled overall. Also, the processor 1300 may perform the functions of the electronic device 1000 described in FIGS. 1 to 17 by executing programs stored in the memory 1700 .

According to an embodiment, the processor 1300 acquires an image by photographing an area to be monitored by executing the one or more instructions, and at a location where the electronic device is installed using at least one sensor. Seismic wave information is acquired by sensing seismic waves, at least one fire index is determined based on the obtained image analysis result, whether or not a fire occurs is determined based on the determined fire index, and at least one fire index is determined based on the seismic wave information. It is possible to determine an earthquake index of , determine whether an earthquake has occurred based on the determined earthquake index, and output notification information according to whether the fire or the earthquake has occurred.

According to an embodiment, the processor 1300 may obtain a plurality of frame images having a predetermined frame interval through a camera device that captures the surveillance target area.

According to an embodiment, the processor 1300 may obtain 3-axis acceleration information from a 3-axis acceleration sensor among the at least one sensor, and obtain geomagnetic information from a geomagnetic sensor among the at least one sensor.

According to an embodiment, the processor 1300 identifies the direction of the earth's magnetism at the location where the electronic device is installed based on the geomagnetism information, identifies the direction of the 3-axis acceleration based on the 3-axis acceleration information, and the direction of the 3-axis acceleration. is matched with the identified geomagnetic direction, and based on the three-axis acceleration direction matched to the geomagnetic direction, the maximum ground acceleration is determined by monitoring the magnitude of the three-axis acceleration, and the three-axis acceleration direction matched with the geomagnetic direction. And it is possible to determine whether or not an earthquake has occurred based on the determined maximum ground acceleration.

According to an embodiment, the processor 1300 determines the seismic intensity based on the maximum ground acceleration, and when the determined seismic intensity is greater than or equal to a preset threshold magnitude, based on the 3-axis acceleration direction matched to the geomagnetic direction, an earthquake occurred. can be determined as

According to an embodiment, when it is determined that the earthquake has occurred, the processor 1300 determines a PS time based on the three-axis acceleration information, determines an epicenter based on the determined PS time, and communicates with the electronic device Obtain epicenter information from two or more connected electronic devices, determine epicenter information on which an earthquake occurred based on the determined epicenter and the epicenter information obtained from the other electronic devices, and determine the determined epicenter information and information about the determined seismic intensity. may be output as the notification information.

According to an embodiment, the processor 1300 determines at least one vector based on the image and a change in position of a reference object in the image appearing in frames before and after the image, and determines a change in direction and size of the at least one vector. It is possible to determine whether or not the earthquake occurred based on the direction and magnitude change of the identified at least one vector and the determined maximum ground acceleration.

According to an embodiment, the processor 1300 converts RGB data of pixels in the acquired image into HSL data, determines a fire candidate area in the image based on the HSL data, and determines a fire candidate area in the determined fire candidate area. The at least one fire index is determined, based on the determined at least one fire index, a number of times the fire candidate area is identified as a fire suspect area is identified, and when the number of times the identified number is equal to or greater than a preset threshold, a fire has occurred. can be determined to have occurred.

The sensing unit 1400 may detect a state of the electronic device 1000 or a state around the electronic device 1000 and transmit the sensed information to the processor 1300 . The sensing unit 1400 generates some of the specification information of the electronic device 1000, the state information of the electronic device 1000, the surrounding environment information of the electronic device 1000, the user's state information, and the user's device use history information. can be used

The sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor. (eg, GPS) 1460, an air pressure sensor 1470, a proximity sensor 1480, and an RGB sensor (illuminance sensor) 1490, but may include at least one, but is not limited thereto. Since a person skilled in the art can intuitively infer the function of each sensor from its name, a detailed description thereof will be omitted.

The network interface 1500 may include one or more components allowing the electronic device 1000 to communicate with other devices (not shown) and the server 2000 . Another device (not shown) may be a computing device such as the electronic device 1000 or a sensing device, but is not limited thereto. For example, the network interface 1500 may include a short-distance communication unit 1510, a mobile communication unit 1520, and a broadcast reception unit 1530.

The short-range wireless communication unit 1510 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, and an infrared ( It may include an infrared data association (IrDA) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, etc., but is not limited thereto.

The mobile communication unit 1520 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the radio signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception.

The broadcast receiver 1530 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. According to an implementation example, the electronic device 1000 may not include the broadcast reception unit 1530. Also, the network interface 1500 may obtain a target object image or a reference object image from an external device including a camera device or a server. According to an embodiment, the network interface 1500 may transmit disaster alarm information determined by the electronic device to a server or an external device.

An audio/video (A/V) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a photographing mode. An image captured through the image sensor may be processed through the processor 1300 or a separate image processing unit (not shown).

The microphone 1620 receives external sound signals and processes them into electrical voice data. For example, the microphone 1620 may receive a sound signal from an external device or a user. The microphone 1620 may receive a user's voice input. The microphone 1620 may use various noise cancellation algorithms to remove noise generated in the process of receiving an external sound signal.

The memory 1700 may store programs for processing and control of the processor 1300 and may store data input to or output from the electronic device 1000 . In addition, the memory 1700 may store a series of methods for determining whether a fire or an earthquake has occurred in the form of instructions in order for the electronic device 1000 to output disaster alarm information.

The memory 1700 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , an optical disk, and at least one type of storage medium.

Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, such as a UI module 1710, a touch screen module 1720, and a notification module 1730. .

The UI module 1710 may provide a specialized UI, GUI, or the like that works with the electronic device 1000 for each application. The touch screen module 1720 may detect a user's touch gesture on the touch screen and transmit information about the touch gesture to the processor 1300 . The touch screen module 1720 according to some embodiments may recognize and analyze the touch code. The touch screen module 1720 may be configured as separate hardware including a controller.

The notification module 1730 may generate a signal for notifying occurrence of an event of the electronic device 1000 . Examples of events occurring in the electronic device 1000 include reception of a call signal, reception of a message, input of a key signal, and notification of a schedule. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210, output a notification signal in the form of an audio signal through the sound output unit 1220, and may output a notification signal in the form of a vibration motor 1230. A notification signal may be output in the form of a vibration signal through

20 is a block diagram of a server connected to an electronic device according to an embodiment.

According to one embodiment, the server 2000 may include a processor 2300, a network interface 2500, and a database 2700. According to an embodiment, the configuration of the processor 2300, the network interface 2500, and the database 2700 in the server 2000 includes the processor 1300, the network interface 1500, and the memory 1700 in the electronic device 1000. Each may correspond to a configuration.

According to an embodiment, the server 2000 may transmit images acquired from a CCTV device or seismic wave information to the electronic device 1000 . In addition, the server 2000 provides disaster information determined by the electronic device 1000 (for example, whether a fire has occurred, whether an earthquake has occurred, fire alarm information, earthquake alarm information, fire occurrence scale, fire occurrence location, earthquake occurrence magnitude, earthquake occurrence location, etc.) information) may be obtained, and visual or audible disaster content may be generated based on the obtained disaster information. The server 2000 may transmit the generated disaster content back to the electronic device 1000 or to another electronic device or server device connected to the electronic device.

21 is a diagram for explaining a process of performing a method of detecting a disaster by interworking an electronic device and a server according to an embodiment.

The electronic device 1000 and the server 2000 according to the present disclosure may perform at least part of a disaster detection method by interoperating with each other based on an artificial intelligence model. However, according to another embodiment, the artificial intelligence model stored in the server 2000 or the instructions for the artificial intelligence model are stored in the electronic device, so that the electronic device 1000 independently uses the artificial intelligence model described above. Of course, a method of detecting a disaster may be performed.

According to an embodiment, the electronic device 1000 according to the present disclosure may perform a disaster detection method using an artificial intelligence model. For example, the electronic device 1000 may determine at least one fire index based on the image analysis result, and identify the number of times the fire candidate region is identified as a fire suspect region based on the at least one fire index. In addition, the electronic device 1000 identifies the number of times it is identified as a suspected fire area through an image analysis result, and when an image obtained by photographing a surveillance target area is input, an artificial intelligence model outputting fire occurrence probability information. Fire occurrence probability information may be obtained by inputting a monitoring target image.

The electronic device 1000 determines that the number of times that a fire candidate area is identified as a fire suspect area based on the fire index is equal to or greater than a preset threshold number of times, and fire occurrence probability information (eg, a probability value) obtained from an artificial intelligence model is a preset threshold probability value. If it is above the value, it can be determined that a fire has occurred. That is, the electronic device 1000 according to the present disclosure can more accurately identify whether or not a fire has occurred in an area to be monitored by using an artificial intelligence model. Also, according to an embodiment, when seismic wave information is input, the electronic device 1000 inputs the seismic wave information to an artificial intelligence model that outputs an earthquake occurrence probability, and the earthquake occurrence probability obtained from the artificial intelligence model. Based on the information and earthquake indices, it is also possible to accurately determine whether an earthquake has occurred.

According to an embodiment, the artificial intelligence model used by the electronic device 1000 may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between an operation result of a previous layer and a plurality of weight values. A plurality of weights possessed by a plurality of neural network layers may be optimized by a learning result of an artificial intelligence model. For example, a plurality of weights may be updated so that a loss value or a cost value obtained from an artificial intelligence model is reduced or minimized during a learning process.

According to an embodiment, the artificial intelligence model used by the electronic device 1000 may include a deep neural network (DNN), for example, a convolutional neural network (CNN), a deep neural network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, but is not limited to the above examples. don't Also, according to an embodiment, the artificial intelligence model used by the electronic device 1000 may include at least one of YOLO and DARKNET.

Although the method for the electronic device 1000 to detect a disaster using an artificial intelligence model has been described above according to an embodiment of the present disclosure, as shown in FIG. 21 , the electronic device 1000 interworks with the server 2000 to Of course, at least some of the methods for detecting disasters using an artificial intelligence model can be performed.

For example, in S2102, the electronic device 1000 may obtain an image of the monitoring target area by capturing an image of the monitoring target area. The electronic device 1000 may acquire an image of a monitoring target including a plurality of images at predetermined frame intervals.

In S2104, the electronic device 1000 may transmit the acquired image of the surveillance target area to the server 2000. According to another embodiment, the electronic device 1000 may transmit RTSP video data to the server 2000. In S2106, the server 2000 may pre-process the image received from the electronic device. For example, the server 2000 may convert an image or video obtained from the electronic device 1000 into a format optimized for an artificial intelligence model by pre-processing the image or video. More specifically, the server 2000 may convert an image or video acquired from the electronic device 1000 into a format optimized for each artificial intelligence model learned for each image type by pre-processing the image or video.

In S2108, the server 2000 may obtain fire occurrence probability information from the artificial intelligence model by inputting the preprocessed image or video to the artificial intelligence model. According to an embodiment, the server 2000 may store a plurality of artificial intelligence models. For example, the server 2000 identifies the preprocessed image or video type, identifies an artificial intelligence model optimized for the image or video type among artificial intelligence models based on the identified type, and identifies the artificial intelligence model. You can also input preprocessed images or videos to the intelligence model. According to an embodiment, the server 2000 may pre-train an artificial intelligence model to output a fire occurrence probability value when an image or video is input, and input an image or image obtained from an electronic device to the learned artificial intelligence model. can

In S2110, the server 2000 may generate notification information based on fire occurrence probability information obtained from the artificial intelligence model. According to an embodiment, the server 2000 may generate notification information when a fire occurrence probability value output from the artificial intelligence model is equal to or greater than a predetermined threshold value. In S2112, the server 2000 may transmit at least one of output information (eg, fire probability information) output from the artificial intelligence model or notification information to the electronic device 1000.

In S2114, the electronic device 1000 may store fire occurrence probability information and notification information received from the server 2000. According to another embodiment, the electronic device 1000 may obtain fire occurrence probability information from the server 2000 and generate notification information by itself when the obtained fire occurrence probability information is equal to or greater than a predetermined threshold probability value. In S2116, the electronic device 1000 may determine whether a fire has occurred based on at least one of the fire index determined based on the analysis result of the image to be monitored, fire occurrence probability information received from the server, and notification information. For example, the electronic device 1000 detects a fire when the number of times that the fire candidate area is identified as a fire suspect area based on the fire index is greater than or equal to a threshold and the fire occurrence probability information received from the server is greater than or equal to a predetermined threshold probability. can be determined to have occurred.

22 is a diagram for explaining a process of performing a method of detecting a disaster by interworking an electronic device and a server according to another embodiment.

Referring to FIG. 22 , the electronic device 1000 may correspond to a FireWatch Camera. The electronic device 1000 may obtain an input video 2210 for a surveillance target area using a camera. In S2202, the electronic device 1000 may determine first result information about whether a fire has occurred by performing the method for determining whether a fire has occurred described above with reference to FIGS. 1 to 20 . In S2204, the electronic device 1000 may video stream image information obtained by photographing the area to be monitored. In S2206, the electronic device 1000 may obtain, as second result information, external input event information on whether or not a fire has occurred, determined by the server using an artificial intelligence model, from the server 2000 . In S2208, the electronic device 1000 may generate an alert based on the first result information and the second result information. For example, the electronic device 1000 issues an alarm when the first result information indicates that the number of times the fire candidate area is identified as a fire suspect area is greater than or equal to a threshold value and the second result information is identified as being greater than or equal to a predetermined threshold probability value. may cause

The electronic device 1000 according to the present disclosure may transmit input video data 2210 obtained by photographing the area to be monitored in the form of RTSP video data 2220 to the server 2000, and the server 2000 may transmit the electronic device ( An alarm event determined by the server 2000 may be generated by inputting the RTSP video data obtained in 1000) to the artificial intelligence model.

For example, in S2210, the server 2000 may generate an image or video having a format suitable for the type of artificial intelligence model by frame-processing the RTSP video data 2220 obtained from the electronic device 1000. In S2212, the server 2000 may input the preprocessed image or video to the trained deep learning neural network model. In S2214, the server 2000 may infer whether a fire has occurred in the image or video based on the output value of the deep learning neural network model. In S2216, the server 2000 generates alarm event data when the result value deduced based on the output value of the neural network model indicates a situation in which a fire has occurred or indicates a probability of a fire occurring greater than or equal to a predetermined threshold probability, and generates alarm event data. Event data 2230 may be transmitted to the electronic device 1000 .

23 is a diagram for explaining a process of generating learning data for learning an artificial intelligence model by an electronic device according to an embodiment.

The electronic device 1000 according to the present disclosure may train an artificial intelligence model that outputs a fire occurrence probability value in an image or video. Also, according to an embodiment, the electronic device 1000 may train an artificial intelligence model for each type of image. Also, the electronic device 1000 may generate learning data for each predetermined type for learning the artificial intelligence model.

For example, the electronic device 1000 may acquire training video data 2302 for training an artificial intelligence model by capturing a surveillance target area. According to another embodiment, the electronic device 1000 may obtain training video data 2302 for AI model learning from the server 2000 or a pre-stored database. The electronic device 1000 may generate learning data 2340 for each predetermined video type by pre-processing the training video data 2302 .

For example, the electronic device 1000 pre-processes the learning video data 2302 to generate a high-pass filter image 2312, a color (RGB) image 2314, a black and white image 2316, an edge image 2318, and motion At least one of the image 2320 and the brightness control image 2322 may be generated. According to an embodiment, the motion image 2320 may be an image or an image in which vectors and motions in an image are expressed in gray scale. According to an embodiment, the brightness control image 2322 may be an image for which the image brightness is adjusted. The electronic device 1000 may generate combined learning data by combining each type of learning data generated by preprocessing the learning video data 2302 or the learning data generated by type, and the learning data generated by type or combined learning. AI models can be trained based on data. The electronic device 1000 according to the present disclosure may learn the artificial intelligence model by modifying and updating weights of nodes and layers in the artificial intelligence model based on learning data.

According to an embodiment, the electronic device 1000 may train an artificial intelligence model using video data selected by human judgment among the training video data 2302 . According to another embodiment, the electronic device 1000 determines whether a fire has occurred based on the fire occurrence determination algorithm described above with reference to FIGS. can be removed from the training video data. The electronic device 1000 removes an image or video in which fire occurrence is incorrectly determined in the fire occurrence algorithm described in FIGS. 1 to 20 from the training data and trains an artificial intelligence model based on the remaining training data, so that the artificial intelligence model accuracy can be further improved.

24 is a diagram for explaining a process of generating an alarm event by using an artificial intelligence model by an electronic device according to an embodiment.

A process of generating an alarm event by obtaining a real-time image 2402 by capturing a region to be monitored by the electronic device 1000 according to an embodiment, pre-processing the real-time image, and then inputting the generated pre-processed image to a neural network model will be described. I'm going to do it.

In S2440, the electronic device 1000 may generate a pre-processed image generated by pre-processing the real-time image 2402. For example, the electronic device 1000 pre-processes the real-time image 2402 to generate a high-pass filter image 2412, an RGB image 2414, a black and white image 2416, an edge image 2418, a motion image 2428, or A brightness adjustment image 2422 may be generated. In S2442, the electronic device 1000 may monitor the estimation result by inputting the preprocessed image to the deep learning neural network model 2430.

In S2444, the electronic device 1000 may determine whether an average accuracy of a value obtained by accumulating inference results of an artificial intelligence model (eg, a neural network model) for a preset time period is equal to or greater than a predetermined threshold value. According to an embodiment, the electronic device 1000 may accumulate inference results of the artificial intelligence model for 1 to 10 seconds to determine whether the average accuracy is greater than or equal to a threshold value. In S2446, the electronic device 1000 may generate an alarm event when the cumulative average accuracy is identified as greater than or equal to a threshold value.

A method for detecting a disaster by an electronic device according to an embodiment may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software.

In addition, a computer program device including a recording medium storing a program for causing an electronic device to perform a method of detecting a disaster may be provided.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

Some embodiments may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media. Also, some embodiments may be implemented as a computer program or computer program product including instructions executable by a computer, such as a computer program executed by a computer.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. fall within the scope of the right

Claims (20)

In the method for an electronic device to detect a disaster,
acquiring an image by photographing a region to be monitored;
obtaining seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor;
determining at least one fire index based on the obtained image analysis result;
obtaining a fire occurrence probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether a fire occurs when an image is input;
determining whether a fire occurs based on the determined fire index and the fire occurrence probability value;
determining at least one earthquake index based on the seismic wave information, and determining whether an earthquake has occurred based on the determined earthquake index; and
outputting notification information according to whether the fire or the earthquake has occurred; Including, method. The method of claim 1, wherein acquiring the image comprises:
obtaining a plurality of frame images having a preset frame interval through a camera device that captures the surveillance target area; Including, method. The method of claim 1, wherein acquiring the seismic wave information comprises:
obtaining 3-axis acceleration information from a 3-axis acceleration sensor among the at least one sensor; and
obtaining geomagnetic information from a geomagnetic sensor among the at least one sensor; Including, method. The method of claim 3, wherein the step of determining whether an earthquake has occurred
identifying a direction of the geomagnetism at a location where the electronic device is installed based on the geomagnetism information;
identifying a 3-axis acceleration direction based on the 3-axis acceleration information;
matching the 3-axis acceleration direction to the identified geomagnetic direction;
determining a maximum ground acceleration by monitoring the magnitude of acceleration in the 3-axis acceleration information based on the 3-axis acceleration direction matched to the geomagnetic direction; and
determining whether an earthquake has occurred based on the 3-axis acceleration direction matched to the geomagnetic direction and the determined maximum ground acceleration; Including, method. The method of claim 4, wherein the step of determining whether the earthquake has occurred
determining a seismic intensity based on the maximum ground acceleration;
determining that an earthquake has occurred based on a three-axis acceleration direction matched to the geomagnetic direction when the determined seismic intensity is greater than or equal to a preset threshold seismic intensity; Including, method. The method of claim 5, wherein outputting the notification information
determining a PS time based on the three-axis acceleration information when it is determined that the earthquake has occurred;
determining an epicenter based on the determined PS time;
obtaining epicenter information from two or more other electronic devices connected to the electronic device;
determining information on an epicenter where an earthquake occurred based on the determined epicenter and epicenter information obtained from the other electronic device; and
outputting the information on the determined epicenter and the information on the determined progress as the notification information; Including, method. The method of claim 5, wherein the step of determining whether an earthquake has occurred
determining at least one vector based on a change in position of a reference object in the image and an image appearing in frames before and after the image;
identifying changes in direction and magnitude of the at least one vector; and
determining whether the earthquake occurred based on the direction and magnitude change of the identified at least one vector and the determined maximum ground acceleration; Including, method. The method of claim 2, wherein the step of determining whether the fire occurs
converting RGB data of pixels in the obtained image into HSL data;
determining a fire candidate region in the image based on the HSL data;
determining the at least one fire index for the determined fire candidate area;
identifying the number of times the fire candidate area is identified as a fire suspect area based on the determined at least one fire index; and
determining that a fire has occurred if the number of times the identification is equal to or greater than a preset threshold and a fire occurrence probability value output from the artificial intelligence model is equal to or greater than a preset threshold probability value; Including, method. 9. The method of claim 8, wherein determining the fire candidate area comprises:
determining a first candidate region including pixels having all of hue, saturation, and lightness values within the HSL data equal to or greater than a predetermined threshold value;
identifying pixels having at least one of the color, saturation, and brightness within the HSL data equal to or greater than the predetermined threshold value;
When all of the hue, saturation, and brightness values of pixels around the identified pixels are equal to or greater than the predetermined threshold, pixels having at least one of the hue, saturation, or brightness equal to or greater than the predetermined threshold and the first candidate region determining a second candidate region including included pixels; and
determining the second candidate area as the fire candidate area; Including, method. 9. The method of claim 8, wherein determining the at least one fire index comprises:
determining a zero cross count index by performing high-pass filtering on a fire candidate region in three adjacent frames among the plurality of frame images;
determining a high-pass filter variable for each pixel of the fire candidate region by comparing the zero cross count index with a predetermined threshold;
determining a high-pass filter area index related to a variation amount for each frame image of the number of pixels in which the high-pass filter parameter is not 0; Including, method. 11. The method of claim 10, wherein determining the at least one fire index comprises:
determining a periodicity index related to periodicity of fire candidate regions in the plurality of frame images;
determining a motion region index related to a degree of change in the number of pixels having a large pixel value variation among pixels in the plurality of frame images; and
determining moving object indices related to positions of pixels whose disparity value of a specific pixel exceeds a predetermined threshold value within the plurality of frame images; Further comprising a method. 11. The method of claim 10, wherein determining the zero cross count index comprises:
High-pass filtering is performed by weighting pixel values for each frame based on -0.25, 0.5, and -0.25 weights determined for successive N-1 frames, N frames, and N+1 frames among the plurality of frame images, respectively. and determining a flicker variable for each pixel according to a result of high-pass filtering;
determining a high-pass filter variable value for each pixel by comparing the determined flicker variable with a predetermined threshold value; and
determining the zero-cross count index based on the high-pass filter variable value for each of the plurality of frame images; Including, method. 11. The method of claim 10, wherein determining the high pass filter area index comprises:
counting the number of pixels for which the high-pass filter parameter is not 0 for each of the plurality of frame images;
determining a minimum value, a standard deviation, and an average of the number of pixels for which the high-pass filter parameter is not 0, counted for each of the plurality of frame images; and
determining the high-pass filter area index using at least one of a minimum value, a standard deviation, and an average of the number of pixels that are not the counted high-pass filter parameters; Including, method. 12. The method of claim 11, wherein determining the periodicity index comprises:
determining a reference frame among the plurality of frame images;
determining a difference variable for each pixel in the plurality of frame images by performing different mask operations on the reference frame and one frame among the plurality of frame images;
quantizing the determined difference variable for each pixel by comparing the difference variable for each of the pixels in the determined plurality of frame images with a predetermined threshold value; and
determining the periodicity index based on a standard deviation and an average of the number of pixels of each frame image in which the quantized difference variable value does not match a predetermined quantum value; Including, method. 12. The method of claim 11, wherein determining the motion region index comprises:
determining a pixel value variance among pixels in the plurality of frame images;
identifying pixels in a plurality of frame images having the determined disparity greater than or equal to a preset threshold as motion pixels; and
determining a motion region index based on an average and a standard deviation of the number of motion pixels identified in each of the plurality of frame images; Including, method. 12. The method of claim 11, wherein determining the moving object index comprises:
identifying the number of times that disparity values of specific pixels at corresponding positions in the plurality of frame images are identified as being equal to or greater than a predetermined threshold value; and
determining location information of pixels for which the number of times the disparity value is equal to or greater than a predetermined threshold is equal to or greater than a predetermined number as the moving object index; Including, method. 12. The method of claim 11, wherein determining that a fire has occurred comprises:
identifying whether an index condition for at least one of the determined zero cross count index, the high-pass filter area index, the periodicity index, the motion region index, or the moving object index is satisfied;
resetting the number of times the fire candidate area is identified as the fire suspect area when even one of the at least one fire indices does not satisfy an index condition;
increasing the number of times that the fire candidate area is identified as a suspected fire area when all predetermined index conditions for the at least one fire index are satisfied; and
determining that the fire has occurred when the increased number of times is greater than or equal to the preset threshold value; Including, method. 9. The method of claim 8, wherein determining the at least one fire index comprises:
determining the at least one fire index for each of the plurality of frame images or for a frame according to a predetermined frame period among the plurality of frame images; Including, method. camera;
network interface;
a memory that stores one or more instructions; and
at least one processor to execute the one or more instructions; including,
By executing the one or more instructions, the at least one processor:
Obtaining an image by photographing the area to be monitored,
obtaining seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor;
Determine at least one fire index based on the obtained image analysis result;
Obtaining a fire probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether a fire occurs when an image is input,
determining whether a fire occurs based on the determined fire index and the fire occurrence probability value;
Determine at least one earthquake index based on the seismic wave information, determine whether an earthquake occurs based on the determined earthquake index,
An electronic device that outputs notification information according to whether the fire or the earthquake has occurred. In the method for an electronic device to detect a disaster,
acquiring an image by photographing a region to be monitored;
obtaining seismic wave information by sensing seismic waves at a location where the electronic device is installed using at least one sensor;
determining at least one fire index based on the obtained image analysis result;
obtaining a fire occurrence probability value for the image from the artificial intelligence model by inputting the obtained image to an artificial intelligence model that outputs a probability value for whether a fire occurs when an image is input;
determining whether a fire occurs based on the determined fire index and the fire occurrence probability value;
determining at least one earthquake index based on the seismic wave information, and determining whether an earthquake has occurred based on the determined earthquake index; and
outputting notification information according to whether the fire or the earthquake has occurred; A computer-readable recording medium in which a program for performing the method is stored, including a.

Images