KR102630183B1 - Generating apparatus and method of image data for fire detection training, and learning apparatus and method using the same - Google Patents

Abstract

The present invention relates to a device and method for generating image data for fire detection training and a learning device and method using the same. A depth map image acquisition unit that acquires a 3D depth map image including RGB color information and depth information for the spatial region captured based on the point through a plurality of cameras arranged toward the point, and a plurality of preset particle system-based Using particle particles, a particle assembly is formed by modeling fire phenomena including flames and smoke through a fluid simulation rendering technique, and then placed at different coordinate positions on a virtual plane having the same area range as the depth map image. A particle image generation module that generates a plurality of particle images, a depth segmentation image generator that generates a depth segmented image by dividing the acquired depth map image into a plurality of RGB images according to the depth level, and a depth segmented image from the depth segmented image, respectively. An RGB image extraction unit that extracts a pair of RGB images corresponding to the depth value of the pixel area for the particle aggregate on the particle image and the adjacent depth level, and each of the particle images and the pair extracted correspondingly. It is characterized by comprising a composite image generator that generates a composite particle image by arranging the RGB images in depth position order.
Accordingly, it is possible to generate a large amount of high-quality learning data by virtually creating fire situations that may occur in various locations, thereby overcoming the limitations of early fire detection capabilities due to the lack of conventional learning data and achieving a very high level of detection rate. There is a possible effect.

Description

Device and method for generating image data for fire detection training and learning device and method using the same {GENERATING APPARATUS AND METHOD OF IMAGE DATA FOR FIRE DETECTION TRAINING, AND LEARNING APPARATUS AND METHOD USING THE SAME}

The present invention generates high-quality and large amounts of image data for fire detection training by combining CCTV images captured in real time inside a building and images modeling various fire situations, and performs fire detection training using the image data for fire detection training. It relates to generation devices and methods and learning devices and methods using the same.

Fires are emerging as an important urban problem in that, due to their nature, when they occur, they are likely to escalate into large-scale fires and cause serious damage to life and property. In particular, early response to fire is very important, and for early response, detecting fire in advance is required first.

A method of detecting such fire situations in advance has mainly been to detect smoke, heat, and radiant energy in the infrared or ultraviolet rays, which are essential in the event of a fire, using various sensors.

However, since the monitoring environment suitable for detection in sensors has different characteristics depending on the detection principle, sufficient detection performance can be achieved only when appropriate sensor equipment is installed according to the type of monitoring environment, and at the same time, it has high redundancy in various monitoring environments. The realization of a fire sensor capable of early detection remains an unresolved problem in the current disaster prevention field.

Meanwhile, video-based fire detection can solve many of the problems that arise from the sensor-based fire detection described above, and can use existing CCTV without separate sensor equipment, minimizing installation costs and enabling dispatch based on alarms. There is an advantage in reducing the rate of erroneous dispatch by checking the fire status at the scene in advance.

However, the conventional image-based fire detection method simply sets a threshold obtained empirically or experimentally and detects fire based on this, making it difficult to apply in real situations and may cause false alarms for objects similar to flames. You can.

In addition, a lot of learning data is needed to increase accuracy using methods such as image-based deep learning, but since it is impossible to secure data on various environments with only actual fire images, it is difficult to actually perform fire detection based on image learning. The situation is difficult.

KR 10-1806503 B1 KR 10-2235308 B1

The purpose of the present invention is to solve the above problems, by combining CCTV images captured in real time inside a building and images modeling various fire situations to generate high quality and large amounts of image data for fire detection training, and detecting fires using the same. The purpose is to provide an apparatus and method for generating image data for fire detection training that performs learning, and a learning apparatus and method using the same.

In order to achieve the above object, an apparatus for generating training image data for fire detection training according to one aspect of the present invention is an apparatus for generating training image data for fire detection in an indoor space, at different points in the indoor space. A depth map image acquisition unit that acquires a 3D depth map image including RGB color information and depth information for the spatial region captured based on the point through a plurality of cameras arranged toward the point, and a plurality of preset particle system-based Using particle particles, a particle assembly is formed by modeling fire phenomena including flames and smoke through a fluid simulation rendering technique, and then placed at different coordinate positions on a virtual plane having the same area range as the depth map image. A particle image generation module that generates a plurality of particle images, a depth segmentation image generator that generates a depth segmented image by dividing the acquired depth map image into a plurality of RGB images according to the depth level, and a depth segmented image from the depth segmented image, respectively. An RGB image extraction unit that extracts a pair of RGB images corresponding to the depth value of the pixel area for the particle aggregate on the particle image and the adjacent depth level, and each of the particle images and the pair extracted correspondingly. It is characterized by comprising a composite image generator that generates a composite particle image by arranging the RGB images in depth position order.

In addition, the method of generating image data for fire detection training according to another aspect of the present invention includes RGB color information and depth for the spatial area captured based on the point through a plurality of cameras arranged toward different points in the indoor space. After acquiring a 3D depth map image containing information and forming a particle assembly by modeling fire phenomena including flames and smoke through a fluid simulation rendering technique using a plurality of preset particle particles based on a particle system, A step of generating a plurality of particle images by arranging them at different coordinate positions on a virtual plane having the same area range as the depth map image, and dividing the obtained depth map image into a plurality of RGB images according to the depth level. generating a segmented image, extracting a pair of RGB images corresponding to a depth level adjacent to a depth value of a pixel area for the particle aggregate on each particle image from the depth segmented image, and It is characterized in that it includes the step of generating a particle composite image synthesized by arranging the particle image and the pair of RGB images extracted correspondingly in order of depth position.

Meanwhile, the image data learning device for fire detection training according to one aspect of the present invention using the above-described features includes a first RGB image based on a 3D depth map image captured through a plurality of cameras arranged toward different points in an indoor space. Against the background, a virtual object corresponding to a particle assembly modeling a fire phenomenon is synthesized based on the depth information included in the depth map image through a particle system-based fluid simulation rendering technique, and then the virtual object is labeled. Based on a learning data storage unit that stores a training image dataset generated according to the data set, and feature points that are a pixel or combination of pixels detected in response to the characteristics of the particle aggregate corresponding to the virtual object included in the first RGB image, A learning model unit pre-trained to enable image classification according to a convolutional neural network by acquiring feature parameters based on the generated training image dataset, and a predetermined point in the indoor space by one of the plurality of cameras An image acquisition unit that acquires an RGB image at the current time based on a second RGB image captured in real time as a reference as an image to be evaluated, and the image to be evaluated is composed of feature points and outline information detected from objects included in the image to be evaluated. a weight calculation module that calculates a weight by calculating the relative ratio of image feature components of the image to be evaluated based on the contrast values of the pixels, and a pre-stored image to be evaluated through a deep learning-based tensor flow technique. An image preprocessor that generates a preprocessed image by normalizing it to the same pixel size and range as the training image dataset, and image learning based on convolution operation using the weights by applying the preprocessed image as input data to the learning model unit. As the process is performed, a probability value for at least one class corresponding to the evaluation target image is generated among a plurality of classes preset in response to the characteristics of the particle aggregate as the flame and smoke included in the fire phenomenon are modeled in various positions and sizes. It is characterized by including a probability value calculation unit that calculates the probability value.

In addition, a method of learning image data for fire detection training according to another aspect of the present invention uses a first RGB image based on a 3D depth map image captured through a plurality of cameras arranged toward different points in an indoor space as a background. , A virtual object corresponding to a particle assembly modeling a fire phenomenon through a particle system-based fluid simulation rendering technique is synthesized based on the depth information included in the depth map image, and then the virtual object is labeled and created for learning. A step of storing an image data set, and the learning image data set generated based on feature points that are a pixel or combination of pixels detected in response to characteristics of the particle aggregate corresponding to the virtual object included in the first RGB image. performing dictionary learning to enable image classification according to a convolutional neural network by acquiring feature parameters based on Acquiring an RGB image at the current time based on the RGB image as an image to be evaluated, based on feature points and outline information detected from objects included in the image to be evaluated and contrast values for pixels constituting the image to be evaluated. Calculating a weight by calculating the relative ratio of image feature components of the image to be evaluated, and calculating the image to be evaluated through a deep learning-based tensor flow technique to have the same pixel size and range as the training image dataset previously stored. A step of generating a pre-processed image according to normalization, and applying the pre-processed image as input data of a convolutional neural network model that performed the pre-training to perform image learning based on a convolution operation using the weights, thereby producing a fire phenomenon. Comprising the step of calculating a probability value for at least one class corresponding to the evaluation target image among a plurality of classes preset in response to the characteristics of the particle aggregate according to modeling the flame and smoke included in various positions and sizes. It is characterized by

According to the present invention, learning data is generated by compositing a virtual fire image modeled by simulating a fire phenomenon based on a particle system with a background of a pre-photographed image in the fire detection target area, thereby providing information on various locations desired by the user. This has the effect of easily securing learning data for fire detection training.

In addition, according to the present invention, as fire detection learning is performed using evaluation target images obtained in the same background space as the training image dataset, there is an effect of providing high accuracy detection results due to the identity of the background space. .

In addition, according to the present invention, it is possible to generate a large amount of high-quality learning data by virtually creating a fire situation that may occur in various places, thereby overcoming the limitations of the early fire detection ability due to the lack of conventional learning data and achieving a very high level of fire detection. This has the effect of achieving a detection rate of .

1 is a block diagram showing the configuration of an image data generating device for fire detection training according to an embodiment of the present invention;
Figure 2 is a diagram showing an example of a 3D depth map image including RGB color information and depth information acquired by the depth map image acquisition unit of Figure 1;
FIG. 3 is a diagram showing an example of a particle assembly constructed by the particle image generation module of FIG. 1;
FIG. 4 is a diagram for explaining the process of generating a particle composite image by the composite image generator of FIG. 1;
FIG. 5 is a diagram for explaining the process of transparently processing some RGB images synthesized with particle images by the composite image generator of FIG. 1;
FIG. 6 is a diagram illustrating the process of generating a 3D bounding box and a 2D boundary area based on it by the bounding box generator and bounding box transformer of FIG. 1;
FIG. 7 is a diagram illustrating a process of repeatedly generating labeling images as many as a preset reference number by the labeling processing unit and control unit of FIG. 1;
Figure 8 is a flowchart showing a method for generating image data for fire detection training according to an embodiment of the present invention;
Figure 9 is a block diagram showing the configuration of an image data learning device for fire detection training according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating an example of displaying a detection area and a detection probability value corresponding thereto on an image to be evaluated by the output unit of FIG. 9;
Figure 11 is a flowchart showing a method of learning image data for fire detection training according to an embodiment of the present invention.

Specific details, including the problem to be solved by the present invention, the means for solving the problem, and the effect of the invention, are included in the examples and drawings described below. The advantages and features of the present invention and how to achieve them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. Like reference numerals refer to like elements throughout the specification.

Figure 1 is a block diagram showing the configuration of an image data generating device for fire detection training according to an embodiment of the present invention, and Figure 2 includes RGB color information and depth information acquired by the depth map image acquisition unit of Figure 1. It is a diagram showing an example of a 3D depth map image, and FIG. 3 is a diagram showing an example of a particle assembly constructed by the particle image generation module of FIG. 1, and FIG. 4 is a diagram showing a particle composite image by the composite image generator of FIG. 1. This is a diagram for explaining the creation process, and FIG. 5 is a diagram for explaining the process of transparently processing some of the RGB images composited with the particle image by the composite image generator of FIG. 1, and FIG. 6 is the bounding box of FIG. 1. It is a diagram to explain the process of generating a 3D bounding box and a 2D boundary area based on it by the generation unit and the bounding box conversion unit, and FIG. 7 shows a labeling image repeated by the labeling processing unit and the control unit of FIG. 1 by a preset reference number. This is a diagram for explaining the generation process, and Figure 8 is a flowchart showing a method of generating image data for fire detection training according to an embodiment of the present invention.

Hereinafter, an apparatus and method for generating image data for fire detection training according to an embodiment of the present invention will be described with reference to the above-described drawings.

The image data generating device 10 according to an embodiment of the present invention relates to a device that generates training image data for fire detection in indoor spaces. As shown in FIG. 1, it largely includes a depth map image acquisition unit ( 100), a particle image generation module 200, a depth segmentation image generation unit 120, an RGB image extraction unit 300, and a composite image generation unit 400.

The depth map image acquisition unit 100 is used to acquire a 3D depth map image (M _D ) for an indoor space (S100).

The depth map image acquisition unit 100 uses a plurality of cameras (Cam1, Cam2,..., CamN, where N is the number of cameras) arranged toward different points in the indoor space to capture the RGB of the spatial area captured based on that point. A 3D depth map image (M _D ) including color information and depth information can be obtained.

Here, the camera may be an RGB-D camera including an RGB sensor and a depth sensor, and the 3D depth map image (M _D ) is divided into an RGB color information image (FIG. 2A) acquired by the RGB sensor and the depth sensor. It may include a depth information image (FIG. 2b) obtained by .

The particle image generation module 200 generates a plurality of particle images for a particle assembly modeling a virtual fire phenomenon including flames or smoke having various sizes and shapes (S200).

Specifically, the particle image generation module 200 configures a particle assembly (Pa) by modeling a fire phenomenon including flame and smoke using a fluid simulation rendering technique using a plurality of preset particle particles based on a particle system. , it is possible to generate a plurality of particle images (Pi) arranged at different coordinate positions on a virtual plane having the same area range as the depth map image.

In this regard, the particle image generation module 200 may include a characteristic information input unit 210, a particle aggregate configuration unit 220, and a particle image creation unit 230 in detailed configuration.

The characteristic information input unit 210 receives particle characteristic information including color, size, and texture coordinates for the particle from the user (S220).

Here, the particle characteristic information is a plurality of particles constituting the visual effect input from the user through Unity, a particle system-based program that simulates the visual effect (VFX; Visual Effect) desired by the user in virtual space. Setting values for each may include the position (_position), size (_size), movement speed (_velocity), acceleration (_acceleration), color (_color), etc. of the particle in the virtual space.

The particle assembly configuration unit 220 generates fluid entities such as flame and smoke, as shown in FIG. 3, based on particle characteristic information input corresponding to each of the plurality of particle particles from the user through the characteristic information input unit 210. A plurality of particle aggregates (31, 32, 33) are formed according to modeling the fire phenomenon (34) by simulating various positions and sizes (S240).

The particle image generator 230 generates a plurality of particle images (Pi) in which the particle aggregate (Pa) is placed at different coordinate positions on a virtual plane having the same area range as the depth map image (M _D ) (S260) .

The depth segmentation image generator 120 divides the depth map image (M _D ) acquired by the depth map image acquisition unit 100 into a plurality of RGB images (I _R ) according to the depth level (S _D ) is created (S400).

_The RGB image extraction unit 300 generates a pair of RGB images (I Extract _R1 , I _R2 ) (S500).

The composite image generator 400 generates a particle composite image (Ipc) synthesized by arranging each particle image (Pi) and a pair of RGB images (I _R1 , I _R2 ) extracted corresponding thereto in order of depth position. Do it (S600).

For example, referring to FIG. 4, a plurality of particle images (P1, P2, and P3) are generated by the particle image generator 230, and the depth value and adjacent depth level of each particle image are generated by the RGB image extractor 300. When a plurality of RGB images (41, 42, 43) corresponding to (Depth Level) are extracted, the composite image generator 400 generates a plurality of particle images (P1, P2, P3) and RGB images (41, 42, 43) is placed in order of depth position, a pair of RGB images with an indoor space background are placed before and after each particle image corresponding to the virtual object 'fire particle', and then they are combined into one to synthesize particles. An image (Ipc) is created.

Here, when generating the particle composite image (Ipc), the composite image generator 400, as shown in FIG. 5, uses the particle image 52 and the pair of RGB images 51 and 53 corresponding thereto in depth position order ( Depth Level), but made transparent by adjusting the transparency of the RGBA color value of the pixel area of the corresponding RGB image (53) whose depth position is relatively distant among the pair of RGB images (51,53). It can be synthesized later.

In addition, the image data generating device 10 according to an embodiment of the present invention includes a storage unit 240, a bounding box generator 250, and a bounding box conversion unit ( 260), a labeling processing unit 500, a control unit 520, and an image data set storage unit 540 may be further included.

The storage unit 240 stores the size and position information of the pixel area for the particle aggregate (Pa) corresponding to each of the plurality of particle images (Pi) generated by the particle image generator 230 for each particle image ( S320).

The bounding box generator 250 creates a boundary corresponding to the hexahedral shape of the minimum volume surrounding the particle assembly based on the size and position information of the pixel area for the particle assembly (Pa) corresponding to each particle image (Pi). Create a box (BB; bounding box) (S340).

The bounding box conversion unit 260 converts each vertex of the bounding box (BB) generated by the bounding box generator 250 into coordinates on a two-dimensional plane and creates a rectangular shape of the minimum size containing all the points corresponding to the result. Create a border area (BA) (S360).

The bounding box conversion unit 260 acquires the size and position information of the generated boundary area BA and stores it in the corresponding particle image Pi.

The labeling processing unit 500 captures, for each particle composite image (Ipc), a specific area corresponding to the size and position information of the border area (BA) for the particle image (Pi) used to synthesize the corresponding particle composite image. A labeled image (I _L ) is generated (S720).

For example, referring to FIG. 6, when bounding boxes (B1, B2, B3) surrounding particle aggregates corresponding to each of a plurality of particle images are generated by the bounding box generator 250 (61), the bounding box conversion unit After (260) creates a boundary area (A1, A2, A3) by converting the coordinates of each vertex of the bounding box (B1, B2, B3) (62), size and position information of the boundary area (A) is obtained, (63), the labeling processing unit 500 captures the boundary area and generates a labeled image, and these operations are implemented through a coding design (64) based on the deep learning-based Python language. You can.

The control unit 520 is a labeling processing unit to repeatedly generate the labeling image 71 until the number (Ni) of the labeling images (I _L ) generated by the labeling processing unit 500 reaches the preset reference number (Nr). Control 500.

Here, the control unit 520 generates the labeling image 71 until the condition that the number (Ni) of the labeling images (I _L ) matches the reference number (Nr) is met, as shown in FIGS. 7 and 8. The labeling processing unit 500 can be controlled by coding the control algorithm for the command to repeat the step (S720) (72) (S740).

The image dataset storage unit 540 creates and stores an image dataset (Ds) for performing deep learning-based image classification learning using labeling images (I _L ) generated as many as the reference number (Nr).

Based on the description of the components shown in FIG. 1 above, a method for generating image data according to an embodiment of the present invention will be described with reference to FIG. 8 as follows.

First, a 3D depth map image (M _D ) containing RGB color information and depth information for the indoor space area was acquired through multiple cameras (S100), and a particle assembly (Pa) modeling the fire phenomenon based on a particle system was obtained. ) is placed at different coordinate positions on the virtual plane having the same area range as the depth map image (M _D ) to generate a plurality of particle images (Pi) (S200).

Specifically, in step S200, when particle characteristic information for particle particles is input from the user (S220), particles are generated by simulating fluid entities such as flame and smoke at various positions and sizes based on the input particle characteristic information. Constructing an aggregate (Pa) (S240) and generating a plurality of particle images (Pi) in which the constructed particle assembly (Pa) is placed at different coordinate positions on a virtual plane having the same area range as the depth map image (S240) S260).

Next, the size and position information of the pixel area for the particle aggregate (Pa) corresponding to each of the plurality of particle images (Pi) generated in step S200 are stored for each particle image (S320), and then each particle image If a bounding box (BB) corresponding to the hexahedral shape of the minimum volume surrounding the particle assembly is created based on the size and position information of the pixel area for the particle assembly (Pa) corresponding to (Pi) (S340), the boundary box (BB) is generated (S340). A rectangular border area (BA) of the minimum size containing all the points corresponding to the result of converting each vertex of the box (BB) into coordinates on a two-dimensional plane is created (S360), and the size of the border area (BA) and The location information is matched and stored in the corresponding particle image.

Next, after generating a depth segmented image (S D) by dividing the depth map image (M _D ) obtained in step S100 into a plurality of RGB images ( _IR ) according to the depth level ( _S400 ), the depth segmented image When extracting a pair of RGB images (I _R1 , I _R2 ) corresponding to the depth value of the pixel area for the particle assembly (Pa) on each particle image (Pi) and the adjacent depth level from (S _D ) (S500) , a particle composite image (Ipc) is generated by arranging the extracted pair of RGB images (I _R1 , I _R2 ) and the corresponding particle image (Pi) in order of depth position (S600).

Next, for each particle composite image (Ipc), a specific area corresponding to the boundary area (BA) of the particle image (Pi) used in the synthesis of the corresponding particle composite image is captured and labeled as a labeling image (I _L ) is generated (S720), and controlled to repeatedly generate labeling images (I _L ) until the number (Ni) of the generated labeling images reaches a preset reference number (Nr) (S740), and 740 After the step is completed, an image dataset (Ds) for performing deep learning-based image classification learning is created and stored using the labeling images (I _L ) generated as many as the reference number (Nr) (S760).

Meanwhile, the present invention will also describe a learning device and method for performing image classification learning for fire detection training using image data generated by the above-described image data generating device 10 for fire detection training.

In this regard, FIG. 9 is a block diagram showing the configuration of an image data learning device for fire detection training according to an embodiment of the present invention, and FIG. 10 is a detection area and a detection area on an image to be evaluated by the output unit of FIG. 9. This is a diagram showing an example of displaying the corresponding detection probability value, and Figure 11 is a flowchart showing a method of learning image data for fire detection training according to an embodiment of the present invention.

First, the image data learning device 80 for fire detection training according to an embodiment of the present invention includes a learning data storage unit 600, a learning model unit 620, and an image acquisition unit 700, as shown in FIG. 9. , it is configured to include a weight calculation module 720, an image preprocessor 740, and a probability value calculation unit 800.

The learning data storage unit 600 is for storing a learning image dataset (Ds) for performing deep learning-based image classification learning.

Here, the learning image dataset (Ds) is a 3D depth map image (M _D ) Based on the first RGB image (I _R-1 ) as the background, a virtual object corresponding to the particle assembly (Pa) modeling the fire phenomenon through a particle system-based fluid simulation rendering technique was created as a depth map image (M _D ) may be generated by synthesizing based on the depth information included in and then labeling the virtual object.

The learning model unit 620 is generated based on feature points that are a pixel or combination of pixels detected in response to the characteristics of the particle assembly (Pa) corresponding to the virtual object included in the first RGB image (I _R-1 ). As feature parameters are acquired based on the training image dataset (Ds), they are pre-trained to enable image classification according to a convolutional neural network.

At this time, the learning model unit 620 may be implemented by any one of the deep learning learning models among the CNN model, YOLO model, and Fast R-CNN model.

The image acquisition unit 700 is a current viewpoint based on a second RGB image ( _IR-2 ) captured in real time based on a predetermined point in the same indoor space as the 3D depth map image (M _D ) by one of a plurality of cameras. The RGB image of is acquired as the evaluation target image (I _E ).

The weight calculation module 720 calculates the evaluation target image (I _E ) based on the feature point and outline information detected from the object included in the evaluation target image (I _E ) and the contrast value for the pixels constituting the evaluation target image (I E). Calculate the weight (w) by calculating the relative ratio of image feature components of I _E ).

The image pre-processing unit 740 normalizes the evaluation target image (I _E ) to the same pixel size and range as the training image dataset (Ds) previously stored in the learning data storage unit 600 through a deep learning-based tensor flow technique. Create a preprocessed image (Ip).

The probability value calculation unit 800 applies the preprocessed image (Ip) as input data to the learning model unit 620 and performs image learning based on convolution operation using weights (w), thereby performing flame and A detection probability value for at least one class corresponding to the evaluation target image (I _E ) is calculated from among a plurality of preset classes corresponding to the characteristics of the particle assembly (Pa) resulting from modeling the smoke in various positions and sizes.

In addition, the image data learning device 80 according to an embodiment of the present invention includes a location data acquisition unit 640, a detection area generation unit 820, and an output unit ( 900) may further be included.

The position data acquisition unit 640 acquires position data by recording position values according to the shooting positions of each of the plurality of cameras.

When the detection probability value calculated for a certain class by the probability value calculation unit 800 is greater than or equal to a preset reference value, the detection area generator 820 creates a contour for a specific object in the evaluation target image (I _E ) corresponding to the class. A detection area (DA) of the minimum size surrounding the object area based on the information is created and location and size information for it is obtained.

The output unit 900 highlights the detection area (DA) to correspond to the position and size information acquired by the detection area generator 820 on the image to be evaluated (I _E ), and is generated by the probability value calculation unit 800. The detection probability value calculated in response to the detection area (BA) is displayed around the boundary of the detection area.

The output unit 900 corresponds to the camera in which the second RGB image (I _R-2 ) including the evaluation target image (I _E ) is captured around the boundary of the detection area (DA) on the evaluation target image (I _E ). The acquired location data can be displayed.

For example, referring to FIG. 10, when a detection area (DA) surrounding a flame object including a flame is generated by the detection area generator 820, the output unit 900 generates the generated detection area 12. is highlighted on the image to be evaluated (I _E ), and around the border of the detection area 12, the detection probability value (0.85) and pixel coordinates (X ₁ , Y ₁ ) corresponding to the detection area and the image to be evaluated (I At least one of the camera position data (Cam5) corresponding to _E ) can be displayed.

Based on the description of the components shown in FIG. 9 described above, the image data learning method according to an embodiment of the present invention will be described with reference to FIG. 11 as follows.

First, with the first RGB image (I _R-1 ) based on the 3D depth map image (M _D ) captured in the indoor space area as the background, a particle assembly (Pa) corresponding to the fire phenomenon modeled based on the particle system was created. After synthesizing a virtual object based on the depth information included in the depth map image (M _D ), storing the training image dataset (Ds) generated by labeling the virtual object (S800), and storing the stored training image data As the feature parameters are acquired based on the set (Ds), dictionary learning is performed to enable image classification according to the convolutional neural network (S810).

Next, location data is obtained by recording the position value according to the shooting position of each of the plurality of cameras (S820), and the second RGB captured in real time based on a predetermined point in the indoor space by one of the plurality of cameras The current RGB image based on the image ( _IR-2 ) is acquired as the evaluation target image (I _E ) (S830).

Next, the evaluation target image (I _E ) is evaluated based on the feature points and outline information detected from the object included in the evaluation target image (I _E ) and the contrast values for the pixels constituting the evaluation target image (I _E ). Preprocessing image (Ip) by calculating the weight (w) (S840) and normalizing the evaluation target image (I _E ) to the same pixel size and range as the previously stored training image dataset (Ds) through deep learning-based tensor flow technique. ) is generated (S850), the pre-processed image (Ip) is applied as input data to the convolutional neural network model that performed pre-training in step S810, and image learning based on convolution operation using weights (w) is performed. Accordingly, a probability value for at least one class corresponding to the image to be evaluated (I _E ) among the plurality of preset classes is calculated (S900).

Next, if the probability value calculated for a given class in step S900 is greater than or equal to a preset reference value, the minimum size surrounding the object area based on the outline information for a specific object in the evaluation target image (I _E ) corresponding to the class is determined. After creating a rectangular detection area (DA) and obtaining its position and size information (S910), the detection area (DA) is highlighted on the image to be evaluated (I _E ) to correspond to the obtained position and size information. Do it (S920).

At this time, in step S920, a detection probability value calculated corresponding to the detection area DA in step S900 is placed around the boundary of the corresponding detection area DA, and a pixel obtained corresponding to the detection area DA in step S910. At least one of coordinates and camera position data obtained in response to the image to be evaluated (I _E ) in step S820 may be displayed.

Accordingly, according to the present invention described above, learning data is generated by synthesizing a virtual fire image modeled by simulating a fire phenomenon based on a particle system with a background image previously taken in the fire detection target area, so that the user can Learning data for fire detection training for various desired locations can be easily obtained.

In addition, according to the present invention, as fire detection learning is performed using an evaluation target image acquired in the same background space as the training image dataset, high accuracy detection results can be provided due to the identity of the background space.

In addition, according to the present invention, it is possible to generate a large amount of high-quality learning data by virtually creating a fire situation that may occur in various places, thereby overcoming the limitations of the early fire detection ability due to the lack of conventional learning data and achieving a very high level of fire detection. A detection rate of can be obtained.

Although the present invention has been described in detail through preferred embodiments, the present invention is not limited thereto and may be implemented in various ways within the scope of the patent claims.

In particular, the foregoing has described the features and technical strengths of the present invention rather broadly to enable a better understanding of the claims of the invention to be described later. Therefore, the concept and specific embodiments of the present invention described above are intended to serve a similar purpose as the present invention. It should be recognized by those skilled in the art that it can be immediately used as a basis for the design or modification of other shapes for use.

In addition, the embodiment described above is only one embodiment according to the present invention, and can be implemented in various modified and changed forms by those skilled in the art within the scope of the technical idea of the present invention. You will understand. Accordingly, the disclosed embodiments should be considered from an explanatory rather than a limiting perspective, and various modifications and changes thereof are also included in the scope of the technical spirit of the present invention and are indicated in the claims of the present invention, and the scope equivalent thereto is thereto. All differences therein should be construed as being included in the present invention.

10: Image data generation device 80: Image data learning device
100: Depth map image acquisition unit 120: Depth segmentation image generation unit
200: Particle image creation module 210: Characteristic information input unit
220: Particle assembly configuration unit 230: Particle image creation unit
240: storage unit 250: bounding box generation unit
260: Bounding box conversion unit 300: RGB image extraction unit
400: composite image generation unit 500: labeling processing unit
520: Control unit 540: Image data set storage unit
600: Learning data storage unit 620: Learning model unit
640: Location data acquisition unit 700: Image acquisition unit
720: Weight calculation module 740: Image preprocessor
800: Probability value calculation unit 820: Detection area generation unit
900: output unit

Claims (16)

In a device that generates training image data for fire detection in indoor spaces,
a depth map image acquisition unit that acquires a 3D depth map image including RGB color information and depth information for a spatial region photographed based on a corresponding point through a plurality of cameras arranged toward different points in the indoor space;
A characteristic information input unit that receives particle characteristic information including color, size, and texture coordinates for a plurality of preset particle systems based on a particle system from the user, and a characteristic information input unit based on the particle characteristic information input corresponding to each of the plurality of particle particles. A particle assembly component that constitutes a particle assembly for modeling a fire phenomenon by simulating fluid entities such as flame and smoke in various positions and sizes through a fluid simulation rendering technique, and a particle assembly component that configures the particle assembly identical to the depth map image. A particle image generation module comprising a particle image generation unit that generates a plurality of particle images arranged at different coordinate positions on a virtual plane having an area range;
a depth segmentation image generator that generates a depth segmentation image by dividing the acquired depth map image into a plurality of RGB images according to depth levels;
an RGB image extraction unit that extracts a pair of RGB images corresponding to a depth value adjacent to a depth value of a pixel area for the particle aggregate on each particle image from the depth segmented image;
a composite image generator that generates a composite particle image by arranging each of the particle images and the pair of RGB images extracted corresponding thereto in order of depth position;
a storage unit that stores size and position information of a pixel area for the particle aggregate corresponding to each of the plurality of particle images for each particle image;
a bounding box generator that generates a bounding box corresponding to a hexahedral shape with a minimum volume surrounding the particle assembly based on the size and position information of the pixel area for the particle assembly corresponding to each particle image; and
After converting each vertex of the generated bounding box into coordinates on a two-dimensional plane, a rectangular border area of the minimum size is created containing all the corresponding points, and then the size and position information of the border area are obtained and corresponding to it. An image data generation device for fire detection training, comprising a bounding box conversion unit that matches and stores the particle image. According to paragraph 1,
The composite image generator,
When generating the particle composite image, the particle image and the pair of RGB images corresponding thereto are arranged in order of depth position, and the pixel on the RGB image whose depth position is relatively distant among the pair of RGB images An image data generation device for fire detection training, characterized in that it adjusts the transparency of the RGBA color value of the area, processes it transparently, and then synthesizes it. delete delete According to paragraph 1,
A labeling processing unit that generates a labeled image by capturing and labeling a specific area corresponding to the size and position information of the boundary area for the particle image used in the synthesis of the particle synthesis image for each particle synthesis image;
a control unit that controls the labeling processing unit to repeatedly generate the labeling images until the number of the labeling images reaches a preset standard number; and
Image data for fire detection training, further comprising: an image dataset storage unit that generates and stores an image dataset for performing deep learning-based image classification learning using the labeling images generated by the reference number. Generating device. delete delete delete delete delete With the first RGB image based on a 3D depth map image captured through multiple cameras placed toward different points in the indoor space as the background, a particle assembly modeling the fire phenomenon was created using a particle system-based fluid simulation rendering technique. a learning data storage unit that stores a training image dataset generated by synthesizing a corresponding virtual object based on depth information included in the depth map image and labeling the virtual object;
By acquiring feature parameters based on the learning image dataset generated based on feature points that are a pixel or combination of pixels detected in response to the characteristics of the particle aggregate corresponding to the virtual object included in the first RGB image, A learning model unit pre-trained to enable image classification according to a convolutional neural network;
an image acquisition unit that acquires a current RGB image based on a second RGB image captured in real time based on a predetermined point in the indoor space by one of the plurality of cameras as an image to be evaluated;
Weights for calculating the relative ratio of image feature components of the image to be evaluated based on feature points and outline information detected from objects included in the image to be evaluated and contrast values for pixels constituting the image to be evaluated. a weight calculation module that calculates;
An image preprocessor that generates a preprocessed image by normalizing the evaluation target image to the same pixel size and range as the previously stored learning image dataset through a deep learning-based tensor flow technique; and
As image learning based on convolution calculation using the weight is performed by applying the pre-processed image as input data of the learning model unit, the particle aggregate according to modeling the flame and smoke included in the fire phenomenon in various positions and sizes An image data learning device for fire detection training, comprising: a probability value calculation unit that calculates a detection probability value for at least one class corresponding to the evaluation target image among a plurality of classes preset corresponding to features. According to clause 11,
If the calculated detection probability value for a given class is greater than the preset reference value, a rectangular detection area of the minimum size surrounding the object area is created based on the outline information for the specific object in the evaluation target image corresponding to the class. a detection area generator that obtains location and size information; and
It further includes an output unit that highlights the detection area to correspond to the acquired position and size information on the evaluation target image, and displays the detection probability value calculated in response to the detection area around the boundary of the detection area. An image data learning device for fire detection training, characterized in that. According to clause 12,
It further includes a location data acquisition unit that acquires location data by recording location values according to the shooting positions of each of the plurality of cameras,
The output unit,
An image data learning device for fire detection training, characterized in that it displays location data obtained in response to a camera that captured the second RGB image including the evaluation target image around the boundary of the detection area on the evaluation target image. . delete delete delete

Images