KR102395440B1 - Cloud-based building management system and method thereof - Google Patents

Abstract

The present invention relates to a cloud-based building management system. The cloud-based building management system relates to an intelligent edge device which generates first metadata by analyzing images provided by cameras installed in buildings according to first event detection conditions, wherein the first event detection condition is a condition for detecting a human object, and the intelligent edge device may include a cloud server which is located in a cloud accessible through a communication network and adjusts environmental variables of a space where an image was captured by analyzing first meta data.

Description

CLOUD-BASED BUILDING MANAGEMENT SYSTEM AND METHOD THEREOF

The present invention relates to a cloud-based building management system.

The building management system is a system that helps the efficient management of buildings by collecting and analyzing various information of building equipment in real time. The building management system measures the indoor environment and energy use by using a plurality of sensors. The measurement data can be used to identify inefficient operating facilities through building facility operation analysis and energy consumption analysis, provide a comfortable environment through optimal facility control, and maximize energy savings.

The building management system uses the measurement data of the sensor, but elements that cannot be measured by the sensor are predicted through statistical data or simulation. In order to efficiently consume building energy, it is necessary that the measurement target is diverse and the measurement data is precise. Energy management by area (eg, floor, shared/private space) or by target (eg, occupant, vehicle) is also possible, which requires a large number of sensors. However, since the sensor is also an electrical device that consumes energy, installing a sensor for energy management may rather increase energy consumption. On the other hand, since the sensor measures only a specific object in a fixed position, it can precisely measure the environmental change within the measurement range, but cannot measure other measurement objects or environmental changes in a space outside the measurement range.

Korean Patent Publication No. 10-2017-0080384

An embodiment of the present invention is to provide a cloud-based building management system that can efficiently manage buildings by measuring environmental changes using image analysis and recognizing changes in circumstances.

On the other hand, the embodiment of the present invention replaces a significant part of the existing sensors of the building management system with image analysis, thereby reducing the number and types of sensors required for the operation of the building management system, and in particular, the measurement range expanded to the viewing range of the camera. want to implement

In addition, an embodiment of the present invention is to facilitate system construction and maintenance by implementing a building facility control function in the cloud according to the analysis and analysis results of images collected inside and outside the building, but to minimize the occurrence of data traffic due to image analysis.

According to one aspect of the present invention, a cloud-based building management system is provided. The cloud-based building management system is an intelligent edge device that generates first metadata by analyzing an image provided by a camera installed in a building according to a first event detection condition, wherein the first event detection condition is a condition for detecting a human object Im- and the intelligent edge device is located in a cloud accessible through a communication network, and may include a cloud server that analyzes the first metadata and adjusts an environment variable of a space in which the image is captured.

In an embodiment, the cloud server analyzes the first metadata to identify a space to adjust the environment variable from among the space where the image is captured, and an environment analysis server for designating an environment variable to be adjusted in the identified space; and an environment control server that transmits a control command to a building equipment controller in charge of the identified space according to the environment variable to be adjusted.

In an embodiment, the intelligent edge device generates second meta data according to a second event detection condition defining a human object-related event, and the environment analysis server analyzes the second meta data to control when an abnormal behavior is detected It can be displayed on the terminal.

In one embodiment, the intelligent edge device generates third metadata according to a third event detection condition defining a building safety-related event, and the environment analysis server analyzes the third metadata to control when an abnormal behavior is detected. It can be displayed on the terminal.

In an embodiment, the environment analysis server may identify a space in which the environment variable is to be adjusted among spaces in which the human object is not detected by comprehensively analyzing a plurality of metadata according to a time sequence or a spatial location.

In one embodiment, the intelligent edge device receives the measurement data including the measurement value of the environment variable from the environmental sensor installed for each space of the building and transmits it to the environment analysis server, wherein the environment analysis server includes the first meta data And by analyzing the measurement data, it is possible to identify a space to adjust the environment variable from among the space in which the image is captured, and to designate an environment variable to be adjusted in the identified space.

According to another aspect of the present invention, a cloud-based building management system is provided. The cloud-based building management system is an intelligent edge device that generates first metadata by analyzing an image provided by a camera installed in a building according to a first event detection condition - Here, the first event detection condition is to detect a human object The condition is that the intelligent edge device is located in a cloud accessible through a communication network, and an environment analysis server that analyzes the first metadata to identify a space to adjust environment variables in the space in which the image is captured and is located in the cloud, , an environment control server that transmits a control command for adjusting the environment variable to the building equipment controller with reference to the current state of the environment variable related to the identified space.

In one embodiment, the intelligent edge device generates second metadata according to a second event detection condition defining a human object related event and third metadata according to a third event detection condition defining a building safety related event, When the environment analysis server analyzes the second metadata and the third metadata and detects an abnormal behavior, it may be displayed on the control terminal.

In one embodiment, the intelligent edge device receives the measurement data including the measurement value of the environment variable from the environmental sensor installed for each space of the building and transmits it to the environment analysis server, wherein the environment analysis server includes the first meta data can be analyzed to identify a space in which the human object is detected from among the spaces in which the image is captured.

In an embodiment, the environment analysis server may identify a space in which the environment variable is to be adjusted among spaces in which the human object is not detected by comprehensively analyzing a plurality of metadata according to a time sequence or a spatial location.

According to another aspect of the present invention, there is provided a building management method using image analysis executed in a cloud-based building management system. The cloud-based building management method includes generating first metadata by analyzing an image provided by a camera installed in a building by an intelligent edge device according to a first event detection condition, wherein the first event detection condition detects a human object condition, wherein the environment analysis server analyzes the first meta data to identify a space to adjust the environment variable from among the space where the image is captured, the environment control server refers to the current state of the environment variable related to the identified space It may include generating a control command for adjusting the environment variable, and the building equipment controller adjusting the environment variable of the identified space according to the control command.

In one embodiment, the cloud-based building management method includes: generating, by the intelligent edge device, second metadata according to a second event detection condition in which a human object-related event is defined; and the environment analysis server using the second metadata and the The method may further include detecting an abnormal behavior by analyzing the third metadata.

In an embodiment, the cloud-based building management method may further include transmitting, by the intelligent edge device, measurement data including a measurement value provided by an environmental sensor installed in the building to the environment analysis server.

In one embodiment, the cloud-based building management method may further include the step of designating, by the environment analysis server, an environment variable to be adjusted in the identified space by analyzing the first metadata and the measurement data.

According to an embodiment of the present invention, the cloud-based building management system can measure environmental changes through image analysis and recognize changes in circumstances, thereby making building management more efficient.

On the other hand, the embodiment according to the present invention can replace a significant portion of the existing sensors of the building management system with image analysis, and can provide a security function that the existing building management system cannot provide.

In addition, the embodiment of the present invention has the effect of facilitating system construction and maintenance by implementing a building facility control function according to the analysis result and analysis of images collected inside and outside the building in the cloud.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described with reference to the embodiments shown in the accompanying drawings. For ease of understanding, like elements have been assigned like reference numerals throughout the accompanying drawings. The configuration shown in the accompanying drawings is merely an exemplary embodiment for explaining the present invention, and is not intended to limit the scope of the present invention. In particular, in the accompanying drawings, some of the elements represented in the drawings are somewhat exaggerated to help the understanding of the invention.
1 is a view for explaining a cloud-based building management system by way of example.
2 is a functional diagram illustrating the configuration of an intelligent edge device of a cloud-based building management system.
3 is a functional diagram illustrating the configuration of a cloud server of a cloud-based building management system.
4 is a diagram for illustratively explaining image analysis by an intelligent edge device.
5 is a flowchart for exemplarily explaining a process of managing a building through image analysis.
6 is a flowchart for exemplarily explaining a process of managing a building through measurement data.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail through the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In particular, functions, features, and embodiments to be described below with reference to the accompanying drawings may be implemented alone or in combination with other embodiments. Therefore, it should be noted that the scope of the present invention is not limited to the forms shown in the accompanying drawings. Meanwhile, throughout the accompanying drawings, the same or similar elements are referred to using the same reference numerals.

1 is a view for explaining a cloud-based building management system by way of example.

The cloud-based building management system according to an embodiment of the present invention can efficiently manage energy as well as maintain safety/security of a building through images captured in the building. The camera 10 is installed inside the building (additionally outside the building), and in recent years, it is installed substantially throughout the inside of the building except for private spaces such as toilets. Therefore, although the image captured by the camera 10 reflects all events occurring inside the building in real time, it is mainly used for follow-up measures in case of unauthorized intrusion or theft.

In general, a building management system uses a motion sensor that detects a motion of an object such as a person or a vehicle. For example, if no movement of people or vehicles is detected in a hallway or parking lot, the lights can be turned off automatically. That is, human object detection and movement tracking are factors that enable more efficient energy management. However, in order to cover a large space, a fairly large number of motion sensors must be installed, which can conversely result in increased energy consumption. In particular, while the movement of the object is easy to detect by the motion sensor, it is not easy to detect a staying object, for example, a person who sits on a chair for a long time.

The cloud-based building management system according to an embodiment of the present invention detects a human object located inside a building and a human object-related event through image analysis, and can efficiently manage the indoor environment of the building in terms of energy based on the detected event. can Image analysis is a process of identifying an object in an image and detecting an event related to the identified object. An object is an object identified in an image for analysis, and may include not only objects that are detected as events, such as people, vehicles, animals, etc. there is. Since the camera 10 installed inside the building already provides images of a public space and a dedicated space requiring indoor environment control, installation of a new facility is not required for image analysis.

The cloud-based building management system according to an embodiment of the present invention is built in the intelligent edge device 100 and the cloud that detects human objects and human object-related events occurring inside a building, and analyzes the events detected by the intelligent edge device 100 Thus, it may be configured as a cloud server 200 that manages the indoor environment. Instead of transmitting the image captured by the camera 10 to the cloud server 200, the intelligent edge device 100 converts any one or both of the human object and the human object-related event detected according to the learned model as metadata. It is converted and transmitted to the cloud server 200 . This can significantly reduce data traffic. Meanwhile, the cloud server 200 may adjust the indoor environment of the building by analyzing the metadata.

Compared to a case where a server having the same function as the cloud server 200 is installed in a building (hereinafter referred to as a built-in server), the cloud server 200 can be used at a relatively low cost. Since it is continuously interlocked with the camera 10 and an image must be stored for analysis, the built-in server requires a relatively large initial cost, but does not need to pay a monthly fee. However, maintenance is required continuously after installation, which is expensive. On the other hand, in the case of the cloud server 200, the initial cost is relatively small. Compared to a built-in server, you have to pay a monthly fee, but considering the initial cost and maintenance cost of a built-in server, it is relatively cheap. This is because, considering the cloud's charging policy that charges according to the amount of data traffic, metadata generates traffic at a level that is almost negligible compared to the image while including information related to images, objects, or events. In particular, the cloud server 200 has the advantage of not requiring maintenance from the user's point of view.

The intelligent edge device 100 is trained to detect a human object in an image captured by the camera 10 . Additionally, the intelligent edge device 100 may be trained to track human objects and detect occurrence of human object-related events. The human object-related events that the intelligent edge device 100 can detect through learning are exemplified in the following table.

Event classification explanation People Counting and Congestion Counting the number of people entering and exiting a specific area and detecting congestion due to access by more than a certain number of people invasion Detects the moment an unauthorized person enters the trespassing, trespassing or prohibited area visit When a detected object is continuously detected in the same place for a certain period of time, it is judged as a residence status prowl After detecting a human object, suspicious behavior such as wandering around or wandering for a certain period of time without a specific event is detected. abandonment Detection of new objects separated from detected human objects fall down Falling to the floor while walking or standing is detected as a fall when the head touches the floor Face recognition and detection A function that recognizes a human face from a camera image and extracts facial feature points (FFV) Similarity analysis Detects authorized/unauthorized persons by comparing the facial feature points detected from the camera image and the similarity with the face DB stored in the DB

Additionally, the intelligent edge device 100 may be trained to detect the occurrence of an event related to building safety. Building safety-related events that the intelligent edge device 100 can learn and detect are exemplified in the following table.

Event classification explanation damage Detects a deformed state of a part of a building in the camera image fire Detected when flame or smoke is detected in the camera image

In addition, the intelligent edge device 100 may be trained to detect the occurrence of an event related to the indoor environment through the action of the human object. The indoor environment-related events that the intelligent edge device 100 can learn and detect are exemplified in the following table.

Event classification explanation high temperature Detects high indoor temperature when multiple people take off their clothes or fan them low temperature Detects low indoor temperature when multiple people are wearing clothes or turning on a heating device air pollution When a large number of people cover their faces with their hands or clothes, a state in which dust or odor has entered the room is detected.

The intelligent edge device 100 may receive measurement data from the sensor 11 that measures an environmental variable affecting the indoor environment and transmit it to the cloud server 200 . The measurement data may include information for identifying the measured value of the environmental sensor 11 and the measured environmental variable. The environment variable may be, for example, temperature, humidity, illuminance, gas concentration, fine dust concentration, and the like. The environment variable may be measured for each space, and the cloud server 200 may control the building equipment according to the measurement data. Unlike human object and human object event detection, the intelligent edge device 100 may transmit the measurement data to the cloud server 200 as it is without analysis. The intelligent edge device 100 may serve as a router that transmits a control command from the cloud server 200 to building equipment that adjusts the indoor environment. FIG. 2 is a functional diagram illustrating the configuration of a cloud-based building management system. am.

The cloud-based building management system according to an embodiment of the present invention includes an intelligent edge device 100 that generates metadata by analyzing an image captured inside a building, and a cloud server 200 that analyzes metadata to adjust the indoor environment. include The cloud server 200 may analyze the metadata received from the plurality of intelligent edge devices 100 .

The intelligent edge device 100 may generate metadata by analyzing an image captured by the camera 10 installed in a building (hereinafter, referred to as primary analysis). Meanwhile, the intelligent edge device 100 may receive measurement data generated by the environmental sensor 11 installed for each space and transmit the measurement data or metadata about the measurement data to the cloud server 200 . The intelligent edge device 100 may learn to suit the location where the camera 10 is installed and field conditions, and may analyze an image based on the learning performed. The intelligent edge device 100 may be trained to detect, for example, any one or a combination of a human object, a human object-related event, a building safety-related event, and an indoor environment-related event. The generated metadata may be analyzed (hereinafter referred to as secondary analysis) by the environment analysis server 210 . In particular, since the image can be transmitted only when the environment analysis server 210 requests it, it is not limited by the type of communication network, accessibility, bandwidth, etc., and it is possible to detect an event through rapid image analysis.

The metadata includes information on any one of a human object detected in the image, a human object-related event, a building safety-related event, an indoor environment-related event, and measurement data, and additionally information about the space where the image was captured (hereinafter, spatial information). ) may be further included. The spatial information can be specified by the location where the camera is installed. Meanwhile, the metadata may further include information on the position of the human object in the image or the actual position of the human object calculated based thereon. In addition, the metadata may include measurement data generated by the environmental sensor.

The intelligent edge device 100 includes an image data processing module 110 , an image analysis module 120 , a face recognition module 130 , and a communication module 140 . The image data processing module 110 and the image analysis module 120 may be software in the form of modules. That is, the intelligent edge device 100 includes physical components such as one or more central processing units (eg, CPU, GPU, etc.), a semiconductor memory, a communication modem chip, and the like, and the image data processing module 110 and image analysis The module 120 functionally divides and expresses the operation of the program loaded into the memory being executed by the CPU.

The image data processing module 110 receives an image from one or more cameras 10 through the communication network 30 , and pre-processes the received image appropriately for image analysis. To this end, the image data processing module 110 may include an RTSP client, a decoder, and an image preprocessing module. The RTSP client is a communication modem supporting an image transmission protocol, for example, RTSP (Real time streaming protocol), and receives image data from the camera 10 through a communication network. The decoder decodes the received image data to restore the image. The image preprocessing module changes the resolution, size, etc. of the reconstructed image or converts a color image into a black and white image so as to be suitable for image analysis.

The image analysis module 120 detects an object in the pre-processed image, tracks the movement of the object, and detects an event by analyzing the behavior of the object being tracked. The image analysis module 120 may include an object detection unit, an object tracking unit, and an event detection unit. Object detection and object tracking may be performed as a series of inseparable processes. Meanwhile, the image analysis module 120 may detect an event using two or more image analysis methods, and for this purpose, detection and tracking of an object may be performed independently.

The object detector may detect one or more objects, in particular, a human object, from the pre-processed image from the pre-processed image, and classify the detected objects by type. The object detection unit may be, for example, an object detection unit learned using an object image or an object detection unit using a template expressing the object. In particular, the object detector may determine a 3D coordinate value capable of expressing the detected object by applying a 3D geometric filter. When the determined 3D coordinate value is used, the detected object may be displayed on an image as a 3D figure or used for object tracking. In addition, the object detector may identify the same object from the images captured by the two or more cameras 10 .

The object tracking unit tracks the motion of the detected object. The object tracker may track the object through motion prediction of the object or track the object by comparing properties of the object. The object tracking method predicts the motion of an object detected in the first image (or frame) or detects a property, and compares it with the object detected in the second image (or frame) to determine whether two objects are the same object. Various object tracking methods may be applied, such as a method of tracking the motion of an object by comparing in units of macroblocks constituting an image, or a method of detecting and tracking an object by removing a background from an image so that only the object remains in the image.

The event detector detects an event that meets an event detection condition based on any one or a combination of the detected object, the movement of the object, and the surrounding environment of the object. Here, the surrounding environment of the object may be obtained from measurement data. The event detector may detect an event using two or more different image analysis methods, for example, a rule-based analysis and a deep learning analysis method. The deep learning event detection unit extracts a property of the detected object, and determines whether it meets a set event detection condition based on the property. The rule-based event detector detects an event that meets an event detection condition based on the movement of the object in the ROI. If there are a plurality of event detection conditions set to perform the rule-based method, there may be a plurality of rule-based event detection units operating according to each event detection condition. Similarly, if there are a plurality of event detection conditions set to be performed by the deep learning analysis method, there may also be a plurality of deep learning event detection units operating according to each event detection condition. Here, the situation in which there are a plurality of event detection conditions may include a case in which two or more event detection conditions are applied to one image as well as a case in which there are two or more images to be analyzed.

The face recognition module 130 may distinguish an authorized person from an unauthorized person by identifying a face of a human object. The face recognition module 130 may detect a face from a human object, detect a landmark from the detected face, and compare the detected landmark with a landmark of a registered face. Through this, it may be determined whether the detected human object is an authorized person. In addition to the method using the landmark, various face recognition methods may be applied.

The communication module 140 transmits the metadata generated by the image analysis module 120 to the cloud server 200 . Additionally, the communication module 140 transmits the measurement data or metadata for the measurement data generated by the environmental sensor 11 to the cloud server 200, and controls the building equipment controller 12 from the cloud server 200 The control command may be received, and the received control command may be provided to the building equipment controller 12 . The communication module 140 supports a communication protocol for transmitting digital data in the form of packets, and may communicate with the cloud server 200 through a wired, wireless, or wired/wireless mixed data communication network. The wired communication network may be a dedicated line or cable network, etc., and the wireless communication network is CDMA, WCDMA, GSM, Evolved Packet Core (EPC), LTE (Long Term Evolution), Wibro, as well as Wi-Fi, such as Bluetooth, Zigbee, etc. It may be a communication system that transmits data using a wireless signal.

The camera 10 includes a CCTV, an IP camera, etc. that generate a 2D image or a 3D image, and may include both a fixed camera and a Pan-Tlit-Zoom (PTZ) camera. The camera 20 is installed in a public space of a building (eg, a building entrance, a hallway, a staircase, an elevator, a parking lot, etc.) as well as a dedicated space (eg, an office to which only authorized persons can enter) to photograph the space. there is.

The environmental sensor 11 is a sensor that measures environmental variables representing the internal environment of a building, for example, temperature, humidity, illuminance, gas concentration, fine dust concentration, and the like. One environmental sensor 11 measures only an environmental variable for one space, one environmental sensor 11 measures an environmental variable across two or more spaces, or two or more kinds of environmental sensors 11 in one space You can also measure environment variables for On the other hand, a device for detecting fire/flame, a power meter for measuring power consumption by space (floor, office), and a device for measuring temperature/noise of building equipment can also be viewed as the environmental sensor 11 in a broad sense.

The building equipment controller 12 controls the building equipment for controlling the indoor environment of the building. For example, the building equipment may be a heating/air conditioning device, a lighting device, an air conditioning device, and the like, and may further include a facility capable of controlling the indoor environment of the building in terms of energy. In order to independently control the indoor environment for each space, building equipment may be provided for each space.

3 is a functional diagram illustrating the configuration of a cloud server of a cloud-based building management system.

Referring to FIG. 3 , the cloud server 200 may include an environment analysis server 210 that analyzes metadata, and an environment control server 220 that adjusts an indoor environment according to the metadata analysis result. Additionally, the cloud server 200 may further include a metadata storage server 230 for storing metadata received from the intelligent edge device 100 .

The metadata storage server 230 stores metadata received through a communication network. The stored metadata may be provided to the environment analysis server 210 or utilized for search by the control terminal 300 . The measurement data transmitted in the meta data format may be transmitted to the environment control server 220 .

The environment analysis server 210 may analyze the metadata to identify a space that needs to adjust the environment variables. Among the detected spaces, the space to adjust the environment variable is provided as an analysis result. Additionally, the environment analysis server 210 may designate an environment variable to be adjusted. The environmental variable to be controlled may be designated based on any one of metadata and measurement data for an image, and may include a measured value of the environment variable. For example, an environmental variable such as illuminance may be measured through image analysis, and a gas concentration may be measured only through the environmental sensor 11 . Since the metadata may include spatial information, the environment analysis server 210 analyzes the environment of the space in which the human object is detected through the metadata. For example, when a human object is detected in an office where the air conditioning and lighting are turned off, the environment analysis server 210 designates "the lake of the office" to identify the space in which the human object is detected, and "temperature" as the adjustment target environment variable -16 degrees” and “brightness-20 lux” can be specified. The space and environment variables designated by the environment analysis server 210 may be transmitted to the environment control server 220 .

The environment analysis server 210 may detect a human object-related event, a building safety-related event, or an indoor environment-related event by analyzing metadata in a rule-based analysis or deep learning analysis method. The environment analysis server 210 analyzes the meta data for the human object to detect the human object-related event exemplified in Table 1, or analyzes the meta data for the building to detect the building safety-related event exemplified in Table 2 there is.

The environment analysis server 210 may comprehensively analyze the metadata transmitted by the intelligent edge device 100 to detect an event having a higher analysis difficulty than the intelligent edge device 100 .

First, the environment analysis server 210 may analyze a plurality of metadata according to a time sequence or spatially. When an object (or event) is detected and the object disappears from the space in which the environment variable is adjusted (or the event is no longer detected), the adjusted environment variable may be maintained. For example, when a human object is detected in a dark hallway, the lighting in the hallway is turned on, and when the human object disappears, it is no longer necessary to turn on the lighting. The metadata generated by the intelligent edge device 100 may indicate human object detection, lighting, and the like. If new metadata for the corridor on which the light is turned on is not received for a certain period of time, the environment analysis server 210 may determine that a person has passed the corridor and designate the lighting as an environment variable to be adjusted.

Next, the environment analysis server 210 separates the abnormal behavior from the normal behavior by considering not only the metadata but also the metadata for the metadata, for example, the generation (or reception) time of the metadata, the frequency of metadata generation, etc. can For example, the handling of metadata generated during working hours on weekdays and metadata generated during late night may be different. If a plurality of human objects are detected in the office during working hours on a weekday, there is a high probability that they are a working authorized person. Therefore, the environment analysis server 210 may perform only the analysis necessary for indoor environment control. If a human object is detected in the office at night, there is a high probability that it is an unauthorized intruder. Therefore, the face recognition of the operator can be performed.

The environment analysis server 210 may analyze the metadata received from the intelligent edge device 100 to detect an abnormal behavior of the human object. Also, the environment analysis server 210 may provide a new or temporary event detection condition to the plurality of intelligent edge devices 100 . For example, when a new unauthorized person is registered with respect to any one of the plurality of intelligent edge devices 100 , the environment analysis server 210 may also register it with the other intelligent edge devices 100 . Meanwhile, the environment analysis server 210 may display an analysis result to the control terminal 300 when an abnormal behavior is detected or according to a request from the control terminal 300 . The control terminal 300 may be a computer, a smartphone, or a tablet that accesses the cloud server 200 through a communication network to perform building management and security tasks.

The environment control server 220 generates a control command for adjusting the indoor environment according to the analysis result of the environment analysis server 210 and transmits it to the building equipment controller 12 . Based on any one or all of the space and environment variables specified by the environment analysis server 210, the environment control server 220 is a space specified by the environment analysis server 210 of the building equipment controller 12 that adjusts the specified environment variables It is possible to select the building equipment controller 12 in charge of , and transmit a driving start or driving stop command to the selected building equipment controller 12 . The environment control server 220 stores the state of the environment variable for each space and generates a control command with reference to the stored state. For example, if the environment analysis server 210 designates lighting for an arbitrary space, the environment control server 220 may generate a control command to turn off the lighting with reference to the current state (eg, lighting is on). there is.

Meanwhile, the environment control server 220 may generate a control command for adjusting the indoor environment according to the measurement data transmitted by the environment sensor 11 . The measurement data may be transmitted to the environment control server 220 without going through the environment analysis server 210 .

4 is a diagram for illustratively explaining image analysis by an intelligent edge device.

The intelligent edge device 100 may generate metadata by analyzing an image using either one of the rule-based analysis 120a and the deep learning analysis method 120b. The rule-based analysis 120a specifies an action of the object based on the movement of the object in a pre-designated area (hereinafter referred to as the area of interest) to detect the object, and determines whether the action meets a preset event detection condition. . The deep learning analysis (120b) is a deep learning algorithm that learns an object and an action related to the object, extracts an image or a property of an object in the video, and determines whether the action of the object with the property meets a preset event detection condition method. In FIG. 4, a rule-based analysis method will be mainly described.

Referring to FIG. 4 , three spaces S701 , S702 , and S703 may be specified from a plan view of the illustrated building. The first space S701 and the third space S703 are an office, that is, a dedicated space, and the second space S702 is a hallway, that is, a common space. Access to and from the first space S701 and the third space S703 and movement to another floor are possible only through the second space S702. The first and second cameras 10a and 10b photograph the second space S702, and the third camera 10c photographs the third space S703. A camera is not installed in the first space S701.

The first camera 10a is disposed toward the elevator and captures the region of interest ROI1. The second camera 10b is disposed toward the second space S702 and captures regions of interest ROI2 and ROI3. The third camera 10c is disposed toward the third space S703 and captures regions of interest ROI4 and ROI5. That is, since a plurality of regions of interest may be defined within a region captured by the camera, event detection conditions may be set differently for each region, or environmental variables may be adjusted differently for each region.

The region of interest ROI1 may be defined for congestion detection. When the degree of congestion increases, human objects overlap, and a tracking error in which a plurality of objects is recognized as one object may occur. If the 3D geometric filter is applied to the region of interest ROI1 having a high degree of congestion, such as in front of an elevator, tracking errors due to the overlap of human objects can be reduced.

The region of interest ROI2 may be defined for counting human objects. The region of interest ROI2 may be used to generate metadata necessary for indirectly determining whether a person resides in the first space S701 in which a camera is not installed.

The region of interest ROI3 may be defined to detect a human object passing through the second space S702. Meta data generated for the region of interest ROI3 may be used to adjust the illumination of the second space S702.

Regions of interest ROI4 and ROI5 may be defined for detecting retention of a human object. Meta data on the residence of the human object may be used to control lighting, air conditioning, ventilation, etc. of the third space S703.

5 is a flowchart for exemplarily explaining a process of managing a building through image analysis.

In S10, the intelligent edge device 100 receives an image from the camera 10 installed in the building.

In S11 , the intelligent edge device 100 analyzes the received image to generate metadata, and transmits the generated metadata to the cloud server 200 . The received image is preprocessed for analysis. The intelligent edge device 100 basically detects the occurrence of a human object and a human object-related event in the pre-processed image, and may additionally or selectively detect a building safety-related event and an indoor environment-related event. The intelligent edge device 100 may detect a plurality of events according to the learned plurality of event detection conditions. The event detection condition may vary for each intelligent edge device 100 according to a location or environment in which the camera 10 is installed.

In S12 , the environment analysis server 210 included in the cloud server 200 analyzes the received metadata and provides the analysis result to the environment control server 220 .

In one embodiment, the environment analysis server 210 may provide a space to adjust the environment variable as an analysis result. The space to adjust the environment variable is a space in which a human object, a human object-related event, or an indoor environment-related event is detected. The method of providing a space may be applied when there is only one adjustable environment variable in the identified space or an environment variable (eg, brightness) that can be simply adjusted such as starting/stop driving.

In another embodiment, the environment analysis server 210 may provide a space and an environment variable to be adjusted as an analysis result. The method of providing the space and environment variables may be applied to a case in which there are a plurality of environment variables that can be adjusted in the identified space.

Meanwhile, the environment analysis server 210 may detect an abnormal behavior by analyzing the metadata. Among the human object-related events, an intrusion, a fall, an unauthorized person, etc. may be included in the abnormal behavior. Meanwhile, all building safety-related events may be included in the abnormal behavior.

In S13 , the environmental control server 220 generates a control command according to the analysis result and transmits it to the building equipment controller 12 . The environment control server 220 sets the state of the environment variable for each building equipment controller 12 and manages the state of the set environment variable. For example, the air conditioning controller may be set to maintain the temperature (environmental variable) at 25 degrees Celsius (state), and the lighting controller may be set to maintain the illuminance (environmental variable) at 100 lux (state). The environment control server 220 may generate a control command for changing to a new state with reference to the current state of the environment variable for the identified space. The building equipment controller 12 may adjust an environment variable according to a control command.

Meanwhile, in S14 , the environment analysis server 210 may provide a new or temporary event detection condition to the plurality of intelligent edge devices 100 .

6 is a flowchart for exemplarily explaining a process of managing a building through measurement data.

In S20 , the environmental sensor 11 may measure an environmental variable related to the indoor environment of the building, and transmit the measured data to the intelligent edge device 100 . The environmental sensor 11 may measure environmental variables such as temperature, humidity, illuminance, gas concentration, fine dust concentration, and the like for each space.

In S21 , the intelligent edge device 100 transmits the measurement data to the cloud server 200 . The intelligent edge device 100 may transmit the measurement data to the cloud server 200 as it is, or may generate metadata for the measurement data and transmit it to the cloud server 200 . Meanwhile, the intelligent edge device 100 may generate metadata by combining the image and measurement data.

In S22 , the environment analysis server 210 included in the cloud server 200 provides the received measurement data to the environment control server 220 . Among the measurement data, the type of measurement data that can be utilized for image analysis may be reflected in the analysis result and provided to the environment control server 220 . Any one or both of a value of an environment variable indicated by the measured data (eg, current illuminance of a specific space) and a type of measured data (eg, illuminance, temperature, etc.) may be reflected in image analysis. On the other hand, the type of measurement data (eg, gas concentration, etc.) that cannot be utilized for image analysis among the measurement data may be provided to the environment control server 220 as it is.

In S23 , the environment control server 220 generates a control command and transmits it to the building equipment controller 12 .

As an embodiment, when the measurement data is reflected in the image analysis, the environment control server 220 may generate a control command according to the space and environment variables and transmit it to the building equipment controller 12 .

In another embodiment, when the measurement data is transmitted as it is, the environmental control server 220 compares the measurement value of the environmental variable indicated by the measurement data with a set range, and generates a control command according to the comparison result to generate a control command ( 12) can be transmitted.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. In particular, the features of the present invention described with reference to the drawings are not limited to the structures shown in the specific drawings, and may be implemented independently or in combination with other features.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. .

Claims (14)

a plurality of cameras installed inside a building and photographing any one or both of a public space and a private space that require adjustment of an indoor environment;
a plurality of environmental sensors installed in one or both of the public space and the dedicated space and measuring two or more different environmental variables affecting the indoor environment;
By analyzing the images provided by the plurality of cameras, a human object located inside the building is detected, first metadata and a human object-related event are detected to generate second metadata, and measurement data from the plurality of environmental sensors is obtained. receiving intelligent edge device;
The intelligent edge device is located in a cloud accessible through a communication network, and the first metadata and the measurement data transmitted by the intelligent edge device are analyzed to identify a space in which the environment variable needs to be adjusted among the spaces where the image is captured an environment analysis server that designates one or more environmental variables to be adjusted in the identified space, and displays the second meta data to a control terminal when an abnormal behavior is detected by analyzing the second metadata; and
an environment control server located in the cloud and transmitting a control command to a building facility controller in charge of the identified space according to the environment variable to be adjusted;
The environment analysis server distinguishes between abnormal behavior and normal behavior based on the generation time and generation frequency of the first meta data, a cloud-based building management system. delete The cloud-based building according to claim 1, wherein the environment analysis server identifies a space to adjust the environment variable among spaces in which the human object is not detected by comprehensively analyzing a plurality of metadata according to time sequence or spatial location. management system. The method according to claim 1, wherein the intelligent edge device generates third metadata according to a third event detection condition defining a building safety-related event,
The environment analysis server analyzes the third meta data and when an abnormal behavior is detected, it is displayed on a control terminal, a cloud-based building management system. The method according to claim 1, wherein the intelligent edge device generates fourth metadata by detecting an indoor environment-related event through the behavior of the human object,
The environment analysis server analyzes the fourth metadata to identify an environment variable to be adjusted in a space in which the indoor environment-related event is detected, a cloud-based building management system. The cloud-based building management system of claim 1 , wherein the first metadata and the second metadata further include information on the location of the person object. a plurality of cameras installed inside a building and photographing any one or both of a public space and a private space that require adjustment of an indoor environment;
a plurality of environmental sensors installed in one or both of the public space and the dedicated space and measuring two or more different environmental variables affecting the indoor environment;
By analyzing the images provided by the plurality of cameras, a human object located inside the building is detected, first metadata and a human object-related event are detected to generate second metadata, and measurement data from the plurality of environmental sensors is obtained. receiving intelligent edge device;
The intelligent edge device is located in a cloud accessible through a communication network, and analyzes the first metadata transmitted by the intelligent edge device to identify a space in which the environment variable needs to be adjusted among the spaces in which the image is captured, 2 An environment analysis server that analyzes meta data and displays it to the control terminal when an abnormal behavior is detected; and
It is located in the cloud, receives the measurement data transmitted by the intelligent edge device, manages the current state of the environment variable, selects the environment variable to be adjusted with reference to the current state of the environment variable related to the identified space, and selects the selected environment An environmental control server that transmits a control command to a building facility controller in charge of the identified space according to a variable,
The environment analysis server distinguishes between abnormal behavior and normal behavior based on the generation time and generation frequency of the first meta data, a cloud-based building management system. The method according to claim 7, wherein the intelligent edge device generates third metadata according to a third event detection condition defining a building safety-related event,
The environment analysis server analyzes the third meta data and when an abnormal behavior is detected, it is displayed on a control terminal, a cloud-based building management system. The cloud-based building of claim 7 , wherein the environment analysis server comprehensively analyzes a plurality of metadata according to a time sequence or a spatial location to identify a space to adjust the environment variable among spaces in which the human object is not detected. management system. delete In the building management method using image analysis executed in the cloud-based building management system,
A plurality of cameras photographing any one or both of a public space and a dedicated space requiring adjustment of an indoor environment;
generating measurement data by a plurality of environmental sensors measuring two or more different environmental variables installed in the same space as the plurality of cameras;
generating, by an intelligent edge device, first metadata and a human object-related event by analyzing the images provided by the plurality of cameras to detect a person object located inside a building;
The environment analysis server analyzes the first metadata received from the intelligent edge device to identify a space to adjust the environment variable in the space where the image is captured, the generation time and frequency of generation of the first metadata, and the second detecting an abnormal behavior by analyzing the metadata;
generating, by the environmental control server, a control command for adjusting the environment variable related to the identified space with reference to the current state of the environment variable managed by the measurement data received from the intelligent edge device; and
The cloud-based building management method comprising the step of a building equipment controller adjusting an environment variable of the identified space according to the control command. In the building management method using image analysis executed in the cloud-based building management system,
A plurality of cameras photographing any one or both of a public space and a dedicated space requiring adjustment of an indoor environment;
generating measurement data by a plurality of environmental sensors measuring two or more different environmental variables installed in the same space as the plurality of cameras;
generating, by an intelligent edge device, first metadata and a human object-related event by analyzing the images provided by the plurality of cameras to detect a person object located inside a building;
The environment analysis server analyzes the first metadata and the measurement data received from the intelligent edge device to identify a space to adjust the environment variable in the space where the image is captured, and designate one or more environment variables to be adjusted in the identified space and analyzing the generation time and frequency of generation of the first metadata and the second metadata to detect an abnormal behavior;
transmitting, by the environmental control server, a control command to a building equipment controller in charge of the identified space according to the environmental variable to be adjusted; and
The cloud-based building management method comprising the step of a building equipment controller adjusting an environment variable of the identified space according to the control command. delete 13. The method according to claim 11 or 12,
Wherein the environment analysis server comprehensively analyzes a plurality of metadata according to a time sequence or a spatial location to identify a space in which the human object is not detected, a space to adjust the environment variable.

Images