KR102596848B1 - The Method that Estimate Energy Consumption according to The Number of Occupants based on Artificial Intelligence, and The System and Computer-readable Medium for Performing The Same - Google Patents

Abstract

The present invention is an artificial intelligence-based method of estimating energy consumption according to the number of occupants, and a system and computer-readable medium for performing the same, wherein the data is recognized through a first model based on an artificial neural network learned from a training image captured by a first camera. Based on the learning object recognition data for the object, a learning histogram containing information on the number of times recognized per number of recognized objects is generated, time information when shooting the learning image by the first camera, and actual measurements from the image captured by the second camera. A second model based on an artificial neural network is trained based on the number of objects and the learning histogram, and the object recognition data for the object recognized through the first model based on an artificial neural network learned from the image captured by the first camera is used. The number of occupants per hour is estimated based on the generated histogram information and the time information at the time of video recording by the first camera, and the estimated number of occupants per hour and the time information are applied to a third model based on a learned artificial neural network. It relates to an artificial intelligence-based energy consumption estimation method according to the number of occupants, which inputs and predicts energy consumption per hour according to the number of occupants, and a system and computer-readable medium for performing the same.

Description

The Method that Estimate Energy Consumption according to The Number of Occupants based on Artificial Intelligence, and The System and Computer-readable Medium for Performing The Same}

The present invention is an artificial intelligence-based method of estimating energy consumption according to the number of occupants, and a system and computer-readable medium for performing the same, wherein the data is recognized through a first model based on an artificial neural network learned from a training image captured by a first camera. Based on the learning object recognition data for the object, a learning histogram containing information on the number of times recognized per number of recognized objects is generated, time information when shooting the learning image by the first camera, and actual measurements from the image captured by the second camera. A second model based on an artificial neural network is trained based on the number of objects and the learning histogram, and the object recognition data for the object recognized through the first model based on an artificial neural network learned from the image captured by the first camera is used. The number of occupants per hour is estimated based on the generated histogram information and the time information at the time of video recording by the first camera, and the estimated number of occupants per hour and the time information are applied to a third model based on a learned artificial neural network. It relates to an artificial intelligence-based energy consumption estimation method according to the number of occupants, which inputs and predicts energy consumption per hour according to the number of occupants, and a system and computer-readable medium for performing the same.

Recently, interest in energy saving has been increasing around the world. Currently, 30% of the world's total energy is consumed in buildings, and as buildings generate enormous energy consumption, energy demand management of buildings is necessary. In particular, the energy consumption generated in a building is greatly influenced by the number of occupants in the building, so it is necessary to analyze the energy consumption generated in the building based on the occupant information.

Sensors used to measure existing occupancy include PIR sensors (Passive Infrared Seonsors) and _CO2 sensors. However, it becomes difficult for the PIR sensor to recognize an object when there is no movement of the object, and the CO- ₂ sensor detects _CO2 concentration based on the mass balance method, which is greatly affected by variables such as ventilation. Additionally, measurement errors occur in the sensors due to external factors, camera installation location, communication environment, etc., resulting in frequent errors in measuring occupancy. Therefore, there is a need to develop technology to improve the accuracy of energy consumption estimation according to the number of occupants by improving the error in measuring the number of occupants.

AI object recognition is a technology that receives images from a camera and identifies the location and type of a specific object in a given image. The existing method of storing and collecting images and personal information to utilize AI object recognition technology may have the side effect of violating privacy. To prevent these side effects, only objects in images are detected based on already learned data without storing image and video data. However, in this case, due to the nature of the camera, recognition errors may occur due to light reflection, objects such as clothes hanging on the neckrest of a chair may be recognized as people, and small people may not be recognized as people because they are obscured by a chair or monitor. Things happen often. There is a need to develop technology that can prevent these errors and increase the accuracy of measuring occupancy.

The present invention is an artificial intelligence-based method of estimating energy consumption according to the number of occupants, and a system and computer-readable medium for performing the same, wherein the data is recognized through a first model based on an artificial neural network learned from a training image captured by a first camera. Based on the learning object recognition data for the object, a learning histogram containing information on the number of times recognized per number of recognized objects is generated, time information when shooting the learning image by the first camera, and actual measurements from the image captured by the second camera. A second model based on an artificial neural network is trained based on the number of objects and the learning histogram, and the object recognition data for the object recognized through the first model based on an artificial neural network learned from the image captured by the first camera is used. The number of occupants per hour is estimated based on the generated histogram information and the time information at the time of video recording by the first camera, and the estimated number of occupants per hour and the time information are applied to a third model based on a learned artificial neural network. The purpose is to provide an artificial intelligence-based energy consumption estimation method according to the number of occupants, which inputs and predicts energy consumption per hour according to the number of occupants, and a system and computer-readable medium for performing the same.

In order to solve the above problems, an embodiment of the present invention is an artificial intelligence-based energy consumption estimation method according to the number of occupants performed by a computing system including one or more memories and one or more processors, which involves taking pictures with a first camera. A learning object recognition data generation step of generating learning object recognition data for an object recognized through a first model based on an artificial neural network learned from the learned learning image; An actual data generation step of generating actual measurement data including the number of objects actually measured in an image captured by a second camera; A learning histogram generation step of generating learning histogram information including information on the number of times the recognized object was recognized based on the learning object recognition data; And a model learning step of learning a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. Provides an artificial intelligence-based energy consumption estimation method according to the number of occupants.

In one embodiment of the present invention, the artificial intelligence-based energy consumption estimation method according to the number of occupants includes object recognition data for an object recognized through a first model based on an artificial neural network learned from an image captured by a first camera. Object recognition data generation step; A histogram generating step of generating histogram information including recognition count information per number of the recognized objects based on the object recognition data; An occupancy estimation step of inputting a first feature vector including the histogram information and time information when capturing an image by a first camera into the second model to derive an estimated occupancy number; And an energy consumption prediction step of predicting energy consumption according to the number of occupants through a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information. You can.

In one embodiment of the present invention, the time information may include information in units of days, information in hours, and information in units of minutes when an image is captured by the first camera.

In one embodiment of the present invention, each of the learning histogram information and the histogram information is generated based on the number of recognized objects at a preset period, includes recognition count information per number of recognized objects, and the horizontal axis The indicator corresponds to the number of recognized objects, and the vertical axis indicator corresponds to the number of times the object was recognized, and can be input to the second model at the preset period.

In one embodiment of the present invention, the second model includes a CNN-based model, and the second model is inputted with time information and corresponding histogram information at a specific point in time, and the estimated number of occupants at that specific point in time. can be output.

In one embodiment of the present invention, the second model includes a CNN-based model, and the second model includes time information at a plurality of viewpoints before a specific point in time and histogram information at a plurality of viewpoints before the specific point in time. By entering the information, the estimated number of occupants at that specific point in time can be derived.

In one embodiment of the present invention, the second model includes an LSTM-based model, and the second model includes time information of a plurality of points in time before a specific point in each LSTM module and a plurality of points in time before the specific point in time. The histogram information in is input respectively, and the estimated number of occupants at that specific point in time can be derived.

In one embodiment of the present invention, the third model includes a CNN-based model, and the third model inputs time information and the estimated number of occupants at a specific point in time, and calculates the predicted energy consumption at that specific point in time. can be calculated.

In one embodiment of the present invention, the third model includes a CNN-based model, and the third model includes time information at a plurality of viewpoints before a specific point in time and estimated presence at a plurality of viewpoints before the specific point in time. The number of people is entered, and the predicted energy consumption at that specific point in time can be calculated.

In one embodiment of the present invention, the third model includes an LSTM-based model, and the third model includes time information of a plurality of viewpoints before a specific point in each LSTM module and a plurality of viewpoints before the specific point in time. The estimated number of occupants in can be input, respectively, to calculate the predicted energy consumption at that specific point in time.

In order to solve the above problems, in one embodiment of the present invention, it is a computing system that includes one or more memories and one or more processors and performs an artificial intelligence-based energy consumption estimation method according to the number of occupants, and captures images with a first camera. a learning object recognition data generator that generates learning object recognition data for an object recognized through a first model based on an artificial neural network learned from the learned learning image; an actual measurement data generator that generates actual measurement data including the number of objects actually measured in an image captured by a second camera; a learning histogram generator that generates learning histogram information including information on the number of times the recognized object is recognized based on the learning object recognition data; And a model learning unit that trains a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. Provides a computing system that

In one embodiment of the present invention, the time information may include information in units of days, information in hours, and information in units of minutes when an image is captured by the first camera.

In one embodiment of the present invention, each of the learning histogram information and the histogram information is generated based on the number of recognized objects at a preset period, includes recognition count information per number of recognized objects, and the horizontal axis The indicator corresponds to the number of recognized objects, and the vertical axis indicator corresponds to the number of times the object was recognized, and can be input to the second model at the preset period.

In one embodiment of the present invention, the second model includes a CNN-based model, and the second model includes time information at a plurality of viewpoints before a specific point in time and the histogram information at a plurality of viewpoints before the specific point in time. is input, and the estimated number of occupants at that specific point in time can be derived.

In order to solve the above problems, an embodiment of the present invention provides an artificial intelligence-based occupancy estimation system performed by an artificial intelligence-based occupancy estimation system in a computing system including one or more memories and one or more processors. A computer-readable medium for implementing a method for estimating energy consumption according to the present invention, wherein the computer-readable medium includes computer-executable instructions that cause the computing system to perform the following steps: A learning object recognition data generation step of generating learning object recognition data for an object recognized through a first model based on an artificial neural network learned from a learning image captured by a first camera; An actual data generation step of generating actual measurement data including the number of objects actually measured in an image captured by a second camera; A learning histogram generation step of generating learning histogram information including information on the number of times the recognized object was recognized based on the learning object recognition data; And a model learning step of learning a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. Provides a computer-readable medium that

In one embodiment of the present invention, an artificial intelligence neural network-based model is trained based on a histogram for the number of occupants recognized in an image captured by a camera and actual measurement data obtained by directly measuring the number of occupants. The artificial intelligence neural network-based model can have the effect of increasing the accuracy of estimation of the number of occupants per hour.

In one embodiment of the present invention, a plurality of estimation results derived by performing a process of estimating the number of occupants through a plurality of learned artificial intelligence neural network-based models are compared with each other, and a highly accurate estimate of the number of occupants per hour is made. It can be effective in producing results.

In one embodiment of the present invention, a plurality of estimation results derived by performing a process of predicting energy consumption through a plurality of learned artificial intelligence neural network-based models are compared with each other, and an estimation result with high accuracy for hourly energy consumption is obtained. It can have the effect of deriving .

In one embodiment of the present invention, when predicting the number of occupants per hour and energy consumption per hour, time information is input into a learned artificial neural network-based model to calculate the number of occupants per hour by taking into account the environment that changes every minute and the living patterns of occupants. And it can have the effect of increasing the accuracy of prediction of energy consumption per hour.

In one embodiment of the present invention, even in buildings without existing Advanced Metering Infrastructure (AMI)-related infrastructure, future energy consumption can be predicted by measuring the number of occupants and temporary energy consumption, and all floors of the building are similar. If it is being used for a specific purpose, installing the intelligent meter reading infrastructure on only some floors can have the effect of reducing costs by estimating energy consumption.

Figure 1 shows a process of learning an artificial neural network-based model among the artificial intelligence-based energy consumption estimation methods according to the number of occupants according to an embodiment of the present invention.
Figure 2 shows the process of performing inference through a learned artificial neural network-based model among the artificial intelligence-based energy consumption estimation method according to the number of occupants according to an embodiment of the present invention.
Figure 3 schematically shows the configuration of histogram information according to an embodiment of the present invention.
Figure 4 shows a configuration for performing the learning process and inference process of an artificial neural network-based model according to an embodiment of the present invention.
Figure 5 schematically shows the process of performing the energy consumption prediction step according to an embodiment of the present invention.
Figure 6 shows the configuration of histogram information according to an embodiment of the present invention.
Figure 7 schematically shows the process of performing the occupancy estimation step through a plurality of artificial neural network-based models according to an embodiment of the present invention.
Figure 8 schematically shows the process of performing the energy consumption prediction step through a plurality of artificial neural network-based models according to an embodiment of the present invention.
Figure 9 exemplarily shows the number of occupants per hour estimated through a second CNN-based model according to an embodiment of the present invention.
Figure 10 exemplarily shows the degree of error in predicted energy consumption per hour according to an embodiment of the present invention.
Figure 11 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that this aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain example aspects of one or more aspects. However, these aspects are illustrative and some of the various methods in the principles of the various aspects may be utilized, and the written description is intended to encompass all such aspects and their equivalents.

Additionally, various aspects and features may be presented by a system that may include multiple devices, components and/or modules, etc. It is also understood that various systems may include additional devices, components and/or modules, etc. and/or may not include all of the devices, components, modules, etc. discussed in connection with the drawings. It must be understood and recognized.

As used herein, “embodiments,” “examples,” “aspects,” “examples,” etc. may not be construed to mean that any aspect or design described is better or advantageous over other aspects or designs. . The terms '~part', 'component', 'module', 'system', 'interface', etc. used below generally refer to computer-related entities, such as hardware, hardware, etc. A combination of and software, it can mean software.

Additionally, the terms “comprise” and/or “comprising” mean that the feature and/or element is present, but exclude the presence or addition of one or more other features, elements and/or groups thereof. It should be understood as not doing so.

Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the embodiments of the present invention, have an ideal or excessively formal meaning. It is not interpreted as

Figure 1 shows a process of learning an artificial neural network-based model among the artificial intelligence-based energy consumption estimation methods according to the number of occupants according to an embodiment of the present invention.

As shown in Figure 1, it is an artificial intelligence-based energy consumption estimation method according to the number of occupants performed by a computing system including one or more memories and one or more processors, and is an artificial neural network learned from training images captured by a first camera. A learning object recognition data generation step (S100) of generating learning object recognition data for an object recognized through a first model based; An actual measurement data generation step (S200) of generating actual measurement data including the number of objects actually measured in an image captured by a second camera; A learning histogram generation step (S300) of generating learning histogram information including information on the number of times the recognized object was recognized based on the learning object recognition data; And a model learning step (S400) of learning a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. Includes ;

Schematically, the execution process shown in FIG. 1 corresponds to a learning step in which the second model based on an artificial neural network is trained to increase accuracy for precise estimation of the number of occupants per hour.

Specifically, the learning object recognition data generation step (S100) is performed by a learning object recognition data generator, and recognizes the object through a first model based on an artificial neural network learned from the learning image captured by the first camera. , generate learning object recognition data for the object. In an embodiment of the present invention, the object recognized through the first model corresponds to an occupant, and the learning object recognition data includes unique information of a bounding box for each recognized occupant, so that each occupant is distinguished. make it possible

The actual data generation step (S200) is performed by an actual data generator, and generates the actual data including the number of occupants per hour measured from an image captured by the second camera. The number of occupants per hour is measured by a person directly monitoring the video captured by the second camera. Meanwhile, the actual measurement data includes the actual number of occupants rather than the actual image of the occupants, which can have the effect of strengthening the security of personal information about occupants captured in the video.

The learning histogram generating step (S300) is performed by a learning histogram generating unit, and generates learning histogram information based on the learning object recognition data generated by the learning object recognition data generating step (S100). The learning histogram information includes information on the number of times the number of recognized objects was recognized by the first camera per number of objects recognized by the first model in the learning image, and the horizontal axis index of the learning histogram is divided into the number of recognized objects. Correspondingly, the vertical axis indicator of the learning histogram corresponds to the number of times the number of recognized objects was recognized. In addition, the learning histogram information is generated at preset intervals from the learning images captured by the first camera, and the learning histogram information generated at each preset interval contains information corresponding to the number of different recognized objects and different recognition times. Includes.

The model learning step (S400) is performed by a model learning unit, generates the first learning feature vector including the learning histogram information and time information when shooting the learning image by the first camera, and generates the actual measurement data. A second model based on an artificial neural network is learned using the measured data generated in step S200. Technology that measures the number of occupants from images captured by a camera through an existing learned artificial neural network-based model generally underestimates the actual number of occupants due to recognition errors due to light reflection due to the nature of the camera and objects blocking people. This problem often occurs. In order to solve this problem, the present invention trains an artificial neural network-based model using the learning histogram information indicating the frequency of the number of occupants recognized per hour and actual measurement data where a person directly measures the number of occupants from the captured image. This can have the effect of increasing accuracy in the process of estimating the number of occupants.

Figure 2 shows the process of performing inference through a learned artificial neural network-based model among the artificial intelligence-based energy consumption estimation method according to the number of occupants according to an embodiment of the present invention.

As shown in FIG. 2, the artificial intelligence-based energy consumption estimation method according to the number of occupants uses object recognition data for objects recognized through a first model based on an artificial neural network learned from an image captured by a first camera. Object recognition data generation step (S500); A histogram generation step (S600) of generating histogram information including recognition count information per number of recognized objects based on the object recognition data; An occupancy estimation step (S700) of inputting a first feature vector including the histogram information and time information when capturing an image by a first camera into the second model to derive an estimated occupancy number; and an energy consumption prediction step (S800) of predicting energy consumption according to the number of occupants through a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information. Includes more.

Schematically, the execution process shown in FIG. 2 corresponds to an inference step of inferring the number of occupants using the second model based on an artificial neural network learned to make a precise prediction of the number of occupants through the actual measurement data. do.

Specifically, the object recognition data generation step (S500) generates object recognition data for an object recognized through the first model in an image captured by the first camera. In an embodiment of the present invention, the object recognized through the first model corresponds to the number of occupants. The object recognition data includes unique information on the bounding box for each recognized occupant, allowing each occupant to be distinguished.

The histogram generation step (S600) generates histogram information including recognition count information per number of recognized objects based on the object recognition data. The histogram information is generated at preset periods in the image captured by the first camera, and the histogram information generated at each preset period due to the position of the object changing over time has a different number of recognized objects, and the number of recognized objects is different. The number of times a number is recognized is also different.

The occupancy estimation step (S700) measures the number of occupancy per hour by inputting the first feature vector into the second model learned by the learning object recognition data and the actual measurement data in the learning step. The first feature vector includes not only the histogram information but also time information when the first camera captures the image in order to consider the influence on the living patterns of the occupants. The second model is learned based on the learning histogram information and the actual measurement data to reduce errors that occur during occupancy measurement and improve accuracy with the histogram information generated based on the occupancy recognized by the first model. can be demonstrated.

The energy consumption prediction step (S800) predicts the energy consumption per hour according to the number of occupants per hour through the third model based on the estimated number of occupants per hour and the second feature vector including the time information. Energy consumption has a significant correlation with the number of occupants, and the energy consumption per hour shows a similar pattern to the number of occupants per hour. Using these characteristics, the energy consumption per hour can be predicted based on the number of occupants per hour estimated in the occupant estimation step (S700).

Meanwhile, the object recognition data and the learning object recognition data are different terms with different meanings and purposes. The learning object recognition data corresponds to data generated based on the learning image captured by the first camera and is used in the learning step, and the object recognition data corresponds to data generated based on the image captured by the first camera. Therefore, it is used in the inference stage.

Figure 3 schematically shows the configuration of histogram information according to an embodiment of the present invention.

As shown in Figure 3, the artificial intelligence-based energy consumption estimation method according to the number of occupants generates learning histogram information including information on the number of times the recognized object is recognized based on the learning object recognition data. Step (S300); and a histogram generation step (S600) of generating histogram information including information on the number of times the recognized objects are recognized based on the object recognition data.

Specifically, the learning histogram is generated for objects recognized in the training images in the learning step, and the histogram is generated for the objects recognized in the images in the inference step. The horizontal axis coordinate of the learning histogram and the histogram corresponds to the number of objects recognized by the first camera, and the vertical axis coordinate corresponds to the number of times the number of recognized objects was recognized by the first camera. In an embodiment of the present invention, the object corresponds to the number of occupants. However, the number of times the recognized number of occupants is recognized is calculated according to the preset cycle. For example, when the preset period is 1 minute and the video captured by the first camera includes images of 60 frames per minute, the number of recognition times for the number of occupants recognized in the image of each frame is for a total of 60 frames. It is calculated by adding up.

Figure 4 shows a configuration for performing the learning process and inference process of an artificial neural network-based model according to an embodiment of the present invention.

As shown in Figure 4, the artificial intelligence-based energy consumption estimation method according to the number of occupants includes the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. A model learning step (S400) of learning a second model based on an artificial neural network using; and an occupancy estimation step (S700) of inputting the histogram information and a first feature vector including time information when capturing an image by the first camera into the second model to derive an estimated occupancy number.

Additionally, the time information includes information in units of days, information in hours, and information in units of minutes when an image is captured by the first camera.

Specifically, Figure 4(a) schematically shows the execution process of the model learning step (S400). The learning object recognition data measures an overall smaller number of objects than the actual measurement data due to measurement errors such as recognition errors caused by light reflection and people being obscured by objects. In order to reduce this error, the model learning step (S400) inputs the learning histogram information generated based on the learning object recognition data and the actual measurement data generated by direct measurement by a person through a second camera into the second model. Thus, learning is performed on estimating the number of occupants per hour. The first learning feature vector is constructed by converting the histogram information into the occupancy frequency in minutes. The first learning feature vector includes the time information to take into account that the frequency of occupants is influenced by the life patterns of occupants that change with time. In addition, the time information can include information ranging from day-level information to minute-level information, which can have the effect of increasing the accuracy of estimating the number of occupants.

Figure 4(b) schematically shows the process of performing the occupancy estimation step (S700). In the occupancy estimation step (S700), the first feature vector including histogram information generated based on the image captured through the first camera and the time information for considering the life patterns of the occupancy are converted to the second feature vector. Enter the model to estimate the number of occupancy per hour. The second model is previously learned through the learning histogram and the actual measurement data in the model learning step (S400), and can estimate the number of occupants per hour with higher accuracy than the prior art using only the histogram information.

Figure 5 schematically shows the process of performing the energy consumption prediction step (S800) according to an embodiment of the present invention.

As shown in Figure 5, the artificial intelligence-based energy consumption estimation method according to the number of occupants is a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information. It further includes an energy consumption prediction step (S800) of predicting the energy consumption according to the number of occupants.

Specifically, since the amount of energy consumed in a building is greatly influenced by the number of occupants in the building, hourly energy consumption can be analyzed based on the number of occupants per hour. In addition, when the hourly energy consumption of a building is predicted, the time information is additionally considered because the hourly energy consumption is influenced by the living patterns of each occupant, which varies depending on the time. The second feature vector includes the time information and the number of occupants per hour estimated by the previous occupant estimation step (S700), and the energy consumption prediction step (S800) applies the second feature vector to the third model. Enter the information to predict the building's hourly energy consumption.

Figure 6 shows the configuration of histogram information according to an embodiment of the present invention.

As shown in FIG. 6, each of the learning histogram information and the histogram information is generated based on the number of recognized objects at a preset period, includes information on the number of times recognized per number of recognized objects, and is displayed on the horizontal axis. The indicator corresponds to the number of recognized objects, and the vertical axis indicator corresponds to the number of times the object was recognized, and is input to the second model at the preset period.

Specifically, in one embodiment of the present invention, as shown in FIG. 6, the preset period is set to 1 minute, and the object recognized by the first model corresponds to the number of occupants. Shooting begins with the first camera, and histogram information about the number of occupants recognized through the first model is generated in a 1- to 2-minute section. The histogram includes information on the number of times recognized per number of objects recognized in a shooting period of 1 to 2 minutes. The histogram shown in FIG. 6 is an embodiment of the present invention, and the number of times one occupant was recognized was two times during the 1 to 2 minutes after shooting with the first camera started, and the number of occupants was recognized two times. The number of times is 3, and it includes information corresponding to the number of times 3 occupants were recognized. The recognized number of times is classified based on the number of frames of the video. Histogram information corresponding to time information of 1 to 2 minutes and histogram information corresponding to time information of 2 to 3 minutes may be different from each other depending on the movement of the occupants.

Figure 7 schematically shows the process of performing the occupancy estimation step (S700) through a plurality of artificial neural network-based models according to an embodiment of the present invention.

As shown in Figure 7, in one embodiment of the present invention, the second model includes a CNN-based model, and the second model inputs time information and corresponding histogram information at a specific point in time, Outputs the estimated number of occupants.

Meanwhile, in another embodiment of the present invention, the second model includes a CNN-based model, and the second model includes time information at a plurality of viewpoints before a specific point in time and histogram information at a plurality of viewpoints before the specific point in time. is entered, and the estimated number of occupants at that specific point in time is derived.

Meanwhile, in another embodiment of the present invention, the second model includes an LSTM-based model, and the second model includes time information of a plurality of points in time before a specific point in each LSTM module and a plurality of points in time before the specific point in time. The histogram information at each point in time is input, and the estimated number of occupants at that specific point in time is derived.

Specifically, as shown in (a) of FIG. 7, the second model includes a CNN-based model, and when time information at a specific point A and histogram information at a specific point A are input, the time information at a specific point A is calculated. The number of occupants is estimated, and when time information at a specific point in time B and histogram information at a specific point in time are input, the number of occupants per hour at a specific point in time B is estimated. The second model derives the number of occupants per hour at the current time based on the current time information and current histogram information.

As shown in (b) of FIG. 7, the second model includes a CNN-based model, and includes time information of a plurality of viewpoints before a specific point A and histogram information of a plurality of viewpoints before a specific point A. When time information up to point A and histogram information up to a specific point A are input, the number of occupants per hour at a specific point A is derived. The second model derives the number of occupants at a specific time point A based on past time information and past histogram information.

As shown in (c) of FIG. 7, the second model includes an LSTM-based model, and the time information of the first time point before a specific point A and the histogram information of the first point in time before the specific point A are LSTM The process of inputting to module #1, time information of a second point in time before specific point A, and histogram information of a second point in time before specific point A is input to LSTM module #2. Execute until histogram information is input to LSTM module #N. Meanwhile, N corresponds to a natural number and corresponds to the number of time information and histogram information input to the LSTM module, and the number of occupants per hour at the specific point A is derived from the LSTM module #N. The second model derives the number of occupants at a specific time point A based on past time information and past histogram information. Meanwhile, in another embodiment of the present invention, the second model shown in (a), (b), and (c) of Figure 7 can receive the histogram image rather than the histogram information and derive the number of occupants per hour. .

Meanwhile, energy consumption per hour at a specific point in time is also predicted through a third model based on an artificial neural network learned in a similar way to the process of deriving the number of occupants per hour at a specific point in time. In (a) of 8, a second feature vector including time information at a specific point in time and the number of occupants per hour at the specific point in time is input to the third model including a CNN-based model, and the energy consumption per hour at a specific point in time is predicted. The process is shown, and Figure 8(b) shows time information from before a specific point in time to the specific point in the third model including a CNN-based model, and the number of occupants per hour from before the specific point in time to the specific point in time. It shows the process of predicting the energy consumption per hour at a specific point in time by inputting a second feature vector including The second feature vector including time information and the number of occupants per hour from before the specific point in time to the specific point in time is input to each LSTM module to predict the hourly energy consumption at the specific point in time.

Figure 9 exemplarily shows the number of occupants per hour estimated through a second CNN-based model according to an embodiment of the present invention.

As shown in FIG. 9, the graph showing the number of occupants per hour includes Sensing data in light blue indicating the object recognition data, and Estimated data in red indicating the number of occupants per hour estimated in the occupancy estimation step (S700). , and ground-truth data in black indicating the actual number of occupancy per hour is included.

Specifically, Figure 9 shows the accuracy of the number of occupants per hour estimated by the CNN-based second model learned in an embodiment of the present invention. The red information corresponding to the number of occupants per hour estimated through the second model based on an artificial neural network learned based on the learning histogram information and the actual measurement data is the actual number of occupants per hour than the light blue information corresponding to the object recognition data. Considering that it is similar to the black information corresponding to , it can be seen that it has the effect of reducing measurement errors occurring in the prior art and increasing the accuracy of measuring the number of occupants. More specifically, when using the second CNN-based model, the accuracy of occupancy measurement improves by an average of 70.9% compared to the accuracy of the object recognition data.

Figure 10 exemplarily shows the degree of error in predicted energy consumption per hour according to an embodiment of the present invention.

As shown in FIG. 10, the graph showing the error in energy consumption per hour occurring per hour includes light blue information indicating the energy consumption predicted based on the object recognition data and the energy consumption predicted in the energy consumption prediction step (S800). Red information indicating is included. Overall, the energy consumption predicted in the energy consumption prediction step (S800) had a smaller error compared to the actual energy consumption than the energy consumption predicted based on the object recognition data.

Specifically, the hourly energy consumption predicted in one embodiment of the present invention is calculated to be a value similar to the actual hourly energy consumption than the hourly energy consumption predicted according to the number of occupants estimated based on the object recognition data. Specifically, the error between the hourly energy consumption predicted in one embodiment of the present invention and the actual hourly energy consumption is improved by 22.5% compared to the hourly energy consumption predicted based on the object recognition data. More specifically, the prediction accuracy of the energy consumption per hour predicted in one embodiment of the present invention is improved by 7.9% compared to the energy consumption per hour predicted based on the object recognition data. Through an embodiment of the present invention, it is possible to prevent excessive input of additional manpower for monitoring the number of occupants and to accurately predict energy consumption.

FIG. 11 exemplarily shows the internal configuration of a computing device 11000 according to an embodiment of the present invention.

The computing system mentioned in the description of FIG. 1 may include components of the computing device 11000 shown in FIG. 11, which will be described later.

As shown in FIG. 11, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, and an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600).

Specifically, the memory 11200 may be, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. may include. The memory 11200 may include software modules, instruction sets, or other various data necessary for the operation of the computing device 11000.

At this time, access to the memory 11200 from other components such as the processor 11100 or the peripheral device interface 11300 may be controlled by the processor 11100. The processor 11100 may be composed of a single processor or a plurality of processors, and may include GPU and TPU type processors to improve calculation processing speed.

The peripheral device interface 11300 may couple input and/or output peripheral devices of the computing device 11000 to the processor 11100 and the memory 11200. The processor 11100 may execute a software module or instruction set stored in the memory 11200 to perform various functions for the computing device 11000 and process data.

The input/output subsystem 11400 can couple various input/output peripheral devices to the peripheral device interface 11300. For example, the input/output subsystem 11400 may include a controller for coupling peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touch screen or sensor to the peripheral device interface 11300. there is. According to another aspect, the input/output peripheral devices may be coupled to the peripheral interface 11300 without going through the input/output subsystem 11400.

The power circuit 11500 may supply power to all or part of the components of the terminal. For example, the power circuit 11500 may include a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or It may include any other components for power generation, management, and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port. Alternatively, as described above, if necessary, the communication circuit 11600 may include an RF circuit to transmit and receive RF signals, also known as electromagnetic signals, to enable communication with other computing devices.

This embodiment of FIG. 11 is only an example of the computing device 11000, and the computing device 11000 may omit some components shown in FIG. 11 or further include additional components not shown in FIG. 11. , may have a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. 11, and the communication circuit 1160 may include various communication methods (Wi-Fi, Circuits for RF communication (3G, LTE, 5G, 6G, Bluetooth, NFC, Zigbee, etc.) may be included. Components that can be included in the computing device 11000 may be implemented as hardware, software, or a combination of both hardware and software, including an integrated circuit specialized for one or more signal processing or applications.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to this embodiment may be composed of a PC-based program or a mobile terminal-specific application. The application to which the present invention is applied can be installed on a user terminal through a file provided by a file distribution system. As an example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request from the user terminal.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used by any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be stored or executed in a standardized manner on a networked computing device. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

In one embodiment of the present invention, an artificial intelligence neural network-based model is learned based on a histogram for the number of occupants recognized in an image captured by a camera and actual measurement data obtained by directly measuring the number of occupants. The artificial intelligence neural network-based model can have the effect of increasing the accuracy of estimation of the number of occupants per hour.

In one embodiment of the present invention, a process of estimating the number of occupants is performed through a plurality of learned artificial intelligence neural network-based models to compare them with each other, which has the effect of deriving highly accurate estimation results for the number of occupants per hour. You can.

In one embodiment of the present invention, a process of predicting energy consumption can be performed through a plurality of learned artificial intelligence neural network-based models, compared with each other, and has the effect of deriving highly accurate estimation results for energy consumption per hour. there is.

In one embodiment of the present invention, when predicting the number of occupants per hour and energy consumption per hour, time information is input into a learned artificial neural network-based model to calculate the number of occupants per hour by taking into account the environment that changes every minute and the living patterns of occupants. And it can have the effect of increasing the accuracy of prediction of energy consumption per hour.

In one embodiment of the present invention, even in buildings without existing Advanced Metering Infrastructure (AMI)-related infrastructure, future energy consumption can be predicted by measuring the number of occupants and temporary energy consumption, and all floors of the building are similar. If it is being used for a specific purpose, installing the intelligent meter reading infrastructure on only some floors can have the effect of reducing costs by estimating energy consumption.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent. Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (15)

An artificial intelligence-based energy consumption estimation method according to the number of occupants performed by a computing system including one or more memories and one or more processors,
A learning object recognition data generation step of generating learning object recognition data for an object recognized through a first model based on an artificial neural network learned from a learning image captured by a first camera;
An actual data generation step of generating actual measurement data including the number of objects actually measured in an image captured by a second camera;
A learning histogram generation step of generating learning histogram information including information on the number of times the recognized object was recognized based on the learning object recognition data; and
A model learning step of learning a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data; ,
The artificial intelligence-based energy consumption estimation method according to the number of occupants is,
An object recognition data generation step of generating object recognition data for an object recognized through a first model based on an artificial neural network learned from an image captured by a first camera;
A histogram generating step of generating histogram information including recognition count information per number of the recognized objects based on the object recognition data;
An occupancy estimation step of inputting a first feature vector including the histogram information and time information when capturing an image by a first camera into the second model to derive an estimated occupancy number; and
It further includes an energy consumption prediction step of predicting energy consumption according to the number of occupants through a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information,
The actual measurement data includes the number of objects measured by a person directly monitoring the image captured by the second camera,
The time information is,
Contains information in units of days, information in units of hours, and information in units of minutes when an image is captured by the first camera,
The first learning feature vector, the first feature vector, and the second feature vector include information about the object's life pattern based on the time information,
Each of the learning histogram information and the histogram information is,
Generated based on the number of recognized objects at each preset cycle,
Contains information on the number of times the recognized object is recognized,
The horizontal axis indicator corresponds to the number of recognized objects,
The vertical axis indicator corresponds to the number of times the object was recognized,
input to the second model for each preset period,
An artificial intelligence-based energy consumption estimation method according to the number of occupants, where the recognized number of times is classified based on the number of frames of the video.
delete delete delete In claim 1,
The second model includes a CNN-based model,
The second model is an artificial intelligence-based energy consumption estimation method according to the number of occupants, where time information and corresponding histogram information at a specific point in time are input and the estimated number of occupants at that specific point in time is input.
In claim 1,
The second model includes a CNN-based model,
The second model is based on artificial intelligence, which inputs time information at a plurality of points in time before a specific point in time and histogram information at a plurality of points in time before the specific point in time, and derives the estimated number of occupants at that specific point in time. Energy consumption estimation method according to the number of occupants.
In claim 1,
The second model includes an LSTM-based model,
The second model inputs time information at a plurality of time points before a specific point in time and histogram information at a plurality of points in time before the specific time point into each LSTM module, and calculates the estimated number of occupants at that specific point in time. Deriving an artificial intelligence-based energy consumption estimation method according to the number of occupants.
In claim 1,
The third model includes a CNN-based model,
The third model is an artificial intelligence-based energy consumption estimation method according to the number of occupants, where time information and the estimated number of occupants at a specific point in time are input to calculate the predicted energy consumption at that specific point in time.
In claim 1,
The third model includes a CNN-based model,
The third model is an artificial intelligence that calculates the predicted energy consumption at a specific point in time by inputting time information at a plurality of points in time before a specific point in time and the estimated number of occupants at a plurality of points in time before the specific point in time. Energy consumption estimation method based on the number of occupants.
In claim 1,
The third model includes an LSTM-based model,
The third model inputs the time information of a plurality of time points before a specific point in time and the estimated number of occupants at a plurality of points in time before the specific point into each LSTM module, respectively, and calculates the predicted energy at that specific point in time. An artificial intelligence-based energy consumption estimation method based on the number of occupants that calculates consumption.
A computing system that includes one or more memories and one or more processors and performs an artificial intelligence-based energy consumption estimation method according to the number of occupants,
A learning object recognition data generator that generates learning object recognition data for an object recognized through a first model based on an artificial neural network learned from a learning image captured by a first camera;
an actual measurement data generator that generates actual measurement data including the number of objects actually measured in an image captured by a second camera;
a learning histogram generator that generates learning histogram information including information on the number of times the recognized object is recognized based on the learning object recognition data; and
It includes a model learning unit that trains a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data. ,
The computing system is,
an object recognition data generator that generates object recognition data for an object recognized through a first model based on an artificial neural network learned from an image captured by a first camera;
a histogram generator that generates histogram information including recognition count information per number of recognized objects based on the object recognition data;
an occupancy estimation unit that inputs the histogram information and a first feature vector including time information at the time of image capture by the first camera into the second model to derive an estimated occupancy number; and
It further includes; an energy consumption prediction unit that predicts energy consumption according to the number of occupants through a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information;
The actual measurement data includes the number of objects measured by a person directly monitoring the image captured by the second camera,
The time information is,
Contains information in units of days, information in units of hours, and information in units of minutes when an image is captured by the first camera,
The first learning feature vector, the first feature vector, and the second feature vector include information about the object's life pattern based on the time information,
Each of the learning histogram information and the histogram information is,
Generated based on the number of recognized objects at each preset cycle,
Contains information on the number of times the recognized object is recognized,
The horizontal axis indicator corresponds to the number of recognized objects,
The vertical axis indicator corresponds to the number of times the object was recognized,
input to the second model for each preset period,
A computing system wherein the recognized number of times is classified based on the number of frames of the image.
delete delete In claim 11,
The second model includes a CNN-based model,
The second model is a computing system that inputs time information at a plurality of points in time before a specific point in time and the histogram information at a plurality of points in time before the specific point in time to derive the estimated number of occupants at that specific point in time. .
A computer-readable medium for implementing an artificial intelligence-based energy consumption estimation method according to occupancy performed by an artificial intelligence-based energy consumption estimation system according to occupancy in a computing system including one or more memories and one or more processors, The computer-readable medium includes computer-executable instructions that cause the computing system to perform the following steps,
The steps below are:
A learning object recognition data generation step of generating learning object recognition data for an object recognized through a first model based on an artificial neural network learned from a learning image captured by a first camera;
An actual data generation step of generating actual measurement data including the number of objects actually measured in an image captured by a second camera;
A learning histogram generation step of generating learning histogram information including information on the number of times the recognized object was recognized based on the learning object recognition data; and
A model learning step of learning a second model based on an artificial neural network using the learning histogram information, a first learning feature vector including time information when shooting a learning image by the first camera, and the actual measurement data; ,
The steps below are:
An object recognition data generation step of generating object recognition data for an object recognized through a first model based on an artificial neural network learned from an image captured by a first camera;
A histogram generating step of generating histogram information including recognition count information per number of the recognized objects based on the object recognition data;
An occupancy estimation step of inputting a first feature vector including the histogram information and time information when capturing an image by a first camera into the second model to derive an estimated occupancy number; and
It further includes an energy consumption prediction step of predicting energy consumption according to the number of occupants through a third model based on an artificial neural network learned based on a second feature vector including the estimated number of occupants and the time information;
The actual measurement data includes the number of objects measured by a person directly monitoring the image captured by the second camera,
The time information is,
Contains information in units of days, information in units of hours, and information in units of minutes when an image is captured by the first camera,
The first learning feature vector, the first feature vector, and the second feature vector include information about the object's life pattern based on the time information,
Each of the learning histogram information and the histogram information is,
Generated based on the number of recognized objects at each preset cycle,
Contains information on the number of times the recognized object is recognized,
The horizontal axis indicator corresponds to the number of recognized objects,
The vertical axis indicator corresponds to the number of times the object was recognized,
input to the second model for each preset period,
The computer-readable medium of claim 1, wherein the recognized number of times is classified based on the number of frames of the image.

Images