CN117708602B

CN117708602B - Building safety monitoring method and system based on Internet of things

Info

Publication number: CN117708602B
Application number: CN202410167258.4A
Authority: CN
Inventors: 张小凤; 万俊飞
Original assignee: Ansheng Xinda Technology Co ltd
Current assignee: Ansheng Xinda Technology Co ltd
Priority date: 2024-02-06
Filing date: 2024-02-06
Publication date: 2024-04-12
Anticipated expiration: 2044-02-06
Also published as: CN117708602A

Abstract

The invention provides a building safety monitoring method and system based on the Internet of things, wherein a target monitoring state recognition model is obtained by training based on a training template obtained by enhancement, and the coverage of a learning sample added into network training can be improved by integrating a monitoring data sample and a disturbance data sample, so that the target monitoring state recognition model can learn the correlation between an original building monitoring data sample and classification annotation data, and simultaneously learn the correlation between the integrated monitoring data sample and the classification annotation data and the correlation between the disturbance data sample and the classification annotation data, and meanwhile, the target monitoring state recognition model has better generalization capability, and the monitoring state recognition accuracy of the target monitoring state recognition model is improved.

Description

Building safety monitoring method and system based on Internet of things

Technical Field

The disclosure relates to the technical field of data processing, and in particular relates to a building safety monitoring method and system based on the internet of things.

Background

Building safety problems are increasingly prominent with the acceleration of the urban process and the increasing number of high-rise buildings. As an important means for guaranteeing building safety, the building monitoring system has important significance in improving the building safety level due to improvement of functions and performances. In conventional building monitoring systems, security monitoring is typically performed by manual inspection or based on simple threshold alarms. However, this approach has many limitations, such as high labor cost, low efficiency, high false alarm rate, and the like. Therefore, how to use advanced technical means to improve the intelligent level of the building monitoring system and realize more efficient and accurate safety monitoring becomes a hot spot problem of current research.

In recent years, with the continuous development of artificial intelligence technologies such as machine learning and deep learning, more and more researchers begin to explore the application of these technologies in the field of building monitoring. Through intelligent analysis of building monitoring data, functions of automatic identification, anomaly detection, risk assessment and the like of building safety states can be realized, and therefore performance and efficiency of a building monitoring system are greatly improved.

However, to achieve intelligent analysis of building monitoring data, a series of technical challenges need to be addressed. First, building monitoring data is often of diversity and complexity, and how to efficiently extract and represent features of such data is an important issue. Secondly, the identification of the building safety state involves comprehensive judgment of a plurality of factors, and how to construct an efficient and accurate identification model is also a challenge. In addition, the real-time performance and stability requirements of the building monitoring system also put higher requirements on the data analysis model.

Disclosure of Invention

In view of this, the embodiments of the present disclosure at least provide a building security monitoring method and system based on the internet of things.

The technical scheme of the embodiment of the disclosure is realized as follows:

in one aspect, an embodiment of the present disclosure provides a building security monitoring method based on the internet of things, where the method includes: acquiring an original building monitoring data sample and a classification annotation data sample corresponding to the original building monitoring data sample; the classification annotation data sample comprises classification annotation data for converting key monitoring data items in the original building monitoring data sample, wherein the classification annotation data is data obtained after monitoring state identification of the key monitoring data items; converting the key monitoring data items into the classification annotation data in the original building monitoring data samples, and taking the converted original building monitoring data samples containing the classification annotation data as integrated monitoring data samples; performing data disturbance on the classified annotation data sample to obtain a disturbance data sample corresponding to the classified annotation data sample; taking the integrated monitoring data sample and the disturbance data sample as enhanced data samples of the original building monitoring data sample, and taking the original building monitoring data sample and the enhanced data sample as training templates of an initial monitoring state recognition model to be subjected to iterative optimization; the initial monitoring state identification model comprises a characteristic extraction network and a state identification network; the training template is transmitted to the initial monitoring state recognition model, and the feature extraction network of the initial monitoring state recognition model is used for carrying out feature extraction on the data features of the data in the training template to obtain a feature extraction array used for describing the training template; and acquiring annotation data characteristics of the classified annotation data samples, transmitting the characteristic extraction array and the annotation data characteristics to the state recognition network of the initial monitoring state recognition model, recognizing and outputting sample loss corresponding to the training template through the state recognition network, optimizing the initial monitoring state recognition model based on the sample loss, and taking the optimized initial monitoring state recognition model as a target monitoring state recognition model for building safety state recognition.

In a second aspect, the present disclosure provides a computer system comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor implementing steps in the method described above when the program is executed.

The present disclosure has at least the beneficial effects: according to the building safety monitoring method and system based on the Internet of things, through obtaining the original building monitoring data sample and the classification annotation data sample corresponding to the original building monitoring data sample, key monitoring data items are converted into classification annotation data in the original building monitoring data sample, the converted original building monitoring data sample containing the classification annotation data is used as an integrated monitoring data sample, information of the classification annotation data is integrated into a training template, an initial monitoring state recognition model is trained based on the integrated monitoring data sample, the initial monitoring state recognition model has the performance of carrying out monitoring state recognition on specific data according to a specific mode during training, and data disturbance is carried out on the classification annotation data sample, so that disturbance data samples corresponding to the classification annotation data sample are obtained; the target monitoring state recognition model obtained based on the noisy classification annotation data sample training has strong recognition performance. Taking the integrated monitoring data sample and the disturbance data sample as enhancement data samples of the original building monitoring data sample, enhancing the training samples through the enhancement data samples, taking the original building monitoring data sample and the enhancement data sample as training templates of an initial monitoring state recognition model to be subjected to iterative optimization, transmitting the training templates into the initial monitoring state recognition model, and carrying out feature extraction on data features of data in the training templates through a feature extraction network of the initial monitoring state recognition model to obtain a feature extraction array used for describing the training templates; the method comprises the steps of obtaining annotation data characteristics of classified annotation data samples, transmitting a characteristic extraction array and the annotation data characteristics to a state recognition network of an initial monitoring state recognition model, recognizing sample loss corresponding to an output training template through the state recognition network, optimizing the initial monitoring state recognition model through the sample loss, and taking the optimized initial monitoring state recognition model as a target monitoring state recognition model for building safety state recognition; the target monitoring state recognition model is obtained by training based on the training template obtained by enhancement, and the coverage range of the learning sample added into the network training can be improved by integrating the monitoring data sample and the disturbance data sample, so that the target monitoring state recognition model can learn the correlation between the original building monitoring data sample and the classification annotation data, and simultaneously learn the correlation between the integrated monitoring data sample and the classification annotation data and the correlation between the disturbance data sample and the classification annotation data, and meanwhile, the target monitoring state recognition model has better generalization capability, and the monitoring state recognition accuracy of the target monitoring state recognition model is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the aspects of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the technical aspects of the disclosure.

Fig. 1 is a schematic implementation flow diagram of a building security monitoring method based on the internet of things according to an embodiment of the present disclosure.

Fig. 2 is a schematic diagram of a hardware entity of a computer system according to an embodiment of the disclosure.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present disclosure more apparent, the technical solutions of the present disclosure are further elaborated below in conjunction with the drawings and the embodiments, and the described embodiments should not be construed as limiting the present disclosure, and all other embodiments obtained by those skilled in the art without making inventive efforts are within the scope of protection of the present disclosure. In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict. The term "first/second/third" is merely to distinguish similar objects and does not represent a particular ordering of objects, it being understood that the "first/second/third" may be interchanged with a particular order or precedence where allowed, to enable embodiments of the disclosure described herein to be implemented in other than those illustrated or described herein.

The embodiment of the disclosure provides a building safety monitoring method based on the Internet of things, which can be executed by a processor of a computer system. The computer system may refer to a server, a notebook computer, a tablet computer, a desktop computer, or other devices with data processing capability.

Fig. 1 is a schematic implementation flow chart of a building safety monitoring method based on the internet of things, which is provided in an embodiment of the present disclosure, as shown in fig. 1, and includes the following operations:

operation 10: the method comprises the steps of obtaining an original building monitoring data sample and a classification annotation data sample corresponding to the original building monitoring data sample, wherein the classification annotation data sample comprises classification annotation data used for converting key monitoring data items in the original building monitoring data sample, and the classification annotation data is data obtained after monitoring state identification of the key monitoring data items.

The raw building monitoring data samples are raw monitoring data collected directly from various sensors and devices within the building, which may be a sequence of data or may be a matrix of data. These data reflect real-time conditions within the building such as temperature, humidity, smoke concentration, etc. For example, a temperature sensor records the temperature at intervals in a room of a building, and these temperature readings are part of the original building monitoring data. The classification annotation data sample is a result of classifying each data item in the original building monitoring data sample, and is a set of classification labels, each label corresponding to one or more data items in the original building monitoring data sample, for example, a data set of 100 temperature readings, each reading being labeled as normal (0) or abnormal (1), and the 100 labels forming a classification label data set. Classification is typically based on predefined rules or machine learning models for identifying whether a data item is normal or abnormal. For example, if a smoke detector's reading exceeds a preset threshold, it may be classified as abnormal (labeled 1) and a normal reading may be classified as normal (labeled 0). A data item is the basic unit that constitutes a data sample, typically a single measurement or observation. For example, in a monitoring system comprising a plurality of sensors, a single reading of each sensor may be considered as one data item. The key monitoring data items refer to data items that are critical to building safety conditions and generally require more intimate attention and monitoring. For example in a fire monitoring scenario, the readings of smoke detectors and flame detectors may be the key monitoring data items, as they can directly reflect the risk of fire. Monitoring status identification refers to the process of determining the current safety status of a building by analyzing the values of data items. This typically involves comparing the data item with a predefined threshold or pattern. For example, if an image analysis of a camera shows that someone is active in a limited area within a building, the monitoring system may identify this as an abnormal condition.

Operation 10 involves the collection of raw data and the generation of its corresponding classified annotation data. In operation 10, a computer system collects raw building monitoring data samples from various internet of things sensors and devices. These data samples may include various building safety related information such as temperature readings, humidity levels, smoke detector readings, door access system status, camera images, etc. These data items typically exist in a continuous data stream that reflects the changes in various physical and logical states within the building. The computer system then needs to obtain classification annotation data samples corresponding to these original building monitoring data samples. Each original data item is assigned a classification tag that indicates its corresponding state category. For example, on a normal temperature reading, it may be marked "0" indicating that this is a normal event; and on an abnormally high smoke detector reading may be marked with a "1", indicating that this is an abnormal event.

These classification annotation data samples not only contain a classification of the status of each individual data item, but also focus on those important monitoring data items that are critical to building safety. For example, in a fire monitoring system, the readings of a smoke detector may be considered to be important monitoring data items. The computer system may perform additional processing and analysis on these items of critical monitoring data to ensure that their status can be accurately identified and categorized.

To generate these classification annotation data, various machine learning models or algorithms may be used. For example, a supervised learning algorithm may be used to train a classifier that automatically assigns classification labels to newly collected data items based on historical data. The classifier can be any machine learning model capable of processing classification problems, such as decision trees, support vector machines, neural networks and the like. In particular, network implementations may be identified based on the state in subsequent embodiments.

Operation 20: and converting the key monitoring data items into classified annotation data in the original building monitoring data samples, and taking the converted original building monitoring data samples containing the classified annotation data as integrated monitoring data samples.

The key monitoring data items are determined according to the requirements of building safety monitoring, and may include, but are not limited to, key indexes such as temperature, humidity, smoke concentration, entrance guard state and the like. These data items are critical for assessing the safety status of a building. In operation 20, these key monitoring data items are converted into classification annotation data. This means that for each item of critical monitoring data, the computer system will determine its corresponding state category based on its value and predefined rules or models and assign it a corresponding classification label. For example, for a smoke concentration data item, if its value exceeds a preset safety threshold, the computer system may flag it as an abnormal state. After the conversion is completed, the original building monitoring data sample containing the classification annotation data is referred to as an integrated monitoring data sample. The integrated monitoring data sample is a mixture of original data and classification annotation data, and not only contains various real-time monitoring data of a building, but also contains state classification information corresponding to the data.

Wherein, as an implementation manner, the operation 20 converts the key monitoring data item into the classification annotation data in the original building monitoring data sample, and uses the converted original building monitoring data sample containing the classification annotation data as the integrated monitoring data sample, which may include the following sub-operations:

operation 21: and determining the key monitoring data item paragraph of the key monitoring data item in the original building monitoring data sample.

The data item paragraph may contain information critical to building safety, such as temperature readings for a particular time period, smoke concentration in a region, etc. The computer system automatically locates these key paragraphs in the raw data by a preset rule or algorithm. For example, assume a building has a plurality of temperature sensors installed therein, some of which are located in a flammable storage area. The readings of these sensors may require more intimate monitoring over a particular period of time (e.g., at night). Thus, the computer system marks these time periods and the data of the corresponding sensors as the section of the item of data that is to be monitored with emphasis.

Operation 22: and in the original building monitoring data sample, converting the key monitoring data item into classification annotation data according to the key monitoring data item paragraph.

After determining the segments of the data items to be monitored with emphasis, the computer system classifies the states of the data items within the segments and adds corresponding classification tags thereto. This process may include application of a predefined threshold comparison, pattern recognition, or machine learning model. By these methods, each item of critical monitoring data is assigned a class label that indicates its security status. For example, continuing with the temperature sensor example, if a temperature reading within a segment of a certain critical monitoring data item exceeds a preset safety threshold (e.g., 60 degrees celsius), then the data item will be marked as abnormal (e.g., label "1"). Readings below this threshold will be marked as normal (e.g., label "0").

Operation 23: and taking the converted original building monitoring data sample containing the classified annotation data as an integrated monitoring data sample.

After the conversion of the key monitoring data item is completed, the computer system integrates the original data with the classification labels to form a new data set, namely an integrated monitoring data sample. The data set not only contains real-time monitoring data of the building, but also contains safety state information corresponding to the data. Such a data set provides a basis for subsequent security monitoring and data analysis. For example, the integrated monitoring data samples may include raw temperature readings, smoke concentration readings, etc., and a classification label (e.g., normal/abnormal) for each reading. Such data sets may be used to train machine learning models to further improve the accuracy and efficiency of building safety monitoring

Based on operations 21-23, when determining the key monitoring data item in the original building monitoring data sample, the method provided by the embodiment of the disclosure may further include:

operation 2a: and acquiring a monitoring scene corresponding to the original building monitoring data sample, and acquiring a scene common abnormal database matched with the monitoring scene, wherein the scene common abnormal database comprises common abnormal events and classification annotation data items corresponding to the common abnormal events.

In this operation, the computer system first identifies and obtains a specific monitoring scenario corresponding to the original building monitoring data sample. This is because different monitoring scenarios (e.g., building entrances, machine rooms, warehouses, etc.) may face different security risks, requiring different exceptions to be of concern. Each monitoring scenario has its own specific common anomalies that consist of multiple data items, for example, an abnormal rise in temperature of the machine room may consist of a series of readings from the temperature sensor. Next, the computer system accesses and obtains a scene common exception database matching the current monitored scene, wherein the scene common exception database is a set which contains exception events frequently occurring in the specific scene and classification annotation data items corresponding to the exception events. The classification annotation data item is flag data obtained by classifying the states of the respective data items in the abnormal event, and is used to indicate whether the data item is normal or abnormal.

For example, if the monitoring scenario is a fire system of a building, the scenario-common anomaly database may contain anomalies such as "abnormally elevated smoke concentration" or "unexpected activation of fire sprinklers". These abnormal events may consist of a series of related data items such as readings of the smoke sensor, status of the spray head, etc., and these data items may be marked as corresponding classified annotation data indicating whether they indicate an abnormal condition.

By acquiring a common abnormal database matched with the monitoring scene, the computer system can analyze and process the original building monitoring data more pertinently, and the recognition and response speed of potential safety risks are improved.

Operation 2b: and carrying out data decomposition on the original building monitoring data sample to obtain X data items of the original building monitoring data sample, wherein X is greater than or equal to 1.

In this operation, the computer system performs data decomposition on the original building monitoring data samples. The raw building monitoring data is typically a continuous data stream or series of data records containing various sensor readings, device status information, and other building safety related parameters. The goal of data decomposition is to segment these continuous or composite data into individual data items for subsequent analysis and processing. Each data item is a separate unit in the raw data and may represent a particular sensor reading, time stamp, device status, or other important information.

Consider, for example, a raw building monitoring data sample containing temperature and humidity readings. These data may be recorded continuously in the form of "25 degrees celsius, 60% humidity". During the data decomposition phase, the computer system will split this continuous data record into two separate data items: one is the temperature reading "25 degrees celsius" and the other is the humidity reading "60% humidity". In this way, the original building monitoring data samples are decomposed into X data items, where X represents the number of data items and X is at least 1, since each sample contains at least one data item. In practice, the value of X may vary depending on the complexity of the monitoring system and the diversity of the data collected.

Notably, data decomposition is a preprocessing step that provides the basis for subsequent data analysis, anomaly detection, and security monitoring. By decomposing the raw data into individual data items, the computer system is able to more accurately identify and analyze the information represented by each data item, thereby more effectively monitoring the security status of the building.

Operation 2c: and acquiring a data conversion mechanism matched with the scene common exception database, and indexing data items in the scene common exception database based on the data conversion mechanism.

In this operation, the computer system obtains a data conversion mechanism that matches the current scene common exception database. This data transformation mechanism defines how data items are looked up and replaced in the database, and the core of the data transformation mechanism is to determine the similarity or degree of matching between data items. To achieve this, various methods such as feature vector alignment may be employed. In this approach, each data item is converted into a feature vector that captures key attributes or features of the data item. Then, by calculating the similarity (such as cosine similarity, euclidean distance, etc.) between the two feature vectors, it can be determined whether they match. If the similarity exceeds a preset threshold, the two data items are considered to be matched.

For example, assume that a scene common anomaly database contains a series of data items related to temperature anomalies. When a new temperature reading data item arrives, the computer system will first convert it into a feature vector, which may include information such as the specific value of the reading, the time stamp of the reading, the ID of the sensor to which the reading belongs, etc. The computer then searches the database for the data item with the highest similarity to this feature vector to determine if there is a matching anomaly. After the data conversion mechanism is obtained, the computer system will utilize this mechanism to index the X data items that were resolved in the original building monitoring data sample. The indexing process is to find an entry matching or similar to each data item in the scene common exception database. In this way, the computer system can quickly locate the most relevant abnormal event and classified annotation data item to the current monitoring data, and provide powerful support for subsequent security state analysis and abnormal detection.

Operation 2d: and taking the indexed data items which are the same as the common abnormal events in the scene common abnormal database as data items to be marked, wherein the number of the data items to be marked is Y, and Y is not more than X.

In operation 2d, the computer system indexes each data item in the original building monitoring data sample using the data conversion mechanism it previously acquired in operation 2c to find a matching or similar data item in the scene common anomaly database. This is achieved by comparing feature vectors of the data items, determining the similarity between them, and determining whether there is a match based on a preset threshold. When a computer system finds data items in the scene common exception database that match data items in the original data sample, it marks those matching data items as "quasi-annotation data items". These to-be-annotated data items are data directly related to common abnormal events, which are of particular importance in the security monitoring context, as they may be indicative of potential security risks or abnormal conditions.

The number of data items to be annotated is noted as Y, which is determined based on the number of data items actually matched. It should be noted that the value of Y does not exceed X, because X represents the total number of data items in the original building monitoring data sample, and the data items to be annotated are only a portion thereof, i.e., the portion of the data items that match the anomaly event in the scene common anomaly database. For example, if the original building monitoring data sample contains 100 data items (x=100), but only 20 data items matching the abnormal event are found in the scene common abnormal database, the number Y of the data items to be annotated is 20. These 20 items of pseudo-annotation data will be considered critical data that requires further attention and processing, as they may be closely related to changes in building security status.

Operation 2e: and determining W quasi-annotation data items in the Y quasi-annotation data items as target conversion data items, taking the determined target conversion data items as key monitoring data items in an original building monitoring data sample, and marking classification annotation data items corresponding to the target conversion data items as classification annotation data in the classification annotation data sample, wherein W is smaller than Y.

In operation 2e, the computer system further selects W data items from the Y pseudo tag data items determined in operation 2d, the selected data items being referred to as target conversion data items. The selection process may be random or may be based on a particular strategy or algorithm, such as selecting based on the importance of the data item, the degree of abnormality, or the relevance to other data items. Importantly, the value of W is set to be less than Y, meaning that not all of the items to be annotated will be selected as target transformation items, but a portion thereof. These data items that are determined to be target conversion data items will be considered as the key monitoring data items in the original building monitoring data sample. This means that computer systems will be particularly concerned with the variation and trend of these data items during subsequent data analysis and security monitoring, as they may be closely related to the security status of the building and are more likely to reveal potential anomalies or risks. Meanwhile, the classification annotation data item corresponding to the target conversion data item will also be labeled as classification annotation data in the classification annotation data sample. Classifying annotation data provides additional information about the status of the data item, such as normal, abnormal, or requiring further investigation, etc. By associating these classified annotation data with the target transformation data items, the computer system is able to more accurately understand and interpret the actual meaning represented by these data items and provide more rich contextual information for subsequent security monitoring and anomaly detection.

For example, assume that 10 temperature reading data items are selected among the Y to-be-annotated data items, 5 of which are randomly determined as target conversion data items. These 5 target conversion data items will be considered as key monitoring objects and the classification annotation data (e.g. "hyperthermia" or "normothermia") associated with them will be marked as part of the classification annotation data sample. In this way, the computer system will be able to more quickly identify and respond to any anomalies or risks associated with these 5 key monitoring data items during subsequent data processing and analysis.

Based on operations 2 a-2 e, the method provided by the present disclosure may further include, in determining to integrate the monitoring data samples:

operation 210: and taking other data items except Y data items to be marked out of the X data items as first data items to be marked, wherein the number of the first data items to be marked is Q, and the sum value of Y and Q is equal to X.

In operation 210, the computer system will process the data items in the original building monitoring data sample, particularly with respect to data that was not labeled as items to be labeled in a previous operation. In particular, operation 210 involves identifying and marking other data items of the X data items, except for the Y data items to be marked, as first data items to be marked. Here, X represents the total number of data items in the original building monitoring data sample, and Y represents the number of data items determined to be to-be-annotated in the previous step. The number of first items to be data is denoted Q. By definition, the sum of Y (the number of data items to be annotated) and Q (the number of first data items to be annotated) must be equal to X (the total number of original data items). This means that all X data items are classified as either to-be-annotated data items or first to-be-data items, without omission.

For example, if the original building monitoring data sample contains 100 data items (x=100), and 20 items to be annotated (y=20) are determined in the previous step, the remaining 80 data items will be marked as the first item to be annotated (q=80).

The purpose of this operation is to perform a preliminary classification and sorting of the raw data in order to more efficiently process and analyze these data items in subsequent steps. The first items to be data, although not directly marked as related to an exception event at the current stage, may still contain valuable information and should therefore not be ignored in subsequent processing.

Operation 220: and taking the quasi-annotation data items except the target conversion data item in the Y quasi-annotation data items as second undetermined data items.

Operation 230: a fixed data item in the original building monitoring data sample is determined based on the first pending data item and the second pending data item.

In operation 220, the computer system further processes data that has been marked as items to be annotated, but that has not been selected as items to be targeted for conversion. In particular, it will identify data from the Y pseudo-annotation data items that are not the target transformation data items and tag those data as the second pending data item. The purpose of this is to distinguish those data items which, although associated with common anomalies (and therefore marked as quasi-tagged data items), are not selected as important monitoring objects at the current stage. These second pending data items may still contain valuable information and should therefore not be completely disregarded in subsequent data processing and analysis.

For example, assume that 10 items of data to be annotated are associated with a temperature anomaly, but only 5 of them are selected for critical monitoring as target conversion data items. Then the remaining 5 items of data to be annotated (although they are also associated with temperature anomalies) will be marked as second pending items of data.

In operation 230, the computer system will determine a "fixed data item" in the original building monitoring data sample based on the first pending data item and the second pending data item determined in the previous step. "fixed data items" herein refer to those data items that will be retained and continued to be used in subsequent data processing and analysis.

In particular, the computer system may treat the first pending data item and the second pending data item directly as fixed data items. This is because, although these data items are not marked as important monitoring objects or directly associated with specific abnormal events at the current stage, they may still contain information that is valuable for overall security state analysis. For example, if the raw building monitoring data sample contains various types of sensor readings (e.g., temperature, humidity, pressure, etc.), then even if some readings are not currently showing anomalies, they may provide important contextual information or help reveal potential security risks in subsequent analysis. It is therefore interesting to keep these readings as fixed data items.

Through operations 220 and 230, the computer system is able to further sort and sort the data in the original building monitoring data samples, providing a clearer and more orderly data base for subsequent data processing and analysis.

At this time, operation 23, taking the converted original building monitoring data sample containing the classified annotation data as the integrated monitoring data sample, includes:

operation 23a: and taking the data item position of the fixed data item in the original building monitoring data sample as the fixed data item position.

Operation 23b: an integrated monitoring data sample is determined based on the fixed data item at the fixed data item location, the classification annotation data at the accent monitoring data item paragraph, and the accent monitoring data item paragraph.

In operation 23a, the computer system will identify and record the specific location of the fixed data item in the original building monitoring data sample. These location information ensure that critical information in the raw data is accurately preserved and used in constructing the integrated monitoring data sample. The fixed data item location may be an index, address, or other form of identifier of the data item in the original data sample that is sufficient for the computer system to accurately locate the data item in subsequent processing. For example, if the original building monitoring data sample is a table containing a plurality of sensor readings, where each column represents a data item, then the fixed data item location may be the column number or column name of those columns. By recording this location information, the computer system can accurately extract and use these fixed data items in a subsequent integration process.

In operation 23b, the computer system constructs an integrated monitoring data sample using the fixed data item and its location information determined in the previous step, and the classification annotation data on the item segment of the critical monitoring data and the item segment of the critical monitoring data itself. The integrated monitoring data sample is a data structure containing various information, which not only contains key information (embodied by fixed data items and position information thereof) in original data, but also integrates classification annotation data related to key monitoring data items. Such a data structure enables the computer system to comprehensively monitor and analyze the security status of the building within a unified framework.

Specifically, the computer system may create a new data structure (e.g., a table, matrix, or custom data object) and populate the new data structure with fixed data items according to their original location information. At the same time, for the section of the data item to be monitored, the computer system integrates the classification annotation data associated therewith into this data structure so that this information can be accessed and used quickly in subsequent analysis.

For example, if the original building monitoring data sample is a time series of data, each data point containing a time stamp and one or more sensor readings, the integrated monitoring data sample may be a new time series of data containing time stamps, fixed sensor readings, and key monitoring sensor readings and their classification annotation data. Such a data structure both preserves the temporal order and key information of the original data, and highlights key data items that are closely related to security monitoring.

Operation 30: and carrying out data disturbance on the classified annotation data samples to obtain disturbance data samples corresponding to the classified annotation data samples.

In operation 30, the computer system will perform data perturbation processing on the annotated data samples that have been marked and categorized. Data perturbation, also known as noise addition, is a commonly used technique for data enhancement. Its main purpose is to generate new data samples that are similar to but not exactly the same as the original data by introducing some random variations to the original data. In particular, the computer system may make minor, random modifications to each data item in the sample of classified annotation data. These modifications may be minor variations in the values (e.g., adding a small random value to the sensor reading) or minor adjustments in the data structure (e.g., randomly exchanging the positions of two data items in the data sample). With these modifications, the computer system is able to generate a series of disturbance data samples that are similar to the original classification annotation data samples, but with some variability.

The purpose of data perturbation is twofold. First, it can increase the diversity of the data set, enabling the machine learning model to be exposed to more different types of data while training, thereby improving the generalization ability of the model. Second, data perturbation can also be used as a regularization means to help prevent model overfitting.

For example, assume that the original categorical annotation data sample contains a sequence of readings from a temperature sensor over a period of time. The computer system may perform data perturbation by adding a small random number (e.g., ±0.5 ℃) to each reading to generate a new, but not exactly identical, perturbed data sample that is similar to the original reading sequence. It should be noted that the amplitude of the data perturbation should be controlled within a certain range to ensure that the resulting perturbed data samples remain similar to the original data. If the disturbance amplitude is too large, the generated data sample may lose the original meaning, thereby negatively affecting the training of the model.

Specifically, the operation 30 of performing data perturbation on the classified annotation data sample to obtain a perturbation data sample corresponding to the classified annotation data sample may include the following sub-operations:

Operation 31: and carrying out data decomposition on the classified annotation data sample to obtain U data items of the classified annotation data sample, wherein U is greater than or equal to 1.

Operation 32: and obtaining a data disturbance mechanism matched with the classified annotation data sample, determining B data items from the U data items through the data disturbance mechanism, and carrying out data disturbance on the classified annotation data sample based on the B data items to obtain a disturbance data sample corresponding to the classified annotation data sample, wherein B is a non-zero natural number smaller than U.

In operation 31, the computer system performs detailed data decomposition on the classified annotation data sample, and splits the entire data sample into a plurality of independent data items for more accurate data perturbation processing. Each data item may be a specific numerical value, a text label, an image pixel, or any other basic unit that constitutes a data sample. For example, if the categorical annotation data sample is a table containing a plurality of sensor readings, each sensor reading may be considered a data item. The computer system will traverse this table and extract each reading individually as the base unit for subsequent data perturbation processing. The term "U" as used herein refers to the total number of all data items obtained after the classification annotation data sample is decomposed. The value of U depends on the complexity and size of the original data sample, which may be a large number or a relatively small number, but in any case U is a natural number greater than or equal to 1.

In operation 32, the computer system utilizes a particular data perturbation mechanism to data perturb the classification annotation data sample. The data disturbance mechanism is determined according to the characteristics of the data sample and the application requirement, and can be a simple random noise adding rule or a more complex data transformation method based on a machine learning model.

First, the computer system will determine B data items from the U data items as the targets of the data perturbation. This selection process may be random or based on some specific rule or algorithm. The value of B is a non-zero natural number that is smaller than U, which means that not all data items will be disturbed, but only a part of them. Once the data items to be perturbed are determined, the computer system modifies the data items according to the data perturbation mechanism. These modifications may be numerical changes (e.g., adding random noise, performing numerical scaling, etc.), or may be structural changes (e.g., reordering of data items, replacement, etc.).

For example, if the data perturbation mechanism is to add Gaussian noise, the computer system will add to each selected data item a noise value that is randomly extracted from its corresponding Gaussian distribution. In this way the original data item is replaced by a new disturbance value which is similar to the original value but not exactly the same.

Through a combination of operations 31 and 32, the computer system is able to generate a series of disturbance data samples that are similar to, but not exactly the same as, the original classification annotation data samples. These perturbation data samples may be used to enhance the training data set of the machine learning model, thereby improving the generalization ability and robustness of the model.

In the above operation 32, as a first implementation manner, the data perturbation mechanism includes a first perturbation mechanism for performing cleaning processing on B data items, and then performing data perturbation on the classified annotation data samples based on the B data items to obtain perturbation data samples corresponding to the classified annotation data samples, including: and cleaning the B data items in the classified annotation data samples, and taking the classified annotation data samples after the cleaning as disturbance data samples.

The computer system employs a first perturbation mechanism to wash selected B data items. The cleaning process herein refers specifically to a process of deleting a corresponding data item, aiming at changing the structure and content of the original data sample by removing part of the data, thereby generating a new, disturbed data sample. In performing this scheme, the computer system first determines the B data items in the categorical annotation data sample that need to be perturbed. The selection of these data items may be based on specific rules, algorithms or random processes, depending on the design goals and application scenarios of the data perturbation scheme. Once the data items to be perturbed are selected, the computer system uses the first perturbation mechanism to purge the data items, i.e., remove them from the original data samples. Such deletion results in a data loss of the data sample at the corresponding location, thereby altering the integrity and distribution of the original data. After the cleaning process is completed, the computer system will obtain a perturbed classification annotation data sample. This perturbed data sample differs in number and location of data items from the original data sample, but the remaining undeleted data items remain unchanged. Such perturbed data samples may be used in a training or validation process of a machine learning model to improve the generalization ability and robustness of the model by introducing data changes.

In the above operation 32, as a second implementation manner, the data perturbation mechanism includes a second perturbation mechanism for performing gaussian blur on the B data items, and then performing data perturbation on the classified annotation data samples based on the B data items to obtain perturbation data samples corresponding to the classified annotation data samples, including:

operation 321a: and taking the data item positions of the B data items in the classified annotation data sample as fuzzy data item positions, and taking the data item positions of other data items except the B data items of the U data items in the classified annotation data sample as non-fuzzy data item positions.

Operation 322a: gaussian blur is performed on the data at the blurred data item positions, and a disturbance data sample is determined by the data at the non-blurred data item positions and the data at the blurred data item positions after Gaussian blur.

In operation 321a, the computer system distinguishes which data items will be affected by the Gaussian blur and which remain unchanged. In particular, it will first identify selected B data items in the categorical annotation data sample whose locations will be marked as ambiguous data item locations. This means that these data items are the target of the following gaussian blur operation. At the same time, the locations of other U-B data items other than the B data items will be marked as non-ambiguous data item locations. These data items will remain unchanged during the perturbation process, providing a stable background or context for the perturbed data.

After determining which data items need to be blurred, the computer system will perform a Gaussian blur operation on the data located at the blurred data item location. The data item is perturbed by gaussian blur, i.e. by changing its value, introducing a kind of "uncertainty" or "noise". Such perturbation helps to enhance the robustness of the machine learning model because it forces the model to learn the ability to maintain a stable output in the presence of small changes in the input data. After performing the gaussian blur, the data at the blurred data item positions will be updated to new, perturbed values. Finally, the computer system will combine these perturbed data (at the blurred data item locations) and unaffected data (at the non-blurred data item locations) to construct the final perturbed data sample.

Thus, by the combination of operations 321a and 322a, the computer system is able to generate a new data sample that both retains the key features of the original data and introduces beneficial perturbations, which is very valuable for enhancing the performance and generalization ability of the machine learning model.

In operation 32 above, as a third implementation, the B data items include a first data item and a second data item; the data perturbation mechanism comprises a third perturbation mechanism for replacing the B data items. Then, performing data perturbation on the classified annotation data sample based on the B data items to obtain a perturbation data sample corresponding to the classified annotation data sample, including:

Operation 321b: and taking the data item position of the first data item in the classified annotation data sample as a first data item position, taking the data item position of the second data item in the classified annotation data sample as a second data item position, and taking the data item positions of other data items except the first data item and the second data item in the B data items in the classified annotation data sample as a third data item position.

In this scheme of data perturbation, the computer system will perform a series of precise data item position exchange operations on the classification annotation data samples to effect the data perturbation. The key here is to understand the data items and their locations and to ascertain which data items are to be swapped. In this step, the computer system first identifies B data items in the categorical annotation data sample, including the first data item and the second data item, as well as other data items that may be present. Each data item has a particular location in the data sample, which may be an index in the sequence, a pixel coordinate in the image, or any other identifiable location in the data structure. The computer system will perform the following tasks:

Locating the first data item location: it will find the first data item in the categorical annotation data sample and record its location. This position is referred to as the first data item position.

Locating the second data item location: likewise, the computer system will find the second data item and record its location, which is referred to as the second data item location.

Determining a third data item location: in addition to the first data item and the second data item, other data items may be included in B. The locations of these other data items are collectively referred to as the third data item location. It is noted that the "third data item location" herein may actually comprise a plurality of locations, since B may be larger than 2.

For example, if the categorical annotation data sample is a sequence of numbers, such as [5, 10, 15, 20, 25], and the first data item is selected to be 10 (position 1, assuming counting from 0) and the second data item is selected to be 20 (position 3), then in this example the first data item position is 1, the second data item position is 3, and the third data item positions may be 0, 2, and 4 (positions corresponding to numbers 5, 15, and 25).

Operation 322b: in the classification annotation data sample, the data item location of the first data item is swapped from the first data item location to the second data item location.

Operation 323b: in the categorical annotation data sample, the data item location of the second data item is swapped from the second data item location to the first data item location.

Based on this, the computer system will perform a position-exchange operation on the particular data item in the categorized annotation data sample. These operations aim to change the order or position of data items in the original samples, thereby generating perturbed data samples that introduce variations while preserving the original data characteristics, helping to enhance the generalization ability of the machine learning model. In operation 322b, the computer system identifies the first data item in the categorized annotation data sample and records its current location (i.e., the first data item location). The device will then find a second data item location, which is another already determined location. The computer system will then perform a data item exchange operation: it moves the data item located at the first data item location to the second data item location, replacing the data item originally located there. Thus, the first data item occupies the original second data item.

In operation 323b, the second data item and its current location (i.e., the second data item location) are again confirmed. It will then find the first data item location (note that the first data item location at this time is already empty, since the first data item was moved to the second data item location in the previous operation). The computer system then moves the second data item from the second data item location to the first data item location, completing another data item exchange. Thus, the second data item occupies the original position of the first data item. Through these two successive exchange operations, the first data item and the second data item in the categorical annotation data sample are interchanged in position, while the other data items remain unchanged. This data perturbation approach helps introduce local variations while maintaining the overall characteristics of the data set, thereby enhancing the generalization ability of the machine learning model in the face of similar but slightly different data.

For example, if there is a sample of image data with a classification annotation, a plurality of pixels are included as data items, each pixel having a particular location (i.e., coordinates). Let us assume that we select two pixels as the first data item and the second data item and determine their positions. By performing operations 322b and 323b, the computer system will swap the positions of the two pixels, thereby generating a new, perturbed image sample. Such perturbed images may be used to train a machine learning model to better accommodate minor variations that may exist in the input image.

Operation 324b: a disturbance data sample is determined based on the second data at the first data item location, the first data at the second data item location, and the data at the third data item location.

In operation 324b, the computer system will construct a disturbance data sample based on the results of the previous steps after the data item locations are swapped, with the intention of generating a new data sample from the updated data item locations, which new sample will be a variant of the original data, for enhancing the training of the machine learning model.

Specifically, the computer system may perform the following tasks:

Collecting data at a location: first, the device will look at the first data item location, which is now the actual second data item at the first data item location, since the first data item has been swapped to the second data item location when operation 322b was performed. Likewise, the second data item location is now the original first data item as a result of performing operation 323 b. As for the third data item locations, the data at these locations is unchanged when operations 322b and 323b are performed.

The combined data generates a perturbation sample: the computer system then combines the collected data items to form a new data sample. The new sample contains the data item after the position exchange, namely the second data at the first data item position, the first data at the second data item position, and the unchanged data at the third data item position.

This process can be analogous to a simple data reordering task in which the order of data items is rearranged to generate new data combinations. In the context of machine learning models, such data perturbation techniques are typically used to augment a data set, helping the model learn robustness to small changes in the data.

For example, assume a building monitoring system records temperature data within a building in seconds: [23 ℃, 24 ℃, 23.5 ℃) ]. This is a time series data set in which each data point represents a temperature observation at a particular point in time. Now, if operations 322b and 323b are performed to position-exchange the temperature data at a certain point in time with the data at other points in time, such as exchanging the first 23 ℃ data point with the third 23.5 ℃ data point, the perturbed data sequence becomes: [23.5 ℃, 24 ℃, 23 ℃) ]. Then, in performing operation 324b, the computer system generates disturbance data samples based on the new time series data sequence. This perturbation sample may be used as a variant of the raw data for training or testing a machine learning model. In this way, the model can learn the ability to correctly identify and process building monitoring data even in the event of a change in the data acquisition sequence. Such data perturbation techniques are very helpful in improving the performance of machine learning models in dealing with time series data upsets or anomalies that may occur in real scenarios. The method can increase the diversity of the data set, help the model learn more robust and generalized characteristic representation, and further improve the prediction capability and stability of the model.

Operation 40: taking the integrated monitoring data sample and the disturbance data sample as enhanced data samples of the original building monitoring data sample, and taking the original building monitoring data sample and the enhanced data sample as training templates of an initial monitoring state identification model to be subjected to iterative optimization.

The initial monitoring state recognition model includes a feature refinement network and a state recognition network. In operation 40, the original building monitoring data sample is expanded by integrating the monitoring data sample and the disturbance data sample. The integrated monitoring data sample refers to the original data collected from the actual building monitoring system, and the disturbance data sample is new data generated by modifying or transforming the data under a certain rule. Integrating the two portions of data can form a larger, richer data set that helps to improve the generalization ability and robustness of subsequent machine learning models.

After the data enhancement is complete, the computer system will prepare a training template for training the initial monitoring state recognition model. These training templates include raw building monitoring data samples and enhanced data samples. Together, they are used as training samples, so that the model can contact more data under different conditions during learning, and various characteristics and modes of building monitoring states can be better captured.

The specific constitution of the initial monitoring state recognition model consists of two main parts: feature refinement network and state recognition network. The feature extraction network, which can be analogically an encoder, has the task of extracting meaningful features from the input building monitoring data. These features may be statistical properties of the data, timing relationships, spatial distribution, etc., depending on the nature of the input data. Once the features are extracted, they are passed to the state recognition network. A condition recognition network, which may be understood as a decoder, receives feature inputs from a feature refinement network and performs the recognition and classification of building monitoring conditions based on those features. In practice, the state recognition network may be a fully connected neural network or other suitable machine learning algorithm. Its function is to map the refined features to specific monitored states, such as normal states, abnormal states, etc.

Illustrating: assuming a building temperature and humidity monitoring system, the raw data samples include temperature and humidity readings over several consecutive days. To enhance the data, these readings are subjected to small-amplitude random perturbations, generating perturbed data samples. The raw data samples and the disturbance data samples are then input together as training templates into an initial monitoring state recognition model. The feature extraction network of the model learns how to extract key features from these readings, such as trends in temperature and humidity, day-to-night differences, etc. The state recognition network then determines the current monitored state of the building, such as whether it is in a comfortable environmental condition, based on these characteristics.

The data set can be expanded through the operation 40, the generalization capability of the model is improved, and a solid foundation can be laid for subsequent model training and state recognition tasks through feature extraction and the structure of the state recognition network.

Operation 50: and transmitting the training template to an initial monitoring state recognition model, and carrying out feature extraction on the data features of the data in the training template through a feature extraction network of the initial monitoring state recognition model to obtain a feature extraction array used for describing the training template.

In operation 50, the feature refinement network is a component of the machine learning model that is specifically responsible for extracting information useful for subsequent tasks, referred to as features, from the input data. This network is capable of converting raw data into a form that is easier to process and analyze through a series of mathematical transformations and operations. The feature refinement network is the aforementioned encoder, such as the encoder in a transformer.

Data characteristics of data refers to information extracted from raw data that can describe the nature or characteristics of some aspect of the data. In machine learning, features are typically represented in the form of values that reflect the inherent regularity and pattern of the data, which is critical to subsequent model training and prediction. For example, in building monitoring data, one data characteristic may be the average temperature over a period of time, which characteristic can reflect the temperature condition of the building over that period of time. Another data characteristic may be the daily rate of change of humidity, which can describe the fluctuations in humidity over the day. A feature refinement array is a set of values that are processed through a feature refinement network and represent features that are refined from data features. This array will typically be used as input to a machine learning model for subsequent classification, regression or other tasks. The feature refinement array may be a vector or a matrix.

In operation 50, the computer system obtains the training template generated in operation 40. The training templates comprise original building monitoring data samples and enhanced data samples subjected to data enhancement processing, and the original building monitoring data samples and the enhanced data samples jointly form a rich data set for training a machine learning model. Next, the computer system passes the training template into an initial monitoring state recognition model. This model is a pre-built machine learning model designed to identify and understand the different monitoring states of the building. The model internally contains a network structure specially used for feature extraction, namely a feature extraction network. To more particularly illustrate the process of feature refinement, it may be assumed that the feature refinement network is a convolutional layer or a loop layer in a deep neural network. These network layers can automatically learn the method of extracting useful features from the input data. For example, in processing time series data, a Recurrent Neural Network (RNN) or a variant thereof, such as a long short term memory network (LSTM), may effectively capture timing dependencies in the data, thereby extracting features of the time series.

After feature refinement is complete, the computer system obtains a feature refinement array. This array contains feature values extracted from the training templates that describe the internal rules and patterns of data in the training templates in the form of values. These feature values will be passed as inputs to other parts of the monitoring state recognition model for subsequent classification or recognition tasks.

Specifically, operation 50, the training template is transferred to the initial monitoring state recognition model, and feature extraction is performed on the data features of the data in the training template through the feature extraction network of the initial monitoring state recognition model, so as to obtain a feature extraction array for describing the training template, which specifically may include the following sub-operations:

operation 51: and acquiring a target data item group chain sample from the data item group chain samples included in the training template.

In operation 50, data feature refinement of the training template involves a number of sub-operations, where the core task of operation 51 is to obtain a target data item set chain sample from the training template. The training template is a collection of data item group chain samples, each data item group chain sample being formed by a series of data items arranged in a particular order. These data items may be sensor readings, image frames, text recordings, etc., depending on the type of data and the manner of acquisition of the monitoring system. To obtain a target data item group chain sample, the computer system first accesses a memory or database that stores training templates. It then selects one or more data item group chain samples from the training template as targets according to a preset rule or algorithm. The selected rules can be based on random sampling, sequential reading, specific condition screening and the like, and depend on training requirements and model design.

For example, the implementation of operation 51 is described: assuming that the training template contains temperature and humidity data for a week of the building, each data item set chain sample represents a data record for a day. The computer system may sequentially select daily data records as the target data item group chain samples in a time sequence. Alternatively, it may also select several days with the greatest temperature fluctuation as the target data item group chain sample according to specific conditions, such as the temperature fluctuation.

After obtaining the target data item group chain samples, the computer system will perform subsequent feature extraction operations to extract useful information from these samples for training and optimizing the monitoring state recognition model. Operation 51 serves as a starting point for the overall feature refinement process, ensuring that subsequent operations can be performed on specific data samples, thereby improving the accuracy and effectiveness of feature refinement.

It should be noted that in practical applications, operation 51 may involve a large amount of data processing and memory operations, so performance and resource allocation of the computer system are critical to the efficiency and stability of the operation. Meanwhile, in order to ensure the accuracy and generalization capability of feature extraction, the diversity and representativeness of the training templates are also key factors to consider.

Operation 52: and acquiring the data characteristics of the target data item group chain sample, and transmitting the data characteristics of the target data item group chain sample to a characteristic extraction network.

In operation 52, the computer system obtains data characteristics of the target data item group chain sample. Data features are abstract representations of information in a chain of data items, which are able to capture the unique properties and patterns of the sample. In the context of machine learning, features are typically numerical or can be converted to numerical for ease of processing and analysis by a computer. For a target data item group chain sample, the characteristics can include various statistics, time series attributes, image characteristics and the like, depending on the type of data and the requirements of the monitoring system. For example, in a building monitoring system, the data characteristics may include an average value of temperature, a maximum value of humidity, a rate of change of illumination intensity, and the like. These features can reflect important aspects of building status and are of critical importance for subsequent status recognition and analysis. Once the data features of the target data item group chain sample are extracted, the computer system will represent these features in vector form, i.e., the data item group chain vector representation. A vector is an ordered list of values, each value corresponding to a particular feature. By representing the data features as vectors, they can be conveniently input into a machine learning model for processing.

Next, the computer system passes the data feature vector of the target data item group chain sample into a feature refinement network. Feature extraction networks are an important component of machine learning models that are responsible for learning and extracting higher level information from the input features. This information is critical to subsequent classification, regression, or other tasks. The feature refinement network may employ various machine learning algorithms or neural network structures, depending on the complexity of the problem and the nature of the data. For example, in processing time series data, a Recurrent Neural Network (RNN) or long short-term memory network (LSTM) may be used to capture timing dependencies in the data. While Convolutional Neural Networks (CNNs) are a common option in processing image data, they can effectively extract spatial features in the image.

Operation 53: and carrying out feature extraction on the data features of the target data item group chain samples through a feature extraction network to obtain a group chain feature extraction array of the target data item group chain samples until each data item group chain sample in the training template is used as the target data item group chain sample to obtain a group chain feature extraction array of each data item group chain sample.

In operation 53, the computer system processes the data features of the target data item group chain sample using the constructed feature refinement network. Feature extraction networks are specifically designed to learn and extract meaningful information from raw data. This information will play a key role in subsequent tasks such as classification, regression or clustering.

In this step, the feature refinement network vector encodes the data features of the target data item group chain samples. This means that the network will convert each feature into a representation of values that are combined to form a feature vector. This process can be seen as a compression or encoding of the data that preserves critical information in the data while removing redundancy and noise. Through the processing of the feature refinement network, the computer system obtains a group chain feature refinement array of the target data item group chain samples. The array is a list of values, each value corresponding to a particular refinement feature. These refined features are obtained by complex computation of the network on the basis of the original features, which are able to describe more accurately the intrinsic properties and patterns of the data item group chain samples.

To ensure that the entire training template is adequately feature refined, the computer system repeats this process until each data item set chain sample in the training template has been processed as a target data item set chain sample. Finally, each data item group chain sample results in a corresponding group chain feature refinement array.

This process can be analogous to a factory pipeline in which each data item group chain sample is a raw material, the feature extraction network is a processing facility, and the group chain feature extraction array is the end product. Through this process, the raw data is converted into a form more suitable for machine learning and analysis.

It should be noted that the specific implementation of the feature refinement network may vary depending on the task requirements and data characteristics. For example, convolutional Neural Networks (CNNs) may be used to extract spatial features when processing image data; while processing time series data, a Recurrent Neural Network (RNN) or long short-term memory network (LSTM) may be used to capture the time dependence. These network structures and algorithms are designed to more efficiently extract useful information from the raw data.

Operation 54: and taking the group chain feature extraction array of each data item group chain sample as a feature extraction array for representing the training template.

After operation 53, the computer system has performed feature refinement on each of the data item group chain samples and has obtained their respective group chain feature refinement arrays. The arrays contain meaningful information extracted from the original data, and can reflect the intrinsic properties and modes of the data item array chain samples.

To integrate these extracted features to characterize the entire training template, the computer system performs operation 54. Specifically, the group chain feature refinement arrays of each data item group chain sample are combined or summarized in a particular manner. This process may be accomplished by simple stitching, weighted averaging, pooling operations, etc., depending on the design of the model and the nature of the data.

Finally, the computer system obtains a feature extraction array that characterizes the entire training template, via operation 54. The array contains key information extracted from all data item group chain samples, and can describe the data characteristics and distribution of the training template more comprehensively and accurately. This feature refinement array will serve as an important input for subsequent machine learning model training and optimization.

It should be noted that the specific form and dimensions of the feature refinement array may vary depending on the task requirements and data characteristics. For example, in processing image data, the feature extraction array may be a multi-dimensional tensor that includes spatial features and texture information extracted from the image; in processing time series data, the feature extraction array may be a two-dimensional matrix, wherein each row corresponds to a feature vector of one time step.

In addition, the implementation of the machine learning model also affects the manner in which the feature extraction array is used. For example, in a classification task, the feature extraction array may be directly input into the classifier for training and prediction; in the generating task, the feature extraction array can be used as an input or condition of a generating model for generating data samples conforming to specific features.

In summary, operation 54 is a process of integrating the feature refinement results of each data item group chain sample to form a feature refinement array that characterizes the entire training template. The process is one of key links of machine learning model training and data analysis, and is beneficial to improving the performance and accuracy of the model.

Operation 60: and acquiring annotation data characteristics of the classified annotation data samples, transmitting the characteristic extraction array and the annotation data characteristics to a state recognition network of an initial monitoring state recognition model, recognizing and outputting sample loss corresponding to a training template through the state recognition network, optimizing the initial monitoring state recognition model based on the sample loss, and taking the optimized initial monitoring state recognition model as a target monitoring state recognition model for building safety state recognition.

Operation 60 is a key step in model training and optimization based on the data processing and feature extraction described above. The aim is to train and optimize a monitoring model capable of identifying the safety state of the building by utilizing the characteristics of the extracted characteristic array and the corresponding classification annotation data sample.

First, the computer system obtains annotation data characteristics that categorize the annotation data samples. These annotation data are labels or classification information corresponding to the data item set chain samples in the training templates. In order for a computer to understand and process such annotation data, it is often necessary to convert it into a numerical representation, i.e. a vector representation of the annotation data. This may be achieved by an embedded network that is capable of converting discrete labels or text information into representations in a continuous vector space, thereby capturing semantic relationships between labels.

The computer system then passes the extracted feature refinement array from the previous operation along with the annotated data features to the state recognition network of the initial monitoring state recognition model. The state recognition network is the core part of the model and is responsible for judging the current safety state of the building according to the input characteristics. In this step, the state recognition network will attempt to predict the class labels for each data item group chain sample in the training template based on the input feature refinement array and annotation data features.

To evaluate the performance of the state recognition network, the computer calculates the difference between its predicted outcome and the actual annotation data, which is referred to as the sample loss. The sample loss is an indicator of how much the model is mispredicted, and the larger it is, the worse the predictive power of the model, and further optimization is required. Based on the calculated sample loss, the computer may optimize the initial monitoring state identification model. The optimization process is to reduce sample loss by adjusting parameters in the model, thereby improving the prediction accuracy of the model. This typically requires iteratively updating the model parameters using various optimization algorithms, such as gradient descent algorithms or variations thereof.

Finally, after repeated iterative optimization, the performance of the initial monitoring state identification model can be improved, and the safety state of the building can be identified more accurately. At this time, the computer system stores the optimized model and uses it as a target monitoring state recognition model for building safety state recognition.

In practice, the state recognition network may employ a variety of different machine learning models or neural network structures, such as Convolutional Neural Networks (CNNs) for processing image data, cyclic neural networks (RNNs) or long-short-term memory networks (LSTM) for processing time-series data, and the like. The choice of a specific model structure depends on the requirements of the task and the nature of the data. In summary, operation 60 is a comprehensive step involving several links to obtain annotation data features, model training, performance assessment, and model optimization. Through the synergistic effect of the steps, the computer system can train a target monitoring state identification model which can accurately identify the safety state of the building.

As one possible implementation, the training template includes m data item group chain samples, the m data item group chain samples include a target data item group chain sample, the feature refinement array is determined for a group chain feature refinement array obtained when each of the m data item group chain samples is taken as the target data item group chain sample, the classification annotation data sample includes m classification annotation data item group chains corresponding to the m data item group chain samples, one data item group chain sample corresponds to one classification annotation data item group chain, the m classification annotation data item group chains include a target classification annotation data item group chain, and the annotation data feature is determined for a group chain monitoring state identification feature obtained when each of the m classification annotation data item group chains is taken as the target classification annotation data item group chain.

In this implementation, the training template is the basis for building and training a machine learning model, and contains a plurality of data item set chain samples that are used to provide the various features and patterns required for model learning. In this scenario, the training template specifically includes m data item group chain samples, each of which is likely to be selected as a target data item group chain sample. The target data item group chain sample is an object that is of particular interest and processing in the feature refinement process. Feature refinement is a key step in machine learning, whose purpose is to extract meaningful and helpful information for model training from the raw data. In this solution, feature refinement is accomplished through a specialized feature refinement network. When the computer system sequentially processes each of the m data item group chain samples as a target data item group chain sample, the feature extraction network performs deep analysis and conversion on the data features of each sample, and finally generates a group chain feature extraction array corresponding to the sample. This process is repeated until all samples in the training template have been processed, resulting in a series of sets of chain feature refinement arrays. At the same time, in order to supervise the learning process of the model and evaluate its performance, it is also necessary to provide a corresponding classification annotation data item group chain for each data item group chain sample. In this solution, each data item group chain sample has a classification annotation data item group chain associated with it, and there is a one-to-one correspondence between them. These chains of classification annotation data item groups form a set of m chains of classification annotation data item groups, including the chain of target classification annotation data item groups of particular interest.

Similar to feature refinement, annotation data features also need to be extracted and transformed by specific methods. In this solution, the extraction of annotation data features is done via an embedded network. When the computer system sequentially processes each of the m classification annotation data item group chains as a target classification annotation data item group chain, the embedded network performs deep analysis and conversion on each annotation data item group chain, and finally generates a group chain monitoring state identification feature corresponding to the annotation data item group chain. This process is likewise repeated until all of the chain of categorized annotation data item groups have been processed, thereby yielding a series of chain monitoring status recognition features.

In summary, the embodiment further analyzes and processes each data item group chain sample and the corresponding classification annotation data item group chain in the training template through the feature extraction network and the embedded network, so as to obtain a series of group chain feature extraction arrays and group chain monitoring state identification features. These features and arrays will serve as important inputs and basis for subsequent machine learning model training and optimization. The specific machine learning model may be a convolutional neural network (for processing image class data), a cyclic neural network (for processing sequence class data), or other model structure and algorithm suitable for the particular task.

Based on the above embodiment, operation 60 above, the feature extraction array and annotation data features are transferred to the state recognition network of the initial monitoring state recognition model, and the sample loss corresponding to the training template is recognized and output through the state recognition network, which may include the following sub-operations:

operation 61: in the feature extraction array, the group chain feature extraction array of the target data item group chain sample is taken as a target group chain feature extraction array, and in the annotation data feature, the group chain monitoring state identification feature of the target classification annotation data item group chain is taken as a target group chain monitoring state identification feature.

Operation 62: and transmitting the target group chain feature extraction array and the target group chain monitoring state identification feature into a state identification network of the initial monitoring state identification model, and identifying and outputting group chain sample loss corresponding to the target data item group chain sample through the state identification network.

In operation 61, the computer system selects a set chain feature refinement array corresponding to the target data item set chain sample from the feature refinement arrays, the array containing key information refined from the target data item set chain sample, capable of reflecting intrinsic features and patterns of the sample. Likewise, the computer selects a group link monitoring status identification feature of the target categorized annotation data item group link corresponding to the target data item group link sample from among the annotation data features. These features are obtained after the annotation data is processed by the embedded network, and they represent the information of the classified annotations in the form of numerical vectors, so that the computer can understand and process these annotation data.

For example, assuming that the feature refinement array is a two-dimensional array in which each row represents the feature refinement result of a data item group chain sample, then the group chain feature refinement array of the target data item group chain sample is one row in the two-dimensional array. Similarly, if the annotation data feature is also a two-dimensional array, then the group chain monitor status identification feature of the target categorized annotation data item group chain is the corresponding row in the two-dimensional array.

In operation 62, the computer system transmits the target set of chain feature refinement arrays and the target set of chain monitor state identification features selected in the previous step to the state identification network of the initial monitor state identification model. The state recognition network is the core part of the model, which is responsible for prediction and classification based on the input features. Through the processing of the state recognition network, the computer obtains the predicted result of the target data item group chain sample, compares the predicted result with the real result, and calculates the group chain sample loss. In the embodiment of the disclosure, the classification annotation data item group chain corresponding to each data item group chain sample in the integration monitoring data sample may be used as the real recognition result of each data item group chain sample in the integration monitoring data sample, and then the group chain sample loss corresponding to each data item group chain sample in the integration monitoring data sample may be determined according to the group chain monitoring state recognition feature of each data item group chain sample in the integration monitoring data sample and the group chain monitoring state recognition feature corresponding to the classification annotation data item group chain corresponding to each data item group chain sample. The loss function used to obtain the loss is not limited, and is, for example, a cross entropy loss function, an absolute value loss function, or the like.

The group chain sample loss is an index for measuring the prediction accuracy of the model, and reflects the difference between the predicted result and the real result of the model on the group chain sample of the target data item. The smaller this loss value, the stronger the predictive power of the model, and the higher the fit to the training template.

In one embodiment, the target set of chain feature extraction arrays include a first set of chain feature extraction arrays, a second set of chain feature extraction arrays, and a third set of chain feature extraction arrays, the first set of chain feature extraction arrays being set of chain feature extraction arrays of the target data item set chain samples in the raw building monitor data samples in the training templates, the second set of chain feature extraction arrays being set of chain feature extraction arrays of the target data item set chain samples in the integrated monitor data samples in the training templates, the third set of chain feature extraction arrays being set of chain feature extraction arrays of the target data item set chain samples in the disturbance data samples in the training templates, the target set of chain feature extraction arrays including a first set of chain feature extraction features corresponding to the first set of chain feature extraction arrays, a second set of chain feature extraction features corresponding to the second set of chain feature extraction arrays, and a third set of chain feature extraction features corresponding to the third set of chain feature extraction arrays in the training templates, the first set of chain feature extraction arrays being set of chain feature extraction arrays in the classification annotation data samples, the first set of chain feature extraction arrays corresponding to the first set of feature extraction arrays in the disturbance data sample, and the third set of chain feature extraction arrays corresponding to the classification feature extraction arrays in the first set of chain feature extraction arrays in the disturbance data sample.

At this time, in operation 62, identifying, by the state identification network, a group chain sample loss corresponding to the output target data item group chain sample may include:

operation 621: and extracting group chain monitoring state identification features of the group chain samples of the target data items in the original building monitoring data samples according to the first group chain feature extraction array through the state identification network, and determining group chain sample loss corresponding to the group chain samples of the target data items in the original building monitoring data samples through the group chain monitoring state identification features of the group chain samples of the target data items in the original building monitoring data samples and the first group chain monitoring state identification features.

Operation 622: and extracting group chain monitoring state identification features of the group chain samples of the target data items in the integrated monitoring data samples according to the second group chain feature extraction array through the state identification network, and determining group chain sample loss corresponding to the group chain samples of the target data items in the integrated monitoring data samples through the group chain monitoring state identification features of the group chain samples of the target data items in the integrated monitoring data samples and the second group chain monitoring state identification features.

Operation 623: and determining the group link sample loss corresponding to the target data item group link sample in the disturbance data sample through the group link monitoring state identification feature of the target data item group link sample in the disturbance data sample and the third group link monitoring state identification feature.

In the above embodiment, the target set of chain feature refinement arrays are no longer a single array, but include a first set of chain feature refinement arrays, a second set of chain feature refinement arrays, and a third set of chain feature refinement arrays, which correspond to the target data item set chain samples in the original building monitoring data samples, the integrated monitoring data samples, and the disturbance data samples, respectively, in the training templates. Likewise, the target set of chain monitor state identification features also includes a first set of chain monitor state identification features, a second set of chain monitor state identification features, and a third set of chain monitor state identification features corresponding to the three sets of chain feature refinement sets.

In operation 621, the computer system first identifies, via the state identification network, a set of chain monitoring state identification features of a set of chain samples of target data items in the output original building monitoring data sample using the first set of chain feature refinement arrays. This feature is refined based on a target data item group chain sample in the original building monitoring data sample, which can reflect the key information of the sample in the building monitoring state.

The computer then compares the set of chain monitoring state identification features of the identification output with the first set of chain monitoring state identification features (i.e., corresponding features in the categorical annotation data sample) to determine differences therebetween. This difference reflects the predictive accuracy of the model for the target data item group chain samples in the original building monitoring data samples, i.e., group chain sample loss. The smaller the loss value, the stronger the predictive ability of the model, and the higher the fitting degree of the original building monitoring data. Similarly, in operations 622 and 623, the computer system performs similar processing on the target data item group chain samples in the integrated monitor data sample and the disturbance data sample, respectively. And outputting corresponding group chain monitoring state identification characteristics through state identification network identification, and comparing the corresponding characteristics in the classified annotation data samples to determine group chain sample loss.

Because of the more detailed data classification and processing mode, the model can capture the differences and characteristics among different types of data samples more accurately. The prediction accuracy and generalization capability of the model are improved, and the model can be better adapted to various complex building monitoring scenes in practical application.

In summary, operations 621, 622, and 623 provide an important basis for subsequent model optimization by utilizing the multi-set chain features to refine the array and the set chain monitor state identification features to determine the set chain sample loss for different types of data samples. The specific machine learning model may be a deep neural network, support vector machine, or other model structure and algorithm suitable for the particular task.

Operation 63: if each data item group chain sample in the m data item group chain samples is used as a target data item group chain sample, and each classification annotation data item group chain in the m classification annotation data item group chains is used as a target classification annotation data item group chain, group chain sample loss corresponding to each data item group chain sample is obtained.

Operation 64: and determining the sample loss corresponding to the training template based on the group chain sample loss corresponding to each data item group chain sample.

In operation 63, the computer system will process the plurality of data item group chain samples and their corresponding classification annotation data item group chains to obtain group chain sample losses for each data item group chain sample. Specifically, assume that there are m data item group chain samples, each of which is considered a target data item group chain sample. Meanwhile, it is assumed that there are m classification annotation data item group chains, each of which is regarded as a target classification annotation data item group chain. For each target data item group chain sample, the computer system performs a process similar to operations 61 and 62. First, a group chain feature refinement array of the data item group chain sample is extracted from the feature refinement array, and group chain monitoring status identification features of the classified annotation data item group chain corresponding to the data item group chain sample are extracted from the annotation data features.

The computer system then transmits the extracted set of chaining feature refinements and the chaining monitoring state identification features to the state identification network of the initial monitoring state identification model. The state recognition network processes the features and outputs a prediction of the data item group chain samples. The computer then compares the predicted outcome to the actual classification annotation data item group chain to calculate the group chain sample loss for that data item group chain sample.

This process is performed once for each of the m data item group chain samples, and thus eventually m group chain sample losses are obtained, each loss corresponding to one data item group chain sample.

In operation 64, the computer system determines a sample loss corresponding to the training template based on the group chain sample loss for each of the data item group chain samples obtained in operation 63. This sample loss is a comprehensive consideration of the group chain sample loss for all data item group chain samples. In particular, the computer system may employ a variety of methods to calculate the sample loss, such as calculating an average, weighted average, or other statistic of all sets of chain sample losses.

This sample loss is an important indicator of how well the model fits to the entire training template. The smaller the sample loss, the higher the fitting degree of the model to the training template, and the stronger the prediction capability.

It should be noted that in this process, the specific implementation of the initial monitoring state identification model may be varied. For example, it may be a deep neural network model that learns and characterizes the data item set chain samples by multi-layer nonlinear transformations; or a support vector machine model, and classifying and predicting by searching the optimal hyperplane in a high-dimensional space. The specific model selection should be determined according to the characteristics and requirements of the actual problem.

In operation 60, the initial monitoring state recognition model is optimized based on the sample loss, and the optimized initial monitoring state recognition model is used as a target monitoring state recognition model for building safety state recognition, which specifically may include: optimizing the initial monitoring state identification model based on sample loss to obtain an optimized result; if the optimized result indicates that the optimized initial monitoring state recognition model reaches the preset convergence evaluation condition, the initial monitoring state recognition model reaching the preset convergence evaluation condition is used as the target monitoring state recognition model.

The computer system optimizes the initial monitoring state identification model based on the previously calculated sample loss. The optimization aims at adjusting parameters and structures of the model so as to improve the fitting degree of the model to the training template, thereby enhancing the prediction capability of the model. In particular, the optimization process may employ a variety of machine learning algorithms or neural network optimization techniques. For example, if the initial monitoring state identification model is a neural network model, then the gradient of the sample loss to the model parameters may be calculated by a back-propagation algorithm and the model parameters updated in the opposite direction of the gradient to reduce the sample loss. This process may be iteratively repeated until the model reaches a predetermined convergence evaluation condition. The convergence evaluation condition is a set of criteria used to determine whether the model has reached an optimization objective. It may include various indicators such as whether the value of the sample loss is less than some preset threshold, whether the model parameters are stable and no significant change occurs, or whether the number of iterations reaches a preset upper limit, etc. When the model reaches these convergence evaluation conditions, the model can be considered to have been optimized.

Once the initial monitoring state recognition model is optimized and reaches a preset convergence evaluation condition, the computer system takes the initial monitoring state recognition model as a target monitoring state recognition model. The target monitoring state recognition model is obtained by optimizing an initial model, has stronger prediction capability and higher fitting degree, and is more suitable for recognizing building safety states.

It should be noted that the optimization process may involve adjustment of multiple super-parameters, such as learning rate, iteration number, batch size, etc. The setting of these hyper-parameters directly affects the optimization effect and model performance. Therefore, in practical applications, multiple experiments and adjustments may be required to find the optimal super parameter setting according to the characteristics and requirements of a specific problem.

Operation 60 optimizes the initial monitoring state recognition model based on sample loss, and takes the optimized model as a target monitoring state recognition model, so that the prediction capability and fitting degree of the model are improved, and more accurate model support is provided for subsequent building safety state recognition.

After training to obtain the target monitoring state recognition model, the target monitoring state recognition model can be used for carrying out safety state recognition reflected by building monitoring data, and specifically, the use process of the model can comprise the following operations:

Operation 100: building monitoring data is obtained.

Operation 200: and transmitting the building monitoring data to a feature extraction network of the target monitoring state recognition model, and carrying out feature extraction on the data features of the building monitoring data based on the feature extraction network to obtain a feature extraction array used for describing the building monitoring data.

Operation 300: and transmitting the feature extraction array of the building monitoring data to a state recognition network of the target monitoring state recognition model to perform monitoring state recognition, so as to obtain annotation data features of the building monitoring data.

Operation 400: classified annotation data of the building monitoring data is obtained based on the annotation data characteristics of the building monitoring data.

In one implementation, operation 100 obtains building monitoring data, which may include various sensor readings, camera images, door access records, etc., that together form a comprehensive view of the building's security status. Such data is typically collected by a monitoring system within the building and stored in a database or file system accessible to the computer system. Next, operation 200 communicates the building monitoring data to a feature refinement network of the target monitoring state identification model. Feature refinement networks are an integral part of the model that is responsible for extracting meaningful features from raw data. These features can describe the inherent properties and relationships of building monitoring data and are important basis for subsequent status recognition. The feature refinement network may employ various machine learning algorithms or neural network structures, such as Convolutional Neural Networks (CNNs) for image feature extraction, or Recurrent Neural Networks (RNNs) for sequence data feature extraction. Through the processing of the feature refinement network, the computer system can obtain a feature refinement array that contains key features that describe building monitoring data. Operation 300 then passes the feature extraction array into a state recognition network of the target monitoring state recognition model for monitoring state recognition. The state recognition network is another key component of the model, which is responsible for recognizing the monitoring state of the building based on the feature extraction array. This process typically involves comparing or matching the feature refinement array with predefined monitored states to determine the state class to which the current data belongs. The state recognition network may employ a classification algorithm such as a Support Vector Machine (SVM), decision tree, or neural network classifier. Through the processing of the status recognition network, the computer system is able to obtain annotated data characteristics of the building monitoring data that describe the status category to which the data belongs. Finally, operation 400 obtains categorized annotation data for the building monitoring data based on the annotation data characteristics. The classification annotation data is a detailed annotation of the security status recognition results of the individual data items, which provides direct information about the security status of the building. For example, the classification annotation data may indicate whether there is abnormal behavior in a certain camera image, whether a certain sensor reading exceeds a safety threshold, and so on. Such information is critical to assessing the safety status of a building and taking corresponding safety measures.

In summary, through operations 100 to 400, the computer system can perform comprehensive security status recognition on building monitoring data using the target monitoring status recognition model. The process involves data acquisition, feature extraction, state identification and classification annotation, providing powerful support for building security management.

It should be noted that, in the embodiment of the disclosure, if the building safety monitoring method based on the internet of things is implemented in the form of a software function module, and is sold or used as an independent product, the building safety monitoring method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present disclosure may be essentially or portions contributing to the related art, and the software product may be stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present disclosure. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes. Thus, embodiments of the present disclosure are not limited to any specific hardware, software, or firmware, or any combination of the three.

The disclosed embodiments provide a computer system comprising a memory storing a computer program executable on the processor and a processor implementing some or all of the steps of the above method when the processor executes the program. The disclosed embodiments provide a computer readable storage medium having stored thereon a computer program which when executed by a processor performs some or all of the steps of the above method. The computer readable storage medium may be transitory or non-transitory. The disclosed embodiments provide a computer program comprising computer readable code which, when run in a computer device, performs some or all of the steps for implementing the methods described above.

Embodiments of the present disclosure provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program which, when read and executed by a computer, performs some or all of the steps of the above-described method. The computer program product may be realized in particular by means of hardware, software or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium, in other embodiments the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like. It should be noted here that: the above description of various embodiments is intended to emphasize the differences between the various embodiments, the same or similar features being referred to each other. The above description of apparatus, storage medium, computer program and computer program product embodiments is similar to that of method embodiments described above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the disclosed apparatus, storage medium, computer program and computer program product, please refer to the description of the embodiments of the disclosed method.

Fig. 2 is a schematic diagram of a hardware entity of a computer system according to an embodiment of the disclosure, as shown in fig. 2, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program executable on the processor 1001, the processor 1001 implementing the steps in the method of any of the embodiments described above when the program is executed.

The memory 1002 stores a computer program executable on a processor, and the memory 1002 is configured to store instructions and applications executable by the processor 1001, and may also cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the processor 1001 and the computer system 1000, which may be implemented by a FLASH memory (FLASH) or a random access memory (Random Access Memory, RAM).

The steps of the building safety monitoring method based on the internet of things according to any one of the above are implemented when the processor 1001 executes a program. The processor 1001 generally controls the overall operation of the computer system 1000.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present disclosure, the size of the sequence numbers of the steps/processes described above does not mean the order of execution, and the order of execution of the steps/processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation of the embodiments of the present disclosure. The foregoing embodiment numbers of the present disclosure are merely for description and do not represent advantages or disadvantages of the embodiments. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Claims

1. The building safety monitoring method based on the Internet of things is characterized by comprising the following steps of:

acquiring an original building monitoring data sample and a classification annotation data sample corresponding to the original building monitoring data sample; the classification annotation data sample comprises classification annotation data for converting key monitoring data items in the original building monitoring data sample, wherein the classification annotation data is data obtained after monitoring state identification of the key monitoring data items;

converting the key monitoring data items into the classification annotation data in the original building monitoring data samples, and taking the converted original building monitoring data samples containing the classification annotation data as integrated monitoring data samples;

performing data disturbance on the classified annotation data sample to obtain a disturbance data sample corresponding to the classified annotation data sample;

taking the integrated monitoring data sample and the disturbance data sample as enhanced data samples of the original building monitoring data sample, and taking the original building monitoring data sample and the enhanced data sample as training templates of an initial monitoring state recognition model to be subjected to iterative optimization; the initial monitoring state identification model comprises a characteristic extraction network and a state identification network;

The training template is transmitted to the initial monitoring state recognition model, and the feature extraction network of the initial monitoring state recognition model is used for carrying out feature extraction on the data features of the data in the training template to obtain a feature extraction array used for describing the training template;

acquiring annotation data characteristics of the classified annotation data samples, transmitting the characteristic extraction array and the annotation data characteristics to the state recognition network of the initial monitoring state recognition model, recognizing and outputting sample loss corresponding to the training template through the state recognition network, optimizing the initial monitoring state recognition model based on the sample loss, and taking the optimized initial monitoring state recognition model as a target monitoring state recognition model for building safety state recognition;

the data perturbation is performed on the classified annotation data sample to obtain a perturbation data sample corresponding to the classified annotation data sample, including:

carrying out data decomposition on the classified annotation data sample to obtain U data items of the classified annotation data sample, wherein U is greater than or equal to 1;

Acquiring a data disturbance mechanism matched with the classified annotation data sample, determining B data items from the U data items through the data disturbance mechanism, and performing data disturbance on the classified annotation data sample based on the B data items to obtain a disturbance data sample corresponding to the classified annotation data sample, wherein B is a non-zero natural number smaller than U;

the data perturbation mechanism comprises a first perturbation mechanism for cleaning the B data items; the data perturbation is performed on the classified annotation data sample based on the B data items to obtain a perturbation data sample corresponding to the classified annotation data sample, which comprises the following steps:

performing cleaning processing on the B data items in the classification annotation data samples, and taking the classification annotation data samples after the cleaning processing as the disturbance data samples;

alternatively, the data perturbation mechanism comprises a second perturbation mechanism for Gaussian blur of the B data items; the data perturbation is performed on the classified annotation data sample based on the B data items to obtain a perturbation data sample corresponding to the classified annotation data sample, which comprises the following steps:

Taking the data item positions of the B data items in the classified annotation data sample as fuzzy data item positions, and taking the data item positions of other data items except the B data items of the U data items in the classified annotation data sample as non-fuzzy data item positions;

performing Gaussian blur on the data in the blurred data item position, and determining the disturbance data sample through the data in the non-blurred data item position and the data in the blurred data item position after Gaussian blur;

alternatively, the B data items include a first data item and a second data item; the data perturbation mechanism comprises a third perturbation mechanism for replacing the B data items; the data perturbation is performed on the classified annotation data sample based on the B data items to obtain a perturbation data sample corresponding to the classified annotation data sample, which comprises the following steps:

taking the data item position of the first data item in the classified annotation data sample as a first data item position, taking the data item position of the second data item in the classified annotation data sample as a second data item position, and taking the data item positions of other data items except the first data item and the second data item in the B data item in the classified annotation data sample as a third data item position;

In the categorical annotation data sample, swapping the data item location of the first data item from the first data item location to the second data item location;

in the categorical annotation data sample, swapping the data item location of the second data item from the second data item location to the first data item location;

the disturbance data sample is determined based on the second data at the first data item location, the first data at the second data item location, and the data at the third data item location.

2. The method of claim 1, wherein the converting the key monitoring data item into the classification annotation data in the original building monitoring data sample, taking the converted original building monitoring data sample containing the classification annotation data as an integrated monitoring data sample, comprises:

determining a key monitoring data item paragraph of the key monitoring data item in the original building monitoring data sample;

in the original building monitoring data sample, converting the key monitoring data item into the classification annotation data according to the key monitoring data item paragraph;

And taking the converted original building monitoring data sample containing the classified annotation data as an integrated monitoring data sample.

3. The method according to claim 2, wherein the method further comprises:

acquiring a monitoring scene corresponding to an original building monitoring data sample, and acquiring a scene common abnormal database matched with the monitoring scene; the scene common abnormal database comprises common abnormal events and classification annotation data items corresponding to the common abnormal events;

performing data decomposition on the original building monitoring data sample to obtain X data items of the original building monitoring data sample, wherein X is greater than or equal to 1;

acquiring a data conversion mechanism matched with the scene common exception database, and indexing the X data items in the scene common exception database based on the data conversion mechanism;

taking the indexed data items which are the same as the common abnormal events in the scene common abnormal database as quasi-annotation data items; the number of the data items to be marked is Y, wherein Y is not more than X;

and determining W quasi-annotation data items in the Y quasi-annotation data items as target conversion data items, taking the determined target conversion data items as the key monitoring data items in the original building monitoring data sample, and marking the classification annotation data items corresponding to the target conversion data items as the classification annotation data in the classification annotation data sample, wherein W is smaller than Y.

4. A method according to claim 3, characterized in that the method further comprises:

taking other data items except the Y data items to be marked as first data items, wherein the number of the first data items to be marked is Q, and the sum of Y and Q is equal to X;

taking the quasi-annotation data items except the target conversion data item out of the Y quasi-annotation data items as second undetermined data items;

determining a fixed data item in the original building monitoring data sample based on the first to-be-determined data item and the second to-be-determined data item;

the taking the converted original building monitoring data sample containing the classified annotation data as an integrated monitoring data sample comprises the following steps:

taking the data item position of the fixed data item in the original building monitoring data sample as a fixed data item position;

the integrated monitoring data sample is determined based on the fixed data item at the fixed data item location, the classification annotation data at the accentuated monitoring data item paragraph, and the accentuated monitoring data item paragraph.

5. The method of claim 1, wherein the importing the training template into the initial monitoring state recognition model, feature extracting data features of data in the training template through the feature extraction network of the initial monitoring state recognition model, obtaining a feature extraction array to describe the training template, comprises:

Acquiring a target data item group chain sample from the data item group chain sample included in the training template;

acquiring data characteristics of the target data item group chain sample, and transmitting the data characteristics of the target data item group chain sample to the characteristic extraction network;

performing feature extraction on the data features of the target data item group chain samples through the feature extraction network to obtain a group chain feature extraction array of the target data item group chain samples until each data item group chain sample in the training template is used as the target data item group chain sample to obtain a group chain feature extraction array of each data item group chain sample;

and taking the group chain feature extraction array of each data item group chain sample as a feature extraction array for representing the training template.

6. The method of claim 1, wherein the training template comprises m data item group chain samples, the m data item group chain samples comprising target data item group chain samples; the feature extraction array is obtained by determining a chain feature extraction array obtained when each data item chain sample in the m data item chain samples is used as the target data item chain sample; the classified annotation data sample comprises m classified annotation data item group chains corresponding to the m data item group chain samples, and one data item group chain sample corresponds to one classified annotation data item group chain; the m classification annotation data item group chains comprise target classification annotation data item group chains, and the annotation data features are obtained by determining group chain monitoring state identification features obtained when each classification annotation data item group chain in the m classification annotation data item group chains is used as the target classification annotation data item group chain;

The step of transmitting the feature extraction array and the annotation data feature to the state recognition network of the initial monitoring state recognition model, and recognizing and outputting the sample loss corresponding to the training template through the state recognition network comprises the following steps:

in the feature extraction array, taking a group chain feature extraction array of the target data item group chain sample as a target group chain feature extraction array, and taking a group chain monitoring state identification feature of the target classification annotation data item group chain as a target group chain monitoring state identification feature in the annotation data feature;

transmitting the target group chain feature extraction array and the target group chain monitoring state identification feature to the state identification network of the initial monitoring state identification model, and identifying and outputting group chain sample loss corresponding to the target data item group chain sample through the state identification network;

if each data item group chain sample in the m data item group chain samples is used as the target data item group chain sample, and each classification annotation data item group chain in the m classification annotation data item group chains is used as the target classification annotation data item group chain, obtaining group chain sample loss corresponding to each data item group chain sample;

And determining the sample loss corresponding to the training template based on the group chain sample loss corresponding to each data item group chain sample.

7. The method of claim 6, wherein the target set of chain feature refinement arrays comprises a first set of chain feature refinement arrays, a second set of chain feature refinement arrays, and a third set of chain feature refinement arrays; the first set of chain feature extraction arrays are set of chain feature extraction arrays of target data item set chain samples in the original building monitoring data samples in the training template, the second set of chain feature extraction arrays are set of chain feature extraction arrays of target data item set chain samples in the integrated monitoring data samples in the training template, and the third set of chain feature extraction arrays are set of chain feature extraction arrays of target data item set chain samples in the disturbance data samples in the training template; the target set of chain monitoring state identification features comprise a first set of chain monitoring state identification features corresponding to the first set of chain feature extraction arrays, a second set of chain monitoring state identification features corresponding to the second set of chain feature extraction arrays, and a third set of chain monitoring state identification features corresponding to the third set of chain feature extraction arrays; the first set of chain monitoring state identification features are set chain monitoring state identification features of a set chain of classification annotation data items corresponding to a set chain of target data items in the original building monitoring data sample, the second set of chain monitoring state identification features are set chain monitoring state identification features of a set chain of classification annotation data items corresponding to a set chain of target data items in the integrated monitoring data sample, and the third set of chain monitoring state identification features are set chain monitoring state identification features of a set chain of classification annotation data items corresponding to a set chain of target data items in the disturbance data sample; and identifying and outputting the group chain sample loss corresponding to the target data item group chain sample through the state identification network, wherein the method comprises the following steps of:

The state recognition network is used for extracting array recognition according to the first set of chain features to output the set chain monitoring state recognition features of the target data item set chain samples in the original building monitoring data samples, and determining the set chain sample loss corresponding to the target data item set chain samples in the original building monitoring data samples through the set chain monitoring state recognition features of the target data item set chain samples in the original building monitoring data samples and the first set chain monitoring state recognition features;

extracting array recognition output chain group monitoring state recognition features of target data item chain group samples in the integrated monitoring data samples according to the second group chain feature through the state recognition network, and determining chain group sample loss corresponding to the target data item chain group samples in the integrated monitoring data samples through the chain group monitoring state recognition features of the target data item chain group samples in the integrated monitoring data samples and the second group chain monitoring state recognition features;

and extracting group chain monitoring state identification features of the group chain samples of the target data items in the disturbance data samples based on the third group chain feature extraction array through the state identification network, and determining group chain sample loss corresponding to the group chain samples of the target data items in the disturbance data samples through the group chain monitoring state identification features of the group chain samples of the target data items in the disturbance data samples and the third group chain monitoring state identification features.

8. The method according to claim 1, wherein optimizing the initial monitoring state recognition model based on the sample loss, using the optimized initial monitoring state recognition model as a target monitoring state recognition model for building safety state recognition, comprises:

optimizing the initial monitoring state identification model based on the sample loss to obtain an optimization result;

and if the optimized result indicates that the optimized initial monitoring state identification model reaches a preset convergence evaluation condition, taking the initial monitoring state identification model reaching the preset convergence evaluation condition as the target monitoring state identification model.

9. A computer system comprising a memory and a processor, the memory storing a computer program executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 8 when the program is executed.