CN116486345A - Property service platform management system and method thereof - Google Patents

Abstract

The application relates to the field of property management, and particularly discloses a property service platform management system and a method thereof, which adopt an artificial intelligent monitoring technology based on deep learning to capture high-dimensional implicit characteristic representation of behavior gesture characteristics of power distribution indoor personnel in behavior action monitoring videos of the power distribution indoor personnel collected by cameras, further perform global context semantic understanding based on the high-dimensional implicit characteristic representation of the behavior gesture characteristics to pay attention to the behavior gesture characteristics of the power distribution indoor personnel on time sequence dimension, and perform behavior anomaly analysis of the power distribution indoor personnel based on the change mode characteristics. Thus, when abnormality is detected, a behavior abnormality early warning prompt is generated in the background to prevent and avoid problems in the distribution room.

Description

Property service platform management system and method thereof

Technical Field

The present application relates to the field of property management, and more particularly, to a property service platform management system and method thereof.

Background

When the property service management is carried out, the power supply system needs to be ensured to be normal, the distribution room is taken as a power supply center, which is an important component for maintaining the normal operation of the power supply system, and the normal operation and the safe operation of the power supply system are not only related to the normal operation of the power supply system, but also related to whether the distribution room or community can normally operate. Therefore, stable and safe operation of the distribution room, abnormal occurrence and failure can respond and process in time, and the problem that important attention is required by each power related department is one of the problems.

The most common solution in the prior art is to send out personnel periodically for maintenance and repair replacement to avoid and reduce faults and accidents, but this can only prevent faults in the internal circuit, but cannot early warn faults caused by abnormal behaviors of personnel in the distribution room.

Accordingly, an optimized property service platform management scheme for a power distribution room is desired.

Disclosure of Invention

The present application has been made in order to solve the above technical problems. The embodiment of the application provides a property service platform management system and a method thereof, which adopt an artificial intelligent monitoring technology based on deep learning to capture high-dimensional implicit characteristic representation of behavior action gesture characteristics of power distribution indoor personnel in behavior action monitoring videos of the power distribution indoor personnel collected by cameras, further perform global context semantic understanding based on the high-dimensional implicit characteristic representation of the behavior action gesture characteristics to pay attention to the behavior action gesture characteristics of the power distribution indoor personnel in time sequence dimension, and further perform behavior anomaly analysis of the power distribution indoor personnel based on the change mode characteristics. Thus, when abnormality is detected, a behavior abnormality early warning prompt is generated in the background to prevent and avoid problems in the distribution room.

Accordingly, according to one aspect of the present application, there is provided a property service platform management system, comprising: the monitoring data acquisition module is used for acquiring behavior and action monitoring videos of personnel in the power distribution room acquired by the camera; the sampling module is used for extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; the behavior feature extraction module is used for enabling the behavior action monitoring key frames to respectively pass through a convolutional neural network model serving as a filter so as to obtain a plurality of behavior action feature vectors; the behavior understanding module is used for enabling the behavior motion feature vectors to pass through a motion semantic understanding model based on a converter to obtain motion understanding feature vectors; the behavior monitoring module is used for enabling the action understanding feature vector to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and the management result generation module is used for generating a management result based on the classification result, wherein the management result is used for indicating whether the abnormal behavior early warning prompt is generated or not.

In the above property service platform management system, the sampling module is configured to extract a plurality of behavior action monitoring key frames from the behavior action monitoring video at a predetermined sampling frequency.

In the above property service platform management system, the behavior feature extraction module is further configured to: each layer using the convolutional neural network model is performed in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the behavior action characteristic vector, and the input of the first layer of the convolutional neural network model is the behavior action monitoring key frame.

In the property service platform management system, the convolutional neural network model serving as the filter is a pyramid network.

In the above property service platform management system, the behavior understanding module includes: a time sequence context understanding unit, configured to input the plurality of behavior motion feature vectors into the motion semantic understanding model based on the converter to obtain a plurality of context behavior motion feature vectors; the aggregation degree optimization unit is used for carrying out inter-vector feature aggregation degree optimization on the context behavior feature vectors so as to obtain a plurality of optimized context behavior feature vectors; and the cascading unit is used for cascading the plurality of optimized context behavior feature vectors to obtain the action understanding feature vector.

In the above property service platform management system, the time sequence context understanding unit is further configured to: arranging the plurality of behavior action feature vectors into an input vector; respectively converting the input vector into a query vector and a key vector through a learning embedding matrix; calculating the product between the query vector and the transpose vector of the key vector to obtain a self-attention correlation matrix; carrying out standardization processing on the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function to activate so as to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each of the plurality of behavior feature vectors as a value vector to obtain the plurality of contextual behavior feature vectors.

In the above property service platform management system, the aggregation degree optimization unit is further configured to: performing inter-vector feature aggregation optimization on the context behavior feature vectors by using the following formula to obtain a plurality of optimized context behavior feature vectors; wherein, the formula is: Wherein->Is each contextual behavior feature vector of the plurality of contextual behavior feature vectors,/v>Is the contextual behavior feature vector of each of the contextual behavior feature vectors +.>Distance between, i.e.)>Less than a predetermined threshold, i.e.)>Contextual behavioral characteristic vector of +.>For weighting superparameters, < >>An exponential operation representing a vector representing a calculation of a natural exponential function value raised to a power by a characteristic value of each position in the vector, ">Representing difference by position +.>And representing each optimized contextual behavior feature vector of the plurality of optimized contextual behavior feature vectors.

In the above property service platform management system, the behavior monitoring module includes: the full-connection coding unit is used for carrying out full-connection coding on the action understanding feature vector by using a full-connection layer of the classifier so as to obtain a coding classification feature vector; and the classification result generating unit is used for inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

According to another aspect of the present application, there is also provided a property service platform management method, including: acquiring a behavior and action monitoring video of personnel in a power distribution room, which is acquired by a camera; extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; the behavior action monitoring key frames are respectively passed through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors; passing the plurality of behavioral action feature vectors through a converter-based action semantic understanding model to obtain action understanding feature vectors; the action understanding feature vector passes through a classifier to obtain a classification result, and the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and generating a management result based on the classification result, wherein the management result is used for indicating whether the behavior action abnormality early warning prompt is generated.

In the above property service platform management method, the extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video includes: and extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video at a preset sampling frequency.

In the above property service platform management method, the step of obtaining a plurality of behavior action feature vectors by passing the plurality of behavior action monitoring key frames through a convolutional neural network model as a filter, respectively, includes: each layer using the convolutional neural network model is performed in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the behavior action characteristic vector, and the input of the first layer of the convolutional neural network model is the behavior action monitoring key frame.

In the property service platform management method, the convolutional neural network model serving as the filter is a pyramid network.

In the above property service platform management method, the step of obtaining the action understanding feature vector by passing the plurality of action feature vectors through an action semantic understanding model based on a converter includes: inputting the plurality of behavioral action feature vectors into the converter-based action semantic understanding model to obtain a plurality of contextual behavioral action feature vectors; performing inter-vector feature aggregation optimization on the context behavior feature vectors to obtain a plurality of optimized context behavior feature vectors; and cascading the plurality of optimized context behavior feature vectors to obtain the action understanding feature vector.

In the above property service platform management method, the inputting the plurality of behavior motion feature vectors into the converter-based motion semantic understanding model to obtain a plurality of contextual behavior motion feature vectors includes: arranging the plurality of behavior action feature vectors into an input vector; respectively converting the input vector into a query vector and a key vector through a learning embedding matrix; calculating the product between the query vector and the transpose vector of the key vector to obtain a self-attention correlation matrix; carrying out standardization processing on the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function to activate so as to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each of the plurality of behavior feature vectors as a value vector to obtain the plurality of contextual behavior feature vectors.

In the above property service platform management method, the performing inter-vector feature aggregation optimization on the plurality of context behavior feature vectors to obtain a plurality of optimized context behavior feature vectors includes: performing inter-vector feature aggregation optimization on the context behavior feature vectors by using the following formula to obtain a plurality of optimized context behavior feature vectors; wherein, the formula is:wherein->Is each contextual behavior feature vector of the plurality of contextual behavior feature vectors,/v>Is the contextual behavior feature vector of each of the contextual behavior feature vectors +.>Distance between, i.e.)>Less than a predetermined threshold, i.e.)>Contextual behavioral characteristic vector of +.>For weighting superparameters, < >>An exponential operation representing a vector representing a calculation of a natural exponential function value raised to a power by a characteristic value of each position in the vector, ">Representing difference by position +.>And representing each optimized contextual behavior feature vector of the plurality of optimized contextual behavior feature vectors.

In the above property service platform management method, the classifying the action understanding feature vector by a classifier to obtain a classification result, where the classification result is used to indicate whether the behavior of personnel in a power distribution room is abnormal, and the method includes: performing full-connection coding on the action understanding feature vector by using a full-connection layer of the classifier to obtain a coding classification feature vector; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform the property service platform management method as described above.

According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform a property service platform management method as described above.

Compared with the prior art, the property service platform management system and the method thereof provided by the application adopt an artificial intelligent monitoring technology based on deep learning to capture high-dimensional implicit characteristic representation of the behavior action gesture characteristics of the power distribution indoor personnel in the behavior action monitoring video of the power distribution indoor personnel collected by the camera, further perform global context semantic understanding based on the high-dimensional implicit characteristic representation of the behavior action gesture characteristics to pay attention to the behavior action gesture characteristics of the power distribution indoor personnel in time sequence dimension, and perform behavior anomaly analysis of the power distribution indoor personnel based on the change mode characteristics. Thus, when abnormality is detected, a behavior abnormality early warning prompt is generated in the background to prevent and avoid problems in the distribution room.

Drawings

The foregoing and other objects, features and advantages of the present application will become more apparent from the following more particular description of embodiments of the present application, as illustrated in the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.

Fig. 1 is a block diagram of a property service platform management system according to an embodiment of the present application.

Fig. 2 is a schematic architecture diagram of a property service platform management system according to an embodiment of the present application.

Fig. 3 is a block diagram of a behavior understanding module in a property service platform management system according to an embodiment of the present application.

Fig. 4 is a flowchart of a property service platform management method according to an embodiment of the present application.

Fig. 5 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

Summary of the application: as described above, the past property can only prevent the occurrence of faults and accidents in the internal circuit by regularly dispatching personnel to perform maintenance and repair replacement and avoiding and reducing the occurrence of faults and accidents, but cannot make timely feedback and response to the situation of faults and accidents caused by external personnel, and an optimized property service platform management scheme for a power distribution room is expected.

In recent years, with the rise of unattended operation, cameras are installed in a power distribution room to perform video monitoring. Accordingly, in the technical scheme of the application, the behavior action monitoring video of personnel in the distribution room can be collected based on the cameras deployed in the distribution room, and the behavior abnormality analysis is performed based on the collected behavior action monitoring video, so that when the behavior action abnormality is detected, a behavior action abnormality early warning prompt can be generated in the background to prevent and avoid the problem of the distribution room. The abnormal behavior detection of the behavior action monitoring video can be realized by artificial intelligence technology based on deep learning and a deep neural network.

Specifically, in the technical scheme of the application, behavior and action monitoring videos of personnel in a power distribution room collected by a camera are firstly obtained. Considering that the behavior movement monitoring video has partial image frames with high similarity and even repeated image frames in a plurality of continuous image frames in a time sequence dimension, if the behavior movement monitoring video is directly used for detecting the behavior abnormality, the behavior movement monitoring video has high information redundancy, so that the detection accuracy is reduced and the detection data processing amount is increased. Based on this, in the technical solution of the present application, before performing behavior anomaly detection, a plurality of behavior action monitoring key frames are extracted from the behavior action monitoring video. That is, the key frame sampling processing is performed on the behavior action monitoring video to perform data distribution sparsification. In one specific example of the present application, the plurality of behavioral monitoring key frames are extracted from the behavioral monitoring video at a predetermined sampling frequency.

And then, the behavior action monitoring key frames are respectively passed through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors. That is, a convolutional neural network model having excellent performance in the field of image feature extraction is used as an image feature extractor to capture a high-dimensional implicit feature representation, i.e., the plurality of behavioral motion feature vectors, of the plurality of behavioral motion monitoring key frames for representing behavioral motion pose features of personnel in the power distribution room. In a specific example of the application, the convolutional neural network model serving as the filter is a pyramid network, and the pyramid network can retain high-dimensional abstract features for representing behavior essence and low-dimensional display features for representing image edges, shapes, textures and the like when feature extraction is performed, so that the richness and accuracy of expression of behavior gesture features are improved.

Further, the plurality of behavior motion feature vectors are passed through a transducer-based motion semantic understanding model to obtain motion understanding feature vectors. That is, the plurality of behavior action feature vectors (i.e., the time series distribution of the behavior action gesture features) are subjected to global-based context semantic understanding by using the converter concept to obtain action understanding feature vectors for representing the change pattern features of the behavior action gesture features of the personnel in the power distribution room in the time series dimension, i.e., the dynamic understanding of the behavior actions of the personnel in the power distribution room.

Specifically, in the encoding process of the motion semantic understanding model based on the converter, the context semantic encoding based on the full-time space of the plurality of behavior motion feature vectors is performed on the behavior motion feature vectors by using a self-attention mechanism so as to obtain a plurality of context behavior motion feature vectors. Further, the plurality of contextual behavior motion feature vectors are concatenated to obtain the motion understanding feature vector.

In particular, in the technical solution of the present application, when the plurality of behavior motion feature vectors are obtained by passing through a motion semantic understanding model based on a converter, the plurality of behavior motion feature vectors are directly cascaded through a plurality of context behavior feature vectors obtained by a context encoder based on the converter to obtain the motion understanding feature vector. Here, although the context encoder based on the converter performs context encoding on the feature semantic layer on the plurality of behavior feature vectors, the obtained plurality of context behavior feature vectors still have a problem of poor explicit association, so that the feature aggregation degree is affected, and the accuracy of the classification result of the action understanding feature vector through the classifier is reduced.

Therefore, if each of the plurality of contextual behavior feature vectors is regarded as a contextual semantic feature representation of a temporal node that is a single action monitor frame, and the temporal semantic topological association of the context codes with the converter between the nodes, the class probability feature aggregation degree between the nodes can be improved based on the topological structure of the node as a whole, specifically, the inter-node class probability matching feature vector of each contextual behavior feature vector is calculated as:，/>is per-context behavioral action feature vector, < > is->Is +.>The distance between, i.e.; j->Less than a predetermined threshold (e.g., threshold is denoted +.>) Contextual behavioral characteristic vector of +.>Is a weighted superparameter.

That is, if a predetermined contextual behavior feature vector is to be usedAs nodes of the topology, the corresponding contextual behavior feature vector +.>Can be considered as being inside the topology with the node +.>Connected nodes, i.e.)>Representing node->And node->With edges therebetween. In this way, the degree of interaction between the nodes in the topological structure and the adjacent nodes under the class probability can be determined by calculating the class probability matching feature vector between the nodes The action understanding feature vector is obtained by replacing the contextual behavior action feature vector cascade, so that the class probability feature aggregation degree among all nodes in a topological structure formed by multiple nodes can be improved, which is equivalent to applying an attention mechanism to node features in the feature aggregation dimension based on internal feature interaction, thereby improving the feature aggregation degree of the action understanding feature vector and improving the accuracy of the classification result of the action understanding feature vector through a classifier.

Further, the action understanding feature vector is passed through a classifier to obtain a classification result, and the classification result is used for representing whether the behavior of personnel in a power distribution room is abnormal or not. That is, the classifier is used to determine a class probability tag to which the action understanding feature vector belongs, the class probability tag including an abnormality in the behavior of the person in the power distribution room (first tag) and an abnormality in the behavior of the person in the power distribution room (second tag). Accordingly, after the classification result is obtained, a management result can be generated on the background server based on the classification result, wherein the management result is used for indicating whether the abnormal behavior action early warning prompt is generated or not. In the actual property service management, after the abnormal behavior early warning prompt is received, corresponding personnel can be assigned to go to the distribution room for processing so as to ensure the safety of the distribution room.

Based on this, the application provides a property service platform management system, which includes: the monitoring data acquisition module is used for acquiring behavior and action monitoring videos of personnel in the power distribution room acquired by the camera; the sampling module is used for extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; the behavior feature extraction module is used for enabling the behavior action monitoring key frames to respectively pass through a convolutional neural network model serving as a filter so as to obtain a plurality of behavior action feature vectors; the behavior understanding module is used for enabling the behavior motion feature vectors to pass through a motion semantic understanding model based on a converter to obtain motion understanding feature vectors; the behavior monitoring module is used for enabling the action understanding feature vector to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and the management result generation module is used for generating a management result based on the classification result, wherein the management result is used for indicating whether the behavior action abnormality early warning prompt is generated or not.

Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.

Exemplary system fig. 1 is a block diagram of a property service platform management system according to an embodiment of the present application. As shown in fig. 1, a property service platform management system 100 according to an embodiment of the present application includes: the monitoring data acquisition module 110 is used for acquiring behavior and action monitoring videos of personnel in the power distribution room acquired by the camera; a sampling module 120, configured to extract a plurality of behavioral monitoring key frames from the behavioral monitoring video; the behavior feature extraction module 130 is configured to pass the plurality of behavior action monitoring key frames through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors; a behavior understanding module 140, configured to pass the plurality of behavior motion feature vectors through a motion semantic understanding model based on a converter to obtain a motion understanding feature vector; the behavior monitoring module 150 is configured to pass the action understanding feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the behavior of the personnel in the power distribution room is abnormal; and a management result generating module 160, configured to generate a management result based on the classification result, where the management result is used to indicate whether to generate an abnormal behavior action early warning prompt.

Fig. 2 is a schematic architecture diagram of a property service platform management system according to an embodiment of the present application. As shown in fig. 2, firstly, acquiring a behavior and action monitoring video of personnel in a power distribution room, which is acquired by a camera; then, extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; then, the behavior action monitoring key frames are respectively passed through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors; then, the action feature vectors are passed through a motion semantic understanding model based on a converter to obtain action understanding feature vectors; then, the action understanding feature vector passes through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the behavior of personnel in a power distribution room is abnormal or not; and finally, based on the classification result, generating a management result, wherein the management result is used for indicating whether the behavior action abnormality early warning prompt is generated.

As described above, the past property can only prevent the occurrence of faults and accidents in the internal circuit by regularly dispatching personnel to perform maintenance and repair replacement and avoiding and reducing the occurrence of faults and accidents, but cannot make timely feedback and response to the situation of faults and accidents caused by external personnel, and an optimized property service platform management scheme for a power distribution room is expected.

In recent years, with the rise of unattended operation, cameras are installed in a power distribution room to perform video monitoring. Accordingly, in the technical scheme of the application, the behavior action monitoring video of personnel in the distribution room can be collected based on the cameras deployed in the distribution room, and the behavior abnormality analysis is performed based on the collected behavior action monitoring video, so that when the behavior action abnormality is detected, a behavior action abnormality early warning prompt can be generated in the background to prevent and avoid the problem of the distribution room. The abnormal behavior detection of the behavior action monitoring video can be realized by artificial intelligence technology based on deep learning and a deep neural network.

In the above-mentioned property service platform management system 100, the monitoring data collection module 110 and the sampling module 120 are configured to obtain a behavior monitoring video of a person in the distribution room collected by a camera, and extract a plurality of behavior monitoring key frames from the behavior monitoring video. That is, the behavior and action monitoring video of the personnel in the distribution room is collected by the camera disposed in the distribution room. Further, considering that the behavior movement monitoring video has a part of image frames with high similarity and even repeated image frames in a plurality of continuous image frames in a time sequence dimension, if the behavior movement monitoring video is directly used for detecting the behavior abnormality, the behavior movement monitoring video has high information redundancy, so that the detection accuracy is reduced and the processing amount of detection data is increased.

Based on this, in the technical solution of the present application, before performing behavior anomaly detection, a plurality of behavior action monitoring key frames are extracted from the behavior action monitoring video. That is, the key frame sampling processing is performed on the behavior action monitoring video to perform data distribution sparsification. In one specific example of the present application, the plurality of behavioral monitoring key frames are extracted from the behavioral monitoring video at a predetermined sampling frequency.

In the property service platform management system 100, the behavior feature extraction module 130 is configured to obtain a plurality of behavior feature vectors by passing the plurality of behavior action monitoring key frames through a convolutional neural network model as a filter, respectively. That is, a convolutional neural network model having excellent performance in the field of image feature extraction is used as an image feature extractor to capture a high-dimensional implicit feature representation, i.e., the plurality of behavioral motion feature vectors, of the plurality of behavioral motion monitoring key frames for representing behavioral motion pose features of personnel in the power distribution room. In a specific example of the application, the convolutional neural network model serving as the filter is a pyramid network, and the pyramid network can retain high-dimensional abstract features for representing behavior essence and low-dimensional display features for representing image edges, shapes, textures and the like when feature extraction is performed, so that the richness and accuracy of expression of behavior gesture features are improved.

Specifically, in the embodiment of the present application, the behavioral characteristic extraction module 130 uses each layer of the convolutional neural network model to perform in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the behavior action characteristic vector, and the input of the first layer of the convolutional neural network model is the behavior action monitoring key frame.

In the property service platform management system 100, the behavior understanding module 140 is configured to obtain the behavior understanding feature vectors by using a behavior semantic understanding model based on a converter. That is, the plurality of behavior action feature vectors (i.e., the time series distribution of the behavior action gesture features) are subjected to global-based context semantic understanding by using the converter concept to obtain action understanding feature vectors for representing the change pattern features of the behavior action gesture features of the personnel in the power distribution room in the time series dimension, i.e., the dynamic understanding of the behavior actions of the personnel in the power distribution room.

Fig. 3 is a block diagram of a behavior understanding module in a property service platform management system according to an embodiment of the present application. As shown in fig. 3, the behavior understanding module 140 includes: a time sequence context understanding unit 141, configured to input the plurality of behavior motion feature vectors into the motion semantic understanding model based on the converter to obtain a plurality of context behavior motion feature vectors; the aggregation level optimization unit 142 is configured to perform inter-vector feature aggregation level optimization on the plurality of context behavior feature vectors to obtain a plurality of optimized context behavior feature vectors; and a concatenation unit 143, configured to concatenate the plurality of optimized context behavior feature vectors to obtain the action understanding feature vector.

Specifically, in the encoding process of the motion semantic understanding model based on the converter, the context semantic encoding based on the full-time space of the plurality of behavior motion feature vectors is performed on the behavior motion feature vectors by using a self-attention mechanism so as to obtain a plurality of context behavior motion feature vectors.

More specifically, in the embodiment of the present application, the timing context understanding unit 141 is further configured to: arranging the plurality of behavior action feature vectors into an input vector; respectively converting the input vector into a query vector and a key vector through a learning embedding matrix; calculating the product between the query vector and the transpose vector of the key vector to obtain a self-attention correlation matrix; carrying out standardization processing on the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function to activate so as to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each of the plurality of behavior feature vectors as a value vector to obtain the plurality of contextual behavior feature vectors.

In particular, in the technical solution of the present application, when the plurality of behavior motion feature vectors are obtained by passing through a motion semantic understanding model based on a converter, the plurality of behavior motion feature vectors are directly cascaded through a plurality of context behavior feature vectors obtained by a context encoder based on the converter to obtain the motion understanding feature vector. Here, although the context encoder based on the converter performs context encoding on the feature semantic layer on the plurality of behavior feature vectors, the obtained plurality of context behavior feature vectors still have a problem of poor explicit association, so that the feature aggregation degree is affected, and the accuracy of the classification result of the action understanding feature vector through the classifier is reduced.

Therefore, if each of the plurality of contextual behavior feature vectors is regarded as a contextual semantic feature representation of a temporal node that is a single action monitor frame, and the temporal semantic topological association of the context codes with the converter between the nodes, the class probability feature aggregation degree between the nodes can be improved based on the topological structure of the node as a whole, specifically, the inter-node class probability matching feature vector of each contextual behavior feature vector is calculated as: Wherein->Is each contextual behavior feature vector of the plurality of contextual behavior feature vectors,/v>Is the plurality of contextsOf the contextual behavioral motion feature vectors of the respective plurality of behavioral motion feature vectors, there is +.>The distance between, i.eLess than a predetermined threshold, i.e.)>Contextual behavioral characteristic vector of +.>For weighting superparameters, < >>An exponential operation representing a vector representing a calculation of a natural exponential function value raised to a power by a characteristic value of each position in the vector, ">Representing difference by position +.>And representing each optimized contextual behavior feature vector of the plurality of optimized contextual behavior feature vectors.

That is, if a predetermined contextual behavior feature vector is to be usedAs nodes of the topology, the corresponding contextual behavior feature vector +.>Can be considered as being inside the topology with the node +.>Connected nodes, i.e.)>Representing node->And node->With edges therebetween. Therefore, the interaction degree between the nodes in the topological structure and the adjacent nodes under the class probability can be determined by calculating the class probability matching feature vectors among the nodes, the action understanding feature vectors are obtained by replacing the contextual behavior action feature vector cascade, the class probability feature aggregation degree among all the nodes in the topological structure formed by multiple nodes can be improved, which is equivalent to applying an attention mechanism to the node features in the feature aggregation dimension based on internal feature interaction, so that the feature aggregation degree of the action understanding feature vectors is improved, and the accuracy of the classification result of the action understanding feature vectors through the classifier is improved.

Furthermore, the plurality of optimized contextual behavior feature vectors are cascaded to obtain the action understanding feature vector.

In the above-mentioned property service platform management system 100, the behavior monitoring module 150 is configured to pass the action understanding feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the behavior of the personnel in the power distribution room is abnormal. That is, the classifier is used to determine a class probability tag to which the action understanding feature vector belongs, the class probability tag including an abnormality in the behavior of the person in the power distribution room (first tag) and an abnormality in the behavior of the person in the power distribution room (second tag).

Specifically, in the embodiment of the present application, the behavior monitoring module 150 includes: the full-connection coding unit is used for carrying out full-connection coding on the action understanding feature vector by using a full-connection layer of the classifier so as to obtain a coding classification feature vector; and the classification result generating unit is used for inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

In the above-mentioned property service platform management system 100, the management result generating module 160 is configured to generate a management result based on the classification result, where the management result is used to indicate whether to generate a behavioral action abnormality early warning prompt. In the actual property service management, after the abnormal behavior early warning prompt is received, corresponding personnel can be assigned to go to the distribution room for processing so as to ensure the safety of the distribution room.

In summary, the property service platform management system 100 according to the embodiment of the present application is illustrated, which adopts an artificial intelligent monitoring technology based on deep learning to capture a high-dimensional implicit characteristic representation of a behavior gesture feature of a power distribution indoor person in a behavior action monitoring video of the power distribution indoor person collected by a camera, and further performs global context semantic understanding based on the high-dimensional implicit characteristic representation of the behavior gesture feature to pay attention to the behavior gesture feature of the power distribution indoor person in a time sequence dimension, and then performs behavior anomaly analysis of the power distribution indoor person based on the change mode feature. Thus, when abnormality is detected, a behavior abnormality early warning prompt is generated in the background to prevent and avoid problems in the distribution room.

As described above, the property service platform management system 100 according to the embodiment of the present application may be implemented in various terminal devices, for example, a server for property service platform management, etc. In one example, the property service platform management system 100 according to embodiments of the present application may be integrated into the terminal device as a software module and/or hardware module. For example, the property service platform management system 100 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the property service platform management system 100 can also be one of a plurality of hardware modules of the terminal device.

Alternatively, in another example, the property service platform management system 100 and the terminal device may be separate devices, and the property service platform management system 100 may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information in a contracted data format.

Exemplary method fig. 4 is a flowchart of a property service platform management method according to an embodiment of the present application. As shown in fig. 4, a property service platform management method according to an embodiment of the present application includes: s110, acquiring behavior and action monitoring videos of personnel in a power distribution room, wherein the behavior and action monitoring videos are acquired by a camera; s120, extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; s130, enabling the behavior action monitoring key frames to respectively pass through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors; s140, the action feature vectors are processed through an action semantic understanding model based on a converter to obtain action understanding feature vectors; s150, the action understanding feature vector passes through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and S160, based on the classification result, generating a management result, wherein the management result is used for indicating whether the abnormal behavior early warning prompt is generated.

In one example, in the above property service platform management method, the extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video includes: and extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video at a preset sampling frequency.

In one example, in the above property service platform management method, the passing the plurality of behavior action monitoring key frames through a convolutional neural network model as a filter to obtain a plurality of behavior action feature vectors includes: each layer using the convolutional neural network model is performed in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the behavior action characteristic vector, and the input of the first layer of the convolutional neural network model is the behavior action monitoring key frame.

In one example, in the property service platform management method, the convolutional neural network model used as the filter is a pyramid network.

In one example, in the property service platform management method, the step of passing the plurality of behavior action feature vectors through a converter-based action semantic understanding model to obtain action understanding feature vectors includes: inputting the plurality of behavioral action feature vectors into the converter-based action semantic understanding model to obtain a plurality of contextual behavioral action feature vectors; performing inter-vector feature aggregation optimization on the context behavior feature vectors to obtain a plurality of optimized context behavior feature vectors; and cascading the plurality of optimized context behavior feature vectors to obtain the action understanding feature vector.

In one example, in the above property service platform management method, the inputting the plurality of behavior motion feature vectors into the converter-based motion semantic understanding model to obtain a plurality of contextual behavior motion feature vectors includes: arranging the plurality of behavior action feature vectors into an input vector; respectively converting the input vector into a query vector and a key vector through a learning embedding matrix; calculating the product between the query vector and the transpose vector of the key vector to obtain a self-attention correlation matrix; carrying out standardization processing on the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function to activate so as to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each of the plurality of behavior feature vectors as a value vector to obtain the plurality of contextual behavior feature vectors.

In an example, in the above property service platform management method, the performing inter-vector feature aggregation optimization on the plurality of context behavior feature vectors to obtain a plurality of optimized context behavior feature vectors includes: vector-to-vector feature aggregation optimization of the plurality of contextual behavioral feature vectors with the following formulaObtaining a plurality of optimized context behavior characteristic vectors; wherein, the formula is:wherein->Is each contextual behavior feature vector of the plurality of contextual behavior feature vectors,/v>Is the contextual behavior feature vector of each of the contextual behavior feature vectors +.>The distance between, i.eLess than a predetermined threshold, i.e.)>Contextual behavioral characteristic vector of +.>For weighting superparameters, < >>An exponential operation representing a vector representing a calculation of a natural exponential function value raised to a power by a characteristic value of each position in the vector, ">Representing difference by position +.>And representing each optimized contextual behavior feature vector of the plurality of optimized contextual behavior feature vectors.

In one example, in the above property service platform management method, the step of passing the action understanding feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the behavior of a person in a power distribution room is abnormal, includes: performing full-connection coding on the action understanding feature vector by using a full-connection layer of the classifier to obtain a coding classification feature vector; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

In summary, the property service platform management method of the embodiment of the application is clarified, which adopts an artificial intelligent monitoring technology based on deep learning to capture a high-dimensional implicit characteristic representation of the behavior action gesture characteristics of the personnel in the power distribution room in a behavior action monitoring video of the personnel in the power distribution room collected by a camera, further carries out global context semantic understanding based on the high-dimensional implicit characteristic representation of the behavior action gesture characteristics to pay attention to the behavior action gesture characteristics of the personnel in the power distribution room in a time sequence dimension, and then carries out behavior anomaly analysis of the personnel in the power distribution room based on the change mode characteristics. Thus, when abnormality is detected, a behavior abnormality early warning prompt is generated in the background to prevent and avoid problems in the distribution room.

Exemplary electronic device an electronic device according to an embodiment of the present application is described below with reference to fig. 5. Fig. 5 is a block diagram of an electronic device according to an embodiment of the present application. As shown in fig. 5, the electronic device 10 includes one or more processors 11 and a memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that can be executed by the processor 11 to implement the functions in the property service platform management methods of the various embodiments of the present application described above and/or other desired functions. Various content such as a behavior action monitoring video of a person in the distribution room may also be stored in the computer readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).

The input means 13 may comprise, for example, a keyboard, a mouse, etc.

The output device 14 may output various information including management results and the like to the outside. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.

Of course, only some of the components of the electronic device 10 that are relevant to the present application are shown in fig. 5 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

In addition to the methods and apparatus described above, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in the functions of the property service platform management method according to the various embodiments of the present application described in the "exemplary methods" section of the present specification.

The computer program product may write program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions, which when executed by a processor, cause the processor to perform steps in the functions of the property service platform management method according to various embodiments of the present application described in the above "exemplary method" section of the present specification.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not intended to be limited to the details disclosed herein as such.

The block diagrams of the devices, apparatuses, devices, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.

It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent to the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims (10)

1. A property service platform management system, comprising: the monitoring data acquisition module is used for acquiring behavior and action monitoring videos of personnel in the power distribution room acquired by the camera; the sampling module is used for extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; the behavior feature extraction module is used for enabling the behavior action monitoring key frames to respectively pass through a convolutional neural network model serving as a filter so as to obtain a plurality of behavior action feature vectors; the behavior understanding module is used for enabling the behavior motion feature vectors to pass through a motion semantic understanding model based on a converter to obtain motion understanding feature vectors; the behavior monitoring module is used for enabling the action understanding feature vector to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and the management result generation module is used for generating a management result based on the classification result, wherein the management result is used for indicating whether the abnormal behavior early warning prompt is generated or not.

2. The property service platform management system of claim 1 wherein the sampling module is configured to extract a plurality of behavior action monitoring keyframes from the behavior action monitoring video at a predetermined sampling frequency.

3. The property service platform management system of claim 2, wherein the behavioral characteristic extraction module is further configured to: each layer using the convolutional neural network model is performed in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the behavior action characteristic vector, and the input of the first layer of the convolutional neural network model is the behavior action monitoring key frame.

4. The property service platform management system of claim 3 wherein the convolutional neural network model as a filter is a pyramid network.

5. The property service platform management system of claim 4 wherein the behavior understanding module comprises: a time sequence context understanding unit, configured to input the plurality of behavior motion feature vectors into the motion semantic understanding model based on the converter to obtain a plurality of context behavior motion feature vectors; the aggregation degree optimization unit is used for carrying out inter-vector feature aggregation degree optimization on the context behavior feature vectors so as to obtain a plurality of optimized context behavior feature vectors; and the cascading unit is used for cascading the plurality of optimized context behavior characteristic vectors to obtain the action understanding characteristic vector.

6. The property service platform management system of claim 5, wherein the time series context understanding unit is further configured to: arranging the plurality of behavior action feature vectors into an input vector; respectively converting the input vector into a query vector and a key vector through a learning embedding matrix; calculating the product between the query vector and the transpose vector of the key vector to obtain a self-attention correlation matrix; carrying out standardization processing on the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function to activate so as to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each of the plurality of behavior feature vectors as a value vector to obtain the plurality of contextual behavior feature vectors.

7. The property service platform management system of claim 6, wherein the aggregation level optimization unit is further configured to: performing inter-vector feature aggregation optimization on the context behavior feature vectors by using the following formula to obtain a plurality of optimized context behavior feature vectors; wherein, the formula is: Wherein->Is the multiple contextual behavior actionEach contextual behavior feature vector of the feature vectors +.>Is the contextual behavior feature vector of each of the contextual behavior feature vectors +.>The distance between, i.eLess than a predetermined threshold, i.e.)>Contextual behavioral characteristic vector of +.>For weighting superparameters, < >>An exponential operation representing a vector representing a calculation of a natural exponential function value raised to a power by a characteristic value of each position in the vector, ">Representing difference by position +.>And representing each optimized contextual behavior feature vector of the plurality of optimized contextual behavior feature vectors.

8. The property service platform management system of claim 7 wherein the behavior monitoring module comprises: the full-connection coding unit is used for carrying out full-connection coding on the action understanding feature vector by using a full-connection layer of the classifier so as to obtain a coding classification feature vector; and the classification result generating unit is used for inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

9. The utility service platform management method is characterized by comprising the following steps: acquiring a behavior and action monitoring video of personnel in a power distribution room, which is acquired by a camera; extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video; the behavior action monitoring key frames are respectively passed through a convolutional neural network model serving as a filter to obtain a plurality of behavior action feature vectors; passing the plurality of behavioral action feature vectors through a converter-based action semantic understanding model to obtain action understanding feature vectors; the action understanding feature vector passes through a classifier to obtain a classification result, and the classification result is used for indicating whether the behaviors of personnel in a power distribution room are abnormal or not; and generating a management result based on the classification result, wherein the management result is used for indicating whether the behavior action abnormality early warning prompt is generated.

10. The property service platform management method of claim 9, wherein the extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video comprises: and extracting a plurality of behavior action monitoring key frames from the behavior action monitoring video at a preset sampling frequency.