CN118411811B

CN118411811B - ChatGPT-based global perception early warning method, chatGPT-based global perception early warning system, medium and electronic equipment

Info

Publication number: CN118411811B
Application number: CN202410869192.3A
Authority: CN
Inventors: 黄怿; 黄辰贇
Original assignee: Zhejiang Fengfeng Intelligent Technology Co ltd
Current assignee: Zhejiang Fengfeng Intelligent Technology Co ltd
Priority date: 2024-07-01
Filing date: 2024-07-01
Publication date: 2024-09-06
Anticipated expiration: 2044-07-01
Also published as: CN118411811A

Abstract

The application discloses a ChatGPT-based global perception early warning method, a ChatGPT-based global perception early warning system, a medium and electronic equipment, wherein the method comprises the following steps: collecting and preprocessing environment sensing data sent by monitoring devices which are pre-arranged in a target area within a preset time period to obtain heterogeneous data to be analyzed; inputting the heterogeneous data to be analyzed into a preset global perception early warning component, and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning component comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the method comprises the steps of presetting ChatGPT models for analyzing multi-dimensional recognition results and generating event keyword sequences of all dimensions; and when the analysis result indicates that the dangerous event exists in the target area, early warning is carried out. Therefore, by adopting the embodiment of the application, the early warning efficiency can be improved, the accuracy of dangerous event prediction can be improved, and key information can not be missed or a response can be timely made.

Description

ChatGPT-based global perception early warning method, chatGPT-based global perception early warning system, medium and electronic equipment

Technical Field

The application relates to the technical field of machine learning, in particular to a ChatGPT-based global perception early warning method, a ChatGPT-based global perception early warning system, a medium and electronic equipment.

Background

Along with the rapid development of the economic society and the continuous promotion of the urban process, the places where people gather in the city and the large-scale activities held are more and more, including large-scale paradises, sporting events, exhibition centers, music festival and the like, people in the areas are relatively dense, and complicated site layout and diversified potential safety hazards exist, so that the dangerous event information is acquired at the first time, and the method has important significance for relieving the dangerous event consequences and avoiding or reducing the casualties.

In the related art, when a complex large area is subjected to safety early warning, a video monitoring system arranged in the large area monitors by means of manual monitoring and a video analysis method, and when an emergency occurs, early warning can be sent out. The prior art has the following defects: 1. a large amount of manpower and material resources are consumed by relying on manual monitoring, so that the early warning efficiency is reduced; 2. the video data with single dimension contains less information, and when colorless and odorless harmful gas or distress audio exists, dangerous event judgment cannot be carried out, so that the accuracy of dangerous event prediction is reduced; 3. the current video analysis method often depends on a traditional data processing algorithm, and the existing video analysis method has limited capability in terms of natural language processing, so that the capability of understanding and analyzing information acquired by monitoring equipment is limited, and key information cannot be reacted in time.

Disclosure of Invention

The embodiment of the application provides a global perception early warning method, a global perception early warning system, a global perception early warning medium and electronic equipment based on ChatGPT. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

In a first aspect, an embodiment of the present application provides a global sensing early warning method based on ChatGPT, where the method includes:

collecting and preprocessing environment sensing data sent by monitoring devices which are pre-arranged in a target area within a preset time period to obtain heterogeneous data to be analyzed;

Inputting the heterogeneous data to be analyzed into a preset global perception early warning component, and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the method comprises the steps of presetting ChatGPT models for analyzing multi-dimensional recognition results and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on event keyword sequences of all dimensions;

And under the condition that the analysis result indicates that the dangerous event exists in the target area, generating dangerous event information of the target area, and reporting the dangerous event information to the early warning terminal for early warning.

Optionally, inputting the heterogeneous data to be analyzed into a preset global perception early warning component, and outputting an analysis result corresponding to the heterogeneous data to be analyzed, including:

Inputting the heterogeneous data to be analyzed into a pre-trained global perception model, and outputting a multi-dimensional recognition result corresponding to the heterogeneous data to be analyzed;

Generating event keyword sequences of each dimension corresponding to the multi-dimension recognition result according to the multi-dimension recognition result and a preset ChatGPT model;

And generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of the dimensions.

Optionally, each monitoring device includes a camera, a monitoring device and each sensor, and the multi-dimensional recognition result includes video description information of the camera, audio description information of the monitoring device and sensor description information of the sensor;

generating event keyword sequences of each dimension corresponding to the multi-dimension recognition result according to the multi-dimension recognition result and a preset ChatGPT model, wherein the event keyword sequences comprise:

and sequentially processing the video description information, the audio description information and the sensor description information through a preset ChatGPT model according to a preset prompt text set for guiding ChatGPT models to extract dangerous events, so as to obtain event keyword sequences of all dimensions.

Optionally, generating a preset prompt text set for guiding ChatGPT a model to extract dangerous events according to the following steps:

acquiring historical dangerous event description texts submitted for different preset dangerous events in a preset period from a preset dynamic dangerous event description text library;

Labeling key information of dangerous events existing in each historical dangerous event description text to obtain each labeled historical dangerous event description text;

acquiring dimension categories corresponding to the historical dangerous event description texts of each label;

inputting the description text of each marked historical dangerous event into a preset ChatGPT model;

outputting prompt texts corresponding to the historical dangerous event description texts of each label and used for guiding ChatGPT models to extract dangerous events;

Acquiring all prompt texts under the same dimension type;

Storing mapping relations between the same dimension category and all prompt texts under the same dimension category;

And taking the mapping relation as a preset prompt text set for guiding ChatGPT a model to extract dangerous events.

Optionally, the dimension category includes a dimension category of the camera, a dimension category of the monitoring device, and a dimension category of the sensor;

according to a preset prompt text set for guiding ChatGPT a model to extract dangerous events, video description information, audio description information and sensor description information are sequentially processed through a preset ChatGPT model to obtain event keyword sequences of all dimensions, wherein the event keyword sequences comprise:

Respectively determining a first dimension category corresponding to the video description information, a second dimension category corresponding to the audio description information and a third dimension category corresponding to the sensor description information;

Respectively acquiring a plurality of first prompt texts corresponding to a first dimension category and used for guiding ChatGPT a model to extract a dangerous event, a plurality of second prompt texts corresponding to a second dimension category and used for guiding ChatGPT a model to extract a dangerous event and a plurality of third prompt texts corresponding to a third dimension category and used for guiding ChatGPT a model to extract a dangerous event from the mapping relation;

Inputting the video description information and a plurality of first prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension type of the camera;

inputting the audio description information and a plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the monitoring equipment;

inputting the sensor description information and a plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the sensor;

and taking the keyword sequence corresponding to the dimension type of the camera, the keyword sequence corresponding to the dimension type of the monitoring equipment and the keyword sequence corresponding to the dimension type of the sensor as event keyword sequences of all dimensions.

Optionally, generating the pre-trained global perception model according to the following steps includes:

collecting historical video data from a camera, historical audio data from monitoring equipment and historical sensor data from each sensor in a preset time period;

Labeling text description information of normal and abnormal behaviors, event types and environmental conditions on event frames, historical audio data and historical sensor data in the historical video data respectively to obtain labeling data; the event frame is a frame representing a shot event and comprises a first frame, a last frame and a plurality of key frames, wherein the key frames are obtained when the similarity is larger than a preset threshold value through sequentially calculating the similarity of all subframes of a shot except the first frame and the last frame and the previous frame;

Cleaning, standardizing and enhancing the labeling data to obtain a model training sample;

Creating a global perception model by adopting a neural network;

Inputting a model training sample into a global perception model so as to enable the global perception model to perform self-learning and obtain a model loss value;

when the model loss value reaches the minimum, a pre-trained global perception model is generated.

Optionally, generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of each dimension includes:

Carrying out format standardization on event keyword sequences of all dimensions to obtain first event keyword sequences of all dimensions;

distributing context information comprising a time stamp, a place mark and an event type aiming at a target event keyword sequence of each dimension to obtain a second event keyword sequence of each dimension;

Normalizing the second event keyword sequences of all the dimensions to eliminate the term difference among different data sources and obtain third event keyword sequences of all the dimensions;

counting the association degree between the occurrence frequency of each third event keyword in each third event keyword sequence of each dimension and the known event type;

determining the importance degree of each third event keyword based on the association degree of the frequency of occurrence of each third event keyword and the known event type;

screening keywords with importance degrees larger than a preset threshold according to the importance degrees of the keywords of each third event to obtain a fourth event keyword sequence of each dimension;

identifying and extracting each dimension target keyword of a cross-dimension common theme or logical connection in a fourth event keyword sequence of each dimension by adopting a theme modeling technology; the topic modeling technology is a dirichlet allocation algorithm;

aggregating the target keywords of each dimension of the cross-dimension common subject or logical connection to obtain a coherent information block;

And carrying out dangerous event inference on the coherent information blocks by adopting an inference component of a preset ChatGPT model to obtain an analysis result corresponding to the heterogeneous data to be analyzed.

In a second aspect, an embodiment of the present application provides a global sensing early warning system based on ChatGPT, where the system includes:

the data collection module is used for collecting and preprocessing environment perception data sent by each monitoring device which is pre-arranged in a target area in a preset time period to obtain heterogeneous data to be analyzed;

The data analysis module is used for inputting the heterogeneous data to be analyzed into a preset global perception early warning assembly and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the method comprises the steps of presetting ChatGPT models for analyzing multi-dimensional recognition results and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on event keyword sequences of all dimensions;

and the early warning module is used for generating dangerous event information of the target area and reporting the dangerous event information to the early warning terminal for early warning under the condition that the analysis result indicates that the target area has dangerous events.

In a third aspect, embodiments of the present application provide a computer storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.

In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps described above.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

In the embodiment of the application, on one hand, the preset global perception early-warning component comprises a preset ChatGPT model, and the preset ChatGPT model has outstanding capabilities in terms of natural language understanding, context perception and information fusion, so that the preset ChatGPT model is fused into the preset global perception early-warning component provided by the application, and a preset prompt text set for guiding ChatGPT the model to extract dangerous events is combined, and the prompt text set can guide ChatGPT the model to accurately extract key information, so that the capability of understanding and analyzing information acquired by monitoring equipment can be improved, and the key information can be reacted in time; on the other hand, the heterogeneous data to be analyzed can be automatically analyzed in real time through the preset global perception early warning assembly, so that an analysis result is obtained, dependence on manual monitoring is avoided, and the early warning efficiency is improved; on the other hand, heterogeneous data to be analyzed is obtained based on environment sensing data sent by monitoring devices which are arranged in a target area in advance, the environment sensing data comprise multi-source data of video dimension, audio dimension and sensor dimension, the information quantity of the multi-source data is large, when colorless and odorless harmful gas or distress audio exists, dangerous event judgment can be carried out, and therefore accuracy of dangerous event prediction is improved, meanwhile, the multi-source data can enable a preset global sensing assembly to have global sensing energy, global sensing can comprehensively, accurately, timely and timely acquire and analyze information of the whole situation or situation, and early warning can be carried out by identifying dangerous events in advance.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic flow chart of a global sensing and early warning method based on ChatGPT according to an embodiment of the present application;

FIG. 2 is a schematic diagram of pre-processed context awareness data according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a multi-dimensional recognition result according to an embodiment of the present application;

FIG. 4 is a block diagram of a global sensing pre-warning based on ChatGPT according to an embodiment of the present application;

FIG. 5 is a flowchart of a global perception model training method according to an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a global sensing and early warning system based on ChatGPT according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the application to enable those skilled in the art to practice them.

It should be understood that the described embodiments are merely some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of systems and methods that are consistent with aspects of the application as detailed in the accompanying claims.

In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application will be understood in specific cases by those of ordinary skill in the art. Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

At present, when a complex large area is subjected to safety early warning, a video monitoring system arranged in the large area monitors by means of manual monitoring and video analysis methods, and when an emergency occurs, early warning can be sent out.

The inventor of the application notices that 1, a great deal of manpower and material resources are consumed by relying on manual monitoring, so that the early warning efficiency is reduced; 2. the video data with single dimension contains less information, and when colorless and odorless harmful gas or distress audio exists, dangerous event judgment cannot be carried out, so that the accuracy of dangerous event prediction is reduced; 3. the current video analysis method often depends on a traditional data processing algorithm, and the existing video analysis method has limited capability in terms of natural language processing, so that the capability of understanding and analyzing information acquired by monitoring equipment is limited, and key information cannot be reacted in time.

In order to solve the above problems, the present application provides a global sensing and early warning method, system, medium and electronic device based on ChatGPT, so as to solve the problems in the related technical problems. In the embodiment of the application, on one hand, the preset global perception early-warning component comprises a preset ChatGPT model, and the preset ChatGPT model has outstanding capabilities in terms of natural language understanding, context perception and information fusion, so that the preset ChatGPT model is fused into the preset global perception early-warning component provided by the application, and a preset prompt text set for guiding ChatGPT the model to extract dangerous events is combined, and the prompt text set can guide ChatGPT the model to accurately extract key information, so that the capability of understanding and analyzing information acquired by monitoring equipment can be improved, and the key information can be reacted in time; on the other hand, the heterogeneous data to be analyzed can be automatically analyzed in real time through the preset global perception early warning assembly, so that an analysis result is obtained, dependence on manual monitoring is avoided, and the early warning efficiency is improved; on the other hand, the heterogeneous data to be analyzed is obtained based on environment sensing data sent by monitoring devices which are arranged in a target area in advance, the environment sensing data comprise multi-source data of video dimension, audio dimension and sensor dimension, the multi-source data is large in information quantity, when colorless and odorless harmful gas or distress audio exists, dangerous event judgment can be carried out, and therefore accuracy of dangerous event prediction is improved, meanwhile, the multi-source data can enable a preset global sensing assembly to have global sensing energy, global sensing can acquire and analyze information which is comprehensive, accurate, timely and available in the whole situation or situation, early warning can be carried out by identifying dangerous events in advance, and an exemplary embodiment is adopted for detailed description.

The following will describe the global sensing and early warning method based on ChatGPT according to the embodiment of the present application in detail with reference to fig. 1 to 5. The method may be implemented in dependence on a computer program, and may be run on ChatGPT-based global perception early warning systems based on von neumann systems. The computer program may be integrated in the application or may run as a stand-alone tool class application.

Referring to fig. 1, a flow chart of a global sensing and early warning method based on ChatGPT is provided in an embodiment of the present application. As shown in fig. 1, the method according to the embodiment of the present application may include the following steps:

S101, collecting and preprocessing environment sensing data sent by monitoring devices which are pre-arranged in a target area within a preset time period to obtain heterogeneous data to be analyzed;

the preset time period is a preset time period, for example, may be one minute or several seconds; the target area is a place needing global perception early warning, such as a large-scale garden, a sports event and an exhibition center; the monitoring devices are, for example, cameras, monitoring devices and sensors, such as temperature sensors, humidity sensors, harmful gas sensors, motion detectors and the like; the environment sensing data is information which is continuously transmitted to the server by each monitoring device according to a preset period.

It should be noted that, each monitoring device may automatically collect data according to a preset parameter and a schedule, where the schedule configures a preset period.

In the embodiment of the application, when the environment-aware data is preprocessed, the environment-aware data can be subjected to preliminary inspection, invalid or erroneous data records are removed, then the data is cleaned, for example, noise is removed, time stamps are corrected and the like, and then the cleaned data is converted into a uniform format so as to facilitate subsequent processing and analysis, and finally heterogeneous data to be analyzed can be obtained.

The preprocessed context aware data, i.e. the heterogeneous data to be analyzed, is e.g. a table as shown in fig. 2, which table comprises device IDs, time stamps, types, data values, status. Device ID: a number uniquely identifying each monitoring device; timestamp: specific date and time of data collection; type (2): types of data such as temperature, humidity, video frames, audio, motion detection, pressure, etc.; data value: the actual collected data values may be numerical values, filenames, or state descriptions, depending on the type; status: status of the device or data, such as normal, warning, error, etc. Remarks: additional description of data or status, such as increased people flow or increased background noise.

It should be noted that the tabular data may vary according to the kind of the monitoring device, the type of data, and the specific requirements of the system. In practice, tabular data may include more columns such as geographic location information, device status history, data quality indicators, etc. In addition, tabular data is typically stored in a database for efficient querying and analysis.

S102, inputting the heterogeneous data to be analyzed into a preset global perception early warning assembly, and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the method comprises the steps of presetting ChatGPT models for analyzing multi-dimensional recognition results and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on event keyword sequences of all dimensions;

The pre-trained global perception model is a mathematical model for analyzing heterogeneous data to be analyzed, the mathematical model can be obtained based on machine learning, and a multi-dimensional recognition result can be generated based on the heterogeneous data to be analyzed; the preset ChatGPT model is a pre-trained language model developed by OpenAI that is based on transducer architecture, particularly good at understanding and generating natural language text.

Wherein, the multi-dimensional recognition result comprises descriptive text such as behavior type, event severity, recommended operation and the like, for example remark content in a black box in fig. 3; the event keyword sequences of the dimensions are obtained by extracting keywords of dangerous events based on ChatGPT models.

In the embodiment of the application, the specific process of inputting the heterogeneous data to be analyzed into the preset global perception early warning component and outputting the analysis result corresponding to the heterogeneous data to be analyzed comprises the following steps: inputting the heterogeneous data to be analyzed into a pre-trained global perception model, and outputting a multi-dimensional recognition result corresponding to the heterogeneous data to be analyzed; generating event keyword sequences of each dimension corresponding to the multi-dimension recognition result according to the multi-dimension recognition result and a preset ChatGPT model; and generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of the dimensions.

In some embodiments of the present application, when a pre-trained global perception model is processed, behavior patterns and events in data, such as abnormal behavior detection, crowd gathering, etc., are first identified; then, the time, the place and other context information of the data are considered to improve the accuracy of the identification; secondly, performing risk assessment on the identified behaviors and events, and determining threat information possibly formed by the identified behaviors and events on safety; finally, threat information is organized into multi-dimensional recognition results, which may include behavior types, event severity, recommended operations, and the like.

In an embodiment of the present application, generating a pre-trained global perception model according to the following steps comprises: collecting historical video data from a camera, historical audio data from monitoring equipment and historical sensor data from each sensor in a preset time period; labeling text description information of normal and abnormal behaviors, event types and environmental conditions on event frames, historical audio data and historical sensor data in the historical video data respectively to obtain labeling data; the event frame is a frame representing a shot event and comprises a first frame, a last frame and a plurality of key frames, wherein the key frames are obtained when the similarity is larger than a preset threshold value through sequentially calculating the similarity of all subframes of a shot except the first frame and the last frame and the previous frame; cleaning, standardizing and enhancing the labeling data to obtain a model training sample; creating a global perception model by adopting a neural network; inputting a model training sample into a global perception model so as to enable the global perception model to perform self-learning and obtain a model loss value; when the model loss value reaches the minimum, a pre-trained global perception model is generated.

Specifically, the event frame in the historical video data is marked, the event frame is a frame representing a shot event and comprises a first frame, a last frame and a plurality of key frames, the key frames are obtained when the similarity is larger than a preset threshold value through carrying out similarity calculation on all subframes of a shot except the first frame and the last frame and the previous frame in sequence, so that the event frame can represent image data independently comprising dangerous events, the number of model analysis is greatly reduced, and the early warning speed is improved.

Each monitoring device comprises a camera, monitoring devices and each sensor, and the multi-dimensional recognition result comprises video description information of the camera, audio description information of the monitoring devices and sensor description information of the sensors.

In the embodiment of the application, when generating the event keyword sequence of each dimension corresponding to the multi-dimension recognition result according to the multi-dimension recognition result and the preset ChatGPT model, the method comprises the following steps: and sequentially processing the video description information, the audio description information and the sensor description information through a preset ChatGPT model according to a preset prompt text set for guiding ChatGPT models to extract dangerous events, so as to obtain event keyword sequences of all dimensions.

In the embodiment of the application, a preset prompt text set for guiding ChatGPT a model to extract dangerous events can be generated according to the following steps: acquiring historical dangerous event description texts submitted for different preset dangerous events in a preset period from a preset dynamic dangerous event description text library; labeling key information of dangerous events existing in each historical dangerous event description text to obtain each labeled historical dangerous event description text; acquiring dimension categories corresponding to the historical dangerous event description texts of each label; inputting the description text of each marked historical dangerous event into a preset ChatGPT model; outputting prompt texts corresponding to the historical dangerous event description texts of each label and used for guiding ChatGPT models to extract dangerous events; acquiring all prompt texts under the same dimension type; storing mapping relations between the same dimension category and all prompt texts under the same dimension category; and taking the mapping relation as a preset prompt text set for guiding ChatGPT a model to extract dangerous events.

The dimension categories comprise dimension categories of the camera, dimension categories of the monitoring equipment and dimension categories of the sensor.

In the embodiment of the application, in a prompt text set for guiding ChatGPT a model to extract dangerous events according to a preset model, video description information, audio description information and sensor description information are sequentially processed through a preset ChatGPT model, and the specific process for obtaining the event keyword sequences of each dimension comprises the following steps: respectively determining a first dimension category corresponding to the video description information, a second dimension category corresponding to the audio description information and a third dimension category corresponding to the sensor description information; respectively acquiring a plurality of first prompt texts corresponding to a first dimension category and used for guiding ChatGPT a model to extract a dangerous event, a plurality of second prompt texts corresponding to a second dimension category and used for guiding ChatGPT a model to extract a dangerous event and a plurality of third prompt texts corresponding to a third dimension category and used for guiding ChatGPT a model to extract a dangerous event from the mapping relation; inputting the video description information and a plurality of first prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension type of the camera; inputting the audio description information and a plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the monitoring equipment; inputting the sensor description information and a plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the sensor; and taking the keyword sequence corresponding to the dimension type of the camera, the keyword sequence corresponding to the dimension type of the monitoring equipment and the keyword sequence corresponding to the dimension type of the sensor as event keyword sequences of all dimensions.

In the embodiment of the application, the specific process of generating the analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of each dimension comprises the following steps: carrying out format standardization on event keyword sequences of all dimensions to obtain first event keyword sequences of all dimensions; distributing context information comprising a time stamp, a place mark and an event type aiming at a target event keyword sequence of each dimension to obtain a second event keyword sequence of each dimension; normalizing the second event keyword sequences of all the dimensions to eliminate the term difference among different data sources and obtain third event keyword sequences of all the dimensions; counting the association degree between the occurrence frequency of each third event keyword in each third event keyword sequence of each dimension and the known event type; determining the importance degree of each third event keyword based on the association degree of the frequency of occurrence of each third event keyword and the known event type; screening keywords with importance degrees larger than a preset threshold according to the importance degrees of the keywords of each third event to obtain a fourth event keyword sequence of each dimension; identifying and extracting each dimension target keyword of a cross-dimension common theme or logical connection in a fourth event keyword sequence of each dimension by adopting a theme modeling technology; the topic modeling technology is a dirichlet allocation algorithm; aggregating the target keywords of each dimension of the cross-dimension common subject or logical connection to obtain a coherent information block; and carrying out dangerous event inference on the coherent information blocks by adopting an inference component of a preset ChatGPT model to obtain an analysis result corresponding to the heterogeneous data to be analyzed.

And S103, when the analysis result indicates that the dangerous event exists in the target area, generating dangerous event information of the target area, and reporting the dangerous event information to the early warning terminal for early warning.

Wherein the analysis result includes the existence or absence of a dangerous event in the target area.

In one possible implementation manner, if the analysis result indicates that the target area does not have a dangerous event, continuing to process the data fed back by the monitoring equipment of the target area; and under the condition that the analysis result indicates that the dangerous event exists in the target area, generating dangerous event information of the target area, and reporting the dangerous event information to the early warning terminal for early warning.

The reporting mode can be terminal monitoring, short message notification, mail notification or APP notification.

For example, as shown in fig. 4, fig. 4 is an overall architecture diagram of a global sensing early warning based on ChatGPT, where data is collected in real time by a camera, a monitoring device, a harmful gas sensor, a humidity sensor and a temperature sensor, then the collected data is sent to a security situation sensing and reporting early warning platform provided by the application, an analysis result is output after the analysis is performed by a preset global sensing early warning component included in the platform, the analysis result is judged by a user big data center to determine whether the analysis result indicates a dangerous event in a target area, if yes, information of the target area carrying the threat or the dangerous event is generated by a processing strategy configured by an emergency treatment tool box, and finally the information is reported to an early warning terminal for early warning by means of terminal monitoring, short message notification, mail notification or APP notification.

Referring to fig. 5, a flowchart of a global perception model training method is provided in an embodiment of the present application. As shown in fig. 5, the method according to the embodiment of the present application may include the following steps:

s201, collecting historical video data from a camera, historical audio data from monitoring equipment and historical sensor data from each sensor in a preset time period;

S202, respectively labeling text description information of normal and abnormal behaviors, event types and environmental conditions for event frames, historical audio data and historical sensor data in historical video data to obtain labeling data; the event frame is a frame representing a shot event and comprises a first frame, a last frame and a plurality of key frames, wherein the key frames are obtained when the similarity is larger than a preset threshold value through sequentially calculating the similarity of all subframes of a shot except the first frame and the last frame and the previous frame;

S203, cleaning, standardizing and enhancing the labeling data to obtain a model training sample;

s204, creating a global perception model by adopting a neural network;

S205, inputting a model training sample into the global perception model so that the global perception model carries out self-learning to obtain a model loss value;

s206, when the model loss value reaches the minimum, generating a pre-trained global perception model.

The following are system embodiments of the present application that may be used to perform method embodiments of the present application. For details not disclosed in the system embodiments of the present application, please refer to the method embodiments of the present application.

Referring to fig. 6, a schematic structural diagram of a global sensing and early warning system based on ChatGPT according to an exemplary embodiment of the present application is shown. The ChatGPT-based global awareness early warning system may be implemented as all or part of an electronic device by software, hardware, or a combination of both. The system 1 comprises a data collection module 10, a data analysis module 20, and an early warning module 30.

The data collection module 10 is used for collecting and preprocessing environment perception data sent by each monitoring device which is pre-arranged in a target area in a preset time period to obtain heterogeneous data to be analyzed;

The data analysis module 20 is configured to input heterogeneous data to be analyzed into a preset global perception early warning component, and output an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the method comprises the steps of presetting ChatGPT models for analyzing multi-dimensional recognition results and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on event keyword sequences of all dimensions;

And the early warning module 30 is used for generating dangerous event information of the target area and reporting the dangerous event information to the early warning terminal for early warning under the condition that the analysis result indicates that the target area has dangerous events.

Optionally, the data analysis module includes:

The identification result output unit is used for inputting the heterogeneous data to be analyzed into a pre-trained global perception model and outputting a multi-dimensional identification result corresponding to the heterogeneous data to be analyzed;

The event keyword sequence generation unit is used for generating event keyword sequences of all dimensions corresponding to the multi-dimensional recognition results according to the multi-dimensional recognition results and a preset ChatGPT model;

And the analysis result generation unit is used for generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of the dimensions.

Optionally, the event keyword sequence generating unit is specifically configured to:

It should be noted that, when the global sensing and early warning system based on ChatGPT provided in the above embodiment performs the global sensing and early warning method based on ChatGPT, only the division of the above functional modules is used for illustration, in practical application, the above functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the global sensing early warning system based on ChatGPT and the global sensing early warning method based on ChatGPT provided in the foregoing embodiments belong to the same concept, and detailed implementation processes of the method embodiments are shown and will not be described herein.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

The application also provides a computer readable medium, on which program instructions are stored, which when executed by a processor, implement the global perception early warning method based on ChatGPT provided by the above method embodiments.

The application also provides a computer program product containing instructions, which when run on a computer, cause the computer to execute the global perception early warning method based on ChatGPT of the above method embodiments.

Referring to fig. 7, a schematic structural diagram of an electronic device is provided in an embodiment of the present application. As shown in fig. 7, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, at least one communication bus 1002.

Wherein the communication bus 1002 is used to enable connected communication between these components.

The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may further include a standard wired interface and a wireless interface.

The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Wherein the processor 1001 may include one or more processing cores. The processor 1001 connects various parts within the overall electronic device 1000 using various interfaces and lines, performs various functions of the electronic device 1000 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and invoking data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1001 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 1001 and may be implemented by a single chip.

The Memory 1005 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). The memory 1005 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. The memory 1005 may also optionally be at least one storage system located remotely from the processor 1001. As shown in fig. 7, an operating system, a network communication module, a user interface module, and ChatGPT-based global awareness early warning application may be included in a memory 1005, which is a type of computer storage medium.

In the electronic device 1000 shown in fig. 7, the user interface 1003 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 1001 may be configured to invoke the ChatGPT-based global awareness early warning application stored in the memory 1005, and specifically perform the following operations:

In one embodiment, when executing the input of the heterogeneous data to be analyzed into the preset global perception early warning component and outputting the analysis result corresponding to the heterogeneous data to be analyzed, the processor 1001 specifically executes the following operations:

In one embodiment, when executing the event keyword sequence of each dimension corresponding to the multi-dimensional recognition result according to the multi-dimensional recognition result and the preset ChatGPT model, the processor 1001 specifically executes the following operations:

In one embodiment, the processor 1001, when executing the generation of a preset set of prompt texts for guiding ChatGPT the model to extract the dangerous event, specifically performs the following operations:

Acquiring all prompt texts under the same dimension type;

In one embodiment, when executing the set of prompt texts for guiding ChatGPT the model to extract the dangerous event according to the preset model, the processor 1001 processes the video description information, the audio description information, and the sensor description information in sequence through the preset ChatGPT model to obtain the event keyword sequence of each dimension, specifically executes the following operations:

In one embodiment, the processor 1001, when executing the generation of the pre-trained global perception model, specifically performs the following operations:

Creating a global perception model by adopting a neural network;

In one embodiment, when executing the event keyword sequence according to each dimension to generate an analysis result corresponding to the heterogeneous data to be analyzed, the processor 1001 specifically performs the following operations:

Those skilled in the art will appreciate that implementing all or part of the above-described methods in the embodiments may be accomplished by a computer program to instruct related hardware, and the program based on ChatGPT global awareness warning may be stored in a computer readable storage medium, where the program, when executed, may include the steps of the embodiments of the methods described above. The storage medium of the program based on ChatGPT global perception warning can be a magnetic disk, an optical disk, a read-only memory or a random memory.

The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.

Claims

1. A global perception early warning method based on ChatGPT, the method comprising:

Inputting the heterogeneous data to be analyzed into a preset global perception early warning component, and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the preset ChatGPT model is used for analyzing the multi-dimensional recognition result and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on the event keyword sequences of the dimensions;

generating dangerous event information of the target area and reporting the dangerous event information to an early warning terminal for early warning under the condition that the analysis result indicates that the dangerous event exists in the target area; wherein,

Inputting the heterogeneous data to be analyzed into a preset global perception early warning component, and outputting an analysis result corresponding to the heterogeneous data to be analyzed, wherein the method comprises the following steps:

generating event keyword sequences of all dimensions corresponding to the multi-dimensional recognition results according to the multi-dimensional recognition results and a preset ChatGPT model;

Generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of the dimensions; wherein,

And generating an analysis result corresponding to the heterogeneous data to be analyzed according to the event keyword sequences of the dimensions, wherein the analysis result comprises:

carrying out format standardization on the event keyword sequences of all the dimensions to obtain first event keyword sequences of all the dimensions;

Counting the association degree between the occurrence frequency of each third event keyword in the third event keyword sequence of each dimension and the known event type;

screening keywords with importance degrees larger than a preset threshold according to the importance degrees of the keywords of the third events to obtain a fourth event keyword sequence of each dimension;

Identifying and extracting each dimension target keyword of a cross-dimension common theme or logical connection in a fourth event keyword sequence of each dimension by adopting a theme modeling technology; wherein the topic modeling technique is a dirichlet allocation algorithm;

2. The method of claim 1, wherein each monitoring device comprises a camera, a listening device, and each sensor, and wherein the multi-dimensional recognition result comprises video description information of the camera, audio description information of the listening device, and sensor description information of the sensor;

generating an event keyword sequence of each dimension corresponding to the multi-dimension recognition result according to the multi-dimension recognition result and a preset ChatGPT model, wherein the event keyword sequence comprises the following steps:

And sequentially processing the video description information, the audio description information and the sensor description information through the preset ChatGPT model according to a preset prompt text set for guiding ChatGPT models to extract dangerous events, so as to obtain event keyword sequences of all dimensions.

3. The method of claim 2, wherein the generating a set of pre-set prompt texts for guiding ChatGPT the model to extract the risk event comprises:

inputting the historical dangerous event description text of each label into a preset ChatGPT model;

Outputting prompt texts which correspond to the marked historical dangerous event description texts and are used for guiding ChatGPT models to extract dangerous events;

Acquiring all prompt texts under the same dimension type;

4. The method of claim 3, wherein the dimension categories include a dimension category of the camera, a dimension category of the listening device, and a dimension category of the sensor;

The step of sequentially processing the video description information, the audio description information and the sensor description information through the preset ChatGPT model according to a preset prompt text set for guiding ChatGPT models to extract dangerous events, so as to obtain event keyword sequences of each dimension, including:

Respectively acquiring a plurality of first prompt texts corresponding to the first dimension category and used for guiding ChatGPT a model to extract a dangerous event, a plurality of second prompt texts corresponding to the second dimension category and used for guiding ChatGPT a model to extract a dangerous event and a plurality of third prompt texts corresponding to the third dimension category and used for guiding ChatGPT a model to extract a dangerous event from the mapping relation;

Inputting the video description information and the plurality of first prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension type of the camera;

inputting the audio description information and the plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the monitoring equipment;

inputting the sensor description information and the plurality of second prompt texts into a preset ChatGPT model, and outputting a keyword sequence corresponding to the dimension category of the sensor;

and taking the keyword sequences corresponding to the dimension categories of the cameras, the keyword sequences corresponding to the dimension categories of the monitoring equipment and the keyword sequences corresponding to the dimension categories of the sensors as event keyword sequences of all dimensions.

5. The method of claim 1, wherein generating the pre-trained global perception model comprises:

labeling the text description information of normal and abnormal behaviors, event types and environmental conditions to event frames, historical audio data and historical sensor data in the historical video data respectively to obtain labeling data; the event frame is a frame representing a shot event and comprises a first frame, a last frame and a plurality of key frames, wherein the key frames are obtained when the similarity is larger than a preset threshold value by sequentially calculating the similarity of all subframes of a shot except the first frame and the last frame and the previous frame;

cleaning, normalizing and enhancing the labeling data to obtain a model training sample;

Creating a global perception model by adopting a neural network;

Inputting the model training sample into the global perception model so as to enable the global perception model to perform self-learning and obtain a model loss value;

and generating a pre-trained global perception model when the model loss value reaches the minimum.

6. A ChatGPT-based global perception early warning system, the system comprising:

The data analysis module is used for inputting the heterogeneous data to be analyzed into a preset global perception early warning assembly and outputting an analysis result corresponding to the heterogeneous data to be analyzed; the pre-set global perception early-warning assembly comprises a pre-trained global perception model and a pre-set ChatGPT model; the pre-trained global perception model is used for generating a multi-dimensional recognition result based on heterogeneous data to be analyzed; the preset ChatGPT model is used for analyzing the multi-dimensional recognition result and generating event keyword sequences of all dimensions; the analysis results corresponding to the heterogeneous data to be analyzed are generated based on the event keyword sequences of the dimensions;

The early warning module is used for generating dangerous event information of the target area and reporting the dangerous event information to the early warning terminal for early warning under the condition that the analysis result indicates that the dangerous event exists in the target area; wherein,

The data analysis module is specifically used for:

7. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method of any one of claims 1-5.

8. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method according to any of claims 1-5.