CN118071112B - Command scheduling system based on voice interaction and large model - Google Patents
Command scheduling system based on voice interaction and large model Download PDFInfo
- Publication number
- CN118071112B CN118071112B CN202410459641.7A CN202410459641A CN118071112B CN 118071112 B CN118071112 B CN 118071112B CN 202410459641 A CN202410459641 A CN 202410459641A CN 118071112 B CN118071112 B CN 118071112B
- Authority
- CN
- China
- Prior art keywords
- monitoring
- evacuation
- data
- real
- evacuation path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 16
- 238000012544 monitoring process Methods 0.000 claims abstract description 87
- 238000012806 monitoring device Methods 0.000 claims abstract description 66
- 239000003245 coal Substances 0.000 claims abstract description 14
- 238000012216 screening Methods 0.000 claims abstract description 4
- 238000004519 manufacturing process Methods 0.000 claims description 62
- 238000005192 partition Methods 0.000 claims description 37
- 230000004927 fusion Effects 0.000 claims description 31
- 230000007613 environmental effect Effects 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 4
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 229910002091 carbon monoxide Inorganic materials 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000005065 mining Methods 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 24
- 238000000034 method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000004080 punching Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Alarm Systems (AREA)
Abstract
The invention relates to the field of coal mine safety, and discloses a command scheduling system based on voice interaction and a large model, which comprises the following components: the monitoring devices are used for collecting monitoring data; a plurality of sensors for acquiring sensor data; the intelligent management background is used for matching and obtaining a data set, and then inputting a downhole dangerous event early warning large model to obtain a dangerous event prediction result; if the prediction probability is greater than a preset probability threshold, acquiring a real-time positioning data set and screening out a target real-time positioning data set from the real-time positioning data set; respectively sending control information to each identification card in the identification card group so as to control each identification card to send out a prompt signal; the intelligent management background is also used for playing prompt voice through the monitoring equipment so as to prompt workers to evacuate. Therefore, the underground dangerous event can be early warned in time.
Description
Technical Field
The invention relates to the field of coal mine safety, in particular to a command scheduling system based on voice interaction and a large model.
Background
The coal mine command and dispatch system is used as an important component in the whole coal mine system, and is related to the production and safety of the coal mine. In the coal exploitation process, the production speed and various preparation works can be checked in time, and the method has a vital effect on improving the production efficiency of the coal mine.
The existing coal mine commanding and dispatching system often has the following technical problems:
Firstly, the existing coal mine command and dispatch system and the early warning system operate independently, so that actual production data such as operation data of production equipment, environmental data of a production area and the like acquired in the command and dispatch system cannot be organically combined with early warning, and underground dangerous events cannot be early warned in time;
secondly, workers often rely on their own experience in the evacuation process, which is very easy to cause crowding or confusion, and thus the workers cannot evacuate safely and efficiently;
Thirdly, the existing coal mine command and dispatch system generally displays real-time images of a plurality of monitoring devices in an indifferent manner through an intelligent large screen, and can not conduct targeted display aiming at key areas;
fourth, if a safety accident occurs, the identification card worn by the worker is easy to drop due to congestion and other reasons in the emergency evacuation process, so that the position of the worker cannot be accurately positioned, and the rescue work is affected.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
The invention provides a command scheduling system based on voice interaction and a large model, which solves one or more of the technical problems mentioned in the background section.
The invention provides a command scheduling system based on voice interaction and a large model, which comprises: the intelligent management system comprises a plurality of monitoring devices, a plurality of intelligent management background management system and a plurality of intelligent management system, wherein the plurality of monitoring devices are deployed at a plurality of actual monitoring points of a production area, each monitoring device comprises an image module and a voice module, the image module is used for collecting monitoring images of the corresponding actual monitoring points, and forming a binary group from the monitoring images and the positions of the corresponding actual monitoring points to obtain monitoring data and upload the monitoring data to the intelligent management background; the system comprises a plurality of sensors, a smart management background and a control system, wherein each sensor is used for collecting operation data of production equipment or environment data of a production area, forming the operation data or the environment data and corresponding sensor positions into a binary group, obtaining sensor data and uploading the sensor data to the smart management background; the intelligent management background is used for matching a plurality of monitoring data with a plurality of sensor data, one monitoring data and one sensor data which are successfully matched form a data set, and each data set comprises a fusion position, wherein the fusion position is obtained by fusing the position of an actual monitoring point and the position of the sensor; the intelligent management background is also used for inputting the data set into a downhole dangerous event early warning large model to obtain a dangerous event prediction result, and the dangerous event prediction result is used for representing the probability of occurrence of a dangerous event; if the probability is greater than a preset probability threshold, determining a fusion position included in the data set as a target fusion position, and acquiring real-time positioning data of an identification card worn by each worker going into the well on the same day to obtain a real-time positioning data set; screening a plurality of real-time positioning data with the distance smaller than a preset distance threshold value from the real-time positioning data set, and forming a target real-time positioning data set, wherein identification cards respectively corresponding to the real-time positioning data in the target real-time positioning data set form an identification card set; respectively sending control information to each identification card in the identification card group so as to control each identification card to send out a prompt signal; the intelligent management background is also used for playing prompt voice through the monitoring equipment so as to prompt workers to evacuate.
Optionally, the intelligent management background is deployed with a three-dimensional model of a production area, the three-dimensional model of the production area has a mapping relationship with the production area, the three-dimensional model of the production area comprises a plurality of monitoring equipment models, the monitoring equipment models are in one-to-one correspondence with the monitoring equipment, the three-dimensional model of the production area comprises a plurality of sensor models, and the sensor models are in one-to-one correspondence with the sensors; the intelligent management background is also used for associating each data set with a monitoring equipment model and a sensor model in the three-dimensional model of the production area according to the fusion position.
Optionally, the prompt voice is generated by: determining a virtual fusion position corresponding to the target fusion position in the three-dimensional model of the production area, and determining a plurality of virtual outlet positions of the three-dimensional model of the production area; constructing a plurality of evacuation paths according to the virtual fusion position and the plurality of virtual exit positions; generating a score for each evacuation path according to the estimated evacuation time of each evacuation path, the lighting condition rating of each evacuation path, and the number of bifurcations included in each evacuation path; selecting the evacuation path with the highest score from the plurality of evacuation paths and determining the evacuation path as a main evacuation path, selecting the evacuation path with the highest score and determining the evacuation path as an alternative evacuation path, and determining the number of people to be evacuated corresponding to the target fusion position, wherein the number of people to be evacuated is determined according to the number of identification cards included in the identification card group; if the number of people to be evacuated is larger than the preset accommodating number of the single evacuation path, generating an evacuation prompt text containing a main evacuation path or an alternative evacuation path; and carrying out voice synthesis on the evacuation prompt text to obtain corresponding prompt voice.
Optionally, the intelligent management background is further configured to: respectively inquiring monitoring equipment arranged along a main evacuation path and an alternative evacuation path to obtain a main monitoring equipment sequence and an alternative monitoring equipment sequence; and generating evacuation prompt text containing the primary evacuation path or the alternative evacuation path, including: aiming at each monitoring device in a main monitoring device sequence corresponding to the main evacuation path, generating a sub evacuation prompt text corresponding to each monitoring device in the main monitoring device sequence according to the serial numbers of the monitoring devices in the main monitoring device sequence and the evacuation prompt text templates corresponding to the main evacuation path; and generating sub evacuation prompt texts corresponding to each monitoring device in the alternative monitoring device sequence according to the serial numbers of the monitoring devices in the alternative monitoring device sequence and the evacuation prompt text templates corresponding to the alternative evacuation paths aiming at each monitoring device in the alternative monitoring device sequence corresponding to the alternative evacuation paths.
Optionally, the invention provides a command scheduling system based on voice interaction and a large model, which further comprises: the intelligent large screen comprises a plurality of display partitions, wherein the display partitions comprise a main display partition and a plurality of auxiliary display partitions; the main display partition is used for displaying the three-dimensional model of the production area; the auxiliary display partitions are in one-to-one correspondence with the monitoring devices, and each auxiliary display partition is used for displaying real-time images acquired by the corresponding monitoring device; when the probability of dangerous events corresponding to the target data set is recognized to be larger than a preset probability threshold, determining the monitoring equipment corresponding to the target data set as target monitoring equipment, folding and displaying the three-dimensional model of the production area displayed by the main display partition, and switching the real-time image of the target monitoring equipment to the main display partition for display; the intelligent large screen comprises a voice recognition module, reference voiceprint features are stored in the voice recognition module, when a voice signal is detected to comprise a preset keyword, voiceprint feature extraction is carried out on the voice signal, user voiceprint features are obtained, and if the user voiceprint features are matched with the reference voiceprint features, an emergency scheduling scheme is displayed in a secondary display partition before target monitoring equipment is switched.
Optionally, the production equipment comprises a coal mining machine, a hydraulic support and a scraper conveyor, and the environmental data comprises gas concentration, carbon monoxide concentration, wind speed, wind pressure and temperature.
The invention has the following beneficial effects:
1. The voice interaction and the large model are utilized to integrate the early warning function in the command scheduling system, so that the organic combination of actual production data such as the operation data of production equipment, the environmental data of a production area and the like and the early warning function is realized, and the early warning of underground dangerous events is further realized in time; in addition, by combining with the real-time positioning data of the identification card worn by each worker in the well, workers in the area with risk are reminded in a key way, and prompt voice is played through a voice module, so that the combination of various prompt modes is realized, and the early warning of the underground dangerous event is realized in time;
2. Constructing a plurality of evacuation paths by utilizing the virtual fusion position and the plurality of virtual exit positions in the three-dimensional model of the production area, and selecting the evacuation path with higher score from the plurality of evacuation paths, thereby being beneficial to realizing safe and efficient evacuation and avoiding congestion or confusion in the evacuation process; in addition, the voice module included in the monitoring equipment in the evacuation path is used for broadcasting evacuation prompt texts, so that the evacuation path can be indicated, and safe and efficient evacuation is further realized;
3. The intelligent large screen is partitioned, and the main display partition is used for displaying the three-dimensional model of the production area; the auxiliary display partitions are in one-to-one correspondence with the monitoring devices, and each auxiliary display partition is used for displaying real-time images acquired by the corresponding monitoring device. On the basis, when the probability of occurrence of the dangerous event is larger than a preset probability threshold, the real-time image of the target monitoring equipment can be switched to be displayed in the main display partition, so that targeted display of the key area can be realized. In addition, the intelligent large screen comprises a voice recognition module, and voice print comparison is carried out through the voice recognition module, so that the dispatching safety is ensured;
4. Free positioning data is identified by clustering the real-time positioning data sets. On the basis, the nearest monitoring equipment is used for collecting real-time images and carrying out image recognition, and further, the states of workers are judged through questioning voices so as to determine different rescue priorities, so that efficient hierarchical rescue is realized.
Drawings
The above and other features, advantages and aspects of embodiments of the present invention will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is an exemplary architecture diagram of a voice interaction and large model based command dispatch system of the present invention.
Detailed Description
The invention will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the invention have been illustrated in the accompanying drawings, it is to be understood that the invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the invention are for illustration purposes only and are not intended to limit the scope of the present invention.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like herein are merely used for distinguishing between different devices, modules, or units and not for limiting the order or interdependence of the functions performed by such devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those skilled in the art will appreciate that "one or more" is intended to be construed as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the devices of the present invention are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The invention will be described in detail below with reference to the drawings in connection with embodiments.
Referring to FIG. 1, an exemplary architecture diagram of a command and dispatch system based on voice interactions and a large model of the present invention is shown. The method specifically comprises the following steps: a plurality of monitoring devices 101, a plurality of sensors 102 and an intelligent management background 103. The monitoring devices 101 are disposed at a plurality of actual monitoring points in the production area, each monitoring device includes an image module and a voice module, the image module is configured to collect a monitoring image of a corresponding actual monitoring point, and form a binary group with the monitoring image and a position of the corresponding actual monitoring point, so as to obtain monitoring data and upload the monitoring data to the intelligent management background 103.
In practice, a plurality of actual monitoring points can be preset in the production area according to production requirements. Each actual monitoring point is provided with a monitoring device. The monitoring device may be an intelligent camera, which includes an image module and a voice module. The image module can realize the functions of image acquisition, processing, transmission and the like. In addition, each monitoring device may be preset with a position of a corresponding actual monitoring point. Thus, each intelligent camera can collect the monitoring image of the corresponding actual monitoring point, and form the monitoring image and the position of the corresponding actual monitoring point into a binary group, and upload the binary group as monitoring data to the intelligent management background 103.
Each of the plurality of sensors 102 is configured to collect operation data of the production facility or environmental data of the production area, and form a binary set from the operation data or environmental data and the corresponding sensor position, so as to obtain sensor data, and upload the sensor data to the intelligent management background 103. It will be appreciated that the sensor data may be a binary set of operational data and sensor locations, or may be a binary set of environmental data and sensor locations.
In practice, the production equipment comprises a coal mining machine, a hydraulic support, a scraper conveyor and the like, and the environmental data comprise gas concentration, carbon monoxide concentration, wind speed, wind pressure, temperature and the like. It will be appreciated that the sensors used will vary depending on the type of data being collected.
The intelligent management background 103 is configured to match the received plurality of monitoring data with the plurality of sensor data, where one monitoring data and one sensor data that are successfully matched form a data set. Specifically, the distance between the position of the actual monitoring point included in the monitoring data and the position of the sensor included in the sensor data can be determined, and if the distance is smaller than or equal to the matching distance threshold value, the matching of the actual monitoring point and the position of the sensor included in the sensor data is considered successful, and one monitoring data and one sensor data which are successfully matched form a data set. The data set may include the monitoring image, the operational data, and the fusion location, or the data set may further include the monitoring image, the environmental data, and the fusion location. The fusion position is obtained by fusing the position of the actual monitoring point and the position of the sensor. For example, it may be an average of the position of the actual monitoring point and the position of the sensor.
The intelligent management background 103 is further used for inputting the data set into a downhole dangerous event early warning large model to obtain a dangerous event prediction result, and the dangerous event prediction result is used for representing the probability of occurrence of a dangerous event; if the probability is greater than a preset probability threshold, determining a fusion position included in the data set as a target fusion position, and acquiring real-time positioning data of an identification card worn by each worker going into the well on the same day to obtain a real-time positioning data set; screening a plurality of real-time positioning data with the distance smaller than a preset distance threshold value from the real-time positioning data set, and forming a target real-time positioning data set, wherein identification cards respectively corresponding to the real-time positioning data in the target real-time positioning data set form an identification card set; respectively sending control information to each identification card in the identification card group so as to control each identification card to send out a prompt signal; the intelligent management background is also used for playing prompt voice through the monitoring equipment so as to prompt workers to evacuate. The prompting voice can be preset or generated in real time.
The underground dangerous event early warning large model can comprise a convolutional neural network, a cyclic neural network and a prediction network. The convolutional neural network takes the monitoring image in the data set as input and outputs image characteristics; the cyclic neural network takes environmental data or operation data as input and outputs data characteristics; the image features and the data features are used as inputs of a prediction network (sofmax layers) to output dangerous event prediction results.
In practice, in order to ensure the safety of the workers, each worker needs to wear the identification card and place the identification card in the card reading equipment for punching the card before the workers go into the well, so that the card reading equipment can count the time of the workers going into the well and the time of the workers going out of the well. In addition, the positioning module is arranged in the identification card, so that real-time positioning data of workers can be acquired. In addition, the identification card can be provided with a light emitting diode so as to send out prompt signals through different light emitting modes.
In some embodiments, the voice interaction and the large model are utilized to integrate the early warning function in the command scheduling system, so that the organic combination of actual production data such as the operation data of production equipment, the environmental data of a production area and the like and early warning is realized, and further, the early warning of underground dangerous events is realized in time; in addition, through combining with the real-time positioning data of the identification card worn by each worker in the well to remind the workers in the area with risk in a key way, and the voice module plays the prompting voice to realize the combination of various prompting modes so as to realize early warning of the dangerous event in the well in time.
In some embodiments, in order to further solve the second technical problem described in the background section, that is, "workers often rely on their own experiences during evacuation, congestion or confusion is very likely to occur, and thus efficient evacuation is impossible," in some embodiments of the present invention, a three-dimensional model of a production area is deployed in a smart management background, the three-dimensional model of the production area has a mapping relationship with the production area, the three-dimensional model of the production area includes a plurality of monitoring device models, the monitoring device models are in one-to-one correspondence with the monitoring devices, the three-dimensional model of the production area includes a plurality of sensor models, and the sensor models are in one-to-one correspondence with the sensors; the intelligent management background is also used for associating each data set with a monitoring equipment model and a sensor model in the three-dimensional model of the production area according to the fusion position.
On the basis, the prompt voice is generated through the following steps:
Step one, determining a virtual fusion position corresponding to a target fusion position in a three-dimensional model of a production area, and determining a plurality of virtual outlet positions of the three-dimensional model of the production area;
step two, constructing a plurality of evacuation paths according to the virtual fusion positions and the virtual exit positions; in practice, the three-dimensional model of the production zone includes a plurality of paths having mapping relation with each channel of the production zone. On the basis, a plurality of evacuation paths can be constructed according to the virtual fusion position and the plurality of virtual exit positions through a path planning technology.
Thirdly, generating a score of each evacuation path according to the estimated evacuation time of each evacuation path, the illumination condition rating of each evacuation path and the number of bifurcation included in each evacuation path; wherein the estimated evacuation time of each evacuation path may be calculated from the distance of the evacuation path, and the lighting condition rating of each evacuation path may be obtained from the number of lighting devices of the evacuation path. On the basis, the corresponding score of each evacuation path is obtained by inquiring a preset evacuation path scoring table or determining the weighted sum of the items according to a preset weighting coefficient.
Selecting the evacuation path with the highest score from the plurality of evacuation paths and determining the evacuation path as a main evacuation path, selecting the evacuation path with the highest score and determining the evacuation path as an alternative evacuation path, and determining the number of people to be evacuated corresponding to the target fusion position, wherein the number of people to be evacuated is determined according to the number of identification cards included in the identification card group; if the number of people to be evacuated is larger than the preset accommodating number of the single evacuation path, generating an evacuation prompt text containing a main evacuation path or an alternative evacuation path;
and fifthly, carrying out voice synthesis on the evacuation prompt text to obtain corresponding prompt voice.
On this basis, the intelligent management background is also used for: respectively inquiring monitoring equipment arranged along a main evacuation path and an alternative evacuation path to obtain a main monitoring equipment sequence and an alternative monitoring equipment sequence; and generating evacuation prompt text containing the primary evacuation path or the alternative evacuation path, including:
Aiming at each monitoring device in a main monitoring device sequence corresponding to the main evacuation path, generating a sub evacuation prompt text corresponding to each monitoring device in the main monitoring device sequence according to the serial numbers of the monitoring devices in the main monitoring device sequence and the evacuation prompt text templates corresponding to the main evacuation path;
And generating sub evacuation prompt texts corresponding to each monitoring device in the alternative monitoring device sequence according to the serial numbers of the monitoring devices in the alternative monitoring device sequence and the evacuation prompt text templates corresponding to the alternative evacuation paths aiming at each monitoring device in the alternative monitoring device sequence corresponding to the alternative evacuation paths.
In some embodiments, a plurality of evacuation paths are constructed by utilizing virtual fusion positions and a plurality of virtual exit positions in a three-dimensional model of a production area, and evacuation paths with higher scores are selected from the plurality of evacuation paths, so that safe and efficient evacuation is realized, and congestion or confusion in the evacuation process is avoided; in addition, the voice module included in the monitoring equipment in the evacuation path is used for broadcasting evacuation prompt texts, so that the evacuation path can be indicated, and safe and efficient evacuation is further realized.
In some embodiments, in order to further solve the third technical problem described in the background section, that is, "the existing coal mine commanding and dispatching system generally displays real-time images of a plurality of monitoring devices in an intelligent large screen without distinction and cannot be displayed in a targeted manner for a key area", in some embodiments of the present invention, as shown in fig. 1, a commanding and dispatching system based on voice interaction and large model further includes: the smart large screen 104, the smart large screen 104 includes a plurality of display partitions, the plurality of display partitions including a main display partition and a plurality of sub display partitions; the main display partition is used for displaying the three-dimensional model of the production area; the auxiliary display partitions are in one-to-one correspondence with the monitoring devices, and each auxiliary display partition is used for displaying real-time images acquired by the corresponding monitoring device; when the probability of dangerous events corresponding to the target data set is recognized to be larger than a preset probability threshold, determining the monitoring equipment corresponding to the target data set as target monitoring equipment, folding and displaying the three-dimensional model of the production area displayed by the main display partition, and switching the real-time image of the target monitoring equipment to the main display partition for display; the intelligent large screen comprises a voice recognition module, reference voiceprint features are stored in the voice recognition module, when a voice signal is detected to comprise a preset keyword, voiceprint feature extraction is carried out on the voice signal, user voiceprint features are obtained, and if the user voiceprint features are matched with the reference voiceprint features, an emergency scheduling scheme is displayed in a secondary display partition before target monitoring equipment is switched.
In these embodiments, the main display section is used to display the three-dimensional model of the production area by partitioning the smart large screen; the auxiliary display partitions are in one-to-one correspondence with the monitoring devices, and each auxiliary display partition is used for displaying real-time images acquired by the corresponding monitoring device. On the basis, when the probability of occurrence of the dangerous event is larger than a preset probability threshold, the real-time image of the target monitoring equipment can be switched to be displayed in the main display partition, so that targeted display of the key area can be realized. In addition, wisdom large screen includes voice recognition module, compares the voiceprint through voice recognition module, guarantees the security of dispatch.
In some embodiments, in order to further solve the fourth technical problem described in the background section, that is, "if a security accident occurs, an identification card worn by a worker is easy to drop due to congestion and other reasons in the process of emergency evacuation, so that the position of the worker cannot be accurately positioned, and the rescue operation is affected", in some embodiments of the present invention, the intelligent management background is further used for:
if a safety accident occurs, acquiring real-time positioning data of an identification card worn by each worker going into the well on the same day, and obtaining a real-time positioning data set;
Clustering the real-time positioning data sets to obtain a plurality of positioning data clusters and a clustering center of each positioning data cluster;
for each piece of real-time positioning data in each positioning data cluster, determining the distance between the real-time positioning data and the corresponding cluster center, and if the distance is greater than a preset free point identification distance threshold value, setting the real-time positioning data as free positioning data to obtain a free positioning data set;
For each piece of free positioning data in the free positioning data set, determining a monitoring device closest to the free positioning data, acquiring a real-time image through the closest monitoring device, performing human body identification on the acquired real-time image, playing a question voice through a voice module if a human body exists in the real-time image and is in a static state, and determining the rescue priority of the human body corresponding to the free positioning data as a first priority and circularly playing a wake-up voice if no answer voice is received within a preset response time; if no human body exists in the real-time image, discarding the free positioning data; if the human body exists in the real-time image and is in a moving state, the voice module plays the questioning voice, and if the answer voice is received within the preset response time, the rescue priority of the human body corresponding to the free positioning data is determined to be the second priority, and the questioning sub-voice is continuously played.
In these embodiments, the free positioning data is identified by clustering the real-time positioning data sets. On the basis, the nearest monitoring equipment is used for collecting real-time images and carrying out image recognition, and further, the states of workers are judged through questioning voices so as to determine different rescue priorities, so that efficient hierarchical rescue is realized.
The above description is only illustrative of the few preferred embodiments of the present invention and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the invention referred to in the present invention is not limited to the specific combinations of the technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the inventive concept described above. Such as the above-mentioned features and the technical features disclosed in the present invention (but not limited to) having similar functions are replaced with each other.
Claims (4)
1. A command and dispatch system based on voice interaction and a large model, comprising:
The intelligent management system comprises a plurality of monitoring devices, a plurality of intelligent management background and a plurality of intelligent management background, wherein the monitoring devices are deployed at a plurality of actual monitoring points of a production area, each monitoring device comprises an image module and a voice module, the image module is used for collecting monitoring images of the corresponding actual monitoring points, and forming the monitoring images and the positions of the corresponding actual monitoring points into a binary group to obtain monitoring data and upload the monitoring data to the intelligent management background;
The system comprises a plurality of sensors, a smart management background and a control system, wherein each sensor is used for collecting operation data of production equipment or environment data of a production area, forming a binary group by the operation data or the environment data and corresponding sensor positions, obtaining sensor data and uploading the sensor data to the smart management background;
The intelligent management background is used for matching a plurality of monitoring data with a plurality of sensor data, one monitoring data and one sensor data which are successfully matched form a data set, and each data set comprises a fusion position, wherein the fusion position is obtained by fusing the position of the actual monitoring point and the position of the sensor;
The intelligent management background is also used for inputting the data set into a downhole dangerous event early warning large model to obtain a dangerous event prediction result, and the dangerous event prediction result is used for representing the probability of occurrence of a dangerous event; if the probability is larger than a preset probability threshold, determining a fusion position included in the data set as a target fusion position, and acquiring real-time positioning data of an identification card worn by each worker going into the well on the same day to obtain a real-time positioning data set; screening a plurality of real-time positioning data with the distance smaller than a preset distance threshold value from the real-time positioning data set, and forming a target real-time positioning data set, wherein identification cards respectively corresponding to the real-time positioning data in the target real-time positioning data set form an identification card set; respectively sending control information to each identification card in the identification card group so as to control each identification card to send out a prompt signal;
The intelligent management background is also used for playing prompt voice through the monitoring equipment so as to prompt workers to withdraw; the intelligent management background is provided with a three-dimensional model of a production area, the three-dimensional model of the production area has a mapping relation with the production area, the three-dimensional model of the production area comprises a plurality of monitoring equipment models, the monitoring equipment models are in one-to-one correspondence with the monitoring equipment, the three-dimensional model of the production area comprises a plurality of sensor models, and the sensor models are in one-to-one correspondence with the sensors;
The intelligent management background is further used for associating each data set with a monitoring equipment model and a sensor model in the three-dimensional model of the production area according to the fusion position;
The prompt voice is generated through the following steps:
Determining a virtual fusion position corresponding to the target fusion position in the three-dimensional model of the production area, and determining a plurality of virtual outlet positions of the three-dimensional model of the production area;
constructing a plurality of evacuation paths according to the virtual fusion position and the plurality of virtual exit positions;
Generating a score for each evacuation path according to the estimated evacuation time of each evacuation path, the lighting condition rating of each evacuation path, and the number of bifurcations included in each evacuation path;
selecting the evacuation path with the highest score from the plurality of evacuation paths and determining the evacuation path as a main evacuation path, selecting the evacuation path with the highest score and determining the evacuation path as an alternative evacuation path, and determining the number of people to be evacuated corresponding to the target fusion position, wherein the number of people to be evacuated is determined according to the number of identification cards included in the identification card group; if the number of people to be evacuated is larger than the preset accommodating number of the single evacuation path, generating an evacuation prompt text containing the main evacuation path or the alternative evacuation path;
and carrying out voice synthesis on the evacuation prompt text to obtain corresponding prompt voice.
2. The voice interaction and large model based command dispatch system of claim 1, wherein the intelligent management background is further configured to:
respectively inquiring monitoring equipment arranged along the main evacuation path and the alternative evacuation path to obtain a main monitoring equipment sequence and an alternative monitoring equipment sequence; and
The generating evacuation prompt text including the primary evacuation path or the alternative evacuation path includes:
Aiming at each monitoring device in a main monitoring device sequence corresponding to the main evacuation path, generating a sub evacuation prompt text corresponding to each monitoring device in the main monitoring device sequence according to the serial numbers of the monitoring devices in the main monitoring device sequence and the evacuation prompt text templates corresponding to the main evacuation path;
And generating a sub evacuation prompt text corresponding to each monitoring device in the alternative monitoring device sequence according to the serial numbers of the monitoring devices in the alternative monitoring device sequence and the evacuation prompt text templates corresponding to the alternative evacuation paths aiming at each monitoring device in the alternative monitoring device sequence corresponding to the alternative evacuation paths.
3. The voice interaction and large model based command dispatch system of claim 2, further comprising:
The intelligent large screen comprises a plurality of display partitions, wherein the display partitions comprise a main display partition and a plurality of auxiliary display partitions; the main display partition is used for displaying the three-dimensional model of the production area; the auxiliary display partitions are in one-to-one correspondence with the monitoring devices, and each auxiliary display partition is used for displaying real-time images acquired by the corresponding monitoring device;
when the probability of dangerous events corresponding to the target data set is recognized to be larger than a preset probability threshold, determining the monitoring equipment corresponding to the target data set as target monitoring equipment, folding and displaying the three-dimensional model of the production area displayed by the main display partition, and switching the real-time image of the target monitoring equipment to the main display partition for display;
The intelligent large screen comprises a voice recognition module, reference voiceprint features are stored in the voice recognition module, when a voice signal is detected to comprise a preset keyword, voiceprint feature extraction is carried out on the voice signal, user voiceprint features are obtained, and if the user voiceprint features are matched with the reference voiceprint features, an emergency scheduling scheme is displayed in a secondary display partition before the target monitoring equipment is switched.
4. The voice interaction and large model based command and dispatch system of claim 3, wherein the production equipment comprises a coal mining machine, a hydraulic support and a scraper conveyor, and the environmental data comprises gas concentration, carbon monoxide concentration, wind speed, wind pressure and temperature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410459641.7A CN118071112B (en) | 2024-04-17 | 2024-04-17 | Command scheduling system based on voice interaction and large model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410459641.7A CN118071112B (en) | 2024-04-17 | 2024-04-17 | Command scheduling system based on voice interaction and large model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118071112A CN118071112A (en) | 2024-05-24 |
| CN118071112B true CN118071112B (en) | 2024-06-25 |
Family
ID=91107912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410459641.7A Active CN118071112B (en) | 2024-04-17 | 2024-04-17 | Command scheduling system based on voice interaction and large model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118071112B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951889A (en) * | 2017-05-23 | 2017-07-14 | 煤炭科学技术研究院有限公司 | Underground high risk zone moving target monitoring and management system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107060881A (en) * | 2017-06-19 | 2017-08-18 | 安徽鼎信科技集团有限公司 | A kind of mine emergency command management system |
| CN113090330A (en) * | 2021-04-14 | 2021-07-09 | 武汉拾易鑫科技有限公司 | Coal mine underground operation safety real-time monitoring and early warning method based on wireless sensor technology |
| CN114726096B (en) * | 2022-04-06 | 2024-05-24 | 国网江苏省电力有限公司连云港供电分公司 | Emergency command centralized control system and monitoring method in intelligent power grid based on IMS |
| CN114673558B (en) * | 2022-04-12 | 2023-01-31 | 中国矿业大学 | Coal mine driving face risk identification and intelligent pre-control system and method |
| CN115150589A (en) * | 2022-06-30 | 2022-10-04 | 合肥恒翔电子科技有限公司 | Video monitoring operation and maintenance management system for coal mine enterprise |
| CN117151640A (en) * | 2023-09-06 | 2023-12-01 | 大连煤科安全技术研究院有限公司 | Intelligent comprehensive management platform for coal mine |
| CN117365647A (en) * | 2023-10-11 | 2024-01-09 | 昆明煤炭设计研究院有限公司 | Comprehensive management and control system for coal mine |
| CN117590763B (en) * | 2024-01-18 | 2024-03-19 | 中网华信科技股份有限公司 | Intelligent park energy data management and control system |
-
2024
- 2024-04-17 CN CN202410459641.7A patent/CN118071112B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951889A (en) * | 2017-05-23 | 2017-07-14 | 煤炭科学技术研究院有限公司 | Underground high risk zone moving target monitoring and management system |
Non-Patent Citations (1)
| Title |
|---|
| A BIM-Based Visual Warning Management System for Structural Health Monitoring Integrated with LSTM Network;Gongyu Hou 等;KSCE Journal of Civil Engineering;20210618;第25卷(第8期);2779-2793 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118071112A (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP0191407B1 (en) | Parallel, multi-unit, adaptive, nonlinear pattern class separator and identifier | |
| CN101795395B (en) | System and method for monitoring crowd situation | |
| CN103699698B (en) | A kind of being based on improves Bayesian rail transit fault identification method and system | |
| CN110288001B (en) | Target recognition method based on target data feature training learning | |
| CN104091070A (en) | Rail transit fault diagnosis method and system based on time series analysis | |
| KR102585634B1 (en) | Method And System for Monitoring Fire | |
| CN111814775A (en) | Target object abnormal behavior identification method, device, terminal and storage medium | |
| CN110245168B (en) | Method and system for extracting abnormal event characteristic signals in power grid historical alarm | |
| CN116308960A (en) | Intelligent park property prevention and control management system based on data analysis and implementation method thereof | |
| JP2019079536A (en) | Data identification method based on associative clustering deep learning neutral network | |
| CN109633369A (en) | One kind being based on the matched electric network failure diagnosis method of multidimensional data similitude | |
| CN104268710A (en) | Method for generating track traffic network dynamic emergency disposal scheme | |
| CN118071112B (en) | Command scheduling system based on voice interaction and large model | |
| CN107579771A (en) | A kind of Optic Fiber Transport Network intelligent alarm method based on wall scroll alarm forward trace | |
| CN117523499B (en) | Forest fire prevention monitoring method and system based on Beidou positioning and sensing | |
| Yu et al. | Identification of significant factors contributing to multi-attribute railway accidents dataset (MARA-D) using SOM data mining | |
| CN110481804A (en) | A kind of flight assisting system and aircraft | |
| KR102116991B1 (en) | Controlling safety accident system using life jacket | |
| CN106056977B (en) | Maritime affairs intelligent remote entangles separated method | |
| JP2010086219A (en) | Information processing apparatus for maintaining remote machine, program and method | |
| CN111027232A (en) | BIM-based underground space personnel safety evacuation simulation method and system | |
| CN109635008A (en) | A kind of equipment fault detection method based on machine learning | |
| Iveta et al. | Prediction of traffic accidents severity based on machine learning and multiclass classification model | |
| CN114398773A (en) | Coal mine dispatching robot system based on artificial intelligence technology | |
| Dang et al. | Fast and accurate fall detection and warning system using image processing technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |