CN113500596B - Fire operation auxiliary robot system and monitoring method thereof - Google Patents

Abstract

The invention discloses a fire work auxiliary robot system and a monitoring method thereof, wherein the auxiliary robot system comprises n fire AI robots, m access devices and a control server, the fire AI robots are connected with the control server through the access devices in a network manner, at least one fire AI robot is accessed into one access device, m and n are integral, and n is more than or equal to m and more than or equal to 1. The invention detects the fire operation area through AI detection or AI cooperation, thereby reducing the labor cost, improving the monitoring efficiency and the monitoring quality and ensuring the safety of the fire operation.

Description

Fire operation auxiliary robot system and monitoring method thereof

Technical Field

The invention relates to the technical field of constructional engineering, in particular to an action operation monitoring robot system and a monitoring method thereof.

Background

The fire operation in the construction site refers to the temporary operation of welding and cutting in an forbidden fire area and using a torch, an electric drill, a grinding wheel and the like in an inflammable and explosive place to generate sparks, flame and a hot surface. The fire operation is a high-risk operation, and in order to ensure the safety of the fire operation in the construction process, the fire operation is mainly characterized in that a safety worker is equipped to perform uninterrupted dead-angle-free site confirmation and safety monitoring on the construction worker in the whole construction process, so that safety accidents are avoided. However, there may be a possibility of losing work or violation when the security officer monitors the scene, and the risk of misjudgment or missed judgment is easy to occur. In addition, when a large amount of ignition operations are carried out simultaneously, more safety personnel are required to be equipped to meet the requirements of safety production, and the defects of low monitoring efficiency and high labor cost are caused. Therefore, how to reduce labor cost, improve monitoring efficiency and improve monitoring quality to ensure the safety of the fire operation becomes an urgent technical problem to be solved in the field.

Disclosure of Invention

The invention aims to solve the technical problem of providing a fire operation auxiliary robot system and a monitoring method thereof, and aims to solve the problem of how to reduce labor cost, improve monitoring efficiency and improve monitoring quality to ensure the safety of fire operation.

In order to solve the technical problems, the technical scheme provided by the invention is as follows: the fire operation auxiliary robot system comprises n fire AI robots, m access devices and a control server, wherein the fire AI robots are connected with the control server through the access devices in a network mode, at least one fire AI robot is connected to one access device, m and n are integral, and m is larger than or equal to 1.

Further, the fire-working AI robot comprises a processor, and a memory, a camera, a network module, a communication module, a GPS module and an infrared sensor which are connected with the processor, wherein the network module comprises a mobile communication module, a wired network module and/or a WIFI module; the access equipment comprises a mobile communication base station, a switch, a router and/or wireless access point equipment; the control server is a computer or a mobile phone.

Further, the fire work auxiliary robot system provided by the invention is characterized in that the fire work AI robot further comprises a positioning tag, a temperature sensor and/or a gas sensor which are connected with the processor.

In order to solve the above technical problems, another technical solution provided by the present invention is: a monitoring method of the fire work assisting robot system according to the above, comprising:

the fire AI robot establishes network connection with a control server through access equipment, wherein the network is a mobile communication network or an Ethernet;

the control server carries out remote management, function configuration and AI model deployment on the fire AI robot through a network;

the fire-driving AI robot collects image information of a fire-driving operation area in real time, performs calculation analysis on the collected image information according to an AI algorithm in combination with an AI model deployed in the fire-driving AI robot, and automatically monitors the fire-driving operation area to judge whether violation or safety risks exist in construction operators;

when the fire AI robot finds that the fire operation is illegal or has a safety risk, the fire AI robot uploads fire operation violation/safety risk warning information to the control server and carries identification information and position information of the fire AI robot.

Further, according to the monitoring method of the fire operation auxiliary robot system provided by the present invention, the method for automatically monitoring the fire operation area by the fire AI robot includes:

carrying out face recognition, safety helmet detection, work clothes detection, goggles detection, gloves detection and defense region warning detection on construction operators in the fire working region through a camera;

the camera is combined with the infrared sensor, so that the open fire and the fire hazard in the fire operation area can be dynamically detected;

positioning the position of the fire AI robot through a GPS module;

the fire AI robot is in bidirectional communication with the control server through the access device by combining the communication module;

further, in the monitoring method of the fire operation auxiliary robot system provided by the present invention, the method for automatically monitoring the fire operation area by the fire AI robot further includes:

the auxiliary positioning of the firing AI robot is realized through the positioning tag and the radio frequency tag reader connected with the access equipment or the management and control server;

static temperature detection of each monitoring point of the fire working area is realized through a temperature sensor;

through the gas sensor, realize the detection to the regional harmful gas of operation of starting a fire.

Further, the monitoring method of the fire work auxiliary robot system provided by the invention further comprises the following steps:

the control server synchronously performs remote management, function configuration and AI model deployment on the fire AI robot and the access equipment through a network;

when computational power resources of the fire AI robot are insufficient, partial or all detection data of the fire AI robot are unloaded to the access equipment, computational analysis is carried out by combining the access equipment with an AI model synchronously deployed in the access equipment according to an AI algorithm, or the access equipment assists the fire AI robot to carry out cooperative computational analysis by combining the access equipment with the AI model synchronously deployed in the access equipment according to the AI algorithm, so that a fire operation area is automatically monitored, and whether violation or safety risks exist in construction operation personnel is judged;

when the fire operation violation or the safety risk is found, the fire operation violation/safety risk warning information is uploaded to the management and control server by the fire operation AI robot or the access device, and the fire operation violation/safety risk warning information carries identification information and position information of the fire operation AI robot.

Further, the monitoring method of the fire operation auxiliary robot system provided by the invention performs cooperative AI monitoring with the fire AI robot through a single access device, and comprises the following steps:

step 201, a management and control server synchronously deploys an AI model to an ignition AI robot and access equipment connected with the robot, and carries computing resources required by the AI model and cooperation strategy configuration of the AI model;

step 202, after receiving the deployment of the AI model of the control server, the fire AI robot estimates whether the local idle computing power resource of the fire AI robot meets the computing power resource requirement required by the AI model; when the fire AI robot does not have enough computing resources for executing the AI model, an AI cooperation request is sent to the connected access equipment according to the corresponding cooperation strategy configuration information, and the AI cooperation request carries the specific AI task information to be unloaded;

the access equipment evaluates whether computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model, and when the access equipment has enough idle computing power resource to execute the AI model, the access equipment sends a cooperation confirmation request to the firing AI robot, carries an unloading task identifier and allows the firing AI robot to send AI data and the unloading task identifier required by executing the AI task information to the access equipment;

step 203, after receiving the AI data and the unloading task identifier, the access device matches the corresponding AI model through the unloading task identifier, inputs the AI data into the matched AI model, executes an AI cooperation detection reasoning process to complete automatic calculation analysis, and obtains an AI detection result;

step 204, judging whether the AI detection result has a fire operation violation/safety risk, wherein the judging mode comprises two options, and the first option is that the access equipment directly sends the event to a control server when finding that the AI detection result is a fire operation violation/safety risk warning event; the second option is that the access equipment sends an AI detection result to the fire AI robot, the fire AI robot judges the AI detection result, and when the AI detection result is a fire operation violation/risk warning event, the fire AI robot uploads the AI detection result to the management and control server.

Further, the monitoring method of the fire operation auxiliary robot system provided by the invention performs cooperative AI monitoring with the fire AI robot through a plurality of access devices, and comprises the following steps:

step 301, a management and control server synchronously deploys an AI model to an ignition AI robot and all access devices connected with the robot, and carries computing resources required by the AI model and cooperation strategy configuration of the AI model;

step 302: after receiving the deployment of an AI model of a control server, the fire AI robot estimates whether the local idle computing power resource of the fire AI robot meets the computing power resource requirement required by the AI model, and when the fire AI robot does not have enough computing power resource execution for the AI model, the fire AI robot sends an AI cooperation request to one of the access devices connected with the fire AI robot according to the corresponding cooperation strategy configuration information and carries the specific AI task information to be unloaded, and at the moment, the access device is used as a first access device;

the method comprises the steps that a first access device evaluates whether computing power resource information required in an AI cooperation request meets computing power resource requirements required by an AI model, when the first access device does not have enough free computing power resources for the AI model, the first access device sends the AI cooperation request to another access device according to corresponding cooperation strategy configuration information, carries AI task information carried in the AI cooperation request sent by a fire-activating AI robot, and at the moment, the other access device serves as a second access device;

after the second access device receives the AI cooperation request of the first access device, evaluating whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model; when the second access device has enough idle calculation resources to execute the AI model, the second access device sends an AI cooperation confirmation message to the first access device;

after the first access equipment receives the AI cooperation confirmation message of the second access equipment, the first access equipment sends the AI cooperation confirmation message to the fire AI robot; the fire-driving AI robot splits and sends the AI task information to a first access device according to a splitting mode, and then the first access device splits and sends the AI task information to a second access device;

step 303, the firing AI robot, the first access device and the second access device respectively execute the splitting task of the corresponding AI task information, or the second access device and the second access device respectively execute the splitting task of the corresponding AI task information, so as to complete the complete detection of the AI task information, execute the AI inference process, complete the automatic calculation analysis, and obtain an AI detection result;

step 304: judging whether the AI detection result has a fire operation violation/safety risk or not, wherein the judging mode comprises two options, and the first option is that when the second access device finds that the AI detection result is a fire operation violation/safety risk alarm event, the event is directly sent to a control server; the second option is that the second access device sends the AI detection result to the fire-driving AI robot, the fire-driving AI robot judges the AI detection result, and when the AI detection result is a fire-driving violation/risk warning event, the fire-driving AI robot uploads the AI detection result to the management and control server.

Further, according to the monitoring method of the fire operation auxiliary robot system provided by the invention, the method for deploying and updating the AI model to the fire AI robot by the management and control server comprises the following steps:

step 401, when the fire AI robot independently executes an AI detection task, the management and control server pushes a new AI model to the relevant fire AI robot; when the fire AI robot and the access equipment perform AI cooperation, the management and control server simultaneously pushes a new AI model to the fire AI robot and the access equipment;

step 402, when the fire AI robot independently executes the AI detection task, after the control server finishes pushing the new AI model, the control server sends a new AI model enabling instruction to the relevant fire AI robot, and the relevant fire AI robot confirms the new AI model and enables the new AI model to the control server; when the fire AI robot and the access equipment perform AI cooperation, the control server simultaneously sends a new AI model enabling instruction to the relevant fire AI robot and the access equipment; the fire AI robot and the access equipment confirm the new AI model and start the new AI model to the control server at the same time;

step 403, when the fire AI robot executes the AI detection task independently, the fire AI robot detects AI task information based on the new AI model, and when the fire operation violation or risk alarm is found based on the new AI model, the fire AI robot sends related alarm information to the management and control server; when the fire AI robot and the access equipment perform AI cooperation, and when a fire operation violation or risk alarm is found, the fire AI robot or the access equipment sends related alarm information to the management and control server.

Compared with the prior art, the invention has the following beneficial effects:

according to the fire operation monitoring robot system and the monitoring method thereof provided by the embodiment of the invention, the fire operation area is monitored through the fire AI robot, and AI detection is locally executed, so that the fire operation area is automatically monitored, and whether violation or safety risk exists in construction operators is judged. Need not to carry out manual monitoring at the regional configuration security personnel of operation of starting a fire to reduce the cost of labor, avoided the risk that the data security privacy was revealed in the transmission course that will detect data upload for the management and control server, improved the security that detects data. The method has the advantages that the original data are detected without transmitting the construction site where the fire AI robot monitoring area is located, and the real-time performance of safety detection and the safety privacy of the data are effectively guaranteed. The fire AI robot can be flexibly and dynamically deployed according to the actual position of the fire operation on the construction site, and an irrelevant scene is effectively shielded, so that the AI detection accuracy is improved, and the actual and effective management requirements of the intelligent construction site are met.

The fire-working monitoring robot system and the monitoring method thereof provided by the embodiment of the invention can be used for carrying out remote management, function configuration and AI model deployment on a fire-working AI robot through a network to realize the initial configuration of the fire-working AI robot, and then the fire-working AI robot carries out calculation analysis on image information in a monitoring area based on an AI algorithm according to an AI model to judge whether violation or safety risk exists in construction workers in the fire-working area. The situation that misjudgment or judgment omission caused by carelessness or violation does not occur during field monitoring of a security officer can be avoided, so that the monitoring efficiency and the monitoring quality are improved, and the safety of a fire working area is guaranteed.

According to the fire operation monitoring robot system and the monitoring method thereof provided by the embodiment of the invention, when the fire operation AI robot finds that the fire operation is illegal or has a safety risk, the fire operation AI robot transmits fire operation violation/safety risk warning information to the control server and carries identification information and position information of the fire operation AI robot, so that a manager can remotely guide and remotely remind construction operators through the control server to ensure the safety of a fire operation area.

Drawings

FIG. 1 is a schematic diagram of a fireworking supervisory robotic system;

FIG. 2 is a block schematic diagram of the main components of the fire AI robot;

fig. 3 is a schematic structural diagram of the main hardware of the firing AI robot and its AI detection function and logic function;

fig. 4 is a schematic diagram of an access device with logic function;

FIG. 5 is a block diagram illustrating the logical functions of the policing server;

FIG. 6 is a schematic flow diagram of a method of monitoring a fireworking supervisory robotic system;

FIG. 7 is a schematic flow diagram of cooperative detection with a fire AI robot by a single access device;

fig. 8 is a schematic flow diagram of cooperative detection with a fire AI robot by two access devices;

FIG. 9 is a schematic flow chart of AI model deployment and update;

shown in the figure:

100. the mobile fire AI robot comprises a fire AI robot, 110, a processor, 120, a memory, 130, a camera, 140, a network module, 150, a communication module, 160, a GPS module, 161, a positioning tag, 170, an infrared sensor, 180, a temperature sensor, 190 and a gas sensor;

200. accessing the device;

300. and a management and control server.

Detailed Description

The invention is described in detail below with reference to the attached drawing figures: the advantages and features of the present invention will become more apparent from the following description. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is merely for the purpose of facilitating and distinctly claiming the embodiments of the present invention.

Referring to fig. 1, an embodiment of the present invention provides a fire work auxiliary robot system, including n fire AI robots 100, m access devices 200, and a management and control server 300, where the fire AI robots 100 are connected to the management and control server 300 through the access devices 200 in a network, and at least one fire AI robot 100 is connected to one access device 200, where m and n are both integral, and n is greater than or equal to m and greater than or equal to 1. For ease of distinction, the n fire AI robots 100 are numbered from 1 to n and the m access devices 200 are numbered from 1 to m.

Referring to fig. 2, in the fire work assisting robot system according to the embodiment of the present invention, the fire AI robot 100 includes a processor 110, and a memory 120, a camera 130, a network module 140, a communication module 150, a GPS module 160, a positioning tag 161, an infrared sensor 170, a temperature sensor 180, and a gas sensor 190, which are optionally connected to the processor 110.

The processor 110 includes, but is not limited to, a CPU of an X86 or ARM, a single chip Microcomputer (MCU) of another type, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a Complex Programmable Logic Device (CPLD), an AI chip, and the like.

Wherein the memory 120 may include a Random Access Memory (RAM) and a Read Only Memory (ROM) and its expansion memory. For storing AI detection raw data, detection result data, and the like locally in the firing AI robot 100.

Wherein the camera 130 includes but is not limited to one.

The network module 140 includes a mobile communication module, a wired network module and/or a WIFI module. The mobile communication module refers to mobile communication of more than third generation, such as 3G, 4G, 5G, etc. The communication module 150 needs to insert a SIM card at this time. The wired network module and the WIFI module are for ethernet.

Wherein the GPS module 160 includes, but is not limited to, a global satellite navigation positioning system, galileo system, glonass system and beidou satellite navigation system.

Where the positioning tag 161 needs to be used in conjunction with a radio frequency tag reader.

Wherein the temperature sensor 180 may be a wired or wireless temperature sensor.

It should be noted that, the fire AI robot 100 may add or delete hardware structures based on customized requirements of different worksites, and the fire AI robot 100 may add or delete partial capabilities, such as bidirectional voice communication or temperature anomaly detection capabilities, due to changes in the hardware structures, which do not affect the core protection scope of the present invention, and changes in these functions do not affect the fire AI robot 100 to implement auxiliary automatic safety detection on fire work, and implement safety management on fire work in cooperation with the access device 200 and the management and control server 300.

Referring to fig. 1, the access device 200 may include a mobile communication base station, a switch, a router, and/or a wireless Access Point (AP) device. Wherein the mobile communication base station is matched with the communication module 150, and the switch, the router and the AP device are used for matching with the ethernet. The number of the access devices 200 is generally smaller than the number of the fire AI robots 100, and considering that the number of the fire AI robots 100 is large and the access number of one access device 200 is limited, a plurality of access devices 200 are required.

Referring to fig. 1, the management and control server 300 may be a computer or a mobile phone. The administration server 300 may be deployed outside a worksite, such as on the cloud, facilitating centralized, remote management of one or more worksites by management personnel.

The fire operation auxiliary robot system provided by the embodiment of the invention is used for realizing automatic safety detection and management of AI of the fire operation in a construction site.

Referring to fig. 6, an embodiment of the present invention further provides a monitoring method based on the fire work auxiliary robot system, which includes:

in step 510, the fire-driving AI robot 100 establishes a network connection with the management and control server 300 through the access device 200, where the network is a mobile communication network or an ethernet network. Wherein the step of establishing a network connection may comprise:

in step 511, after the fire AI robot 100 is started, it tries to connect to the network through the access device 200, so as to implement an access procedure between the fire AI robot 100 and the access device 200.

In step 512, after networking the fire AI robot 100, establishing a network connection with the management and control server 300 through the access device 200, that is, sending an access request to the management and control server 300 through the access device 200. In order to support that only the authorized fire-activating AI robot 100 accesses the network through the access device 200, and improve the security, the access device 200 provides an access control function, and performs authentication and authorization on the fire-activating AI robot 100 in conjunction with the management and control server 300, the management and control server 300 feeds back an access confirmation to the access device 200, and the access device 200 feeds back an access confirmation to the fire-activating AI robot 100, so as to implement an access process of networking.

In step 520, the management and control server 300 performs remote management, function configuration, and AI model deployment on the fire AI robot 100 through a network.

Step 530, the fire-driving AI robot 100 collects image information of a fire-driving working area in real time, performs calculation and analysis on the collected image information according to an AI algorithm in combination with an AI model deployed therein, and automatically monitors the fire-driving working area to judge whether violation or safety risks exist in construction workers.

In step 540, when the fire AI robot 100 finds that the fire operation is illegal or has a safety risk, the fire AI robot 100 uploads fire operation violation/safety risk warning information to the management and control server 300, and carries identification information and location information of the fire AI robot 100.

The monitoring method described above is to implement AI monitoring, i.e., AI detection, only by the firing AI robot 100 itself.

Referring to fig. 3, in the monitoring method of the fire operation assisting robot system according to the embodiment of the present invention, the method for automatically monitoring the fire operation area by the fire operation AI robot 100 may include:

carrying out face recognition, safety helmet detection, work clothes detection, goggles detection, gloves detection and defense area warning detection on construction workers in the fire working area through the camera 130; so as to judge whether the construction worker has violation and safety risk.

The camera 130 is combined with the infrared sensor 170, so that dynamic detection of open fire and fire hidden danger in a fire working area is realized; to determine if a security risk exists.

Positioning the position of the firing AI robot 100 itself by the GPS module 160; the convenience management and control server 300 performs management of the firing AI robot 10010 and monitoring management of a firing work area. The management and control server 300 is convenient to present fire operation violation/risk warning information and the specific position where the fire operation violation/risk occurs to the manager, so that the manager can conveniently implement subsequent management measures.

The communication module 150 is combined to realize the bidirectional communication between the fire AI robot 100 and the management and control server 300 through the access device 200; so as to remotely guide and remind the construction operators. The fire AI robot 100 can establish bidirectional voice communication with the management and control server 300, so that remote centralized managers can conveniently perform direct voice communication with field fire workers, and if the remote fire operation managers receive a field violation alarm, the fire workers can point out violation of the fire workers in time through voice, and can correct violation behaviors in time, so that safety accidents are avoided.

The auxiliary positioning of the firing AI robot 100 is realized by the positioning tag 161 and the radio frequency tag reader connected to the access device 200 or the management and control server 300. The location tag 161 may also enable tracking of hazardous materials.

Static temperature detection of each monitoring point of the fire working area is realized through the temperature sensor 180; to determine if a security risk exists. In particular, abnormal temperatures are detected.

The detection of harmful gases in the working area of a fire is realized through the gas sensor 190. To determine if a security risk exists.

And a management and control logic function for interacting with the management and control server 300 and performing control and management instructions of the management and control server 300, including but not limited to information collection, establishing two-way voice communication, setting an early warning rule, log management, AI model management, function configuration, and the like.

The fire AI robot 100 has an AI cooperation management (Agent) logic function for performing AI cooperation detection inference in association with the access device 200.

That is, the firing AI robot 100 can realize the above-described AI detection function by the above-described hardware configuration. The AI detection function can be configured reasonably as required, and can be increased or decreased as required. Wherein the AI detection is specifically performed based on a machine learning algorithm.

Referring to fig. 4, the access device 200 has both communication and computing capabilities, wherein the communication may be wireless communication, in which case the access device 200 may be a wireless base station, AP, etc., and the communication may be active communication, in which case the access device 200 may be a switch, router, gateway, etc.; whether wireless or wired communication is adopted, the communication function of the access device 200 mainly completes a data transmission function, so that the fire AI robot 100 can transmit relevant data to other places through the access device 200, such as the management and control server 300 or other cloud servers, and a network connection service is provided for the fire AI robot 100. In order to support only authorized sparking AI robots 100 to access the network, and improve security, the access device 200 may further provide an access control function, and perform authentication and authorization on the sparking AI robots 100 in association with the management and control server 300. The computing power includes computing power, and the computing power can support the fire AI robot 100 to take over part of AI computing tasks of the fire AI robot 100 when computing resources are insufficient, that is, AI detection and inference of the access device 200 and the fire AI robot 100 are realized in cooperation; meanwhile, the access device 200 will also have certain storage capability for storing relevant AI models and data. The access device 200 has an AI cooperation management (Agent) function, and is used in association with the fire AI robot 100 to perform AI cooperation detection and inference. The access device 200 is further provided to interact with the management server 300 and perform control and management instructions of the management server 300, including information collection, access management, log management, AI model management, function configuration, and the like.

The computing capability of the access device 200 includes computing power and storage, and may include various types of computing resources, where the computing power is physically based on a Central Processing Unit (CPU), including a CPU of X86 or ARM, or may be based on a heterogeneous computing chip such as a Graphics Processing Unit (GPU), an AI chip, a Field Programmable Gate Array (FPGA), and the like. The computing power service is provided by providing soft-cutting resources with various granularities, such as Virtual machines (Virtual machines), containers (containers), virtual nodes (PODs), and the like, in a software virtualization manner based on the physical computing facilities. The storage may be the same as the memory of the firing AI robot 100. The communication capability of the access device 200 may be an existing cellular wireless network such as 4G/5G, or ethernet communication connected by WiFi, short-range wireless access, wired access, etc., and the access device 200 may have a different form for different communication technologies, such as a base station, an AP, a switch, a router, an access gateway, etc.

Referring to fig. 5, the management server 300 according to the embodiment of the present invention may include the following logic function modules:

and the interface supports the firer operation management personnel to remotely manage one or more firer operations. The fire-driving AI robot 100 has an automatic AI safety detection capability, and reports detected safety violations or risks and other events to the management and control server 300, so that the management difficulty of fire-driving operation managers can be greatly reduced, the misjudgment and misjudgment accidents caused by manual job loss or errors can be reduced, and the safety supervision efficiency can be improved. Specifically, the management and control server 300 provides user management, device management, remote management, log/alarm management, device location management, duplex communication such as duplex voice or text, data synchronization and query such as video, image, and sensor data, AI model management, and AI cooperation management.

According to the fire operation monitoring robot system and the monitoring method thereof provided by the embodiment of the invention, the fire operation area is monitored through the fire operation AI robot 100, and AI detection is locally executed, so that the fire operation area is automatically monitored, and whether violation or safety risk exists in construction operators is judged. Need not to carry out manual monitoring at the regional configuration security personnel of operation of starting a fire to reduce the cost of labor, avoided the risk that data security privacy was revealed in the transmission course that will detect data upload to management and control server 300, improve the security that detects data. That is, the original data is detected without being transmitted out of the construction site where the fire AI robot 100 monitors the area, so that the real-time performance of the safety detection and the safety privacy of the data are effectively guaranteed. The fire AI robot 100 can be flexibly and dynamically deployed according to the actual position of the site fire operation, effectively shield irrelevant scenes, improve the accuracy of AI detection, and meet the actual and effective management requirements of an intelligent site.

The fire-working monitoring robot system and the monitoring method thereof provided by the embodiment of the invention perform remote management, function configuration and AI model deployment on the fire-working AI robot 100 through a network to realize the initial configuration of the fire-working AI robot 100, and then the fire-working AI robot 100 performs calculation analysis on image information in a monitoring area based on an AI algorithm according to the AI model to judge whether violation or safety risk exists in construction workers in the fire-working area. The situation that misjudgment or judgment omission caused by carelessness or violation does not occur during field monitoring of a security officer can be avoided, so that the monitoring efficiency and the monitoring quality are improved, and the safety of a fire working area is guaranteed.

According to the fire operation monitoring robot system and the monitoring method thereof provided by the embodiment of the invention, when the fire operation AI robot 100 finds that the fire operation is illegal or has a safety risk, the fire operation AI robot 100 transmits fire operation violation/safety risk warning information to the control server 300, and carries identification information and position information of the fire operation AI robot 100, so that a manager can remotely guide and remotely remind construction operators through the control server 300 to ensure the safety of a fire operation area.

Referring to fig. 7, the monitoring method of the fire work auxiliary robot system according to the embodiment of the present invention further includes an AI cooperation monitoring method, which specifically includes:

in step 610, the management and control server 300 performs remote management, function configuration, and AI model deployment on the fire AI robot 100 and the access device 200 synchronously through the network. The step 610 is essentially the initial configuration of the fire AI robot 100, and this configuration process may be automatically completed by setting rules or manually completed by a fire operation manager through an interface of the management and control server 300.

Step 620, when the computing resources of the fire AI robot 100 are insufficient, offloading part or all of the detection data of the fire AI robot 100 into the access device 200, performing computational analysis by combining the access device 200 with an AI model deployed synchronously therein according to an AI algorithm, or assisting the fire AI robot 100 with an AI model deployed synchronously therein through the access device 200 according to an AI algorithm to perform collaborative computational analysis, so as to automatically monitor a fire work area and determine whether a construction worker has violation or safety risk; the detection data comprises image data and sensor data, wherein the image data comprises pictures and video data, and the sensor data comprises data detected by each sensor.

In step 630, when a fire operation violation or a security risk is found, the fire operation violation/security risk warning information is uploaded to the management and control server 300 by the fire operation AI robot 100 or the access device 200, and carries the identification information and the location information of the fire operation AI robot 100.

The AI cooperation monitoring method solves the problem that when the computing resources of the fire AI robot 100 are insufficient, the AI cooperation monitoring is carried out through the access equipment 200, and at the moment, the AI detection function is executed in the access equipment 200 or is simultaneously executed in the fire AI robot 100 and the access equipment 200, namely, the AI detection function is still locally executed, and the data safety is ensured without uploading to the management and control server 300.

Referring to fig. 7, the monitoring method of the fire work auxiliary robot system according to the embodiment of the present invention, which performs cooperative AI monitoring with the fire AI robot 100 through a single access device 200, may include the following steps:

step 201, the management and control server 300 deploys an AI model to the fire-fighting AI robot 100 and the access device 200 connected thereto synchronously, and carries computing resources required by the AI model and a cooperation policy configuration of the AI model.

Step 202, after receiving the AI model deployment of the management and control server 300, the fire-activated AI robot 100 estimates whether the local idle computational resources of the fire-activated AI robot 100 meet the computational resource requirements required by the AI model; when the fire-activating AI robot 100 does not have enough computing resources for executing an AI model, sending an AI cooperation request to the access device 200 connected thereto according to the corresponding cooperation strategy configuration information, and carrying specific AI task information to be unloaded; the access device 200 evaluates whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model, and when the access device 200 has enough idle computing power resources for the AI model to execute, sends a cooperation confirmation request to the fire AI robot 100, carries an unloading task identifier, and allows the fire AI robot 100 to send AI data and the unloading task identifier required for executing the AI task information to the access device 200; the AI task information includes AI model information, task resource requirement information, and the like, where the AI cooperation policy configuration information may include a splitting manner of the AI model, and the AI data may enable the fire AI robot 100 to execute gradient data or raw data such as video, picture, or sensor data after a part of the AI model is executed. When the AI model is split in a 10-layer DNN (deep neural network), for example, the AI cooperation policy configuration information may indicate which layer is to be split, for example, at layer 5, which means that layers 1 to 5 are executed on the firing AI robot 100, and the remaining layers 6 to 10 are executed on the access device 200, and gradient information after the AI model is split is transmitted between the firing AI robot 100 and the access device 200. If, in the extreme, all AI tasks are offloaded to be performed at the access device 200, the split level may be configured to be 0. It should be noted that the AI cooperation policy configuration information may carry one or more cooperation modes, each of which has different requirements for cooperation calculation power of the access device 200, and the firing AI robot 100 may determine which one is specifically selected based on its own idle resources. The embodiment of the present invention utilizes the detachability of the AI model, and realizes smooth execution of the AI task by coordinating part or all of the AI task information to the access device 200 in the case where the firepower AI robot 100 is short of computational resources. An example of collaboration policy configuration information is illustrated in table one below, where three splitting options are given, the splitting level of each option and the proportion of computing resources that can be offloaded:

options for	Splitting hierarchy	Computing power collaboration
			Separation method 1	0	100％
Resolution method
	2	5	70％
Splitting method 3				8	40％

TABLE 1

Thus, when the fire-driving AI robot 100 obtains that more than 60% of computing power resources need to be unloaded by comparing the local idle resources with the AI model computing power resource requirements, the splitting mode 2 is selected; and transmits the selected splitting manner to the connected access device 200.

Step 203, after receiving the AI data and the offload task identifier, the access device 200 matches the corresponding AI model with the offload task identifier, inputs the AI data into the matched AI model, and executes an AI cooperative detection inference process to complete automatic calculation and analysis, thereby obtaining an AI detection result.

Step 204, judging whether the AI detection result has a fire operation violation/safety risk, wherein the judging mode includes two options, and the first option is to directly send the event to the management and control server 300 when the access device 200 finds that the AI detection result is a fire operation violation/safety risk warning event; the second option is that the access device 200 transmits the AI detection result to the fire AI robot 100, the fire AI robot 100 determines the AI detection result, and when the AI detection result is a fire violation/risk warning event, the fire AI robot 100 uploads the AI detection result to the management and control server 300.

The above steps 201 to 204 implement a manner of performing AI cooperation monitoring by a single access device 200.

Referring to fig. 8, the monitoring method of the fire work auxiliary robot system according to the embodiment of the present invention, which performs cooperative AI monitoring with the fire AI robot 100 through a plurality of access devices 200, may include the following steps:

step 301, the management and control server 300 deploys an AI model to the fire-fighting AI robot 100 and all the access devices 200 connected thereto synchronously, and carries computational resources required by the AI model and a cooperation policy configuration of the AI model.

Step 302, after receiving the AI model deployment of the management and control server 300, the fire AI robot 100 estimates whether the local idle computation resource of the fire AI robot 100 meets the computation resource requirement required by the AI model, and when the fire AI robot 100 does not have enough computation resource execution for the AI model, sends an AI cooperation request to one of the access devices 200 connected thereto according to the corresponding cooperation policy configuration information, and carries AI task information that needs to be unloaded specifically, where the access device 200 serves as the first access device 200.

The first access device 200 evaluates whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model, and when the first access device 200 does not have enough free computing power resource for the AI model, sends the AI cooperation request to another access device 200 according to the corresponding cooperation policy configuration information, and carries the AI task information carried in the AI cooperation request sent by the fire AI robot 100, and at this time, the other access device 200 serves as a second access device 200.

After the second access device 200 receives the AI cooperation request from the first access device 200, it is evaluated whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model; when the second access device 200 has enough idle computation resources to execute the AI model, the second access device 200 sends an AI cooperation confirmation message to the first access device 200; when the second access apparatus 200 does not have the required AI model, the second access apparatus 200 needs to request deployment of the corresponding AI model from the management controlling server 300.

When the first access device 200 receives the AI cooperation confirmation message of the second access device 200, the first access device 200 sends the AI cooperation confirmation message to the fire AI robot 100; the fire AI robot 100 distributes the AI task information to the first access device 200 according to the splitting manner, and then the first access device 200 distributes the AI task information to the second access device 200.

Step 303, the firing AI robot 100, the first access device 200, and the second access device 200 respectively execute the splitting task of the corresponding AI task information, or the second access device 200 and the second access device 200 respectively execute the splitting task of the corresponding AI task information, so as to complete the complete detection of the AI task information, execute the AI inference process, complete the automatic calculation analysis, and obtain the AI detection result.

Step 304, judging whether the AI detection result has a fire operation violation/safety risk, wherein the judging mode includes two options, and the first option is that the second access device 200 directly sends the event to the management and control server 300 when finding that the AI detection result is a fire operation violation/risk warning event; the second option is that the second access device 200 transmits the AI detection result to the fire AI robot 100, the fire AI robot 100 determines the AI detection result, and when the AI detection result is a fire violation/risk warning event, the fire AI robot 100 uploads the AI detection result to the management and control server 300.

The above steps 301 to 304 are supplementary to the single access apparatus 200, and when the computational power of the single access apparatus 200 is insufficient, the AI cooperation detection function is performed by two or more access apparatuses 200, wherein the AI cooperation detection performed by only two access apparatuses 200 is illustrated in the steps 301 to 304. Based on the same method extension, the AI cooperative task can be coordinated to more access devices 200, and to multiple fire AI robots 100. Please refer to the above steps 201 to 204 for the parts not described in detail in steps 301 to 304.

Referring to fig. 9, since the precise, real-time, and the like of the AI monitoring are strongly related to the AI model, when a new AI model is used, the new AI model may be updated to the fire AI robot 100 or the access device 200 by the management and control server 300, so as to further improve the efficiency of the automatic AI fire detection. In the monitoring method of the fire operation auxiliary robot system provided in the embodiment of the present invention, the method for deploying and updating the AI model to the fire AI robot 100 by the management and control server 300 includes:

step 401, when the fire AI robot 100 executes an AI detection task alone, the management and control server 300 pushes a new AI model to the relevant fire AI robot 100; when the fire AI robot 100 performs AI cooperation with the access device 200, the management and control server 300 simultaneously pushes a new AI model to the fire AI robot 100 and the access device 200;

step 402, when the fire AI robot 100 executes an AI detection task alone, after the management and control server 300 completes pushing a new AI model, it sends a new AI model enabling instruction to the relevant fire AI robot 100, and the relevant fire AI robot 100 confirms the new AI model and enables to the management and control server 300; when the fire AI robot 100 and the access device 200 perform AI cooperation, the management and control server 300 simultaneously sends a new AI model enabling instruction to the relevant fire AI robot 100 and the access device 200; the fire AI robot 100 and the access device 200 simultaneously confirm the new AI model and enable to the management and control server 300;

step 403, when the fire AI robot 100 executes an AI detection task alone, the fire AI robot 100 performs detection of AI task information based on the new AI model, and when a fire operation violation or risk alarm is found based on the new AI model, sends related alarm information to the management and control server 300; when the fire AI robot 100 performs AI cooperation with the access device 200, and when a fire operation violation or risk warning is found, the fire AI robot 100 or the access device 200 transmits related warning information to the management and control server 300.

Through the above steps 401 to 403, the management and control server 300 completes updating and deployment of the new AI model, and the fire AI robot 100 or the access device 200 performs AI monitoring based on the new AI model and sends alarm information to the management and control server 300 when detecting a fire violation/risk alarm event. The updated or deployed AI model can improve the efficiency, quality and accuracy of AI detection.

According to the fire operation auxiliary robot system and the monitoring method thereof provided by the embodiment of the invention, the fire AI robot 100 can cooperate with the access device 200 to implement AI joint detection of fire operation, so that the problem of resource limitation of the fire AI robot 100 is solved. The fire AI robot 100 can be remotely managed and remotely configured and deployed and updated by the management and control server 300. Management of the firing AI robot 100 is facilitated.

The fire work auxiliary robot system and the monitoring method thereof provided by the embodiment of the invention support deployment of more AI detection models and functions without increasing hardware resources of the fire AI robot 100 through AI cooperative detection of the access device 200.

According to the fire operation auxiliary robot system and the monitoring method thereof provided by the embodiment of the invention, the single access device 200 or the plurality of access devices 200 execute the AI cooperation detection function, so that the labor cost is reduced, the monitoring efficiency and the monitoring quality are improved, and the safety of fire operation is ensured.

The fire operation auxiliary robot system and the monitoring method thereof provided by the embodiment of the invention have the image and infrared signal acquisition capacity, and realize auxiliary safety detection on fire operation based on a local AI model. Meanwhile, the AI model and the scene requiring AI detection may be dynamically changed according to the severity of the fire operation safety management requirement and the increase of the scene, so that the computational resource of the fire AI robot 100 may be insufficient to support the real-time AI fire operation detection requirement, and at this time, the fire AI robot 100 may unload part of the AI detection tasks to the access device 200 to cooperate with the access device 200 to implement AI detection required by the fire operation safety management.

The present invention is not limited to the above-described specific embodiments, and it is apparent that the above-described embodiments are some, not all, of the embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the invention, are within the scope of the invention. Other levels of modification and variation of the present invention may be made by those skilled in the art. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims.

Claims (6)

1. The monitoring method of the fire work auxiliary robot system is characterized in that the fire work auxiliary robot system comprises n fire work AI robots, m access devices and a control server, wherein the fire work AI robots are connected with the control server through the access devices in a network mode, at least one fire work AI robot is connected to one access device, m and n are integral, and m is larger than or equal to 1; the fire-fighting AI robot comprises a processor, and a memory, a camera, a network module, a communication module, a GPS module and an infrared sensor which are connected with the processor, wherein the network module comprises a mobile communication module, a wired network module and/or a WIFI module; the access equipment comprises a mobile communication base station, a switch, a router and/or wireless access point equipment; the control server is a computer or a mobile phone; the system also comprises a positioning label, a temperature sensor and/or a gas sensor which are connected with the processor;

the monitoring method comprises the following steps:

the fire AI robot establishes network connection with a control server through access equipment, wherein the network is a mobile communication network or an Ethernet;

the control server carries out remote management, function configuration and AI model deployment on the fire AI robot through a network;

the fire-driving AI robot collects image information of a fire-driving operation area in real time, performs calculation analysis on the collected image information according to an AI algorithm in combination with an AI model deployed in the fire-driving AI robot, and automatically monitors the fire-driving operation area to judge whether violation or safety risks exist in construction operators;

when the fire AI robot finds that the fire operation is illegal or has a safety risk, the fire AI robot transmits fire operation violation/safety risk warning information to the control server and carries identification information and position information of the fire AI robot;

the monitoring method further comprises the following steps:

the control server synchronously performs remote management, function configuration and AI model deployment on the fire AI robot and the access equipment through a network;

when computational power resources of the fire AI robot are insufficient, partial or all detection data of the fire AI robot are unloaded to the access equipment, computational analysis is carried out by combining the access equipment with an AI model synchronously deployed in the access equipment according to an AI algorithm, or the access equipment assists the fire AI robot to carry out cooperative computational analysis by combining the access equipment with the AI model synchronously deployed in the access equipment according to the AI algorithm, so that a fire operation area is automatically monitored, and whether violation or safety risks exist in construction operation personnel is judged;

when the fire operation violation or the safety risk is found, the fire operation violation/safety risk warning information is uploaded to the management and control server by the fire operation AI robot or the access device, and the fire operation violation/safety risk warning information carries identification information and position information of the fire operation AI robot.

2. The method for monitoring a fire work assisting robot system according to claim 1, wherein cooperative AI monitoring with a fire AI robot through a single access device comprises the steps of:

step 201, a management and control server synchronously deploys an AI model to an ignition AI robot and access equipment connected with the robot, and carries computing resources required by the AI model and cooperation strategy configuration of the AI model;

step 202, after receiving the deployment of the AI model of the control server, the fire AI robot estimates whether the local idle computational power resources of the fire AI robot meet the computational power resource requirements required by the AI model;

when the fire AI robot does not have enough computing resources for executing the AI model, an AI cooperation request is sent to the connected access equipment according to the corresponding cooperation strategy configuration information, and the AI cooperation request carries the specific AI task information to be unloaded;

the access equipment evaluates whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model, and when the access equipment has enough idle computing power resources for the AI model to execute, sends a cooperation confirmation request to the firing AI robot, carries an unloading task identifier, and allows the firing AI robot to send AI data and the unloading task identifier required for executing the AI task information to the access equipment;

step 203, after receiving the AI data and the unloading task identifier, the access device matches the corresponding AI model through the unloading task identifier, inputs the AI data into the matched AI model, executes an AI cooperation detection reasoning process to complete automatic calculation analysis, and obtains an AI detection result;

step 204, judging whether the AI detection result has a fire operation violation/safety risk, wherein the judging mode comprises two options, and the first option is that the access equipment directly sends the event to a control server when finding that the AI detection result is a fire operation violation/safety risk warning event; the second option is that the access equipment sends an AI detection result to the fire AI robot, the fire AI robot judges the AI detection result, and when the AI detection result is a fire operation violation/risk warning event, the fire AI robot uploads the AI detection result to the management and control server.

3. The method for monitoring a fire work assisting robot system according to claim 1, wherein cooperative AI monitoring with the fire AI robot through a plurality of access devices includes the steps of:

step 301, a management and control server synchronously deploys an AI model to a fire-activating AI robot and all access devices connected with the fire-activating AI robot, and carries computing resources required by the AI model and cooperation strategy configuration of the AI model;

step 302, after receiving the deployment of the AI model of the management and control server, the fire AI robot estimates whether the local idle computing power resource of the fire AI robot meets the computing power resource requirement required by the AI model, and when the fire AI robot does not have enough computing power resource for the AI model, the fire AI robot sends an AI cooperation request to one of the access devices connected with the fire AI robot according to the corresponding cooperation strategy configuration information and carries the specific AI task information to be unloaded, and at this time, the access device serves as a first access device;

the method comprises the steps that a first access device evaluates whether computing power resource information required in an AI cooperation request meets computing power resource requirements required by an AI model, when the first access device does not have enough free computing power resources for the AI model, the first access device sends the AI cooperation request to another access device according to corresponding cooperation strategy configuration information, carries AI task information carried in the AI cooperation request sent by a fire-activating AI robot, and at the moment, the other access device serves as a second access device;

after the second access device receives the AI cooperation request of the first access device, evaluating whether the computing power resource information required in the AI cooperation request meets the computing power resource requirement required by the AI model; when the second access device has enough idle calculation resources to execute the AI model, the second access device sends an AI cooperation confirmation message to the first access device;

after the first access device receives the AI cooperation confirmation message of the second access device, the first access device sends the AI cooperation confirmation message to the fire AI robot; the fire AI robot distributes the AI task information to first access equipment according to a splitting mode, and then the first access equipment distributes the AI task information to second access equipment;

step 303, the firing AI robot, the first access device and the second access device respectively execute the splitting task of the corresponding AI task information, or the second access device and the second access device respectively execute the splitting task of the corresponding AI task information, so as to complete the complete detection of the AI task information, execute the AI inference process, complete the automatic calculation analysis, and obtain an AI detection result;

step 304, judging whether the AI detection result has a fire operation violation/safety risk, wherein the judging mode comprises two options, and the first option is that the second access equipment directly sends the event to the management and control server when finding that the AI detection result is a fire operation violation/risk warning event; the second option is that the second access device sends the AI detection result to the fire AI robot, the fire AI robot judges the AI detection result, and when the AI detection result is a fire operation violation/risk warning event, the fire AI robot uploads the AI detection result to the management and control server.

4. The method for monitoring a fire work assisting robot system according to claim 1, wherein the method for the management and control server to deploy and update the AI model to the fire AI robot includes:

step 401, when the fire AI robot independently executes an AI detection task, the management and control server pushes a new AI model to the relevant fire AI robot; when the fire AI robot and the access equipment perform AI cooperation, the control server simultaneously pushes a new AI model to the fire AI robot and the access equipment;

step 402, when the fire AI robot independently executes an AI detection task, after the control server finishes pushing a new AI model, sending a new AI model enabling instruction to the relevant fire AI robots, and confirming the new AI model and enabling the relevant fire AI robots to the control server; when the fire AI robot and the access equipment perform AI cooperation, the control server simultaneously sends a new AI model enabling instruction to the relevant fire AI robot and the access equipment; the fire AI robot and the access equipment confirm the new AI model and start the new AI model to the control server at the same time;

step 403, when the fire AI robot executes the AI detection task independently, the fire AI robot detects AI task information based on the new AI model, and when the fire operation violation or risk alarm is found based on the new AI model, the fire AI robot sends related alarm information to the management and control server; when the fire AI robot and the access equipment perform AI cooperation, and when a fire operation violation or risk alarm is found, the fire AI robot or the access equipment sends related alarm information to the management and control server.

5. The method for monitoring a fire work assisting robot system according to claim 1, wherein the method for automatically monitoring the fire work area by the fire AI robot includes:

carrying out face recognition, safety helmet detection, work clothes detection, goggles detection, gloves detection and defense region warning detection on construction operators in the fire working region through a camera;

the camera is combined with the infrared sensor, so that the open fire and the fire hazard in the fire operation area can be dynamically detected;

positioning the position of the fire AI robot through a GPS module;

and the fire AI robot is in two-way communication with the control server through the access device by combining the communication module.

6. The monitoring method of a fire work auxiliary robot system according to claim 5, wherein the method of the fire AI robot automatically monitoring the fire work area further comprises:

the auxiliary positioning of the firing AI robot is realized through the positioning tag and the radio frequency tag reader connected with the access equipment or the management and control server;

static temperature detection of each monitoring point of the fire working area is realized through a temperature sensor;

through the gas sensor, realize the detection to the regional harmful gas of operation of starting a fire.

Images