CN109640032B

CN109640032B - Five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection

Info

Publication number: CN109640032B
Application number: CN201810332637.9A
Authority: CN
Inventors: 冯保国; 张静
Original assignee: Hebei Deguroon Electronic Technology Co ltd
Priority date: 2018-04-13
Filing date: 2018-04-13
Publication date: 2021-07-13
Anticipated expiration: 2038-04-13
Also published as: CN109640032A

Abstract

The invention provides a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection, which comprises: at least one front end artificial intelligence multi-element panorama monitoring detection device and central management device, every front end device includes central control processing unit, the camera unit that links to each other with central control processing unit, multi-element sensor, communication module, mobile terminal and interface unit, and the multi-element sensor includes: a brightness light intensity sensor, a temperature and humidity sensor, a rainfall sensor, a wind speed and direction sensor and a comprehensive gas sensor; the camera set comprises a plurality of zoom cameras and IPC infrared thermal imaging cameras; the central control processing unit mainly completes splicing fusion, data acquisition, logical operation and logical control output of the received image information according to a splicing algorithm; and the central management device displays the on-site five-dimensional graph in real time through the big data cloud processing platform. The invention outputs 5 groups of video streams externally and supports a 180-degree or 360-degree panoramic seamless spliced video stream.

Description

Five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection

Technical Field

The invention relates to the technical field of panoramic monitoring, in particular to a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection.

Background

First, high definition surveillance cameras are widely used in various fields including: the system is an indispensable important device in the fields of transportation, banks, airports, oil fields, railways, safe cities, rail transit, public places, national defense and the like. In addition, new life is injected into the camera with single function based on important technical means such as video analysis processing, detection, snapshot, data extraction and the like.

Although the monitoring camera is endowed with a plurality of functions, the monitoring camera still has various disadvantages due to the limits of installation environment, position, angle, height and the like, so that the use requirement of the monitoring camera can be met to the maximum extent only in special occasions, special environments, special installation positions, special angles and heights. For example, in order to realize uninterrupted monitoring for 24 hours a day, a light supplement device must be installed on the camera. For example, in order to enable the camera to be normally used in various weather environments (rain, snow, fog, haze, dust and the like), a specific camera such as an infrared thermal imaging camera must be used, and in order to accurately extract and capture vehicle information, the capture camera must be installed right above a road lane and light is supplemented by light supplementation and the like. In order to monitor a wider area and eliminate blind corners or dead corner zones, a larger number of cameras must be arranged to meet the actual use requirements, even in a manner of linkage of a gunshot and a ball machine. For example, in a safe city, 3 cameras are arranged every 50-100 meters, 1 monitoring camera is arranged almost every 1 kilometer on a highway, one camera is arranged every 100-200 meters in a tunnel, and one camera is arranged every 50-100 meters of an extra-large bridge including a pier, so that the arrangement density of devices in an important area is higher, although the actual use requirement can be met in the technical aspect, the phenomenon of 'under-lamp black' (a blind area is generated below a camera rod is installed) cannot be avoided. In addition, the costs, both tangible and intangible, of such a large number of camera installations are immeasurable, such as long-term power supply, foundations required for installation of the installation, masts, maintenance, servicing, etc. Furthermore, although a 180 ° panoramic stitching or 360 ° panoramic stitching surveillance camera has appeared in the market, there still exist many disadvantages in practical use, such as 1 180 ° stitched camera, when installed in east and west directions, due to different lighting effects (backlight or backlight) in different time periods, different stitched cameras produce different image effects on the picture, the whole picture of the backlit camera becomes dark, wherein the monitored object loses the original image information and becomes darker as a whole, and the prepared target information cannot be identified, the backlit camera becomes brighter, and the color of the monitored object is distorted. However, for a 180 ° panoramic stitching camera, the picture is a big disaster, it is more difficult to acquire data by image acquisition and analysis, although the image quality can be changed by adjusting parameters manually, the operation flow is very complicated, and the picture can be always available only by continuously adjusting in different lighting environments, which is impossible to realize. Furthermore, the existing panoramic stitching technology has solved some customer demands to some extent, but due to the fact that the cameras are installed in different environments, different monitoring detection objects, different distances between the monitored detection objects, different illumination, different installation height angles and the like, popularization and use of the novel panoramic stitching technology are greatly limited, and secondly, the existing panoramic stitching technology still has some defects, all lenses used when the cameras are out of the field are fixed-focus lenses, even if the zoom lenses need to be disassembled by a person to manually adjust lens parameters, the devices can be recovered, and then the parameters can be corrected through software, so that the best stitching effect can be achieved, even if the installation height and the installation angle of the cameras are large in limitation, and the complicated operation process is difficult to master and process by non-professional persons. At present, many users or construction units in monitoring and detecting areas are still willing to select a monitoring and detecting application scheme of a plurality of cameras to a large area range for convenience, stability and reliability.

Secondly, the full-element meteorological station follows the design and specification of the international meteorological organization (WMO) meteorological observation standard, and mainly aims at the following steps: the system comprises a solar energy collecting device, a solar.

Although the all-element weather station can meet the use requirements of most of personnel or departments, the all-element weather station cannot be widely used due to high cost, and even a plurality of functions are not utilized, so that great waste is caused. The professional weather stations are all arranged at fixed points and cannot move at any time, the acquired data can only be single-point weather data, but not continuous area or large-range accurate data, and although local weather conditions of several days or the day in the future can be acquired through weather stations, the weather stations can often appear in the conditions of 'raining at this side and sunny at the side' in the same city or unified area. This situation can have a great impact on important institutions or departments, and even cause unnecessary economic loss or life risks. For example, in winter, the freeway or the urban main traffic road may cause ice on the road surface after raining and snowing, and the ice on the road surface is not fixed, so that the ice condition and the weather condition of the whole road need to be effectively detected to obtain real and effective data, and a more effective method can be adopted to avoid various disasters. The fixed-point weather detection device can also be used in the haze weather, the rain weather, the snow weather or the fog weather, and the fixed-point weather detection device can be careless under the condition. Although the smoke sensors are also arranged in the tunnel, the smoke sensors are arranged at fixed points, the harmful gas detection and the environment detection in the tunnel adopt fixed point detection, the quantity is small, the data of a certain node and/or a section can be reflected on one surface, the accurate data in a continuous area in a large range cannot be represented, particularly, once a traffic accident happens in the tunnel, the data is more important, the effective data with accurate and reliable continuity acquired at an early point provides technical support for critical warning and life saving, and the method is very important for a traffic manager.

Furthermore, tunnel fires are the most dangerous safety accidents in public transportation, which not only cause traffic congestion in tunnels, but also may cause serious losses and life safety. Once explosion or fire disaster occurs in the tunnel, the tunnel is difficult to rescue, the rapid detection, accurate early warning, the reduction of false alarm and missed alarm, the rapid and accurate processing of major events are critical for rapidly controlling the fire condition, rescuing personnel and reducing loss. The conventional fire detection technology mainly includes: the detection system comprises a temperature sensing detection device, a smoke detection device, a temperature sensing optical fiber detection device, an optical fiber grating detection device, a dual-wavelength detection device, a video detection device, an infrared detection device and the like, wherein the temperature sensing detection device, the smoke detection device, the temperature sensing optical fiber detection device, the optical fiber grating detection device, the dual-wavelength detection device, the video detection device, the infrared detection device and the like are easily influenced by a tunnel complex environment, fire real-time detection and early warning time lag are difficult to realize, in addition, the existing detection technology is difficult to guarantee.

The region that smog stretchs, harmful gas constitution condition to and the evacuation of personnel condition in the tunnel, the distribution condition all can lose original function under the condition of dense smoke strong fire, and rescue personnel and monitoring personnel can't develop effective rescue when unable acquisition tunnel in accurate information. Once this condition occurs, only the on-site personnel can be expected to save themselves. In addition, the existing fire alarm detection equipment almost completely alarms in a point mode, has no visual video for guidance, monitoring, observation and other auxiliary functions, and the existing video detection alarm device fails under the condition of dense smoke. The road surface of the tunnel at the entrance and the exit is frozen, particularly after the tunnel enters winter, even in alpine regions, once the road surface is frozen, serious damage can be caused to vehicles running on the road in plains, even the economic loss or casualties caused by the serious traffic accident are the largest, although a plurality of road surface freezing detection technologies or equipment can meet certain requirements of users, the road surface detection equipment which is fast, accurate, simple, stable, reliable, wide in coverage range and visual is always pursued by the owners continuously.

In summary, the various devices described above have respective advantages and disadvantages, but the greatest disadvantage is that each device is independent and independent, so how the huge data information can be integrated to provide help for safe travel or rescue of human beings is a technical problem which needs to be solved at present.

Disclosure of Invention

The object of the present invention is to solve at least one of the technical drawbacks mentioned.

Therefore, the invention aims to provide a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection.

In order to achieve the above object, an embodiment of the present invention provides a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection, including: at least one front end artificial intelligence many key elements panorama monitoring detection device and central management device, every front end artificial intelligence many key elements panorama monitoring detection device includes central control processing unit, with a set of zoom camera that central control processing unit links to each other (realize 180 or 360 panorama concatenation functions, can provide real-time video road conditions on a large scale and look over, provide various monitoring detection function based on video image analysis), IPC infrared thermal imaging camera (realize that high temperature object detects, road dark ice detects, fire early warning alarm detects, night vision function etc. belong to the perception unit), many key elements sensor, communication module and interface unit and power module, wherein, many key elements sensor includes: the system comprises a brightness light intensity sensor, a temperature and humidity sensor, a rainfall sensor, a wind speed and direction sensor and a comprehensive gas sensor, wherein the brightness light intensity sensor is used for acquiring a brightness light intensity value of the current environment; the temperature and humidity sensor is used for collecting temperature and humidity data of the current environment, the rainfall sensor is used for collecting rainfall value of the current environment, the wind speed and direction sensor is used for collecting wind speed and direction data of the current environment, the comprehensive gas sensor is used for collecting concentration values of various gases in the current environment, each front-end artificial intelligent multi-element panoramic monitoring detection device is installed at a preset position in a road section, then the collected information of various environment data is sent to the central control processing unit, each front-end artificial intelligent multi-element panoramic monitoring detection device is uploaded to a central management device (a five-dimensional early warning management platform) by the central control processing unit, simulation of actual environments (such as rain, snow, fog, wind, sunshine and the like) in a simulation site is carried out, the actual environments are fused into a five-dimensional graph to be displayed in real time, and the real-time display of road conditions is carried out, Providing a basic basis for road rescue, environmental disaster prediction and evaluation and emergency scheme generation; the group of zoom cameras comprise a plurality of zoom cameras for realizing a 180-degree or 360-degree panoramic stitching function, providing real-time large-range video road condition viewing and various monitoring and detecting functions based on video image analysis, wherein each front-end artificial intelligent multi-element panoramic monitoring and detecting device is arranged at a preset position in a road section and respectively collects image information of different positions on the road section, each camera adopts a variable-focus lens with different focal lengths to monitor objects with different distances and collect corresponding image information, then the collected image information is sent to the central control processing unit, the central control processing unit uploads each front-end artificial intelligent multi-element panoramic monitoring and detecting device to the central management device for secondary video image stitching and blending into a five-dimensional graph for real-time display, abnormal event detection and alarm are realized, Positioning, traffic state preview and evaluation, full path video viewing roaming and other functions; the IPC infrared thermal imaging camera is used for realizing the detection of high-temperature objects, the detection of road dark ice, the early warning and alarm of fire and providing a night vision perspective function, firstly demarcating an area to be detected, then detecting objects, pedestrians and roadbed and road surfaces in the detected area in real time, and playing a role in night vision perspective monitoring when a common camera is blind at night or in dark environment; each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section, an IPC infrared thermal imaging camera respectively collects data information and image information of objects, pedestrians, road beds and pavements at different positions on the road section, then the collected data information is sent to the central control processing unit, the central control processing unit uploads each front-end artificial intelligence multi-element panoramic monitoring detection device to the central management device for centralized gathering analysis and is fused into a five-dimensional graph for real-time display, and the functions of abnormal vehicle, object, pedestrian and road bed and pavement accident detection alarm, night vision positioning, traffic state preview and evaluation and perspective are achieved; the central control processing unit is used for splicing and fusing the received image information of the group of zoom cameras according to a splicing algorithm, acquiring data, performing logical operation and outputting logical control to form a 180-degree or 360-degree panoramic spliced image, uploading the image to a central management device, combining various data acquired by the multi-element sensor when receiving alarm information from the IPC infrared thermal imaging camera, outputting alarm information through the interface unit according to preset alarm parameters or an alarm plan automatically generated after artificial intelligence deep analysis and learning of the central control processing unit, sending the alarm information to local alarm equipment or third-party equipment, and uploading the alarm information to a central server through a network port for summarizing alarm or alarm triggering; the central control processing unit sends the panoramic mosaic video image, the alarm information of the IPC infrared thermal imaging camera, the video, the image, the data collected by the multi-element sensor and the alarm information to a central management device or a mobile terminal through the communication module, and a maintainer checks real-time monitoring information through a rear end platform of the mobile terminal, checks the states of all online devices in an access local area network, the positions of fault devices and fault conditions and then quickly repairs the on-line devices; the central management device receives videos, images, data and various data and alarm information acquired by the multi-element sensors and sent by each front-end artificial intelligent multi-element panoramic monitoring detection device, N front-end artificial intelligent multi-element panoramic monitoring detection devices are cascaded and associated to acquire front-end data, a large data cloud processing platform is used for carrying out on-site five-dimensional graphic display, full-path traffic state video preview roaming, actual environment simulation, simulation and playback to obtain an environment simulation result, and managers, rescue personnel and fire fighters are informed in the form of graphics and sound to take maintenance measures; the central management device carries out on-site five-dimensional graph real-time display through a big data cloud processing platform, and the method comprises the following steps: the method comprises the steps of on-site five-dimensional graphic display, actual weather perception environment simulation dynamic display, traffic state preview, event accident prejudgment, full-path video splicing three-dimensional infinite roaming viewing, event accident occurrence position real-time positioning and information gathering and uploading.

Furthermore, each front-end artificial intelligence multi-element panoramic monitoring and detecting device adopts a group of variable-focus or fixed-focus cameras to realize 180-degree or 360-degree panoramic splicing and fusion; the IPC camera adopts an infrared thermal imaging camera with the functions of temperature detection and region demarcation.

Furthermore, a group of zoom cameras adopts an intelligent cascade adjustment technology, wherein the intelligent cascade adjustment technology comprises the adjustment of parameters of focal length, aperture, shutter, white light balance, exposure control, minimum illumination, wide dynamic, orientation and coding format of the cameras;

when the parameters of any camera are manually adjusted, the parameters of other cameras are automatically and synchronously changed, including forward increase or backward decrease according to preset programs and parameters, so as to ensure the integrity of the splicing and fusion of the 180-degree or 360-degree panoramic video images.

Further, the set of zoom cameras includes: 3 zoom cameras, wherein the central control processing unit outputs 5 sets of video streams:

group 1: a video stream of a first zoom camera; group 2: a video stream of a second zoom camera; group 3: a video stream of a third zoom camera; group 4: a 180 DEG or 360 DEG panoramic stitched video stream formed by the first to third zoom cameras; group 5: video streams of IPC infrared thermal imaging cameras;

in order to reduce network bandwidth occupied by equipment, the 1 st group to the 3 rd group of video streams adopt a passive calling mode, the system defaults to actively output the 4 th group of video streams and the 5 th group of video streams, and a third-party system ensures that the 4 th group of video streams and the 5 th group of video streams cannot be interrupted when calling the 1 st to 3 rd group of video streams.

Further, the interface unit includes: a power supply interface and an input-output interface, wherein,

the power supply interface adopts a 48VPOE power supply interface; the input-output interface includes: IO interface, audio input interface, audio output interface, CAN interface, RS485 interface and 100/1000M self-adaptation RJ45 interface.

Further, the comprehensive gas sensor is used for collecting the gas concentrations of sulfur dioxide, carbon monoxide, methane, formaldehyde, natural gas and liquefied gas.

Further, the central control processing unit includes: the device comprises a storage unit, a clock unit, a GPU (graphics processing Unit), a multi-core high-speed ARM (advanced RISC machine) processor, a multi-core high-speed DSP (digital signal processor) and a central control logic operation and processing unit; the communication module adopts three types, namely a wireless communication module, a 4G communication module and an 100/1000M self-adaptive network port communication module.

Furthermore, after the image information and the data information collected by each sensor are obtained in real time, the system starts big data analysis and simulates the obtained data in a graph mode to provide basic guarantee for commanders in a monitoring center and emergency rescue and relief workers, in addition, the system finally forms a display graph with five-dimensional real-time dynamic state for decision, positioning, checking, full-path roaming and preview use of operators by regularly self-learning and data summarization of various data, takes a three-dimensional map as a bottom layer preview mode and combines management strategies and periodic action schemes of different time periods, different meteorological conditions, different traffic states, real-time panoramic video images, different places and different hazard degrees, once an alarm critical point is reached or an emergency occurs, the system immediately sends alarm information and previews the information for related workers in a five-dimensional view display mode, so that corresponding rescue measures can be taken as soon as possible, and the life is saved with less damage.

Further, the center management apparatus includes: the system comprises a management workstation and a private cloud server, wherein the private cloud server is arranged in a monitoring sub-center machine room and executes the following work: the method comprises the following steps of data exchange, a storage function, system communication, maintenance, building of each function of the system, parameter setting, centralized alarm, plan generation, artificial intelligence deep learning analysis, linkage scheme execution issuing, three-dimensional map generation, five-dimensional graph generation, field environment data summarization, field environment simulation, big data macroscopic operation, video monitoring, image storage, alarm event video storage, report production, full-path video image secondary splicing, full-path video image viewing, roaming, preview generation, calling of video images and third-party linkage instruction sending, preset scheme sending and daily operation human-computer interface centralized alarm; the management workstation is arranged on a control platform of the monitoring sub-center and executes the following work: the method comprises the following steps of data exchange, system communication, centralized alarming, plan generation, linkage scheme issuing execution, five-dimensional graph real-time display, image display, big data macro display, video monitoring, full-path video image viewing, roaming, preview generation, video image calling, alarm event video recording and checking, report production, third-party linkage instruction sending, preset scheme sending, daily operation, human-computer interface interaction and centralized alarming.

The artificial intelligence multi-element panoramic monitoring detection-based five-dimensional early warning system provided by the embodiment of the invention has the following beneficial effects:

1) the product can output 5 groups of video streams to the outside through a gigabit network port of the product, and each group of video streams can support 5 clients to access simultaneously; the system comprises 3 independent camera video streams for monitoring different angles and different directions, a 180-degree or 360-degree panoramic seamless splicing video stream and an infrared thermal imaging camera video stream. The 5 video streams can all work independently and are sent to different clients to be used for different function detection or graphic analysis and storage calling.

2) The 4 cameras (3 zoom cameras and 1 PLC infrared thermal imaging camera) can carry out independent parameter setting modification or receive an overall modification setting scheme in a cascading mode according to actual use conditions, and intelligent automatic parameter linkage adjustment can also be realized.

3) The coding and communication modes of the video stream conform to the coding and communication modes of the international or domestic mainstream, the unique encryption coding format and communication mode of the user can be adopted according to the user requirements, and the image quality can be switched or called to be checked randomly from 200 ten thousand pixels to 2400 ten thousand pixels.

4) All the 5 groups of video streams are output through one gigabit network port, and each parameter can be managed, set and modified remotely.

5) The equipment has the functions of abnormal event detection (pedestrians, vehicles stop against traffic regulations, traffic accidents, retrograde motion, object throwing, congestion and queuing), license plate snapshot (overspeed, slow motion, lane change against traffic regulations and path identification), dangerous and suspicious vehicle detection, fire detection, object high and low temperature detection, icing detection on a dark ice road surface, detection of various abnormal weather disasters, harmful gas, combustible gas and the like, and can also add other functions such as face identification, perimeter precaution and the like based on image analysis and processing according to user demands.

6) Three cameras all adopt starlight level customized imaging devices with ultralow illumination target surfaces of 1/1.8' to realize effect presentation and dark environment backlight control; the remote micro-focusing and precise aperture adjusting functions are realized, and the working mode of 'black under light' of a single camera or a plurality of spliced cameras is thoroughly solved by adopting a special physical hardware combination and a mechanical framework.

7) And the seamless splicing mode is adopted to ensure that the spliced pictures are more smooth, continuous and uninterrupted.

8) Two or N continuous artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system front-section equipment can be combined together through a network, when certain installation requirements (such as spacing distance, height and the like) are met, the 180-degree or 360-degree panoramic video images sent by each front-section equipment are subjected to wider-range image splicing and fusion, and therefore the condition that the vehicles are in a tunnel, outside the road or in a monitored continuous area range without losing image information is guaranteed. The secondary splicing and fusion technology can realize the continuous monitoring and detection of long distance, whole road section and large area without blind angles and the tracking of individual objects, vehicles and pedestrians and the analysis of motion tracks, and even can realize a full-scene 3D tracking, monitoring and detecting mechanism.

9) The 180-degree panoramic stitching fusion adopts a multi-focal-length compound eye stitching mode, 3 cameras adopt variable-focus lenses with different focal lengths to monitor objects with different distances, and the equipment stitches and fuses 3 video images together through a specific graph stitching algorithm to form a complete monitoring picture.

10) The 3 cameras adopt an intelligent cascading focusing technology, when the focal length of any one of the 3 cameras is manually adjusted, the focal lengths of the other two cameras can be automatically and synchronously changed (positively increased or reversely decreased through a preset processing mechanism and a linkage algorithm), and the output panoramic stitching fusion image is not influenced.

11) The 3 cameras adopt an intelligent cascade adjustment technology, and when important parameters of any one camera, such as color reduction, white balance, ultralow illumination, wide dynamic, exposure intensity, shutter speed, video coding format, color uniformity and the like, are considered to be adjusted, parameters corresponding to the other two cameras can be automatically and synchronously changed (positively increased or negatively decreased through a preset processing mechanism and a linkage algorithm).

12) The 3 cameras appear as a whole, and the wide-angle image distortion of the three cameras needs to be corrected and output; the exposure, white balance, color uniformity and color restoration adopt a central wide-angle camera (camera 2) as a reference mechanism, and when image parameters of the central wide-angle camera (camera 2) are adjusted, other two cameras can be synchronously and automatically adjusted so as to meet the image splicing requirement when different light incoming quantities are met and achieve the optimal image splicing effect.

13) When any one front-end device in the network cluster causes interruption of wired communication or interruption of wired power supply due to special reasons, the system immediately starts an energy-saving mode and an ad hoc network mode and depends on a storage battery of the front-end device to perform maintenance work, and disconnection is avoided within a range of 24 hours. Meanwhile, the system sends alarm information to the end user in a 4G (future support for 5G mode or higher wireless transmission mode).

14) The user can check and access all online equipment states, positions of fault equipment and fault conditions in the cluster network in real time through a mobile phone APP or a computer, and then quick repair is carried out.

15) When fire or explosion occurs in the tunnel, one or more devices in the tunnel are damaged, even when a power supply system or a communication system in the tunnel cannot work, a fire fighter or a rescue worker can access the wireless network of the device through a notebook or a mobile phone at the entrance and exit of the tunnel, and immediately start an ad hoc network and an energy-saving mode in the tunnel for the device with communication interruption or power supply interruption, so that the situation of live images (the image form is updated once every 5-60 seconds, and the time can be adjusted) in the tunnel can be checked in real time under the continuous 24-hour low-power-consumption mode, and precious time is provided for rescue and rescue of the fire fighter.

16) The system collects various data such as multi-element sensors and comprehensive gas sensors, the working state of the system (temperature, voltage, current, communication condition, on-line or damage condition of each sensor and mechanical fault condition) and compressed video images or thumbnails; the wireless network or the 4G communication module (supporting a 5G mode or a higher wireless transmission mode in the future) transmits the information to a device manager or maintainers and leaders, and the personnel can master the device condition and the field condition at any time and any place by the method, so that the online management of mobile office demands can be easily realized, and the effects of quick checking, quick knowing, quick device troubleshooting, quick repairing and the like can be realized.

17) Under the condition that a field maintenance worker of the equipment does not approach the equipment, the detailed fault condition of the equipment can be preliminarily judged by using a mobile phone APP through a flashing mode or 4G communication (a 5G mode or a higher wireless transmission mode is supported in the future) of an indicator light of the equipment, and even the detailed fault condition is accurate to main information such as the damage degree of each sensor or mechanical parts.

18) The comprehensive sensor and the multi-element sensor analyze, summarize and judge real-time data acquired on site through front-end equipment to form an early warning scheme or alarm information, transmit the early warning scheme or the alarm information to a central server to perform on-site environment simulation reduction, deeply analyze the real-time data through private cloud big data in a manual way, combine a five-dimensional early warning system display platform to perform real-time display, and send out an alarm to start a corresponding emergency or rescue scheme.

19) The central cloud processing platform carries out real, accurate and real-time environment five-dimensional simulation reduction and simulation (perfectly fusing and displaying the weather conditions of a three-dimensional map, a real-time video image, a certain point, a certain area, a certain road, a certain city and a certain province) on a large amount of data sent by N pieces of front-end equipment through deep learning and analysis, provides basic data, graphic information, image information and detailed plans for road congestion, traffic accidents, perimeter precaution, natural disasters, emergency rescue and relief, emergency accident processing, command scheduling, early warning precaution, fire rescue and the like, and provides correct precious time for related personnel and leaders.

20) The central management server can remotely set, revise, acquire and issue a brand new scheme and the like for the front-end equipment.

21) The device can comprehensively analyze through the visible light camera, the infrared thermal imaging camera, the temperature sensor and the gas sensor, and provides a more friendly visual graphic alarm viewing interface, and in addition, the sensors can perform area division on the detected object and the area needing to be detected, so that the defects of misinformation, missing report, invisible alarm, inaccurate positioning, invisible and few reference quantity of the traditional device are greatly reduced, and the reaction time of the abnormal event to be generated is prolonged. And an integrated solution is provided for more abnormal events, such as the alarm temperature of a high-temperature object, the alarm temperature of a low-temperature object, the duration time of the temperature of the detected object, the spread range, the characteristics of flame or dark ice and the like, so that the investment of repeated equipment and unnecessary equipment is reduced, and the waste of a large amount of invisible resources is reduced. The device can detect open fire, high-temperature objects and dark ice in tunnels, roads, bridges, perimeter areas, airports, plants and specific areas in real time, and the sensor has a detection and alarm function and plays a role in perspective monitoring when a fire disaster occurs in the tunnels or a common camera raising dense smoke is blinded. The method provides accurate real-time video images for tunnel firefighters, rescue workers and managers. When the equipment is installed in an external field, the equipment also plays a role of perspective night vision when the common camera is blinded at night, in dense fog, haze, heavy snow and heavy rain.

22) The device can also provide a visual solution for early warning and alarming fire, high temperature, low temperature, dark ice and icy road whether in the day or at night and under dense smoke haze

23) Because the equipment has the functions of splicing and tracking the whole path video, the equipment can accurately position, identify and playback the driving path of the vehicles or pedestrians driving on the expressway, and can provide effective guarantee for accurate path charging

24) After the front-end equipment detects an abnormal event and deeply analyzes various data collected by the sensor, the alarm plan is automatically generated in a harm degree priority mode, alarm information is output through an IO (input/output), R485 or CAN (controller area network) interface and sent to local alarm equipment or third-party equipment, and the alarm information is uploaded to a central server through a network port to be subjected to summary alarm or alarm triggering, so that a full-automatic unmanned intervention integrated solution or mechanism is formed.

23) The full path panoramic image video event detection cost is one third of the cost of a common single camera; the cost of the full path panoramic image vehicle violation snapshot is one third of the cost of a common single camera.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a structural diagram of a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection according to an embodiment of the invention;

fig. 2 is a hardware logic architecture diagram of a front-end device of a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection according to an embodiment of the present invention;

fig. 3 is an architecture diagram of a front-end device of a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a front-end device of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system, in which a group of variable-focus cameras implement 180 ° or 360 ° panoramic video image stitching, automatic synchronization, cascade adjustment and focusing technologies according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a front-end device of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system and a third-party fire alarm device according to an embodiment of the invention;

FIG. 6 is a schematic diagram of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system for identifying a vehicle running accurate path according to an embodiment of the invention;

FIG. 7 is a schematic diagram of a tunnel fire, smoke, harmful gases, and combustible gases detection alarm, perspective or night vision function and link connection of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the invention;

FIG. 8 is a schematic diagram of a night vision perspective function implemented by an artificial intelligence multi-element panoramic monitoring and detecting five-dimensional early warning system for detecting dark ice and frozen road surfaces and under severe weather conditions of fog, haze and dust according to an embodiment of the invention;

fig. 9 is a schematic diagram of a front-end device of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the present invention, illustrating a function of displaying super images in a mosaic manner for roads covering a larger area or farther away;

FIG. 10 is a schematic diagram of an artificial intelligence multi-element panoramic monitoring and detecting five-dimensional early warning system for simulating and restoring the on-site environment and weather conditions according to an embodiment of the invention;

fig. 11 is a schematic diagram of a front-end device of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system, panoramic video event detection and violation vehicle snapshot functions according to an embodiment of the invention;

fig. 12 is a link diagram of an interface of a central control processing unit and main components of a front-end device of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system according to an embodiment of the invention;

fig. 13 is a flow chart of artificial intelligence learning of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the present invention;

fig. 14 is a schematic communication diagram of a front-end device and a mobile client of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system according to an embodiment of the present invention;

FIG. 15 is a flowchart of the operation of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the present invention;

fig. 16 is a schematic diagram of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system in fire monitoring according to an embodiment of the invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to explain the manner and meaning of implementing the invention, but may be implemented in other ways or by hardware structures in practical product applications and should not be construed as limiting the invention.

The invention provides a five-dimensional early warning system based on artificial intelligence multi-element panoramic monitoring detection, wherein front-end equipment integrally adopts an embedded system architecture, and the main body adopts a shell protection grade not lower than IP 66. The hardware architecture building of the scheme can be realized in various ways, including: RAM + DSP, RAM + GPU, RAM + DSP + GPU, RAM + FPGA, RAM + DSP + FPGA, etc., or other hardware architecture combination processing manners, but is not limited to these manners, architectures, or implementation processes. The invention adopts RAM + DSP hardware architecture as an example for description, and the scheme does not describe the product appearance, mechanical architecture, transmission device and sensor parameter selection in detail, and the part is described in detail in the patent of the special invention. As shown in fig. 2 and 3, the multi-element sensor can be added, removed or replaced according to the actual use requirements of the user and the application environment (such as important areas or infrastructures of expressways, tunnels, bridges, airports, city expressways, provincial roads, national roads, perimeter security, oil fields, forest fire prevention, frontier defense, factory buildings, etc.), and the types of the sensor are not limited to the following types.

As shown in fig. 1, the artificial intelligence-based multi-element panoramic monitoring and detection five-dimensional early warning system according to the embodiment of the present invention includes: at least one front-end device 1 and a central management device (namely, a five-dimensional early warning and alarming management platform) 2, wherein each front-end device 1 comprises a central control processing unit, a camera set (comprising 3 variable-focus cameras and 1 IPC infrared thermal imaging camera) connected with the central control processing unit, a multi-element sensor, a communication module, an interface unit and a power supply unit.

Referring to fig. 2 and 3, the multi-element sensor may detect the ambient weather environment and the integrated gas detection belongs to the sensing unit. The multi-element sensor includes: the system comprises a brightness light intensity sensor, a temperature and humidity sensor, a rainfall sensor, a wind speed and direction sensor and a comprehensive gas sensor, wherein the brightness light intensity sensor is used for acquiring a brightness light intensity value of the current environment; the temperature and humidity sensor is used for collecting temperature and humidity data of the current environment, the rainfall sensor is used for collecting rainfall values of the current environment, the wind speed and direction sensor is used for collecting wind speed and direction data of the current environment, and the comprehensive gas sensor is used for collecting concentration values of various gases of the current environment. .

Each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section, then various collected environmental data information is sent to the central control processing unit, each front-end artificial intelligence multi-element panoramic monitoring detection device is uploaded to the central management device through the central control processing unit, the actual environment of a simulation site is simulated and fused into a five-dimensional graph to be displayed in real time, and a basis is provided for road condition checking, road rescue, environmental disaster prediction and evaluation and emergency scheme generation.

In one embodiment of the invention, the integrated gas sensor is used to collect the gas concentrations of sulfur dioxide, carbon monoxide, methane, formaldehyde, natural gas, liquefied gas.

The invention supports supporting access: the system comprises a 1-path composite gas sensor (serial port UARTTTL3.3v), a 1-path temperature and humidity sensor (R485 interface), a 1-path wind direction and wind speed sensor (R485 interface), a 1-path air quality sensor (serial port UARTTTL3.3v) and a 1-path brightness and light intensity sensor (I2C interface).

Data information is reported by port data of each sensor in a round-robin mode, a local equipment data processing unit collects the uploaded data, the data are uploaded to an artificial intelligent multi-element panoramic monitoring and detecting five-dimensional early warning system of a main center through a network to be analyzed in real time, the system simulates and restores the field meteorological conditions in a real-time dynamic graph mode through the acquired data of various sensors and is fused into a three-dimensional map, and a brand new video image and real-time data which are spliced and fused with a full-path video image are jointly completed to realize a human-computer interaction interface of the five-dimensional early warning system. And secondly, the local data processing unit carries out logical operation according to various setting instructions issued by the upper end center, and when the preset judgment condition is met, alarm information is output through an IO port and a CAN port.

The highway tunnel or urban tunnel and other tunnels such as air-raid shelter, power supply corridor and the like are all of closed structure, most of the tunnels are in one-out passing mode, wherein the special passing tunnel for the highway or the urban tunnel crossing the river generally comprises a car-avoiding hole, a fire-fighting facility, an emergency communication facility, a water-proof and drainage facility, a ventilation facility and a lighting facility for the safety of drivers and passengers, once a fire disaster happens in the tunnel, if the fire disaster happens, the economic loss and a large number of casualties can be caused, if the fire disaster does not give an early warning in time and an effective measure is taken for rescue or refuge, the fire disaster happens, high temperature, dense smoke, sulfur dioxide, carbon monoxide, volatile matters and other harmful gases exceed the standards, all the people in the tunnel are directly damaged, and in addition, when dangerous vehicles drive into the tunnel and gas leakage (such as methane) occurs, Combustible gases such as formaldehyde, natural gas, liquefied gas and the like) and can easily explode when the concentration exceeds the standard, thereby causing greater disasters to tunnels or personnel.

The comprehensive sensor and the multi-element sensor analyze, summarize and judge real-time data acquired on site through the central control processing unit, form an early warning scheme or alarm information, transmit the early warning scheme or the alarm information to the central server for environment simulation and demonstration, and send out an alarm to start a corresponding emergency or rescue scheme. The central management device 2 can remotely set, revise, acquire and issue a new scheme and the like for the front-end equipment. The central control processing unit is connected with a local fire fighting system, an alarm device and other third-party detection devices through an IO interface, a CAN interface or an R485 interface, and outputs data or switching value to the local alarm device or the third-party devices.

The central management device 2 can cascade and associate N front-end devices to obtain front-end data, form a real-time dynamic cloud picture by combining data acquired by various sensors through a big data cloud processing platform, perform all-dimensional fusion with a three-dimensional map, a panoramic spliced video image and dynamic data, perform simulation and restoration of an actual environment on site, realize functions of five-dimensional early warning management, viewing, positioning, tracking and the like, and inform managers, rescue personnel and firefighters in the form of graphics and sound. And various rescue schemes can be provided according to the simulation result.

Referring to fig. 2 and 3, each front-end device employs a group of zoom cameras (which implement a 180 ° or 360 ° panoramic stitching function, provide real-time large-range video road condition viewing, provide various monitoring and detection functions based on video image analysis), and IPC infrared thermal imaging cameras (which implement high-temperature object detection, road dark ice detection, fire early warning and alarm detection, night vision function, and the like, and belong to sensing units). Each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section and respectively collects image information of different positions on the road section, wherein each camera adopts a variable-focus lens with different focal lengths to monitor objects with different distances and collects corresponding image information, then the collected image information is sent to a central control processing unit, each front-end artificial intelligence multi-element panoramic monitoring detection device is uploaded to a central management device by the central control processing unit, secondary video images are spliced and fused into a five-dimensional graph to be displayed in real time, and functions of abnormal accident detection alarm, positioning, traffic state preview and evaluation, full-path video viewing roaming and the like are achieved.

In one embodiment of the invention, a set of zoom cameras comprises: comprising 3 zoom cameras. Namely, each front-end artificial intelligent multi-element panoramic monitoring and detecting device adopts a group of variable-focus or fixed-focus cameras to realize 180-degree or 360-degree panoramic splicing and fusion; the IPC camera adopts an infrared thermal imaging camera with the functions of temperature detection and region definition. Wherein, the central control processing unit outputs 5 sets of video streams:

group 1: a video stream of a first zoom camera;

group 2: a video stream of a second zoom camera;

group 3: a video stream of a third zoom camera;

group 4: a 180 DEG or 360 DEG panoramic stitched video stream formed by the first to third zoom cameras;

group 5: video streams of IPC infrared thermal imaging cameras;

The artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system front-end equipment is installed at a preset position in a road section, image information on the road section is collected, each camera adopts a variable-focus lens with different focal lengths to monitor objects with different distances, corresponding image information is collected, and the collected image information is sent to the central control processing unit.

The intelligent multi-element panoramic monitoring detection five-dimensional early warning system front-end equipment adopts an intelligent cascade adjustment technology for each zoom camera (including important parameters such as focal length, aperture, shutter, white light balance, exposure control, minimum illumination, wide dynamic, azimuth, coding format and the like of the camera), when the parameters of any camera are manually adjusted, the parameters of other cameras can be automatically and synchronously changed (forward increase or reverse decrease is carried out according to preset programs and parameters), the integrity of splicing and fusion of 180-degree or 360-degree panoramic video images is ensured, and the continuous monitoring effect without blind angles in large areas and whole road sections can be realized.

1) 5 groups of video streams are output through a gigabit network port of the central control processing unit, and each group of video streams can support 5 clients to access simultaneously;

zoom camera 1 video stream-group 1;

zoom camera 2 video stream-group 2;

zoom camera 3 video stream-group 3;

the system comprises a zoom camera 1, a zoom camera 2 and a zoom camera 3, wherein the three cameras output a video stream-group 4 after being subjected to 180-degree or 360-degree panoramic stitching fusion;

the 1-3 groups of video streams are output in a passive calling mode;

the 3 zooming cameras can be independently accessed and parameter set, and intelligent automatic parameter linkage adjustment can also be realized;

the fixed-focus IPC camera video stream-group 5 (IPC infrared thermal imaging camera) is directly output through an onboard gigabit network port without any image processing, but the camera can be remotely accessed and set with functional parameters.

The IPC infrared thermal imaging camera is used for realizing the detection of high-temperature objects, the detection of road dark ice, the early warning and alarm of fire and providing a night vision perspective function, firstly demarcating an area to be detected, then detecting objects, pedestrians, roadbed pavements and the like in the detected area in real time, and playing a role in night vision perspective monitoring when a common camera is blind at night or in dark environment; each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section, an IPC infrared thermal imaging camera respectively collects objects, pedestrians, roadbed and pavement data information and image information at different positions on the road section, then the collected data information is sent to a central control processing unit, each front-end artificial intelligence multi-element panoramic monitoring detection device is uploaded to a central management device through the central control processing unit, centralized gathering analysis is carried out, the data information is fused into a five-dimensional graph to be displayed in real time, and the functions of abnormal accident detection alarm, positioning, traffic state preview and evaluation and night vision perspective are achieved.

The system defaults to actively output the 4 th group of video stream and the 5 th group of video stream, and when the third-party system calls the 1 st to 3 rd group of video streams, the 4 th group of video stream and the 5 th group of video stream cannot be interrupted, and the number of video output frames cannot be less than 30 frames.

2) Camera analysis and detection function (Central processing Unit can carry out remote parameter setting and data reporting)

The camera 1 has the functions of parking and pedestrian analysis and detection, and the camera 3 has the function of license plate snapshot. The detection function can output alarm information, pictures and videos through the gigabit network port.

Three cameras all adopt starlight level customized imaging devices with ultralow illumination target surfaces of 1/1.8' to realize effect presentation and dark environment backlight control; the functions of remote micro-focusing and accurate aperture adjustment are realized, the working mode of 'black under lamp' of a single camera or a plurality of spliced cameras can be solved, and the spliced images are more smooth, continuous and uninterrupted by adopting a seamless splicing mode. The invention can combine two or N continuous artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system front-section devices together through a network, and when certain installation requirements (such as spacing distance, height and the like) are met, the 180-degree or 360-degree panoramic video images sent by each front-section device are subjected to larger-range image splicing and fusion, so that the vehicles can be ensured not to lose image information in tunnels, road outfields or monitored continuous area ranges. The secondary splicing and fusion technology can realize the continuous monitoring and detection of long distance, whole road section and large area without blind angles and the tracking of individual objects, vehicles and pedestrians and the analysis of motion tracks, and even can realize a full-scene 3D tracking, monitoring and detecting mechanism.

The 180-degree or 360-degree panoramic stitching fusion mode adopts a multi-focal-length compound eye stitching mode, 3 cameras adopt focusable lenses with different focal lengths to monitor objects with different distances, and the central control processing unit stitches and fuses 3 video images together through a specific graph stitching algorithm to form a complete monitoring picture.

The 3 cameras adopt an intelligent cascading focusing technology, when the focal length of any one of the 3 cameras is manually adjusted, the focal lengths of the other two cameras can be automatically and synchronously changed (positively increased or reversely decreased through a preset processing mechanism and a linkage algorithm), and the output panoramic stitching fusion image is not influenced.

The 3 cameras adopt the third technology of intelligent cascade color restoration, white balance, exposure and color uniformity adjustment, the 3 cameras appear as a whole, and the wide-angle image distortion of the three cameras needs to be corrected and output; the exposure, white balance, color uniformity and color restoration are performed by taking a wide-angle camera (camera 2) as a center, and when image parameters of the center camera or other two cameras are adjusted, the other two cameras can be synchronously and automatically adjusted to meet the image splicing requirements (shown in figure 4) at different light incoming quantities so as to achieve the optimal image splicing effect.

3) Code stream compression mode of camera

The encoding format of the video stream output by the encoding module of the central processing unit and the communication protocol conform to the common format of the international and national relevant standards, and can be individually customized according to the project requirements. The 4 th group code stream is 30 frames, and the super-high definition video with 200-2400 ten thousand pixels.

The IPC camera carries out region division on the detected object and the region needing to be detected, carries out real-time detection on various objects and dark ice by the IPC camera, carries out detection alarm, and plays a role in perspective monitoring when a common camera is blinded. The IPC camera realizes the functions of fire, high temperature, low temperature and road icing detection alarm, perspective and night vision.

The IPC camera forms an independent video stream 5, the camera is an infrared thermal imaging graphic sensor, firstly, the sensor carries out region planning on a detected object and a region needing to be detected, and sets various alarm parameters such as high-temperature object alarm temperature, low-temperature object alarm temperature, detected object temperature duration, swept range, flame or dark ice characteristics and the like on the sensor, after the setting is finished, the sensor can carry out open fire, high-temperature object and dark ice real-time detection on the tunnel, a road, a bridge, a perimeter region, an airport, a factory building and a specific region, besides the sensor has a detection alarm function, and the sensor plays a role of perspective monitoring when a fire disaster occurs in the tunnel or a common camera raising dense smoke is blinded. The method provides accurate real-time video images for tunnel firefighters, rescue workers and managers. When the equipment is installed in an external field, the equipment also plays a role of perspective night vision when the common camera is blinded at night, in dense fog, haze, heavy snow and heavy rain.

The IPC camera can also provide a visual solution for early warning and alarming of fire, high temperature, low temperature, dark ice and road icing in daytime or at night and under dense smoke haze. The central control processing unit can independently output IPC camera video stream to the server or the management platform for centralized management through the gigabit network port

The center can independently access the IPC camera through the central control processing unit and can set various parameters, acquire data and analyze results.

When the central control processing unit detects that the IPC camera sends alarm information, and combines various data collected by the local sensor, the central control processing unit outputs the alarm information to be sent to local alarm equipment or third-party equipment through an IO (input/output), R485 or CAN (controller area network) interface according to preset alarm parameters or an alarm plan automatically generated by the central control processing unit, and the alarm information is uploaded to a central server through a network port to perform summary alarm or alarm triggering.

The hardware interface characteristics and the operation mode of the central control processing unit are further described below, and the RAM + DSP hardware architecture is used as an example for description, but not limited to this mode, architecture or implementation process.

The central control processing unit adopts a high-speed RAM board, the high-speed RAM board runs various algorithms, and controls parameter setting, storage, analysis, alarm output power-off memory, power-on self-test, self-starting, local storage of data network interruption, and automatic uploading to the central management device after the network is connected.

For example, the integrated gas sensor and environmental sensor transmit real-time data collected on site to the ARM board, which analyzes, aggregates and uploads the data to a central server (i.e., central management device). And when the data exceeds the standard and the alarm is needed, the high-speed RAM board outputs alarm information through an IO or RS232 interface and sends the alarm information to local alarm equipment or third-party equipment.

The vibration sensor can transmit real-time data collected on site to the ARM board, and the ARM board analyzes, gathers and uploads the data to the central server. And the high-speed RAM board outputs alarm information through an IO or RS232 interface and sends the alarm information to local alarm equipment or third-party equipment.

The high-speed RAM board can independently output video streams of the cameras to a server or a management platform through the gigabit network port, and the center can independently access the cameras through the high-speed RAM board and can set various parameters to acquire data and analysis results. And the high-speed RAM board outputs alarm information through an IO or 232 interface and sends the alarm information to local alarm equipment or third-party equipment.

The present invention was based on the AM5728 embedded processor of texas instruments, usa, and a central control processing unit hardware device was designed and developed. The hardware system technology is as follows:

1. support 2-path gigabit Ethernet communication, RJ45 interface;

support for network protocols: IPv4/v6, HTTP, HTTPS, FTP, CIFS/SMB, SMTP, DNS, DynDNS, NTP, RTSP, RTP, TCP, UDP, IGMP, RTCP, ICMP, DHCP, ARP, and SOCKS; the standard ONVIF2.4 protocol is supported.

2. Supporting 1 path of microphone to input 3.5mm JACK MIC-IN;

3. supporting 1-channel audio output of 3.5mm JACK Line-Out;

4. the system supports 1-path band address (the system leaves factory and is unified and defaulted to be a unified address code which CAN be modified and edited through software, and the address code is not lost after power failure) CAN ports, supports 1-path trunk node IO (supports 24V/1A) output (IPC outputs signals, and simultaneously triggers the CAN ports and the IO ports to output through AM5728 board level conversion);

5.1 RS485 serial port is supported;

6. 1-path wireless and 1-path 4G modules are supported;

7. 5 groups of video streams are output through the AM5728 board kilomega network port, and each group of video streams can support 5 clients to access simultaneously

Zoom Camera 1 video stream-group 1

Zoom Camera 2 video stream-group 2

Zoom Camera 3 video stream-group 3

Zoom camera 1, zoom camera 2, zoom camera 3 three cameras 180 ° stitched video stream-group 4

1-3 groups of video streams are all output in a passive calling mode

All 3 zoom cameras can be individually accessed and parameter set

Fixed focus camera 5 video stream-group 5 (IPC camera), output directly through onboard gigabit portal without any image processing, with independent access and functional parameter settings for the camera

And the system actively outputs the 4 th group video stream and the 5 th group video stream by default, and when the third-party system calls the 1 st to 3 rd group video streams, the 4 th group video stream and the 5 th group video stream are ensured not to be interrupted, and the video output frame number is not less than 25 frames.

8. And (3) executing self-diagnosis state detection, wherein the self-diagnosis state detection comprises 3 fixed cameras, 4 sensors, 1 IPC camera, and important information such as board power supply voltage, temperature, network transmission working state and the like, uploading the important information to a third party platform through a gigabit network port in real time or at regular time, and once an abnormal component fails, giving an alarm and informing the information of the abnormal component and a damaged part. The data can also be self-diagnosed by the APP viewing system through a local wireless (wireless or Bluetooth) connection, and the function only allows a user to read the data and does not allow the modification or access to the internal network of the device.

9. The equipment needs high and low temperature and lightning protection treatment. The requirements are clear for high and low temperature working and storage ranges and lightning protection grades, and the POE power supply is supported.

In summary, the central control processing unit can at least perform the following functions: the central control processing unit uploads each front-end artificial intelligent multi-element panoramic monitoring detection device to the central management device, and the front-end artificial intelligent multi-element panoramic monitoring detection device simulates and simulates the actual environment (such as rain, snow, fog, wind, sunshine and the like) of a site and is integrated into a five-dimensional graph to be displayed in real time, so that important basic basis is provided for road condition viewing, road rescue, environmental disaster prediction and evaluation and emergency scheme generation. The central control processing unit uploads each front-end artificial intelligence multi-element panoramic monitoring detection device to the central management device, secondary video images are spliced and merged into a five-dimensional graph to be displayed in real time, and functions of abnormal event accident detection alarm, positioning, traffic state preview and evaluation, full-path video viewing roaming and the like are achieved. The central control processing unit uploads each front-end artificial intelligent multi-element panoramic monitoring and detecting device to a central management device (five-dimensional early warning and alarming management platform), centralized collection and analysis are carried out, and the devices are integrated into a five-dimensional graph to be displayed in real time, so that the functions of abnormal accident detection and alarm, positioning, traffic state preview and evaluation, night vision perspective and the like of abnormal vehicles, objects, pedestrians, roadbed pavements and the like are realized

In one embodiment of the present invention, a central control processing unit includes: the device comprises a storage unit, a clock unit, a multi-core high-speed ARM processor, a multi-core high-speed DSP processor, a high-speed GPU processor and a central control logic operation and processing unit; the communication module adopts three types, namely a wireless communication module, a 4G communication module and an 100/1000M self-adaptive network port communication module. The central control processing unit is used for carrying out splicing fusion and data acquisition, logical operation and logical control output on the received image information according to a splicing algorithm to form a 180-degree or 360-degree panoramic spliced image and a full-road section blind-angle-free continuous monitoring super spliced image, and combining various data acquired by the multi-element sensor when receiving alarm information from the IPC camera, outputting the alarm information through the interface unit according to preset alarm parameters or an alarm plan automatically generated by the central control processing unit, sending the alarm information to local alarm equipment or third-party equipment, and uploading the alarm information to the central server through the internet access for summarizing alarm or alarm triggering; the central control processing unit sends the compressed panoramic stitching image, the super stitching image and data and alarm information acquired by the multi-element sensor to the mobile terminal through the communication module, and a user checks real-time monitoring information through the mobile terminal, checks and accesses all online equipment states, positions of fault equipment and fault conditions in the local area network and then quickly repairs the on-line equipment states, the positions of the fault equipment and the fault conditions. Specifically, the invention supports handset APP remote status query (device operating status indication and wireless/4G communication (future support of 5G communication mode or higher wireless communication mode)). The central control processing unit collects various data such as multi-element sensors and comprehensive gas sensors, the working state (temperature, voltage, current, communication condition, on-line or damage condition of each sensor and mechanical fault condition) of the central control processing unit and compressed video images or thumbnails; the wireless network or the 4G communication module (supporting a 5G communication mode or a higher wireless communication mode in the future) transmits the information to a device manager or maintainers and leaders, and the personnel can master the device condition and the field condition at any time and any place in the way, so that the online management of mobile office demands can be easily realized, and the effects of quick checking, quick knowing, quick device troubleshooting, quick repairing and the like can be realized. Under the condition of not disassembling the machine, the on-site maintenance personnel of the equipment can preliminarily judge the probable fault condition of the equipment through the flashing mode of the indicating lamp of the equipment.

In addition, the invention supports that the ad hoc network mode (wireless communication) can be realized after the interruption of wired communication or the interruption of wired power supply. Specifically, after one device in the network cluster is interrupted in wired communication or wired power supply due to special reasons, the system immediately starts the energy-saving mode and the ad hoc network mode and depends on a storage battery of the system to maintain the system, and the device is prevented from being disconnected within 24 hours. Meanwhile, the system sends alarm information to the terminal user in a 4G mode (a mode of supporting 5G communication or a higher wireless communication mode in the future), and the user can check the states of all online devices in the access local area network, the positions of fault devices and the fault conditions in real time through a mobile phone APP or a computer so as to repair the online devices quickly.

When a fire or explosion occurs in the tunnel, one or more devices in the tunnel are damaged, even when a power supply system or a communication system in the tunnel cannot work, a fireman or an ambulance man can access the wireless network of the devices through a notebook at the entrance and exit of the tunnel, so that the fireman or the ambulance man can conveniently conduct and dispatch the scene through the notebook, and check the scene video situation in the tunnel in real time, and the fireman or the ambulance man can visit the wireless network through a mobile phone or the notebook.

The central control processing unit collects data of each sensor, remote parameter setting, local storage, data analysis, data summarization, alarm output, periodically and automatically generating an early warning plan, power-off memory, power-on self-check, power-on self-start, self-diagnosis execution, energy-saving function starting after power-off, self-networking function starting after network disconnection, data breakpoint continuous transmission, pushing a data image to a terminal through a wireless or 4G network (supporting a 5G communication mode or a higher wireless communication mode in the future), network disconnection data local storage, automatic uploading after network connection, running various algorithms including 180-degree image splicing, event detection, license plate snapshot, fire detection, dark ice and road icing detection, connection with third-party alarm equipment, connection with a third-party platform, execution of an AI deep learning function and the like. Fig. 16 is a schematic diagram of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system in fire monitoring according to an embodiment of the invention.

In one embodiment of the invention, the interface unit comprises: the power supply interface and the input/output interface. The power supply interface adopts a 48VPOE power supply interface. The input/output interface includes: the network interface, IO port, audio input interface, audio output interface, CAN interface, RS485 interface and RJ45 interface. Fig. 12 is a link diagram of an interface and main components of a front-end device of an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system according to an embodiment of the invention.

The invention uses cadence16.x software to develop a hardware circuit, and uses a Linux operating system and a TI Processor SDK v3.3 as a software development platform; the programming is implemented using the C/C + + programming language. The invention adopts the scheme of the TI latest embedded processor, and has the characteristics of high performance, low power consumption, flexible configuration and the like. The hardware supports H.264 HighProfile coding, so that the high-definition video obtains the optimal compression ratio and video quality, and the performance of network transmission and hard disk storage is improved. A multi-core, single-chip solution is formed by combining a variety of embedded technologies, including a dual-core ARM Cortex-a15 high-performance ARM processor, and a dual-core C66x floating-point DSP digital signal processor, as shown in fig. 12.

And board card hardware configuration:

1) CORE board CORE57 x:

CPU：TI Sitara AM5728 SOC

a dual-core Cortex-A15ARM core with the highest dominant frequency of 1.5GHz

Double-core C66x floating-point DSP core with dominant frequency of 750MHz at most

RAM：2GByte DDR3

FLASH：8GByte eMMC

Power: dedicated power management

And (3) expanding an interface: 2-path 160PIN board-to-board connector

1-path 40PIN high-speed inter-board connector

2) Interface bottom plate:

the 3-path (3-path FFC36P-0.5mm connector camera module supports BT.1120 interface, separates synchronous signal, outputs YUV422 format) flat cable interface high definition video input, supports 1080P/30fps resolution at maximum,

2-way 1000Mbps Ethernet, RJ45 interface and lamp;

a 1-way gigabit network for transmitting H.264 video streams and data communications;

the 1 path of kilomega network is used for accessing an IPC network camera;

path 1 RS485, phoenix terminal;

1 path of I2C interface, 4pin wiring terminal (VCC 5V power supply);

a 2-path UART TTL3.3V interface and a 4pin wiring terminal (with VCC 5V power supply);

2 paths of RS485 interfaces and 4pin wiring terminals (with VCC 5V for power supply);

1-path miniCICIE interface (expansion 4G)

1 path of UART TTL serial port;

1-way USB 2.0;

the 3-way LED status lamp, the power indicator lamp is a green lamp, (the warning lamp is red, and the status lamp is green) is realized through GPIO;

the 1-path trunk node IO output (the IO alternative output of IPC and AM 5728) supports 24V and 1A load access;

1 path of CAN bus;

1-channel wireless & bluetooth module, TI WL1837MOD (dual-frequency WIFI + BT 4.1);

1-path RTC real-time clock, I2C interface, with battery seat;

1 path TF card interface;

1 path of UART (universal asynchronous receiver transmitter) console 3.3v TTL level, and an onboard 3PIN contact PIN;

1-way DC +48v power input or POE power supply, 3PIN contact PIN

12V voltage required by the camera is provided by an AM5728 board card;

5V voltage required by the sensor is provided by an AM5728 board card;

light sensitive electrically controlled IPC power supply

Board card size: 160mm x 120mm (pending)

3) Video input

The 3-path FFC36P-0.5mm connector camera module supports a BT.1120 interface, separates synchronous signals, outputs YUV422 format and inputs the YUV422 format to a video interface VIP of an AM 5728; the function parameter setting of the camera is realized through a serial port

1 path of IPC network camera inputs H.264 video stream through network port;

4) data storage

An onboard TF card interface for programming system firmware;

5) data communication:

5.1) serial port communication:

1 path UART realizes a1 path UART console system debugging port, and carries a 3PIN contact PIN on board;

a 2-path UART (universal asynchronous receiver/transmitter) external sensor module (an air quality sensor and a composite gas sensor);

the 1-path I2C is externally connected with a sensor module (a brightness and light intensity sensor);

2 paths of RS485 external sensor modules (a temperature and humidity sensor and a wind speed and direction sensor);

the 1-channel RS485 interface is communicated with an upper computer;

1 path of CAN bus for outputting alarm information;

the 3-path serial port is communicated with the 3-path camera module branch of the FFC36P-0.5mm connector to set the functional parameters of the camera

5.2) network communication:

2-path gigabit Ethernet, RJ45 interface, with lamp;

5.3) USB communication:

1-way USB2.0, minippice connector;

5.4) IO control

1-way digital output (dry node IO);

6) system status light

3 way LED status indicator: the power state is green; the warning lamp is red, and the status lamp is green.

7) System start-up mode

The system is booted through the eMMC, and the programming firmware is booted through TF.

8) Real-time clock

1-path independent RTC chip, i2c interface and battery seat; keeping a clock after the system is powered off;

9) system power supply

The board card adopts DC +48V power supply or POE power supply.

The complete system software and hardware architecture of the present invention is shown in table 1 below: the top layer is functional module software, the middle two layers are driving layer software, and the lower two layers are hardware systems.

TABLE 1

The software design of the invention is as follows:

hardware platform transplantation and drive development, embedded U-boot transplantation, embedded Linux4.4 kernel transplantation, embedded Linux file system transplantation and hardware peripheral drive program development.

Wherein, the development of the hardware peripheral driving program comprises the following steps: BT.1120 high-definition video acquisition driver, IPC access acquisition driver, gigabit Ethernet driver, TF card driver, GPIO driver, RS232 driver, USB2.0 driver, RTC real-time clock driver, power management driver, data acquisition, summarization, uploading, alarm logic parameter setting, alarm output, wireless device connection and data transmission, video protocol transmission protocol compilation: pv4/v6, HTTP, HTTPS, FTP, CIFS/SMB, SMTP, DNS, DynDNS, NTP, RTSP, RTP, TCP, UDP, IGMP, RTCP, ICMP, DHCP, ARP, and SOCKS; the standard ONVIF2.4 protocol is supported.

Referring to fig. 5, the central management device receives videos, images, data and various data and alarm information acquired by the multi-element sensor sent by each front-end device, cascades and associates N front-end devices to acquire the front-end data, performs on-site five-dimensional graphic display, full-path traffic state video preview roaming, simulation, and playback of an actual environment through the big data cloud processing platform to acquire an environment simulation result, and informs a manager, rescue personnel, and fire fighters in the form of graphics and sound to take maintenance measures. The central management device carries out on-site five-dimensional graph real-time display through a big data cloud processing platform, and the method comprises the following steps: the method comprises the steps of on-site five-dimensional graphic display, actual weather perception environment simulation dynamic display, traffic state preview, event accident prejudgment, full-path video splicing three-dimensional infinite roaming viewing, event accident occurrence position real-time positioning and information gathering and uploading.

Specifically, the center management device includes: the system comprises a management workstation, a private cloud server and an analog simulation early warning, alarming, commanding and monitoring management platform. The center management device includes: management workstations and private cloud servers.

The private cloud server is arranged in the monitoring sub-center machine room and executes the following work: the method comprises the following steps of data exchange, a storage function, system communication, maintenance, building of each function of the system, parameter setting, centralized alarm, plan generation, artificial intelligence deep learning analysis, linkage scheme execution issuing, three-dimensional map generation, five-dimensional graph generation, field environment data summarization, field environment simulation, big data macroscopic operation, video monitoring, image storage, alarm event video storage, report production, full-path video image secondary splicing, full-path video image viewing, roaming, preview generation, calling of video images and third-party linkage instruction sending, preset scheme sending and daily operation human-computer interface centralized alarm;

the management workstation is arranged on a control platform of the monitoring sub-center and executes the following work: the method comprises the following steps of data exchange, system communication, centralized alarming, plan generation, linkage scheme issuing execution, five-dimensional graph real-time display, image display, big data macro display, video monitoring, full-path video image viewing, roaming, preview generation, video image calling, alarm event video recording and checking, report production, third-party linkage instruction sending, preset scheme sending, daily operation, human-computer interface interaction and centralized alarming.

Specific applications of the artificial intelligence-based multi-element panoramic monitoring and detecting five-dimensional early warning system in different occasions according to the embodiment of the invention are described below with reference to fig. 6 to 15.

(1) Vehicle precision path travel identification

As shown in fig. 6, when a vehicle continuously passes through a plurality of artificial intelligent multi-element panoramic monitoring and detecting front-end devices of the five-dimensional early warning system with license plate snapshot recognition, the system can accurately judge the actual driving route of the vehicle in a punctuation connection mode

(2) The tunnel fire, smoke, harmful gas and combustible gas can be detected and the functions of perspective and night vision can be provided

As shown in fig. 7, the infrared thermal imaging image sensor and the high-definition panoramic camera of the device are used for detecting the flame and smoke states of the open fire, and the data collected by the comprehensive gas sensor and the temperature and humidity sensor are integrally analyzed, so that whether fire, dense smoke, harmful gas, combustible gas and the like occur in the tunnel can be accurately detected, real-time alarming and early warning functions can be performed, and a perspective or night vision image can be provided in the tunnel filled with dense smoke or under the condition that power supply and illumination of the tunnel are failed.

(3) The detection of icing, fog, haze, dust and severe weather of the dark ice on the road surface can provide a night vision perspective function

As shown in fig. 8, the infrared thermal imaging image sensor and the high-definition panoramic camera of the device are used for integrally analyzing the shape, temperature, reflective area and image contrast of the ice-dark and the ice-road surface in combination with various data collected by the temperature and humidity sensor, so that whether the ice-dark and the ice-road surface and the snow-road surface occur in the detected area can be accurately detected, real-time alarming and early warning functions can be performed, the weather conditions around the device can be obtained through analyzing various data collected by the comprehensive gas sensor and the multi-element gas sensor, and the night vision infrared thermal imaging camera is used for providing perspective images at night or in dark light

(4) Super image splicing display function covering larger area or being farther away

As shown in fig. 9, the panoramic mosaic video images output by the front-end device of the N continuous artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system installed at a certain distance and height are output to the comprehensive management platform for secondary fusion and mosaic, so that super mosaic monitoring video images with a larger area and a longer coverage distance can be formed for the use of a monitoring manager.

(5) Simulation function for field environment and weather condition

As shown in fig. 10, N continuous intelligent multi-element comprehensive detection devices installed at a certain distance form a cluster, and a large amount of data acquired by a sensor and a comprehensive gas sensor of each device is used to simulate the actual environment on site through a cloud processing server of a management center, so that the real and accurate weather conditions of a road section or an area can be obtained, and the cluster provides favorable guarantee for emergency rescue and relief work, and rapid processing and response of major accidents.

(6) Panoramic video event detection and violation vehicle snapshot function

As shown in fig. 11, N continuous intelligent multi-element comprehensive detection devices installed at a certain distance and height form a cluster, and after panoramic video images of each device are subjected to secondary fusion and splicing, large-range video event detection and illegal vehicle snapshot are carried out, so that functions of day and night monitoring, monitoring and detection can be realized in a seamless manner within 24 hours day and night, and the traffic state of a road can be marked in a graphical manner in real time.

The artificial intelligence multi-element panoramic monitoring detection-based five-dimensional early warning system can be applied to the following fields: an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system, an artificial intelligence multi-element panoramic monitoring detector, an artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system all-in-one machine, a multi-element monitoring detection early warning all-in-one machine (a camera, a system, a device or a technology), a multi-element panoramic fire monitoring detection early warning all-in-one machine (a camera, a system, a device or a technology), a multi-element panoramic road icing, dark ice monitoring detection early warning all-in-one machine (a camera, a system, a device or a technology), an artificial intelligence multi-element panoramic monitoring detection site environment simulation early warning system (a device or a technology), a device wireless inspection technology, a device power failure and network failure wireless ad hoc network technology, a multi-element panoramic comprehensive monitoring detection site environment simulation reduction technology, a continuous area, Compound eye panoramic stitching technology, multi-element panoramic integrated video event detection monitoring all-in-one machine (camera, system, equipment or technology), multi-element panoramic integrated illegal snapshot monitoring detection all-in-one machine (camera, system, equipment or technology), multi-element panoramic integrated path identification monitoring detection all-in-one machine (camera, system, equipment or technology), multi-element panoramic integrated monitoring detection all-in-one machine (camera, system, equipment or technology) fire smoke detection technology, multi-element panoramic integrated road surface dark ice icing monitoring detection all-in-one machine (camera, system, equipment or technology), multi-element panoramic integrated night vision monitoring detection all-in-one machine (camera, system, equipment or technology), artificial intelligent deep analysis multi-element panoramic integrated stitching integrated monitoring detection early warning alarm technology, artificial intelligent deep analysis multi-element panoramic integrated stitching integrated monitoring detection field environment simulation technology, artificial intelligent deep analysis multi-element panoramic stitching integrated, The system comprises a tunnel rescue site environment simulation and simulation technology, a severe weather environment simulation and early warning technology, an intelligent traffic multi-element site environment simulation and simulation technology, a multi-element environment simulation and simulation technology, an integrated monitoring and detection machine (a camera, a system, equipment or technology), a panoramic spliced video event detection technology, a panoramic spliced vehicle violation snapshot technology, a panoramic spliced vehicle path identification technology, a continuous N-piece panoramic spliced equipment large-area seamless splicing technology and a panoramic spliced camera automatic cascade adjustment technology or scheme.

The artificial intelligence-based multi-element panoramic monitoring and detecting five-dimensional early warning system provided by the embodiment of the invention has the function of artificial intelligence deep learning process. In addition, the system forms management strategies and periodic action schemes in different time periods, different meteorology, different traffic states and different hazard degrees after regularly self-learning and data summarization are carried out on various data, once an alarm critical point is reached or an emergency occurs, the system immediately sends alarm information to related working personnel so as to take corresponding rescue measures as soon as possible and reduce damage to life.

The invention combines a customized multi-element meteorological detection device, a fire alarm detection device and an icy road detection device, collects a large amount of acquired image data, meteorological data, environmental data, gas data and the like, reports the collected data in a most appropriate real-time scheme after AI artificial intelligence deep learning analysis, provides great help for a final user by combining a plurality of receiving means such as images, graphs, data, field simulation and the like, provides a large-range continuous area real-time comprehensive monitoring and detecting device, provides a device with low cost and a plurality of early warning schemes by artificial intelligence analysis, has simple and quick operation, durability, stability and reliability, can realize personalized customization, small volume and wide application range, and becomes a comprehensive monitoring and detecting device in various industries (including a highway, a fire alarm and the like) at present, City traffic, area security, airport perimeter, etc.).

Fig. 13 is a flow chart of artificial intelligence learning of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the present invention.

After image information and data information collected by each sensor are obtained in real time, the system starts big data analysis and simulates the obtained data in a graph mode to provide basic guarantee for commanders in a monitoring center and emergency rescue and relief workers, in addition, the system finally forms a display graph with five-dimensional real-time dynamic state for decision making, positioning, checking, full-path roaming and preview use of operators by regularly self-learning and data summarization of various data, takes a three-dimensional map as a bottom layer preview mode and combines management strategies and periodic action schemes of different periods, different meteorological conditions, different traffic states, real-time panoramic video images splicing and fusion, different places and different hazard degrees, once an alarm critical point is reached or an emergency occurs, the system immediately sends alarm information and watches the preview for related workers in a five-dimensional view display mode, so that corresponding rescue measures can be taken as soon as possible, and the life is saved with less damage.

Fig. 14 is a schematic communication diagram of a front-end device and a mobile client of an artificial intelligence multi-element panoramic monitoring and detection five-dimensional early warning system according to an embodiment of the present invention.

And the mobile phone app or the computer checks and accesses the states of all the devices in the local area network in real time. The staff can use cell-phone APP to pass through the wireless network connection (user, password authentication) of system and acquire all equipment operating condition parameters in the LAN like: working voltage, working current, working state of various sensors, and fault conditions, and viewing video images. The devices can be interconnected through a wired network or a wireless device. The wireless access is generally arranged at two sections of the tunnel, so that rescue workers, fire fighters and leaders can conveniently conduct and dispatch on site, and the site video condition in the tunnel can be checked in real time only by accessing through a mobile phone or a notebook computer.

Fig. 15 is a flowchart of the operation of the artificial intelligence multi-element panoramic monitoring detection five-dimensional early warning system according to the embodiment of the present invention.

Firstly, a system is set up and parameters are set as follows:

(1) and acquiring data of a front-end sensor, and simulating a simulation field environment to generate a dynamic attempt. For example: rain, snow, fog, temperature, wind direction, light, etc.;

(2) acquiring a front-end 180-degree panoramic stitching image, generating a full-path real-time preview video image through secondary stitching fusion, and performing abnormal event accident detection and license plate snapshot information extraction;

(3) acquiring front-end IPC infrared thermal imaging camera images to realize the functions of high-temperature object detection, road surface dark ice, icing detection, fire detection, night vision and perspective;

(4) and acquiring longitude and latitude coordinate information of the front end to generate a three-dimensional coordinate map.

And then, storing important information such as data, audio, video, alarm information, coordinate information, data report forms, license plates, alarm records, vehicle motion tracks and the like in the process.

It should be noted that the above processes can all be deeply learned through artificial intelligence, and automatically generate a plan.

A five-dimensional graphic man-machine interaction interface integrating audio, video, data, coordinates, a three-dimensional map and alarm information is formed by fusing a three-dimensional map generated in the process, a full-path panoramic video preview image, a dynamic attempt generated by simulating a field environment, data and the like. When the data exceeds the standard and an alarm is needed, the front-end device outputs an alarm signal, and meanwhile, the third-party equipment outputs the alarm signal in a linkage manner.

13) When any one front-end device in the network cluster causes interruption of wired communication or interruption of wired power supply due to special reasons, the system immediately starts an energy-saving mode and an ad hoc network mode and depends on a storage battery of the front-end device to perform maintenance work, and disconnection is avoided within a range of 24 hours. Meanwhile, the system sends alarm information to the end user in a 4G (future support of 5G mode or higher wireless transmission mode),

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and not to be construed as limiting the present invention, and that those skilled in the art can make changes, modifications, substitutions and alterations to the above embodiments without departing from the principles and spirit of the present invention. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims

1. The utility model provides a five-dimensional early warning system based on artificial intelligence multi-element panorama monitoring detection which characterized in that includes: at least one front-end artificial intelligence multi-element panoramic monitoring detection device and a central management device, wherein each front-end artificial intelligence multi-element panoramic monitoring detection device comprises a central control processing unit, a group of zoom cameras, an IPC infrared thermal imaging camera, a multi-element sensor, a communication module, an interface unit and a power supply module, wherein the group of zoom cameras, the IPC infrared thermal imaging camera, the communication module, the interface unit and the power supply module are connected with the central control processing unit,

the multi-element sensor includes: the system comprises a brightness light intensity sensor, a temperature and humidity sensor, a rainfall sensor, a wind speed and direction sensor and a comprehensive gas sensor, wherein the brightness light intensity sensor is used for acquiring a brightness light intensity value of the current environment; the temperature and humidity sensor is used for collecting temperature and humidity data of the current environment, the rainfall sensor is used for collecting rainfall value of the current environment, the wind speed and direction sensor is used for collecting wind speed and direction data of the current environment, the comprehensive gas sensor is used for collecting concentration values of various gases in the current environment,

each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section, then various collected environmental data information is sent to the central control processing unit, the central control processing unit uploads the data obtained by each front-end artificial intelligence multi-element panoramic monitoring detection device to a five-dimensional early warning management platform of a central management device, the actual environment of a simulation site is simulated and merged into a five-dimensional graph to be displayed in real time, and a basis is provided for road condition viewing, road rescue, environmental disaster prediction and evaluation and emergency scheme generation;

the group of zoom cameras comprises a plurality of zoom cameras for realizing a 180-degree or 360-degree panoramic stitching function and providing real-time large-range video road condition viewing and various monitoring and detecting functions based on video image analysis, wherein each front-end artificial intelligent multi-element panoramic monitoring and detecting device is arranged at a preset position in a road section and respectively collects image information of different positions on the road section, each camera adopts a variable focus lens with different focal lengths to monitor objects with different distances and collect corresponding image information, then the collected image information is sent to the central control processing unit, the central control processing unit uploads the data obtained by each front-end artificial intelligent multi-element panoramic monitoring and detecting device to a five-dimensional early warning management platform of the central management device for secondary video image stitching and is fused into a five-dimensional graph for real-time display, the functions of abnormal event accident detection alarm, positioning, traffic state preview and evaluation and full-path video viewing roaming are realized;

the IPC infrared thermal imaging camera is used for realizing high-temperature object detection, road dark ice detection, fire early warning alarm detection and night vision perspective function providing, firstly, an area needing to be detected is defined, then, objects, pedestrians and roadbed pavements in the detected area are detected in real time, and the IPC infrared thermal imaging camera plays a role in night vision perspective monitoring when a common camera is blind at night or in dark environment; each front-end artificial intelligence multi-element panoramic monitoring detection device is installed at a preset position in a road section, an IPC infrared thermal imaging camera respectively collects data information of objects, pedestrians and road beds and road surfaces at different positions on the road section, then the collected data information is sent to the central control processing unit, the central control processing unit uploads the data obtained by each front-end artificial intelligence multi-element panoramic monitoring detection device to a five-dimensional early warning management platform of a central management device, centralized gathering analysis is carried out, and the data are fused into a five-dimensional graph to be displayed in real time, so that the functions of abnormal vehicle, object, pedestrian and road bed abnormal event accident detection warning, positioning, traffic state preview and evaluation and perspective are realized;

the central control processing unit is used for splicing and fusing image information shot by a group of zoom cameras and acquiring data, performing logical operation and outputting logical control according to a splicing algorithm to form a 180-degree or 360-degree panoramic spliced image, uploading the image information to a central management device, combining various data acquired by the multi-element sensor when receiving alarm information from the IPC infrared thermal imaging camera, outputting alarm information through the interface unit according to preset alarm parameters or an alarm plan automatically generated after artificial intelligence deep analysis and learning of the central control processing unit, sending the alarm information to local alarm equipment or third-party equipment, and uploading the alarm information to a central server through a network port for summarizing alarm or alarm triggering; the central control processing unit sends the panoramic stitching image, the alarm information of the IPC infrared thermal imaging camera, the video, the image, the data collected by the multi-element sensor and the alarm information to a central management device or a mobile terminal through the communication module, and a maintainer checks the real-time monitoring information through a rear end platform of the mobile terminal, checks the states of all online devices in an access local area network, the positions of fault devices and the fault conditions and then quickly repairs the on-line devices;

the central management device receives videos, images and various data and alarm information acquired by the multi-element sensors and sent by each front-end artificial intelligent multi-element panoramic monitoring detection device, N front-end artificial intelligent multi-element panoramic monitoring detection devices are cascaded and associated to acquire front-end data, a large data cloud processing platform is used for carrying out on-site five-dimensional graphic display, full-path traffic state video preview roaming and actual environment simulation, simulation and playback to obtain an environment simulation result, managers, rescue personnel and fire fighters are informed in the form of graphics and sound, and maintenance measures are taken;

the central management device carries out on-site five-dimensional graph real-time display through a big data cloud processing platform, and the method comprises the following steps: the method comprises the steps of on-site five-dimensional graphic display, actual weather perception environment simulation dynamic display, traffic passage state preview, event accident prejudgment, full-path video splicing three-dimensional infinite roaming viewing, event accident occurrence position real-time positioning and information gathering and uploading;

after image information and data information acquired by each sensor are acquired in real time, the system starts big data analysis and simulates the acquired data in a graph mode to provide basic guarantee for commanders in a monitoring center and emergency rescue and relief personnel, in addition, the system finally forms a display graph with five-dimensional real-time dynamic state for decision, positioning, checking, full-path roaming and preview use of operators by regularly self-learning and data summarization of various data, takes a three-dimensional map as a bottom layer preview mode and combines management strategies and periodic action schemes of different time periods, different meteorological conditions, different traffic states, real-time panoramic video image splicing and fusion, different places and different hazard degrees, and immediately sends alarm information once an alarm critical point is reached or an emergency occurs, and relevant workers watch the preview in a five-dimensional view display mode, so that corresponding rescue measures can be taken as soon as possible, and the life is saved with less damage; wherein, the five-dimensional view display mode adopts: the three-dimensional map, the image information and the data information acquired by each sensor jointly form a five-dimensional view display.

2. The artificial intelligence multi-element panoramic monitoring and detection-based five-dimensional early warning system as claimed in claim 1, wherein each front-end artificial intelligence multi-element panoramic monitoring and detection device adopts a group of variable focus or fixed focus cameras to realize 180 ° or 360 ° panoramic stitching fusion; the IPC infrared thermal imaging camera adopts an infrared thermal imaging camera with the functions of temperature detection and region demarcation.

3. The artificial intelligence-based multi-element panoramic monitoring and detection five-dimensional early warning system as claimed in claim 1 or 2, wherein a group of variable focus cameras adopts an intelligent cascade calibration technology, wherein the intelligent cascade calibration comprises adjustment of focal length, aperture, shutter, white light balance, exposure control, minimum illumination, wide dynamic, orientation and encoding format parameters of the cameras;

4. The artificial intelligence based multi-element panoramic surveillance detection five-dimensional pre-warning system of claim 2, wherein the set of zoom cameras comprises: 3 zoom cameras, wherein the central control processing unit outputs 5 sets of video streams:

group 1: a video stream of a first zoom camera;

group 2: a video stream of a second zoom camera;

group 3: a video stream of a third zoom camera;

group 5: video streams of IPC infrared thermal imaging cameras;

5. The artificial intelligence multi-element panorama monitoring detection based five-dimensional early warning system of claim 1, wherein the interface unit comprises: a power supply interface and an input-output interface, wherein,

the power supply interface adopts a 48V POE power supply interface;

the input-output interface includes: IO interface, audio input interface, audio output interface, CAN interface, RS485 interface and 100/1000M self-adaptation RJ45 interface.

6. The artificial intelligence based multi-element panoramic monitoring and detection five-dimensional early warning system as claimed in claim 1, wherein the integrated gas sensor is used for collecting gas concentrations of sulfur dioxide, carbon monoxide, methane, formaldehyde, natural gas and liquefied gas.

7. The artificial intelligence multi-element panorama monitoring detection based five-dimensional early warning system of claim 1, wherein the central control processing unit comprises: the device comprises a storage unit, a clock unit, a GPU (graphics processing Unit), a multi-core high-speed ARM (advanced RISC machine) processor, a multi-core high-speed DSP (digital signal processor) and a central control logic operation and processing unit; the communication module adopts three types, namely a wireless communication module, a 4G communication module and an 100/1000M self-adaptive network port communication module.

8. The artificial intelligence multi-element panorama monitoring detection based five-dimensional early warning system of claim 1, wherein the central management apparatus comprises: a management workstation and a private cloud server, wherein,

the private cloud server is arranged in a monitoring sub-center machine room and executes the following work: the method comprises the following steps of data exchange, a storage function, system communication, maintenance, building of each function of the system, parameter setting, centralized alarm, plan generation, artificial intelligence deep learning analysis, linkage scheme execution issuing, three-dimensional map generation, five-dimensional graph generation, field environment data summarization, field environment simulation, big data macroscopic operation, video monitoring, image storage, alarm event video storage, report production, full-path video image secondary splicing, full-path video image viewing, roaming, preview generation, calling of video images and third-party linkage instruction sending, preset scheme sending and daily operation human-computer interface centralized alarm;