CN115941909B

CN115941909B - Driving safety monitoring system, method and device

Info

Publication number: CN115941909B
Application number: CN202310108768.XA
Authority: CN
Inventors: 张彭园; 周磊; 郭辉
Original assignee: XCMG Hanyun Technologies Co Ltd
Current assignee: XCMG Hanyun Technologies Co Ltd
Priority date: 2023-02-14
Filing date: 2023-02-14
Publication date: 2023-05-09
Anticipated expiration: 2043-02-14
Also published as: CN115941909A

Abstract

The application provides a driving safety monitoring system, method and device, which belong to the technical field of crane safety, and the system comprises: the system comprises an image acquisition subsystem, an alarm subsystem and a driving safety control subsystem which is in communication connection with the image acquisition subsystem and the alarm subsystem; the image acquisition subsystem is used for acquiring a first real-time video picture, a second real-time video picture and a first real-time heat induction picture; the driving safety control subsystem is used for determining current operation parameters of the driving and current postures of cargoes based on the first real-time video picture and the second real-time video picture, a forbidden area corresponding to the driving and admittance crowd of the forbidden area, determining whether staff exists in the forbidden area, alarming under the condition that the staff exists in the forbidden area and is a non-admitted crowd, sending an emergency action command to the driving based on action information of the staff, and improving production efficiency while guaranteeing the safety of the staff.

Description

Driving safety monitoring system, method and device

Technical Field

The application relates to the technical field of crane safety, in particular to a driving safety monitoring system, method and device.

Background

The crane is also called bridge crane, and is widely used in various industrial plants. The existing industrial factory building is usually synchronously operated with the production operation and the lifting operation, so that operators often stay or pass through the travelling path of the travelling crane, and if the objects lifted by the travelling crane drop carelessly, the personal safety of the operators is seriously threatened. In order to ensure the personnel safety of the running vehicle in the industrial factory building to the maximum extent, the prior art generally uses the movable range of the running vehicle as a forbidden area when the running vehicle works, and alarms once a worker enters the forbidden area, so that the running vehicle can be braked emergently, and the running vehicle can be restored after the worker leaves the forbidden area.

The technical proposal can furthest ensure the personal safety of the staff, but the working frequency of the crane in the industrial factory building is higher, which leads the staff to only carry out production operation in the running clearance of the crane, thereby greatly reducing the production efficiency.

Disclosure of Invention

The application provides a driving safety monitoring system, method and device, so as to furthest improve the production efficiency on the basis of ensuring the safety of personnel in an industrial factory building.

The application provides a safety monitoring system of driving, the driving sets up in the industrial factory building, the system includes:

the system comprises an image acquisition subsystem, an alarm subsystem and a driving safety control subsystem which is in communication connection with the image acquisition subsystem and the alarm subsystem;

the image acquisition subsystem is used for acquiring a first real-time video picture, a second real-time video picture and a first real-time heat induction picture; the first real-time video picture and the first real-time heat induction picture are respectively a real-time video picture and a real-time heat induction picture of a preset area below a traveling crane, and the second real-time video picture is a real-time video picture of different viewing angles of the traveling crane;

the driving safety control subsystem is used for determining current operation parameters of the driving and current postures of goods based on the first real-time video picture and the second real-time video picture, determining a forbidden area corresponding to the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current postures of the goods, determining whether staff exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling the alarm subsystem to alarm and send an emergency action instruction to the driving based on action information of the staff under the condition that the staff exists in the forbidden area and the staff is a non-admittance crowd.

According to the driving safety monitoring system provided by the application, the image acquisition subsystem is arranged at the first optical camera and the infrared thermal imager at the preset position of the driving, the first optical camera is used for acquiring the first real-time video picture, and the infrared thermal imager is used for acquiring the first real-time thermal induction picture; the image acquisition subsystem further comprises a second optical camera arranged on the side wall of the industrial factory building, and the second optical camera is used for acquiring the second real-time video picture.

According to the driving safety monitoring system provided by the application, the alarm subsystem comprises a plurality of audible and visual alarm modules arranged at the edge of the driving operation range.

The application also provides a driving safety monitoring method, which is applied to the driving safety control subsystem of the driving safety monitoring system, and comprises the following steps:

acquiring a first real-time video picture and a second real-time video picture, and determining current operation parameters of the travelling crane and current postures of cargoes based on the first real-time video picture and the second real-time video picture;

determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of goods;

And acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

According to the method for monitoring the safety of the travelling crane, the operation parameters comprise positions, movement directions and movement speeds corresponding to the cart, the trolley and the object taking device respectively, and correspondingly, the current operation parameters of the travelling crane and the current gesture of goods are determined based on the first real-time video picture and the second real-time video picture, and the method specifically comprises the following steps:

determining a real-time video picture sequence corresponding to a cargo overlooking view angle based on the first real-time video picture, and determining real-time video picture sequences corresponding to a driving main view angle and a driving side view angle respectively based on the second real-time video picture;

and determining the position, the movement direction and the movement speed corresponding to the trolley and the object taking device based on the real-time video picture sequence corresponding to the driving main view angle, determining the position, the movement direction and the movement speed corresponding to the trolley based on the real-time video picture sequence corresponding to the driving side view angle, and determining the current gesture of the goods based on the real-time video picture sequence corresponding to the goods overlooking view angle.

According to the method for monitoring the safety of the traveling crane, the forbidden area of the traveling crane and the admitted crowd of the forbidden area are determined based on the current operation parameters of the traveling crane and the current gesture of goods, and the method specifically comprises the following steps:

determining a current operation stage of the travelling crane based on the current operation parameters of the travelling crane;

and determining a forbidden area of the traveling crane and an admittance crowd of the forbidden area based on the current operation stage, the current operation parameters and the current gesture of the goods of the traveling crane.

According to the method for monitoring the safety of the traveling crane, the forbidden area is a rectangular area including a projection area of the cargo, each boundary of the rectangular area is perpendicular to or parallel to the moving direction of the cargo, and accordingly, the forbidden area of the traveling crane and an admittance crowd of the forbidden area are determined based on the current operation stage of the traveling crane, the current operation parameters and the current posture of the cargo, and the method specifically comprises the following steps:

and determining the distance between each boundary of the rectangular area corresponding to the forbidden area of the travelling crane and the projection area of the goods and the admittance crowd of the forbidden area based on the current working stage, the current working parameters and the current gesture of the goods of the travelling crane.

According to the method for monitoring the safety of the traveling crane, the operation stage comprises a loading stage, a transporting stage and a unloading stage, and correspondingly, the distance between each boundary of a rectangular area corresponding to a forbidden area of the traveling crane and a projection area of the cargo and the admittance crowd of the forbidden area are determined based on the current operation stage, the current operation parameters and the current posture of the cargo of the traveling crane, and the method specifically comprises the following steps:

if the current operation stage of the travelling crane is a loading stage, the distances between the boundaries of the rectangular area corresponding to the forbidden area of the travelling crane and the projection area of the goods are all first preset distances, and the admitted people are loading people;

if the current operation stage of the travelling crane is a transportation stage, determining a target boundary of a rectangular area corresponding to a forbidden area of the travelling crane and a target distance between the target boundary and a projection area of the goods based on a position, a movement direction and a movement speed corresponding to an object taking device, wherein the distances between other boundaries of the rectangular area corresponding to the forbidden area of the travelling crane and the projection area of the goods are second preset distances;

and if the current operation stage of the traveling crane is a discharging stage, the distances between the boundaries of the rectangular area corresponding to the forbidden area of the traveling crane and the projection area of the goods are all third preset distances, and the admitted people are discharging people.

According to the method for monitoring the safety of the traveling crane, which is provided by the application, the emergency action instruction is sent to the traveling crane based on the action information of the staff, and the method specifically comprises the following steps:

judging the movement trend of the staff based on the action information of the staff;

if the movement trend of the staff member is based on the judgment that the staff member is still and the first preset duration is continued, a braking instruction is sent to the service;

and if the movement trend of the staff is based on the judgment that the staff moves along the movement direction of the cart, and the movement speed is smaller than the current movement speed of the cart, sending a speed reduction instruction to the cart.

The application also provides a driving safety monitoring device, the device is applied to the driving safety control subsystem in the driving safety monitoring system, and the device comprises:

the operation information determining unit is used for acquiring a first real-time video picture and a second real-time video picture and determining the current operation parameters of the travelling crane and the current gesture of goods based on the first real-time video picture and the second real-time video picture;

the forbidden information determining unit is used for determining forbidden areas of the travelling crane and admittance crowds of the forbidden areas based on the current operation parameters of the travelling crane and the current postures of cargoes;

The safety control unit is used for acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area or not based on the first real-time video picture and the first real-time heat induction picture, and controlling the alarm subsystem to alarm and sending an emergency action instruction to the traveling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of safety monitoring of a vehicle as described in any of the above.

The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of a method for monitoring the safety of a vehicle as described in any one of the above.

The application provides a driving safety monitoring system, method and device, driving sets up in the industry factory building, the system includes:

the system comprises an image acquisition subsystem, an alarm subsystem and a driving safety control subsystem which is in communication connection with the image acquisition subsystem and the alarm subsystem; the image acquisition subsystem is used for acquiring a first real-time video picture, a second real-time video picture and a first real-time heat induction picture; the first real-time video picture and the first real-time heat induction picture are respectively a real-time video picture and a real-time heat induction picture of a preset area below a traveling crane, and the second real-time video picture is a real-time video picture of different viewing angles of the traveling crane; the driving safety control subsystem is used for determining current operation parameters of the driving and current postures of goods based on the first real-time video picture and the second real-time video picture, determining a forbidden area corresponding to the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current postures of the goods, determining whether staff exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling the alarm subsystem to alarm and send an emergency action instruction to the driving under the condition that staff exists in the forbidden area and is not the admittance crowd, so that the production efficiency can be improved to the maximum extent on the basis of ensuring the safety of staff in an industrial factory building.

Drawings

For a clearer description of the present application or of the prior art, the drawings that are used in the description of the embodiments or of the prior art will be briefly described, it being apparent that the drawings in the description below are some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic structural diagram of a safety monitoring system for a vehicle in accordance with the present application;

FIG. 2 is a schematic view of the viewing angle of a driving vehicle provided in the present application;

FIG. 3 is a schematic diagram of an alarm subsystem provided herein;

fig. 4 is a flow chart of a method for monitoring driving safety provided by the application;

FIG. 5 is a schematic illustration of a determination flow of job information provided herein;

FIG. 6 is a schematic illustration of a determination flow of forbidden information provided herein;

FIG. 7 is a schematic illustration of the shape of the keep-in area provided herein;

FIG. 8 is a schematic structural view of a safety monitoring device for a traveling crane provided by the present application;

fig. 9 is a schematic structural diagram of an electronic device provided in the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the present application will be clearly and completely described below with reference to the drawings in the present application, and it is apparent that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

Fig. 1 is a schematic structural diagram of a safety monitoring system for a traveling crane provided in the present application, where the traveling crane is disposed in an industrial factory building, as shown in fig. 1, and the system includes:

Specifically, based on the foregoing, in order to maximally ensure the safety of personnel during the running of the vehicle in the industrial factory building, the prior art generally uses the movable range of the vehicle as a forbidden area during the running of the vehicle, and the personnel is forbidden to enter the forbidden area during the running of the vehicle, which can be understood as the movable range of the vehicle, namely the ground range which can be covered during the moving process of the large vehicle, which can lead the personnel to only perform the production operation during the idle period of the vehicle, thereby greatly reducing the production efficiency. To this problem, in order to furthest improve production efficiency on the basis of guaranteeing personnel security in the industrial factory building, this application embodiment has proposed a safety monitoring system of driving, can set up the forbidden region and carry out accurate monitoring to forbidden region based on the running state developments of driving rationally, carries out timely emergency treatment under the circumstances that has the staff in forbidden region.

More specifically, the image acquisition subsystem is arranged at a first optical camera and an infrared thermal imager at a preset position of the travelling crane, the first optical camera is used for acquiring the first real-time video picture, and the infrared thermal imager is used for acquiring the first real-time thermal induction picture. The first real-time video picture and the first real-time thermal induction picture are respectively a real-time video picture and a real-time thermal induction picture of a preset area below a traveling crane cart, and it can be understood that the preset area preferably uses an object taking device of the traveling crane as a center, and correspondingly, the first optical camera and the infrared thermal imager are preferably arranged on a cart of the traveling crane so as to ensure that the first optical camera and the infrared thermal imager can track and acquire the real-time video picture and the real-time thermal induction picture of the preset area in real time. Based on the method, whether a worker exists near the object taking device or not and whether the worker is in a forbidden area or not can be accurately determined according to the real-time picture of the preset area, and then timely alarming and driving control can be carried out. Under normal conditions, whether a worker exists in the preset area can be judged based on the first real-time video picture, however, considering that the operation environment in part of industrial plants is severe, impurities affecting the visual field of the optical camera due to dust and the like are accompanied in the plants, or whether the worker exists in the preset area can not be accurately judged based on the first real-time video picture due to insufficient light at night and shielding of the worker by goods and the like. To this problem, the embodiment of the application further adopts the infrared thermal imaging system to gather first real-time thermal induction picture, because the formation of image of infrared thermal imaging system does not receive the influence of factors such as light and object shelter from, consequently combine first real-time thermal induction picture and first real-time video picture, can furthest guarantee the accuracy of the judgement result of whether there is the staff in the area of predetermineeing.

Meanwhile, in consideration of the association relation between the running state of the travelling crane and the forbidden area, the image acquisition subsystem of the embodiment of the application further comprises a second optical camera arranged on the side wall of the industrial factory building, wherein the second optical camera is used for acquiring a second real-time video picture, and the second real-time video picture is a real-time video picture of different viewing angles of the travelling crane and comprises a main viewing angle and a side viewing angle. Fig. 2 is a schematic view of a driving observation view angle provided in the present application, as shown in fig. 2, a position, a movement direction and a movement speed corresponding to a trolley and an object taking device may be determined based on a real-time video image sequence corresponding to a driving main view angle, and a position, a movement direction and a movement speed corresponding to a cart may be determined based on a real-time video image sequence corresponding to a driving side view angle. Meanwhile, based on the foregoing, the first real-time video frame and the first real-time heat sensing frame correspond to a top view angle of the cargo. According to the method and the system, research discovers that as the movement modes of different operation stages of the travelling crane are different, the operation parameters of the same operation stage are possibly different, corresponding areas with risks are different, meanwhile, the change of the gesture of the goods also causes the change of the risk areas, so that the travelling crane safety control subsystem of the embodiment of the invention determines the current operation parameters of the travelling crane and the current gesture of the goods based on the first real-time video picture and the second real-time video picture, and determines the forbidden area corresponding to the travelling crane and the admitted crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of the goods. Based on this, the rationality of the no-entry area setting can be ensured. Moreover, in consideration of the fact that staff is needed to assist in the loading and unloading stages, the embodiment of the application further determines the admittance crowd of the forbidden area based on different operation stages of the driving, and based on the fact, the situation that the admittance crowd enters the forbidden area to cause false triggering of an alarm can be avoided. It may be understood that the image acquisition subsystem may send the first real-time video frame, the second real-time video frame, and the first real-time heat induction frame to the driving safety control subsystem in real time, or may send the first real-time video frame, the second real-time video frame, and the first real-time heat induction frame based on an acquisition instruction of the driving safety control subsystem.

After the forbidden area and the admittance crowd of the forbidden area are determined, the driving safety control subsystem can determine whether a worker exists in the forbidden area or not based on the first real-time video picture and the first real-time heat induction picture, and when the worker exists in the forbidden area and is a non-admittance crowd, the alarming subsystem is controlled to alarm and send an emergency action instruction to the driving based on action information of the worker. It is understood that the forbidden area is smaller than the aforementioned preset area. Meanwhile, considering that the disposal modes of corresponding staff after alarming are possibly different, therefore, the driving safety control subsystem of the embodiment of the application further sends an emergency action instruction to the driving based on action information of the staff, namely, the driving control is carried out in a targeted mode according to the disposal modes of the corresponding staff after alarming, driving braking is controlled when potential danger is avoided each time, and based on the fact, production efficiency and driving working efficiency can be guaranteed.

Fig. 3 is a schematic diagram of a setting mode of an alarm subsystem provided in the present application, as shown in fig. 3, where the alarm subsystem includes a plurality of audible and visual alarm modules disposed at an edge of a driving range. The driving operation range edge is the area corresponding to the vertical projection of the driving operation track, and because the driving corresponds to two operation tracks, the driving operation range edge is two, the audible and visual alarm modules arranged on the two driving operation range edges are preferably arranged at equal intervals along the driving operation range edge, the audible and visual alarm modules at the same positions on the two driving operation range edges form an audible and visual alarm module group, therefore, the audible and visual alarm module group related to the forbidden area can be rapidly determined under the condition that staff is in the forbidden area and the staff is a non-admittance crowd, and an alarm instruction is sent to the alarm subsystem to control the audible and visual alarm module group to alarm, so that the alarm information can be accurately conveyed to the non-admittance crowd in the forbidden area, and the non-admittance crowd can be ensured to leave the forbidden area as soon as possible. It will be appreciated that the number of the audible and visual alarm modules and the setting interval in fig. 3 are only examples, and the number of the audible and visual alarm modules and the setting interval can be freely set according to needs in practical applications.

The embodiment of the application provides a driving safety monitoring system, the system includes: the system comprises an image acquisition subsystem, an alarm subsystem and a driving safety control subsystem which is in communication connection with the image acquisition subsystem and the alarm subsystem; the image acquisition subsystem is used for acquiring a first real-time video picture, a second real-time video picture and a first real-time heat induction picture; the first real-time video picture and the first real-time heat induction picture are respectively a real-time video picture and a real-time heat induction picture of a preset area below a traveling crane, and the second real-time video picture is a real-time video picture of different viewing angles of the traveling crane; the driving safety control subsystem is used for determining current operation parameters of the driving and current postures of goods based on the first real-time video picture and the second real-time video picture, determining a forbidden area corresponding to the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current postures of the goods, determining whether staff exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling the alarm subsystem to alarm and send an emergency action instruction to the driving under the condition that staff exists in the forbidden area and is not the admittance crowd, so that the production efficiency can be improved to the maximum extent on the basis of ensuring the safety of staff in an industrial factory building.

Fig. 4 is a flow chart of a method for monitoring driving safety provided in the present application, where the method is applied to a driving safety control subsystem of a driving safety monitoring system described in the foregoing embodiment, as shown in fig. 4, and the method includes:

step 101, a first real-time video picture and a second real-time video picture are obtained, and the current operation parameters of the travelling crane and the current gesture of goods are determined based on the first real-time video picture and the second real-time video picture.

Specifically, as can be seen from the foregoing embodiments, the image acquisition subsystem is configured to acquire a first real-time video frame, a second real-time video frame, and a first real-time heat induction frame, based on which the driving safety control subsystem may acquire the first real-time video frame and the second real-time video frame, and determine a current operation parameter of the driving and a current posture of the cargo based on the first real-time video frame and the second real-time video frame. Fig. 5 is a schematic view of determining a flow of the operation information provided in the present application, as shown in fig. 5, where the operation parameters include positions, movement directions and movement speeds corresponding to a cart, a trolley and an object taking device, respectively, and correspondingly, determining, based on the first real-time video frame and the second real-time video frame, a current operation parameter of the travelling crane and a current posture of a cargo specifically includes:

Step 201, determining a real-time video picture sequence corresponding to a cargo overlooking view angle based on the first real-time video picture, and determining real-time video picture sequences corresponding to a driving main view angle and a driving side view angle respectively based on the second real-time video picture;

step 202, determining the position, the movement direction and the movement speed corresponding to the trolley and the object taking device based on the real-time video picture sequence corresponding to the driving main view angle, determining the position, the movement direction and the movement speed corresponding to the trolley based on the real-time video picture sequence corresponding to the driving side view angle, and determining the current gesture of the goods based on the real-time video picture sequence corresponding to the goods overlooking view angle.

It can be understood that, because the running state is continuous when the travelling crane works, it is required to determine a real-time video picture sequence corresponding to the overlooking view angle of the goods based on the first real-time video picture, determine real-time video picture sequences corresponding to the driving main view angle and the side view angle respectively based on the second real-time video picture, determine positions, movement directions and movement speeds corresponding to the trolley and the object taking device based on the real-time video picture sequences corresponding to the driving main view angle, determine positions, movement directions and movement speeds corresponding to the trolley based on the real-time video picture sequences corresponding to the side view angle of the travelling crane, and determine the current posture of the goods based on the real-time video picture sequences corresponding to the overlooking view angle of the goods. It will also be appreciated that, since the attitude of the cargo changes during transportation due to various external factors, such as rotation or tilting of the cargo, the change in attitude of the cargo will directly result in a change in the risk area, in order to accurately determine the extent of the no-entry area, it is necessary to determine the current state of the cargo in addition to the operation parameters.

And 102, determining a forbidden area of the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current gesture of goods.

Specifically, fig. 6 is a schematic diagram of a flow chart of determining forbidden information provided in the present application, as shown in fig. 6, where determining a forbidden area of a driving vehicle and an admittance crowd of the forbidden area based on a current operation parameter of the driving vehicle and a current gesture of goods specifically includes:

step 301, determining a current operation stage of the travelling crane based on current operation parameters of the travelling crane;

step 302, determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current working stage, the current working parameters and the current gesture of goods of the travelling crane.

The operation stage comprises a loading stage, a transporting stage and a unloading stage, and it is understood that operation parameters corresponding to different stages of the travelling crane are different, wherein in the loading stage, the positions of the cart, the trolley and the object taking device are all right above a loading point, the cart and the trolley are all static, and the object taking device moves upwards; in the transportation stage, the cart moves, the trolley moves or is static, and the object taking device is static; in the unloading stage, the positions of the cart, the trolley and the object taking device are all right above the unloading point, the cart and the trolley are all static, and the object taking device moves downwards. Based on the method, the current operation stage of the travelling crane can be accurately determined according to the current operation parameters of the travelling crane. It can be understood that the corresponding positions of the cart, the trolley, the object taking device, the loading point and the unloading point are three-dimensional coordinates, which can be determined based on a three-dimensional model of a cube area corresponding to a pre-generated movable range of the vehicle, and the specific coordinates of the loading point and the unloading point can be obtained from a corresponding vehicle transportation task management system, wherein the vehicle transportation task management system comprises transportation tasks corresponding to different time points of the vehicle, and the cargo parameters, the loading point and the unloading point coordinates corresponding to the transportation tasks, and based on the three-dimensional coordinates, the rapid and accurate determination of the boundary point coordinates of the forbidden area of the vehicle can be ensured.

Based on the foregoing embodiments, it can be known that, because the movement modes of the driving in different operation stages are different, and the operation parameters in the same operation stage may also be different, corresponding regions with risks are also different, and meanwhile, the change of the gesture of the cargo also causes the change of the risk region, so in the embodiment of the present application, the current operation stage of the driving is determined based on the current operation parameters of the driving, and then the forbidden region of the driving and the admittance crowd of the forbidden region are determined based on the current operation stage of the driving, the current operation parameters and the current gesture of the cargo. More specifically, fig. 7 is a schematic shape diagram of a forbidden area provided in the present application, as shown in fig. 7, where the forbidden area in the embodiment of the present application is a rectangular area including a projection area of the cargo, each boundary of the rectangular area is perpendicular to or parallel to a moving direction of the cargo, and according to the foregoing embodiment and a working manner of the traveling vehicle, it is known that a movable range of the traveling vehicle, that is, a ground rectangular range that can be covered in a moving process of the traveling vehicle, is a ground rectangular range that can be covered in a moving process of the traveling vehicle, during a traveling vehicle working period, and this can lead to a substantial reduction in production efficiency, and in this embodiment of the present application, the forbidden area is a rectangular area consistent with a boundary direction of the movable area, but the rectangular area is a rectangular area including the projection area of the cargo, and a distance between each boundary and the projection area of the cargo is determined based on a current working stage of the traveling vehicle, a current working parameter and a current posture of the cargo, and based on this, a forbidden area can be reduced to a maximum extent on a basis of ensuring safety of personnel, and further reducing an influence of the traveling vehicle on production. It will be appreciated that the shape of the projection area of the cargo in fig. 7 is merely an example, and the projection area of the cargo may be any shape during the actual production process, which is not specifically limited in this embodiment of the present application.

Correspondingly, the determining the forbidden area of the traveling crane and the admittance crowd of the forbidden area based on the current operation stage, the current operation parameters and the current gesture of the goods of the traveling crane specifically comprises:

As shown in fig. 7, it is noted that the distance between each boundary of the rectangular area corresponding to the forbidden area of the vehicle and the projected area of the cargo refers to the distance between each boundary of the rectangular area corresponding to the forbidden area of the vehicle and the corresponding boundary of the circumscribed rectangle of the projected area of the cargo. The circumscribed rectangle is also a rectangular area consistent with the boundary direction of the movable area, so that the circumscribed rectangle of the projection area of the goods is in one-to-one correspondence and parallel with each boundary of the rectangular area corresponding to the forbidden area of the travelling crane. Based on this, more specifically, the determining, based on the current operation stage of the traveling crane, the current operation parameter, and the current posture of the cargo, the distance between each boundary of the rectangular area corresponding to the forbidden area of the traveling crane and the projection area of the cargo, and the admittance crowd of the forbidden area specifically includes:

It can be appreciated that, because the cargo moves only in the vertical direction in the loading stage and the unloading stage, the range of the risk area is smaller, based on this, in the embodiment of the present application, each boundary of the rectangular area corresponding to the forbidden area of the travelling crane is kept consistent with the distance of the projection area of the cargo in the loading stage and the unloading stage, the first preset distance is a loading operation distance, the loading operation distance is determined based on the operation area required by the staff to perform the loading operation, the third preset distance is an unloading operation distance, the unloading operation distance is determined based on the operation area required by the staff to perform the unloading operation, and both the first preset distance and the third preset distance can be determined based on historical experience. It is worth noting that, although the range of the risk areas of the loading and unloading phases is relatively small, there is still a danger, so that the loading phase only allows professional loading personnel to enter the forbidden area of the travelling crane (i.e. the admittance crowd is loading personnel), and correspondingly, the unloading phase only allows professional unloading personnel to enter the forbidden area of the travelling crane (i.e. the admittance crowd is unloading personnel). It can be further appreciated that the embodiment of the application may determine whether a worker is in the forbidden area and whether the worker is an admitted crowd based on the first real-time video frame and the first real-time heat induction frame. Specifically, facial recognition or work clothes recognition (different types of work clothes can be different in style or color) can be adopted for the identification means of the staff. Of course, the identification of the staff may also be performed based on other means after the staff in the forbidden area is determined based on the first real-time video image and the first real-time heat induction image, for example, the identification is performed based on the RFID on the article worn by the worker, and the identification device may be disposed in an area such as the edge of the driving range.

Different from loading and unloading stages, if the current operation stage of the travelling crane is a transportation stage, the risk area corresponding to the movement direction of the cargoes is larger than that of the stationary state because the cargoes are in the movement state in the horizontal direction. The reason is that if the cargo falls at this time, the cargo still moves forward (i.e. in the moving direction) by a distance due to inertia, so that the distance between the boundary (i.e. the target boundary) of the vehicle in the moving direction of the cargo and the target of the projection area of the cargo should be greater than the maximum distance that the cargo can move after falling, for the rectangular area corresponding to the forbidden area of the vehicle. Based on this, the embodiment of the application determines, based on the position, the movement direction and the movement speed corresponding to the object taking device, the target boundary of the rectangular area corresponding to the forbidden area of the travelling crane and the target distance between the target boundary and the projection area of the goods, that is, determines, based on the movement direction corresponding to the object taking device (the same as the movement direction of the goods), the target boundary of the rectangular area corresponding to the forbidden area of the travelling crane, and determines, based on the position and the movement speed corresponding to the object taking device, the target distance between the target boundary and the projection area of the goods. It can be understood that, since the position corresponding to the object taking device is close to the object, the movement speed corresponding to the object taking device is the same, so the position and the movement speed corresponding to the object taking device can be equal to the position and the movement speed of the object, and the position of the object can be determined based on the second real-time video image in consideration of further improving the precision. After the position and the movement speed of the goods are determined, the maximum distance can be calculated based on a corresponding displacement calculation formula, and the target distance can be increased by a safety guarantee distance based on the maximum distance so as to furthest avoid personal injury to staff caused by continuous forward movement after the goods fall onto the ground. The safety guarantee distance can be obtained based on big data analysis or can be determined empirically, and the embodiment of the application is not particularly limited.

The distance between the other boundaries of the rectangular area corresponding to the forbidden area of the travelling crane (i.e. the boundaries outside the target boundary) and the projection area of the goods can be set to be a fixed second preset distance, and the second preset distance can be determined based on the response time of the staff receiving the alarm information and the moving speed of the staff, namely, the situation that the staff has response lag after receiving the alarm information, so that the staff continues to move forward for a small distance (called as the response distance hereinafter), and the staff may possibly enter the projection area of the goods with little carelessness, so that potential safety hazards exist. Therefore, the embodiment of the application takes the reaction distance as the second preset distance, and based on the second preset distance, the potential safety hazard can be reduced to the greatest extent in the transportation stage. The maximum value of the reaction time and the maximum value of the moving speed can be obtained based on big data analysis by the staff receiving the alarm information, and the maximum value of the reaction time and the maximum value of the moving speed are multiplied to obtain the reaction distance. It is noted that if the current operation stage of the travelling crane is a transportation stage, all the staff are not allowed to enter the forbidden area, i.e. no admittance crowd exists.

Step 103, a first real-time heat induction picture is obtained, whether a worker exists in the forbidden area or not is determined based on the first real-time video picture and the first real-time heat induction picture, and an alarm subsystem is controlled to alarm and send an emergency action instruction to the traveling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

Specifically, the detailed description of the specific means and effects of controlling the alarm subsystem to alarm and determining whether the staff exists in the forbidden area and whether the staff is a non-admitted group based on the first real-time video picture and the first real-time heat induction picture is omitted herein. The sending an emergency action instruction to the travelling crane based on the action information of the staff specifically comprises the following steps:

It will be appreciated that after receiving an alarm, normally the staff member will leave the forbidden area as soon as possible, for which case the vehicle can remain operating normally. However, for some staff members may have no action for a long time or have mistakes in evacuation direction due to tension emotion, and for similar situations, the embodiment of the application further judges the movement trend of the staff members based on the action information of the staff members after controlling the alarm subsystem to alarm;

and if the operator keeps static and lasts for the first preset time period based on the movement trend of the operator, sending a braking instruction to the service. It can be appreciated that, for the loading and unloading phases, since the falling risk of the cargo is relatively small, the first preset duration can be set as required on the basis of ensuring the production efficiency, and for the transportation phase, since the falling risk of the cargo is relatively large, the first preset duration is the time required for the cargo to move the safety guarantee distance, and based on this, the safety of the staff can be guaranteed to the maximum extent.

And if the movement trend of the staff is based on the judgment that the staff moves along the movement direction of the cart, and the movement speed is smaller than the current movement speed of the cart, sending a speed reduction instruction to the cart. It can be appreciated that this situation is mainly aimed at the transportation stage, when judging that the staff moves along the movement direction of the cart, and the movement speed is smaller than the current movement speed of the cart, it means that the cart will gradually approach the staff, based on this, the embodiment of the application sends a deceleration instruction to the driving to maintain normal operation as much as possible on the basis of ensuring the safety of the staff, and the high-speed operation is resumed after the staff leaves the forbidden area, so as to ensure the transportation efficiency.

The method provided by the embodiment of the application comprises the steps of obtaining a first real-time video picture and a second real-time video picture, and determining the current operation parameters of the travelling crane and the current gesture of goods based on the first real-time video picture and the second real-time video picture; determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of goods; the method comprises the steps of obtaining a first real-time heat induction picture, determining whether staff exists in a forbidden area or not based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to give an alarm and sending an emergency action instruction to a traveling crane based on action information of the staff under the condition that the staff exists in the forbidden area and is a non-admitted crowd, so that production efficiency can be improved to the maximum extent on the basis of guaranteeing personnel safety in an industrial factory building.

The following describes the safety monitoring device of the travelling crane provided by the application, and the safety monitoring device of the travelling crane described below and the safety monitoring method of the travelling crane described above can be referred to correspondingly.

Based on any of the above embodiments, fig. 8 is a schematic structural diagram of a driving safety monitoring device provided in the present application, where the device is applied to a driving safety control subsystem in a driving safety monitoring system according to the foregoing embodiment, as shown in fig. 8, and the device includes:

a job information determining unit 401, configured to obtain a first real-time video frame and a second real-time video frame, and determine a current job parameter of the travelling crane and a current gesture of a cargo based on the first real-time video frame and the second real-time video frame;

a forbidden information determining unit 402, configured to determine a forbidden area of the driving vehicle and an admittance crowd of the forbidden area based on a current operation parameter of the driving vehicle and a current gesture of goods;

the safety control unit 403 is configured to obtain a first real-time heat induction picture, determine whether a worker is in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and control the alarm subsystem to alarm and send an emergency action instruction to the driving based on action information of the worker when the worker is in the forbidden area and the worker is a non-admitted crowd.

In the device provided by the embodiment of the present application, the operation information determining unit 401 acquires a first real-time video frame and a second real-time video frame, and determines a current operation parameter of the travelling crane and a current gesture of goods based on the first real-time video frame and the second real-time video frame; the forbidden information determining unit 402 determines a forbidden area of the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current gesture of goods; the safety control unit 403 obtains the first real-time heat induction picture, and based on the first real-time video picture and the first real-time heat induction picture, determines whether there is a staff in the forbidden area, in the situation that there is a staff in the forbidden area and the staff is non-admitted crowd, controls the alarm subsystem to alarm and based on the action information of the staff to send an emergency action instruction to the driving, can furthest improve production efficiency on the basis of guaranteeing the personnel safety in the industrial factory building.

Based on the above embodiment, the operation parameters include positions, movement directions and movement speeds corresponding to the cart, the trolley and the object taking device respectively, and correspondingly, the determining the current operation parameters of the travelling crane and the current gesture of the goods based on the first real-time video picture and the second real-time video picture specifically includes:

Based on any one of the above embodiments, the determining the forbidden area of the driving and the admittance crowd of the forbidden area based on the current operation parameter of the driving and the current gesture of the goods specifically includes:

Based on any of the foregoing embodiments, the forbidden area is a rectangular area including a projection area of the cargo, each boundary of the rectangular area is perpendicular to or parallel to a moving direction of the cargo, and accordingly, the determining, based on a current operation stage of the traveling crane, a current operation parameter, and a current posture of the cargo, the forbidden area of the traveling crane and an admittance crowd of the forbidden area specifically includes:

Based on any of the foregoing embodiments, the operation stage includes a loading stage, a transporting stage and a unloading stage, and correspondingly, the determining, based on the current operation stage of the travelling crane, the current operation parameters and the current posture of the cargo, the distance between each boundary of the rectangular area corresponding to the forbidden area of the travelling crane and the projection area of the cargo, and the admittance crowd of the forbidden area specifically includes:

Based on any one of the above embodiments, the sending an emergency action instruction to the driving based on action information of the staff specifically includes:

Fig. 9 illustrates a physical schematic diagram of an electronic device, as shown in fig. 9, which may include: a processor (processor) 501, a communication interface (Communications Interface) 502, a memory (memory) 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other via the communication bus 504. The processor 501 may invoke logic instructions in the memory 503 to execute the method for monitoring the safety of driving provided by the above methods, where the method includes: acquiring a first real-time video picture and a second real-time video picture, and determining current operation parameters of the travelling crane and current postures of cargoes based on the first real-time video picture and the second real-time video picture; determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of goods; and acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

Further, the logic instructions in the memory 503 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present application further provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, where the computer program, when executed by a processor, can perform a method for monitoring driving safety provided by the above methods, where the method includes: acquiring a first real-time video picture and a second real-time video picture, and determining current operation parameters of the travelling crane and current postures of cargoes based on the first real-time video picture and the second real-time video picture; determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of goods; and acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

In yet another aspect, the present application further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a method for monitoring safety of a driving provided by the above methods, the method comprising: acquiring a first real-time video picture and a second real-time video picture, and determining current operation parameters of the travelling crane and current postures of cargoes based on the first real-time video picture and the second real-time video picture; determining a forbidden area of the travelling crane and an admittance crowd of the forbidden area based on the current operation parameters of the travelling crane and the current gesture of goods; and acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. A safety monitoring system for a trolley, the trolley being disposed in an industrial plant, the system comprising:

the driving safety control subsystem is used for determining current operation parameters of the driving and current postures of goods based on the first real-time video picture and the second real-time video picture, determining a forbidden area corresponding to the driving and an admittance crowd of the forbidden area based on the current operation parameters of the driving and the current postures of the goods, determining whether staff exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling the alarm subsystem to alarm and send an emergency action instruction to the driving based on action information of the staff under the condition that the staff exists in the forbidden area and the staff is a non-admitted crowd;

The operation parameters comprise positions, movement directions and movement speeds corresponding to the cart, the trolley and the object taking device respectively, and correspondingly, the current operation parameters of the travelling crane and the current gesture of goods are determined based on the first real-time video picture and the second real-time video picture, and the operation parameters specifically comprise:

determining the position, the movement direction and the movement speed corresponding to the trolley and the object taking device based on the real-time video picture sequence corresponding to the driving main view angle, determining the position, the movement direction and the movement speed corresponding to the trolley based on the real-time video picture sequence corresponding to the driving side view angle, and determining the current gesture of the goods based on the real-time video picture sequence corresponding to the goods overlooking view angle;

the determining the forbidden area corresponding to the driving and the admittance crowd of the forbidden area based on the current operation parameters of the driving and the current gesture of the goods specifically comprises the following steps:

2. The vehicle safety monitoring system of claim 1, wherein the image acquisition subsystem is disposed at a first optical camera and a thermal infrared imager at a preset position of the vehicle, the first optical camera is configured to acquire the first real-time video frame, and the thermal infrared imager is configured to acquire the first real-time thermal sensing frame; the image acquisition subsystem further comprises a second optical camera arranged on the side wall of the industrial factory building, and the second optical camera is used for acquiring the second real-time video picture.

3. The traffic safety monitoring system according to claim 2, wherein the alarm subsystem comprises a plurality of audible and visual alarm modules disposed at edges of the traffic operating range.

4. A method for monitoring the safety of a vehicle, the method being applied to the vehicle safety control subsystem of the vehicle safety monitoring system of claim 3, the method comprising:

acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd;

The determining the forbidden area of the driving and the admittance crowd of the forbidden area based on the current operation parameters of the driving and the current gesture of the goods specifically comprises the following steps:

5. The method for monitoring the safety of a traveling crane according to claim 4, wherein the forbidden area is a rectangular area including a projection area of the cargo, each boundary of the rectangular area is perpendicular to or parallel to a moving direction of the cargo, and the method for determining the forbidden area of the traveling crane and an admittance crowd of the forbidden area based on a current operation stage of the traveling crane, a current operation parameter and a current posture of the cargo specifically comprises:

6. The method for monitoring the safety of a traveling crane according to claim 5, wherein the operation stage includes a loading stage, a transporting stage and a unloading stage, and the determining, based on the current operation stage of the traveling crane, the current operation parameters and the current posture of the cargo, the distance between each boundary of the rectangular area corresponding to the forbidden area of the traveling crane and the projection area of the cargo, and the admittance crowd of the forbidden area specifically includes:

7. The method for monitoring the safety of a traveling crane according to claim 6, wherein the step of sending an emergency action command to the traveling crane based on the action information of the staff member specifically comprises:

8. A traffic safety monitoring device, characterized in that the device is applied to a traffic safety control subsystem in a traffic safety monitoring system according to claim 3, the device comprising:

The safety control unit is used for acquiring a first real-time heat induction picture, determining whether a worker exists in the forbidden area or not based on the first real-time video picture and the first real-time heat induction picture, and controlling an alarm subsystem to alarm and sending an emergency action instruction to the travelling crane based on action information of the worker under the condition that the worker exists in the forbidden area and the worker is a non-admitted crowd;