CN112287721A

CN112287721A - Method and device for tracking falling object, computer equipment and storage medium

Info

Publication number: CN112287721A
Application number: CN201910666668.2A
Authority: CN
Inventors: 刘洲
Original assignee: Changsha Intelligent Driving Research Institute Co Ltd
Current assignee: Changsha Intelligent Driving Research Institute Co Ltd
Priority date: 2019-07-23
Filing date: 2019-07-23
Publication date: 2021-01-29

Abstract

The application relates to a method, a device, a computer device and a storage medium for falling object tracking. The method comprises the following steps: the method comprises the steps of obtaining a multi-frame captured image set, identifying a falling object according to the multi-frame captured image set, obtaining a captured image set comprising images of the falling object, obtaining an image acquisition equipment identifier corresponding to the captured image set comprising the images of the falling object according to the captured image set comprising the images of the falling object, obtaining an equipment coordinate parameter of image acquisition equipment corresponding to the image acquisition equipment identifier according to the image acquisition equipment identifier, locking the position of the falling object according to the equipment coordinate parameter, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, determining a target falling track of the falling object according to the coordinate parameters, and tracking the falling object according to the target falling track. By adopting the method, the falling object can be responded according to the tracking result, and the risk of injury caused by the falling object is obviously reduced.

Description

Method and device for tracking falling object, computer equipment and storage medium

Technical Field

The present application relates to the field of building engineering technologies, and in particular, to a method and an apparatus for tracking a falling object, a computer device, and a storage medium.

Background

With the development of modern large-scale building systems, more and more high-rise buildings and large-scale building sites appear, and when pedestrians are near the high-rise buildings and the large-scale building sites, great risks of being injured by falling objects exist.

Taking a construction site as an example, the construction site refers to a site where a construction project is being developed and civil engineering is carried out, and the range of the site is often closed by enclosing plates, wire nets or enclosing walls so as to limit the access of people, materials, machines and vehicles. On the building site, the phenomenon that workers are injured by falling materials due to the building of a high-rise often happens, and the workers on the building site have great risks of injury by falling objects.

Disclosure of Invention

In view of the above, it is necessary to provide a method, an apparatus, a computer device, and a storage medium for tracking a falling object, which can accurately track the falling object and significantly reduce the risk of being injured by the falling object.

A method of fall tracking, the method comprising:

acquiring a multi-frame captured image set, and identifying a falling object according to the multi-frame captured image set to obtain a captured image set comprising images of the falling object;

acquiring an image acquisition equipment identifier corresponding to a captured image set comprising falling object images according to the captured image set comprising the falling object images, and acquiring equipment coordinate parameters of image acquisition equipment corresponding to the image acquisition equipment identifier according to the image acquisition equipment identifier;

the method comprises the steps of locking the position of a falling object according to equipment coordinate parameters, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and determining a target falling track of the falling object according to the coordinate parameters;

and tracking the falling object according to the falling track of the target.

In one embodiment, the obtaining a set of multi-frame captured images from which the identifying the falling object comprises:

acquiring a capturing time interval corresponding to a multi-frame captured image set and a frame number identifier carried by each frame captured image;

obtaining an object image set to be recognized in each frame of captured image according to a preset target detection algorithm;

determining the capturing sequence of the captured images according to the frame number identification carried by each frame of captured image;

according to the capturing sequence of the captured images, calculating first distance information and second distance information of each object image to be recognized in the object image set to be recognized, wherein the first distance information and the second distance information are position distance information of each object image to be recognized in any two frames of captured images;

and calculating a distance difference value of the first distance information and the second distance information, and identifying the falling object according to the distance difference value and the capturing time interval.

In one embodiment, identifying the falling object based on the distance difference and the capture time interval comprises:

calculating the acceleration of each object to be recognized according to the distance difference and the capturing time interval;

calculating an acceleration difference value between the acceleration and the gravity acceleration;

when the acceleration difference is smaller than a preset difference threshold value, determining that the object to be identified is a falling object;

when the acceleration difference is larger than the difference threshold, acquiring the tracking times, calculating the acceleration difference between the acceleration of the object to be recognized and the gravity acceleration within a plurality of capturing time intervals of the tracking times according to the capturing sequence of the captured images, and when the acceleration difference within any capturing time interval is smaller than the difference threshold, determining that the object to be recognized is a falling object.

In one embodiment, the method for determining the target falling track of the falling object according to the coordinate parameters comprises the following steps of:

determining an image acquisition equipment set belonging to the same group according to the image acquisition equipment identification;

acquiring equipment coordinate parameters of each image acquisition equipment in each group of image acquisition equipment set;

according to the captured images and the equipment coordinate parameters acquired by the image acquisition equipment, locking the position of the falling object, and obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object;

fitting a paraboloid of the falling object according to the coordinate parameters, and determining an initial falling track of the falling object according to the paraboloid of the falling object;

and continuously acquiring a to-be-analyzed captured image set from the image acquisition equipment according to a preset acquisition interval, and acquiring a target falling track of the falling object according to the to-be-analyzed captured image set and the initial falling track.

In one embodiment, fitting a parabolic surface of the falling object according to the coordinate parameters, and determining an initial falling trajectory of the falling object according to the parabolic surface of the falling object comprises:

fitting the paraboloid of the falling object according to the coordinate parameters and the corresponding relation between the paraboloid of the falling object and a preset coordinate system in a preset range of the position of the falling object;

obtaining coordinate information of a plurality of falling objects according to the coordinate parameters, and obtaining a horizontal displacement distance and a vertical displacement distance according to the coordinate information of the falling objects, wherein the horizontal displacement distance and the vertical displacement distance are displacement difference values between any two falling object coordinates in a paraboloid of the falling objects;

acquiring displacement time corresponding to the horizontal displacement distance and the vertical displacement distance;

and determining the initial falling track of the falling object according to the horizontal displacement distance, the vertical displacement distance and the displacement time.

In one embodiment, the continuously acquiring the set of captured images to be analyzed from the image acquisition device according to the preset acquisition interval, and the obtaining the target falling trajectory of the falling object according to the set of captured images to be analyzed and the initial falling trajectory includes:

continuously acquiring a set of captured images to be analyzed from the image acquisition equipment according to a preset acquisition interval;

obtaining a plurality of coordinate parameters to be processed of the falling object within a preset range of the position of the falling object according to the set of captured images to be analyzed and the equipment coordinate parameters of the image acquisition equipment;

and correcting the initial falling track according to the coordinate parameters to be processed to obtain the target falling track of the falling object.

In one embodiment, after tracking the falling object according to the target falling trajectory, the method further comprises the following steps:

determining horizontal coordinate information and vertical coordinate information of the falling object on a paraboloid of the falling object according to the falling object track;

comparing a preset building coordinate system according to the horizontal coordinate information and the vertical coordinate information, and determining target coordinate information of the falling object on the building coordinate system;

acquiring the number of a capture net corresponding to the falling track on a building coordinate system according to the target coordinate information;

outputting a control command to a capturing net corresponding to the falling track according to the number of the capturing net, and capturing the falling object by the capturing net;

and determining a dangerous area according to the target coordinate information, and updating a preset environment map according to the determined coordinate information of the dangerous area.

A fall tracking device, the device comprising:

the first acquisition module is used for acquiring a multi-frame captured image set, identifying the falling object according to the multi-frame captured image set and obtaining a captured image set comprising images of the falling object;

the second acquisition module is used for acquiring an image acquisition equipment identifier corresponding to the captured image set comprising the falling object image according to the captured image set comprising the falling object image, and acquiring an equipment coordinate parameter of the image acquisition equipment corresponding to the image acquisition equipment identifier according to the image acquisition equipment identifier;

the processing module is used for locking the position of the falling object according to the equipment coordinate parameters, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and determining a target falling track of the falling object according to the coordinate parameters;

and the tracking module is used for tracking the falling object according to the falling track of the target.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

and tracking the falling object according to the falling track of the target.

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

and tracking the falling object according to the falling track of the target.

The method, the device, the computer equipment and the storage medium for tracking the falling object identify the falling object by acquiring and comparing the multi-frame captured image set to obtain the captured image set comprising the images of the falling object, acquire the image acquisition equipment identification corresponding to the captured image set comprising the images of the falling object according to the captured image set comprising the images of the falling object, acquire the corresponding equipment coordinate parameter according to the image acquisition equipment identification, lock the position of the falling object according to the equipment coordinate parameter, obtain a plurality of coordinate parameters of the falling object within the preset range of the position of the falling object, determine the target falling track of the falling object according to the coordinate parameters, and track the falling object according to the target falling track. Whole process, accurate discernment fallen object and the falling object of tracking according to the target orbit that falls to can reply the fallen object according to the pursuit result, showing and reducing and being injured the risk by the falling object pounding.

Drawings

FIG. 1 is a schematic flow diagram of a method for fall tracking in one embodiment;

FIG. 2 is a schematic sub-flow chart illustrating step S102 of FIG. 1 according to an embodiment;

FIG. 3 is a schematic sub-flow chart illustrating step S210 in FIG. 2 according to an embodiment;

FIG. 4 is a schematic illustration of a sub-flow chart of step S106 in FIG. 1 according to an embodiment;

FIG. 5 is a schematic sub-flow chart illustrating step S408 of FIG. 4 according to an embodiment;

FIG. 6 is a schematic sub-flow chart illustrating step S410 of FIG. 4 according to an embodiment;

FIG. 7 is a schematic flow chart of a method of fall tracking in another embodiment;

FIG. 8 is a block diagram of a device for drop tracking in one embodiment;

FIG. 9 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In one embodiment, as shown in fig. 1, there is provided a method of fall tracking, the method comprising the steps of:

s102: acquiring a multi-frame captured image set, and identifying the falling object according to the multi-frame captured image set to obtain a captured image set comprising images of the falling object.

A frame is a single video frame of the smallest unit in a video animation. One frame is a still picture, and the continuous frames form the animation. The multi-frame captured image set is obtained from image acquisition equipment, the image acquisition equipment is equipment which is arranged in a preset range and is used for capturing an object image, and the multi-frame captured image set at least comprises three captured images captured by the same image acquisition equipment. After acquiring the multi-frame captured image set, the server identifies the falling object according to the multi-frame captured image set to obtain a captured image set comprising images of the falling object. Specifically, the server is in communication with the image capture device, and the server can acquire data such as captured images from the image capture device by using 5G technology.

Identifying the falling object according to the multi-frame captured image set comprises obtaining an image of the object to be identified from the multi-frame captured images, and determining whether the object to be identified is the falling object according to the position of the image of the object to be identified in each frame captured image and the capturing time interval. Since the object to be recognized is affected only by gravity and resistance in the vertical direction, it is possible to determine whether the object to be recognized is a falling object by calculating the acceleration of the object to be recognized in the vertical direction.

S104: according to the captured image set comprising the falling object images, acquiring image acquisition equipment identifications corresponding to the captured image set comprising the falling object images, and according to the image acquisition equipment identifications, acquiring equipment coordinate parameters of image acquisition equipment corresponding to the image acquisition equipment identifications.

The captured image carries an image capture device identification that is used to determine the image capture device corresponding to the captured image. After the server obtains the captured image set comprising the images of the falling objects, the server can obtain image acquisition equipment identifications corresponding to the captured image set comprising the images of the falling objects according to the captured image set comprising the images of the falling objects, can determine the image acquisition equipment capturing the falling objects according to the image acquisition equipment identifications, and further can obtain the equipment coordinate parameters of the image acquisition equipment. The device coordinate parameter of the image acquisition device refers to a coordinate parameter of the image acquisition device in a preset coordinate system in a preset range. The preset coordinate system can be set as required, and is usually a spatial rectangular coordinate system. For example, if the preset environment refers to a construction site, the device coordinate parameter of the image capturing device refers to a position parameter of the image capturing device on the construction site.

S106: and locking the position of the falling object according to the coordinate parameters of the equipment, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and determining the target falling track of the falling object according to the coordinate parameters.

The image acquisition equipment takes a group as a unit, the server firstly determines the position information of the falling object in a captured image corresponding to the image acquisition equipment, and then the position of the falling object is locked according to the equipment coordinate parameters of the group of image acquisition equipment and the position information of the falling object in the captured image captured by the group of image acquisition equipment, and a plurality of coordinate parameters of the falling object in a preset range of the position of the falling object are obtained. Specifically, a coordinate parameter of the falling object can be determined by a set of captured images of each group of image acquisition devices, and the coordinate parameter refers to a group of coordinate values of the falling object within a preset range. The number of groups of image acquisition devices can be determined according to the size of the preset range and the sight range of the image acquisition devices. If the advance range of the image acquisition equipment can cover the whole preset range, only one group of image acquisition equipment is needed, and if the advance range of the image acquisition equipment can not cover, a plurality of groups of image acquisition equipment are needed.

The falling object falls from the high altitude, so that a corresponding relation exists between a paraboloid where the falling object is located and a preset coordinate system in a preset range, specifically, the corresponding relation can be that the falling surface where the falling object is located is perpendicular to a horizontal coordinate plane in the preset range, after a plurality of coordinate parameters are obtained, the server can fit the paraboloid of the falling object according to the plurality of coordinate parameters and the corresponding relation between the falling object and the preset coordinate system in the preset range, further convert the plurality of coordinate parameters into a plurality of falling object coordinate information with the paraboloid as a reference coordinate system according to the position relation between the paraboloid and the preset coordinate system, and determine the initial falling track of the falling object according to the falling object coordinate information.

After the initial falling trajectory is obtained, the initial falling trajectory needs to be corrected in real time to obtain a target falling trajectory due to the influence of wind resistance and errors. The specific method can be that, in the process that the falling object continuously falls, the server continuously acquires a set of images to be analyzed and captured from the image acquisition equipment according to a preset acquisition interval, and a plurality of coordinate parameters to be processed of the falling object in a preset range of the position where the falling object is located are obtained through analysis of the set of images to be analyzed and the equipment coordinate parameters of the image acquisition equipment, so that the paraboloid of the falling object is continuously fitted according to the coordinate parameters to be processed, the error between the fitted paraboloid and the actual paraboloid is reduced, and finally the target falling track of the falling object is determined according to the fitted paraboloid and the plurality of coordinate parameters to be processed, the error is reduced, and accurate calculation of the target falling track is achieved. Wherein, the preset acquisition interval can be set according to the requirement.

S108: and tracking the falling object according to the falling track of the target.

The falling trajectory refers to the falling path of the falling object. After obtaining the falling track of the falling object, can track the falling object according to the falling track of the falling object, predict the region that the falling object probably passes through in predetermineeing the within range, and then send out the warning to this region, reduce and injure the risk by the falling object pounding. Furthermore, if a falling object catching net is installed in the area, the catching net can be triggered to catch the falling object.

The falling object tracking method includes the steps of identifying a falling object by acquiring and comparing a multi-frame captured image set to obtain a captured image set comprising images of the falling object, acquiring an image acquisition equipment identifier corresponding to the captured image set comprising the images of the falling object according to the captured image set comprising the images of the falling object, acquiring corresponding equipment coordinate parameters according to the image acquisition equipment identifier, locking the position of the falling object according to the equipment coordinate parameters, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, determining a falling track of the falling object according to the coordinate parameters, and tracking the falling object according to a target falling track. Whole process, accurate discernment fallen object and the falling object of tracking according to the target orbit that falls to can reply the fallen object according to the pursuit result, showing and reducing and being injured the risk by the falling object pounding.

In one embodiment, as shown in fig. 2, S102 includes:

s202: acquiring a capturing time interval corresponding to a multi-frame captured image set and a frame number identifier carried by each frame captured image;

s204: obtaining an object image set to be recognized in each frame of captured image according to a preset target detection algorithm;

s206: determining the capturing sequence of the captured images according to the frame number identification carried by each frame of captured image;

s208: according to the capturing sequence of the captured images, calculating first distance information and second distance information of each object image to be recognized in the object image set to be recognized, wherein the first distance information and the second distance information are position distance information of each object image to be recognized in any two frames of captured images;

s210: and calculating a distance difference value of the first distance information and the second distance information, and identifying the falling object according to the distance difference value and the capturing time interval.

The capture time interval refers to the time interval of capturing the captured images by the image acquisition device, and the frame number identification refers to the sequence identification carried by each frame of captured image and used for determining the capture sequence of the captured images. The server can obtain an object image set to be recognized in each frame of captured image according to a preset target detection algorithm, the object image to be recognized refers to an abnormal object appearing in the captured image, the abnormal object is recognized by the target detection algorithm, and specifically, a common high-altitude falling object can be used as a sample set for training when the target detection algorithm is trained. The target detection algorithm may be a common target detection algorithm based on deep learning, and includes R-CNN (Region-CNN), yolo (young Only Look one), ssd (single Shot multi box detector), and so on.

After obtaining the set of the object images to be recognized, the server determines a capturing sequence of the captured images according to the frame number identifiers carried by the captured images of the frames, and further calculates first distance information and second distance information of the object images to be recognized in the set of the object images to be recognized according to the capturing sequence of the captured images, wherein the first distance information and the second distance information are position distance information of the object images to be recognized in any two captured images, and specifically, the position distance information refers to distance information of the object to be recognized in the vertical direction of any two captured images. The first distance information and the second distance information may be obtained by comparing positions of the images of the objects to be recognized in the captured images of the frames. Specifically, the distance difference may refer to a pixel distance between positions of the respective images of the object to be recognized in the respective captured images. After obtaining the distance difference, the server further identifies the falling object according to the distance difference and the capturing time interval.

In one embodiment, as shown in fig. 3, S210 includes:

s302: calculating the acceleration of each object to be recognized according to the distance difference and the capturing time interval;

s304: calculating an acceleration difference value between the acceleration and the gravity acceleration;

s306: when the acceleration difference is smaller than a preset difference threshold value, determining that the object to be identified is a falling object;

s308: when the acceleration difference is larger than the difference threshold, acquiring the tracking times, calculating the acceleration difference between the acceleration of the object to be recognized and the gravity acceleration within a plurality of capturing time intervals of the tracking times according to the capturing sequence of the captured images, and when the acceleration difference within any capturing time interval is smaller than the difference threshold, determining that the object to be recognized is a falling object.

Because the first distance information and the second distance information refer to the position distance information of each image of the object to be recognized in any two captured images, when the falling object is recognized, the distance difference value needs to be converted to obtain the corresponding distance value of the distance difference value in the preset coordinate system. Specifically, a scaling relationship between a unit distance in a preset coordinate system and a unit pixel distance in a captured image can be determined in a pre-calibration manner, so that the distance difference value is converted according to the scaling relationship to obtain a corresponding distance value in the preset coordinate system. And then calculating the acceleration of each object to be recognized according to the distance value and the capturing time interval. Furthermore, when the captured image contains a scaffold background, the scaffold background can be used to form a natural image size grid, and the corresponding distance value of the distance difference value in the preset coordinate system is obtained by calibrating the scaling relationship between the pixel distance and the scaffold grid distance in advance.

Because the object to be identified is only under the action of gravity and resistance after falling, and the time interval of capturing images in every two frames is extremely short, the object to be identified can be considered to do uniform motion in the horizontal direction, and then the object to be identified can do uniform motion in the vertical direction according to a common displacement formula s-0.5 a t²And calculating the acceleration of the object to be identified. For example, the calculation process may be: first distance information s1 ═ 0.5 ═ a ═ t1²The second distance information s2 is 0.5 a t2²Since t2-t1 can be obtained from the capturing order of captured images and the capturing time interval, the acceleration of the object to be recognized can be obtained by calculating s2-s 1. After the acceleration of the object to be recognized is obtained, the object to be recognized can be recognized by calculating the acceleration difference between the acceleration and the gravity acceleration. Specifically, when the acceleration difference is smaller than a preset difference threshold, it is determined that the object to be identified is a falling object. When the acceleration difference is greater than the difference threshold, the server needs to continuously track the object to be identified, and eliminate calculation errors and accidental errors, and the specific tracking mode can be as follows: acquiring tracking times, calculating the acceleration difference value between the acceleration of the object to be identified and the gravity acceleration within a plurality of capturing time intervals of the tracking times according to the capturing sequence of the captured images, and determining that the object to be identified is a falling object when the acceleration difference value within any capturing time interval is smaller than a difference threshold value. The threshold value of the difference here can be determined from the resistance to which the falling object is subjected. The resistance of the falling object mainly refers to air resistance, the air resistance is in direct proportion to the square of the falling speed of the object to be identified, and is also related to the air friction coefficient and the shape of the object to be identified, and in addition, the influence of wind speed and wind direction. Wherein, the information of wind speed and wind direction can be released from local meteorological departmentThe data acquisition of (2).

In one embodiment, as shown in fig. 4, S106 includes:

s402: determining an image acquisition equipment set belonging to the same group according to the image acquisition equipment identification;

s404: acquiring equipment coordinate parameters of each image acquisition equipment in each group of image acquisition equipment set;

s406: according to the captured images and the equipment coordinate parameters acquired by the image acquisition equipment, locking the position of the falling object, and obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object;

s408: fitting a paraboloid of the falling object according to the coordinate parameters, and determining an initial falling track of the falling object according to the paraboloid of the falling object;

s410: and continuously acquiring a to-be-analyzed captured image set from the image acquisition equipment according to a preset acquisition interval, and acquiring a target falling track of the falling object according to the to-be-analyzed captured image set and the initial falling track.

The server may determine, according to the image capturing device identifier, an image capturing device set belonging to the same group, and specifically, the image capturing device set of the same group may include two or three image capturing devices for capturing images in the preset range in an all-around manner. The server acquires equipment coordinate parameters of each image acquisition equipment in each group of image acquisition equipment set, wherein the equipment coordinate parameters comprise view angle parameter information and position parameter information of the image acquisition equipment. The server can determine the position information of the falling object in the captured image according to the captured image acquired by each image acquisition device, and further can convert the position information of the falling object in the captured image into the coordinate parameter of the falling object in the preset position range according to the view angle parameter information and the position parameter information. The position information refers to a set of coordinate values of the falling object in the captured image and comprises horizontal position information and vertical coordinate information, the coordinate parameter refers to a set of coordinate values of the falling object within a preset range, and the conversion of the position information into the coordinate parameter refers to the representation of the coordinate parameter by the position information. For example, when the set of image capturing devices includes a first image capturing device and a second image capturing device, the position information of the falling object captured by the first image capturing device is (x3, y3), and the position parameter information of the first image capturing identifier corresponding device is (x1, y1, z1), the viewing angle parameter information is (θ 1), and the direction (α 1, β 1, γ 1), the coordinate parameter of the XZ plane of the falling object within the preset range is obtained as:

p1(x，z)＝f(x1，y1，z1)(θ1)(α1，β1，γ1)(x3，y3)

when the position information of the falling object collected by the second image collecting device is (x4, y4), and the position parameter information (x2, y2, z2), the viewing angle parameter information (θ 2), and the direction (α 2, β 2, γ 2) of the second image collecting identification corresponding device, the coordinate parameters of the YZ plane of the falling object within the preset range are obtained as follows:

p1(y，z)＝f(x2，y2，z2)(θ2)(α2，β2，γ2)(x4，y4)

since the same coordinate information z2 exists in p1(x2, z2) and p1(y2, z2), the coordinate parameter of the falling object is p1(x2, y2, z2) according to p1(x2, z2) and p1(y2, z 2).

In one embodiment, as shown in fig. 5, S408 includes:

s502: fitting the paraboloid of the falling object according to the coordinate parameters and the corresponding relation between the paraboloid of the falling object and a preset coordinate system in a preset range of the position of the falling object;

s504: obtaining coordinate information of a plurality of falling objects according to the coordinate parameters, and obtaining a horizontal displacement distance and a vertical displacement distance according to the coordinate information of the falling objects, wherein the horizontal displacement distance and the vertical displacement distance are displacement difference values between any two falling object coordinates in a paraboloid of the falling objects;

s506: acquiring displacement time corresponding to the horizontal displacement distance and the vertical displacement distance;

s508: and determining the initial falling track of the falling object according to the horizontal displacement distance, the vertical displacement distance and the displacement time.

Because the falling object falls from the high altitude, a corresponding relation exists between the paraboloid where the falling object is located and a preset coordinate system in a preset range, specifically, the corresponding relation can be that the falling surface where the falling object is located is perpendicular to a horizontal coordinate plane in the preset range, and after a plurality of coordinate parameters are obtained, the server can fit the paraboloid of the falling object according to the coordinate parameters and the corresponding relation between the falling object and the preset coordinate system in the preset range. For example, because the paraboloid where the falling object is located has a corresponding relationship with a preset coordinate system in a preset range, a linear equation corresponding to the paraboloid where the falling object is located can be obtained through calculation of a plurality of coordinate parameters, an included angle between the paraboloid where the falling object is located and a certain coordinate plane of the preset coordinate system is calculated through the plurality of coordinate parameters, and finally the corresponding paraboloid is determined through the linear equation and the included angle.

After the paraboloid is obtained, firstly, the server converts a plurality of coordinate parameters into a plurality of pieces of falling object coordinate information by taking the paraboloid as a reference coordinate system according to the position relation between the paraboloid and a preset coordinate system, specifically the included angle between the paraboloid and a certain coordinate plane of the preset coordinate system, and obtains the horizontal displacement distance and the vertical displacement distance of the falling object according to the falling object coordinate information, wherein the horizontal displacement distance and the vertical displacement distance are displacement difference values between any two falling object coordinates in the paraboloid of the falling object. The server then obtains the displacement time corresponding to the horizontal displacement distance and the vertical displacement distance, where the displacement time corresponding to the horizontal displacement distance and the vertical displacement distance are the same. And finally, the server determines the initial falling track of the falling object according to the horizontal displacement distance, the vertical displacement distance and the displacement time. Specifically, the server firstly calculates the initial velocity v1 of the falling object in the horizontal direction according to the horizontal displacement distance and the displacement time, and calculates the initial velocity v2 of the falling object in the vertical direction according to the vertical displacement distance and the displacement time, so that a falling object curve is obtained according to v1 and v 2.

For example, the calculation method may be: first the server gets two points (m1, n1), (m2, n2) within the parabola, and the displacement time t between these two points. Because the wind resistance is neglected in a very short time t, the horizontal direction can be approximately regarded as that the falling object does uniform linear motion, and m2-m1 is v1 t, and v1 is the horizontal initial speed. In the vertical direction, n2-n1 ═ v2 ═ t +0.5 ═ a ^ t²Neglecting wind resistance, a can be approximated as gravity acceleration in the vertical direction, and then v2 can be calculated, so that the expression of the coordinate of any falling object on the falling object curve is (v1 t, v2 t +0.5 a t)²)。

In one embodiment, as shown in fig. 6, S410 includes:

s602: continuously acquiring a set of captured images to be analyzed from the image acquisition equipment according to a preset acquisition interval;

s604: obtaining a plurality of coordinate parameters to be processed of the falling object within a preset range of the position of the falling object according to the set of captured images to be analyzed and the equipment coordinate parameters of the image acquisition equipment;

s606: and correcting the initial falling track according to the coordinate parameters to be processed to obtain the target falling track of the falling object.

In one embodiment, as shown in fig. 7, after S108, the method further includes:

s702: determining horizontal coordinate information and vertical coordinate information of the falling object on a paraboloid of the falling object according to the falling object track;

s704: comparing a preset building coordinate system according to the horizontal coordinate information and the vertical coordinate information, and determining target coordinate information of the falling object on the building coordinate system;

s706: acquiring the number of a capture net corresponding to the falling track on a building coordinate system according to the target coordinate information;

s708: outputting a control command to a capturing net corresponding to the falling track according to the number of the capturing net, and capturing the falling object by the capturing net;

s710: and determining a dangerous area according to the target coordinate information, and updating a preset environment map according to the determined coordinate information of the dangerous area.

The server can determine horizontal coordinate information and vertical coordinate information of the falling object on a paraboloid of the falling object according to the falling track, and then compares the horizontal coordinate information and the vertical coordinate information with a preset building coordinate system to determine target coordinate information of the falling object on the building coordinate system. Specifically, the building coordinate system corresponds to the preset coordinate system, and the corresponding relationship between the paraboloid and the building coordinate system can be determined through the corresponding relationship between the paraboloid and the preset coordinate system. After the target coordinate information of the falling object on the building coordinate system is determined, the server can calculate a parabolic curve of the falling object under the building coordinate system according to the target coordinate information, prolong the parabolic curve to obtain an intersection point of the parabolic curve and a preset capturing net plane, and determine a capturing net number corresponding to the falling track according to the intersection point.

Specifically, catch the net and set up on building site apart from the plane of ground preset height on, preset height and can set up as required by oneself. The length and the width of the catching net are known, so that the catching range of each catching net can be defined in the plane of the catching net, the corresponding catching net can be determined according to the intersection point of the parabolic curve and the plane of the catching net, the falling object can be predicted to fall into the catching range of the catching net, the number of the catching net corresponding to the falling track is further obtained, a control command is output to the catching net corresponding to the falling track according to the number of the catching net, the falling object is caught by the catching net, and therefore the risk of being injured by the falling object can be reduced. Specifically, the server communicates with the corresponding capture net, and the server outputs the control command to the capture net corresponding to the falling trajectory by using a 5G technology.

After the target coordinate information of the falling object on the building coordinate system is determined, the server can calculate a parabolic curve of the falling object under the building coordinate system according to the target coordinate information, the parabolic curve is prolonged to obtain an intersection point of the parabolic curve and the ground, a possible falling point of the falling object is predicted according to the intersection point of the parabolic curve and the ground, a dangerous area is defined according to the possible falling point, and a preset environment map is updated according to the determined coordinate information of the dangerous area.

It should be understood that although the various steps in the flow charts of fig. 1-7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-7 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 8, there is provided a falling object tracking device comprising: a first acquisition module 802, a second acquisition module 804, a processing module 806, and a tracking module 808, wherein:

a first obtaining module 802, configured to obtain a set of multiple frames of captured images, identify a falling object according to the set of multiple frames of captured images, and obtain a set of captured images including images of the falling object;

a second obtaining module 804, configured to obtain, according to the captured image set including the falling object image, an image capturing device identifier corresponding to the captured image set including the falling object image, and obtain, according to the image capturing device identifier, a device coordinate parameter of the image capturing device corresponding to the image capturing device identifier;

the processing module 806 is configured to lock a position of the falling object according to the device coordinate parameters, obtain a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and determine a target falling track of the falling object according to the coordinate parameters;

and the tracking module 808 is used for tracking the falling object according to the target falling track.

The device for tracking the falling object comprises a capturing image set, an image acquisition equipment identifier, an equipment coordinate parameter, a plurality of coordinate parameters, a target falling track and a falling object tracking device, wherein the capturing image set comprises falling object images, the image acquisition equipment identifier corresponds to the capturing image set comprising the falling object images, the corresponding equipment coordinate parameter is acquired according to the image acquisition equipment identifier, the position of the falling object is locked according to the equipment coordinate parameter, the coordinate parameters of the falling object in the preset range of the position of the falling object are acquired, the target falling track of the falling object is determined according to the coordinate parameters, and the falling object is tracked according to the target falling track. Whole process, accurate discernment fallen object and the falling object of tracking according to the target orbit that falls to can reply the fallen object according to the pursuit result, showing and reducing and being injured the risk by the falling object pounding.

In one embodiment, the first obtaining module is further configured to obtain a capturing time interval corresponding to the multiple frames of captured image sets and a frame number identifier carried by each frame of captured image, obtain an image set of an object to be recognized in each frame of captured image according to a preset target detection algorithm, determine a capturing order of the captured images according to the frame number identifier carried by each frame of captured image, calculate first distance information and second distance information of each image of the object to be recognized in the image set of the object to be recognized according to the capturing order of the captured images, where the first distance information and the second distance information are position distance information of each image of the object to be recognized in any two frames of captured images, calculate a distance difference between the first distance information and the second distance information, and recognize a falling object according to the distance difference and the capturing time interval.

In one embodiment, the first obtaining module is further configured to calculate an acceleration of each object to be recognized according to the distance difference and the capturing time interval, calculate an acceleration difference between the acceleration and the gravitational acceleration, determine that the object to be recognized is a falling object when the acceleration difference is smaller than a preset difference threshold, obtain the tracking number when the acceleration difference is larger than the difference threshold, calculate an acceleration difference between the acceleration of the object to be recognized and the gravitational acceleration within a plurality of capturing time intervals according to a capturing sequence of captured images, and determine that the object to be recognized is the falling object when the acceleration difference within any capturing time interval is smaller than the difference threshold.

In one embodiment, the processing module is further configured to determine, according to the image acquisition device identifier, an image acquisition device set belonging to the same group, acquire device coordinate parameters of each image acquisition device in each image acquisition device set, lock the position of the falling object according to a captured image and the device coordinate parameters acquired by each image acquisition device, obtain a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, fit a paraboloid of the falling object according to the coordinate parameters, determine an initial falling trajectory of the falling object according to the paraboloid of the falling object, continuously acquire, according to a preset acquisition interval, a captured image set to be analyzed from the image acquisition device, and obtain a target falling trajectory of the falling object according to the captured image set to be analyzed and the initial falling trajectory.

In one embodiment, the processing module is further configured to fit the paraboloid of the falling object according to the coordinate parameters and the corresponding relationship between the paraboloid of the falling object and a preset coordinate system in a preset range of the position of the falling object, obtain coordinate information of a plurality of falling objects according to the coordinate parameters, obtain a horizontal displacement distance and a vertical displacement distance according to the coordinate information of the falling object, obtain displacement time corresponding to the horizontal displacement distance and the vertical displacement distance, and determine the initial falling track of the falling object according to the horizontal displacement distance, the vertical displacement distance and the displacement time, wherein the horizontal displacement distance and the vertical displacement distance are displacement difference values between any two falling object coordinates in the paraboloid of the falling object.

In one embodiment, the processing module is further configured to continuously acquire a set of captured images to be analyzed from the image acquisition device according to a preset acquisition interval, obtain a plurality of coordinate parameters to be processed of the falling object within a preset range of the position of the falling object according to the set of captured images to be analyzed and the device coordinate parameters of the image acquisition device, and correct the initial falling trajectory according to the coordinate parameters to be processed to obtain a target falling trajectory of the falling object.

In one embodiment, the tracking module is further configured to determine horizontal coordinate information and vertical coordinate information of a paraboloid of the falling object according to the falling object trajectory, compare a preset building coordinate system according to the horizontal coordinate information and the vertical coordinate information, determine target coordinate information of the falling object on the building coordinate system, obtain a number of a capture net corresponding to the falling trajectory on the building coordinate system according to the target coordinate information, output a control command to the capture net corresponding to the falling trajectory according to the number of the capture net, capture the falling object by the capture net, determine a dangerous area according to the target coordinate information, and update a preset environment map according to the coordinate information of the determined dangerous area.

In one embodiment, the device for tracking falling objects further comprises a communication module for communicating with other devices, and specifically, the communication module may be at least one of a 3G module, a 4G module, a 5G module and a bluetooth module.

For the specific definition of the means for fall tracking, reference may be made to the above definition of the method for fall tracking, which is not described in detail here. The modules in the above described falling object tracking apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of fall tracking.

Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, there is provided a computer device comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program:

and tracking the falling object according to the falling track of the target.

The computer equipment for tracking the falling object identifies the falling object by acquiring and comparing a multi-frame captured image set to obtain a captured image set comprising images of the falling object, acquires an image acquisition equipment identifier corresponding to the captured image set comprising the images of the falling object according to the captured image set comprising the images of the falling object, acquires a corresponding equipment coordinate parameter according to the image acquisition equipment identifier, locks the position of the falling object according to the equipment coordinate parameter, obtains a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, determines a target falling track of the falling object according to the coordinate parameters, and tracks the falling object according to the target falling track. Whole process, accurate discernment fallen object and the falling object of tracking according to the target orbit that falls to can reply the fallen object according to the pursuit result, showing and reducing and being injured the risk by the falling object pounding.

In one embodiment, the processor, when executing the computer program, further performs the steps of:

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

and tracking the falling object according to the falling track of the target.

The storage medium for tracking the falling object identifies the falling object by acquiring and comparing a multi-frame captured image set to obtain a captured image set comprising images of the falling object, acquires an image acquisition device identifier corresponding to the captured image set comprising the images of the falling object according to the captured image set comprising the images of the falling object, acquires a corresponding device coordinate parameter according to the image acquisition device identifier, locks the position of the falling object according to the device coordinate parameter, obtains a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, determines a target falling track of the falling object according to the coordinate parameters, and tracks the falling object according to the target falling track. Whole process, accurate discernment fallen object and the falling object of tracking according to the target orbit that falls to can reply the fallen object according to the pursuit result, showing and reducing and being injured the risk by the falling object pounding.

In one embodiment, the computer program when executed by the processor further performs the steps of:

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of fall tracking, the method comprising:

acquiring an image acquisition equipment identifier corresponding to the captured image set comprising the falling object image according to the captured image set comprising the falling object image, and acquiring an equipment coordinate parameter of the image acquisition equipment corresponding to the image acquisition equipment identifier according to the image acquisition equipment identifier;

locking the position of the falling object according to the equipment coordinate parameters, obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and determining a target falling track of the falling object according to the coordinate parameters;

and tracking the falling object according to the target falling track.

2. The method of claim 1, wherein the obtaining a set of multi-frame captured images from which identifying a fall comprises:

acquiring capture time intervals corresponding to the multi-frame captured image set and frame number identifications carried by each frame captured image;

3. The method of claim 2, wherein identifying a fall based on the distance difference and the capture time interval comprises:

when the acceleration difference value is larger than the difference threshold value, acquiring tracking times, calculating the acceleration difference value between the acceleration of the object to be recognized and the gravity acceleration within a plurality of capturing time intervals of the tracking times according to the capturing sequence of the captured images, and when the acceleration difference value within any capturing time interval is smaller than the difference threshold value, determining that the object to be recognized is a falling object.

4. The method of claim 1, wherein the step of locking the position of the falling object according to the device coordinate parameters and obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object, and the step of determining the target falling trajectory of the falling object according to the coordinate parameters comprises:

according to the captured images acquired by each image acquisition device and the device coordinate parameters, locking the position of the falling object, and obtaining a plurality of coordinate parameters of the falling object within a preset range of the position of the falling object;

5. The method of claim 4, wherein the fitting a parabolic surface of the fall to the coordinate parameter, and wherein determining the initial fall trajectory of the fall from the parabolic surface of the fall comprises:

6. The method of claim 4, wherein the continuously acquiring the set of captured images to be analyzed from the image acquisition device according to a preset acquisition interval, and the obtaining the target falling trajectory of the falling object according to the set of captured images to be analyzed and the initial falling trajectory comprises:

and correcting the initial falling track according to the coordinate parameters to be processed to obtain a target falling track of the falling object.

7. The method of claim 1, further comprising, after said tracking said fall according to said target fall trajectory:

acquiring a catching net number corresponding to the falling track on the building coordinate system according to the target coordinate information;

outputting a control command to a capturing net corresponding to a falling track according to the number of the capturing net, and capturing the falling object by the capturing net;

8. A fall tracking device, the device comprising:

the first acquisition module is used for acquiring a multi-frame captured image set, identifying a falling object according to the multi-frame captured image set and obtaining a captured image set comprising images of the falling object;

and the tracking module is used for tracking the falling object according to the target falling track.

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.