CN116309795A

CN116309795A - Swimming pool drowning-prevention human head three-dimensional positioning method and device, computer equipment and storage medium

Info

Publication number: CN116309795A
Application number: CN202211638463.1A
Authority: CN
Inventors: 任小枫; 谢欣; 郭羽; 张剑华; 王振华; 郭东岩; 张都思
Original assignee: Hangzhou Juyan Xincheng Technology Co ltd
Current assignee: Hangzhou Juyan Xincheng Technology Co ltd
Priority date: 2022-12-20
Filing date: 2022-12-20
Publication date: 2023-06-23

Abstract

The embodiment of the invention discloses a swimming pool drowning-prevention head three-dimensional positioning method, a swimming pool drowning-prevention head three-dimensional positioning device, computer equipment and a storage medium. The method comprises the following steps: restoring the plane parameters of the water surface of the swimming pool; determining a mapping relation of multiple cameras; triangulating the human head point pairs to obtain the spatial position of the human head; determining the distance from the head to the water surface according to the space position; judging whether the head is positioned on water or underwater by combining the direction of the normal vector of the water surface with the distance from the head to the water surface so as to obtain a judgment result; and generating corresponding information according to the judging result. By implementing the method of the embodiment of the invention, the three-dimensional space positioning of the head of the person can be realized, and the head of the swimmer can be accurately judged to be on water or under water.

Description

Swimming pool drowning-prevention human head three-dimensional positioning method and device, computer equipment and storage medium

Technical Field

The invention relates to a human head positioning method, in particular to a swimming pool drowning prevention human head three-dimensional positioning method, a swimming pool drowning prevention human head three-dimensional positioning device, a swimming pool drowning prevention human head three-dimensional positioning computer device and a swimming pool drowning prevention human head storage medium.

Background

57000 people die each year in China after drowning; the Chinese drowning mortality rate is 8.77% which is equivalent to 150 people per day, wherein the death rate is 56.58% of the ages of 0 to 14, the first cause of death of the age group is more prominent in rural areas, and the drowning rate is more found in swimming pools in developed countries.

The existing swimming pool has no specific intelligent security measures, and is characterized in that management personnel monitor swimming guests in water through human eyes, visual fatigue is easily caused through human eye monitoring, and drowning is easily caused by untimely rescue of people falling into water; although some swimming pools can be intelligently judged according to cameras, swimming guests are easily affected by water waves under a complex water surface background, the problems of water and water are difficult to directly judge, and the judgment accuracy is affected.

Therefore, a new method is needed to be designed to realize three-dimensional space positioning of the head and accurately judge whether the head of the swimming guest is on water or under water.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a three-dimensional positioning method, a device, computer equipment and a storage medium for preventing drowning of a human head in a swimming pool.

In order to achieve the above purpose, the present invention adopts the following technical scheme: a swimming pool drowning prevention three-dimensional positioning method comprises the following steps:

restoring the plane parameters of the water surface of the swimming pool;

determining a mapping relation of multiple cameras;

triangulating the human head point pairs to obtain the spatial position of the human head;

determining the distance from the head to the water surface according to the space position;

judging whether the head is positioned on water or underwater by combining the direction of the normal vector of the water surface with the distance from the head to the water surface so as to obtain a judgment result;

And generating corresponding information according to the judging result.

The further technical scheme is as follows: the restoration of the plane parameters of the swimming pool water surface comprises the following steps:

acquiring images of a plurality of cameras;

performing de-distortion treatment on the image to obtain a treatment result;

extracting swimming pool water surface characteristic points from the processing results;

matching the swimming pool water surface characteristic points and calculating the space position;

fitting the swimming pool water surface by using the space position;

and carrying out multi-view optimization on plane parameters according to the water surface of the swimming pool.

The further technical scheme is as follows: the matching of the swimming pool water surface characteristic points and the calculation of the space position comprise the following steps:

matching the swimming pool water surface characteristic points by utilizing a multi-camera space fusion technology to obtain matched characteristic points;

and triangulating the matched characteristic points to obtain three-dimensional coordinates of the characteristic points, wherein the dimensions of the three-dimensional coordinates are consistent with those of the relative pose between cameras.

The further technical scheme is as follows: the multi-view optimization plane parameters according to the swimming pool water surface comprise:

and constructing an optimization problem through the fact that the plane point-to-plane distance of the swimming pool water surface is zero, and determining a residual error to optimize plane parameters.

The further technical scheme is as follows: the triangularizing the human head point pairs to obtain a spatial position of the human head includes:

determining the position of the head of the person based on the YOLO to obtain a target pixel;

mapping the target pixel to the corresponding area of the pixel plane of other cameras to obtain a target area;

judging the confidence coefficient of the head of a person detected by each camera under a target area according to the area confidence coefficient priori of the plurality of cameras, and filtering out the cameras with the area confidence coefficient meeting the requirement so as to obtain candidate cameras;

according to the orientation priori of the candidate camera, finely dividing a target area and the neighborhood of the target area, and searching the matched head to obtain a search result;

and triangulating the pixel points in the search result, and determining the three-dimensional coordinates of the human head in a unified space coordinate system to obtain the space position of the human head.

The further technical scheme is as follows: the step of determining the distance from the head to the water surface according to the space position comprises the following steps:

according to the spatial position of the head and the spatial expression of the swimming pool water surface, the distance from the head to the horizontal plane and the direction of the normal vector of the horizontal plane are calculated.

The invention also provides a swimming pool drowning-prevention human head three-dimensional positioning device, which comprises:

The parameter recovery unit is used for recovering the plane parameters of the water surface of the swimming pool;

a relationship determining unit for determining a mapping relationship of the plurality of cameras;

the position acquisition unit is used for triangulating the human head point pairs so as to acquire the spatial position of the human head;

the distance determining unit is used for determining the distance from the head to the water surface according to the space position;

the judging unit is used for judging whether the head is positioned on water or the head is positioned under water according to the direction of the normal vector of the water surface and the distance from the head to the water surface so as to obtain a judging result;

and the information generating unit is used for generating corresponding information according to the judging result.

The further technical scheme is as follows: the parameter recovery unit includes:

an image acquisition subunit for acquiring images of a plurality of cameras;

a processing subunit, configured to perform de-distortion processing on the image to obtain a processing result;

the characteristic point extraction subunit is used for extracting characteristic points of the swimming pool water surface from the processing result;

the matching subunit is used for matching the swimming pool water surface characteristic points and calculating the space position;

a fitting subunit, configured to fit the swimming pool water surface with the spatial position;

and the optimizing subunit is used for optimizing plane parameters at multiple angles according to the water surface of the swimming pool.

The invention also provides a computer device which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the method when executing the computer program.

The present invention also provides a storage medium storing a computer program which, when executed by a processor, implements the above method.

Compared with the prior art, the invention has the beneficial effects that: according to the invention, by integrating the space positioning technology of multi-view geometry and bundle set optimization, combining the optimization of the swimming pool water surface, the fusion of the mapping relation of multiple cameras and the common view space, the space position of the head is determined, and the relationship between the head and the water surface is determined by adding the swimming pool water surface, so that the three-dimensional space positioning of the head is realized, and the judgment of the head of a swimming guest on water or under water is accurately carried out.

The invention is further described below with reference to the drawings and specific embodiments.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an application scenario of a method for three-dimensional positioning of a swimming pool drowning prevention head according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a method for three-dimensional positioning of a swimming pool anti-drowning head according to an embodiment of the present invention;

FIG. 3 is a schematic view of a sub-flow of a method for three-dimensional positioning of a swimming pool anti-drowning head according to an embodiment of the present invention;

FIG. 4 is a schematic view of a sub-flowchart of a method for three-dimensional positioning of a swimming pool anti-drowning head according to an embodiment of the present invention;

FIG. 5 is a schematic view of a sub-flowchart of a method for three-dimensional positioning of a swimming pool anti-drowning head according to an embodiment of the present invention;

FIG. 6 is a schematic block diagram of a three-dimensional positioning device for preventing drowning of a human head in a swimming pool according to an embodiment of the present invention;

FIG. 7 is a schematic block diagram of a parameter recovery unit of a three-dimensional positioning device for preventing drowning of a human head in a swimming pool according to an embodiment of the present invention;

FIG. 8 is a schematic block diagram of a matching subunit of a three-dimensional positioning device for preventing drowning of a human head in a swimming pool, provided by an embodiment of the present invention;

FIG. 9 is a schematic block diagram of a position acquisition unit of a three-dimensional positioning device for preventing drowning of a human head in a swimming pool, provided by an embodiment of the present invention;

fig. 10 is a schematic block diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

Referring to fig. 1 and fig. 2, fig. 1 is a schematic diagram of an application scenario of a method for three-dimensional positioning of a swimming pool anti-drowning head according to an embodiment of the present invention. Fig. 2 is a schematic flow chart of a three-dimensional positioning method for preventing drowning of a human head in a swimming pool according to an embodiment of the present invention. The method for three-dimensionally positioning the drowned head of the swimming pool is applied to the server. The server, the camera and the terminal perform data interaction, a space positioning technology integrating multi-view geometry and bundling optimization is realized, three-dimensional space positioning is performed on the head of a person, and the head of a swimming guest is accurately judged to be on water or under water.

Fig. 2 is a schematic flow chart of a method for three-dimensional positioning of a swimming pool drowning prevention head according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S160.

S110, restoring the plane parameters of the water surface of the swimming pool.

In this embodiment, the plane parameters of the water surface of the swimming pool are restored, the distance from the head point to the water surface is obtained, and the head is determined to be on the water or under the water according to the direction of the normal vector of the water surface.

The plane parameters of the water surface of the swimming pool comprise mathematical expressions for fitting the water surface of the swimming pool as

A, B, C, D in (a).

In one embodiment, referring to fig. 3, the step S110 may include steps S111 to S116.

S111, acquiring images of a plurality of cameras.

In this embodiment, the images of the plurality of cameras refer to images of the water surface of the swimming pool taken by the plurality of cameras.

S112, performing de-distortion processing on the image to obtain a processing result.

In the present embodiment, the processing result refers to a result formed after the image is subjected to the de-distortion processing.

S113, extracting the characteristic points of the water surface of the swimming pool from the processing result.

In this embodiment, mask masks are added to the images, and horizontal plane feature points of two camera images are extracted.

And S114, matching the swimming pool water surface characteristic points and calculating the space position.

In this embodiment, the spatial location refers to the location of the characteristic points of the water surface of the pool determined in conjunction with a plurality of cameras.

In one embodiment, referring to fig. 4, the step S114 may include steps S1141 to S1142.

S1141, matching the swimming pool water surface characteristic points by utilizing a multi-camera space fusion technology so as to obtain the matched characteristic points.

In this embodiment, the matched feature points refer to matching and spatial fusion of the feature points of the water surface of the swimming pool obtained by different cameras, so as to determine the final position.

Specifically, mapping the region where the target pixel is located to a certain region of the pixel plane of other cameras by means of natural region division of lanes and priori information of camera orientations so as to obtain a mapping relation; and carrying out common view space fusion according to the mapping relation to obtain a fusion result.

Specifically, after feature points of two camera images are extracted and matched, three-dimensional coordinates of the feature points can be obtained through triangulation, but because of large parallax between every two cameras in a swimming pool and similar horizontal plane environments of the swimming pool, most of matching point pairs obtained by a system are wrong, and when the RANSAC algorithm is used for matching and screening SIFT features, correct matching point pairs cannot be screened. In addition, in a swimming pool environment with a complex large scene, the ultra-large parallax among multiple cameras can also cause that the feature point pairs cannot be screened by using global homography transformation. And constructing a mask map of the fine area division of the camera image by means of the natural area division of the lanes and the prior information of the camera orientation.

Specifically, because the floating balls in each area of the lane line of the swimming pool are inconsistent in color, a mask diagram of the fine area division of the camera image is constructed by means of the natural area division of the lane and the prior information of the camera orientation.

And determining a layering region mapping relation between two cameras according to the mask map, and mapping the region where the target pixel is located to a certain region of the pixel plane of other cameras to obtain the mapping relation.

Specifically, because the swimming pool camera is static and not moving, the constructed mask map determines swimming pool areas corresponding to pixels in the field of view of the camera, determines a hierarchical area mapping relationship between two cameras according to the mask map, and maps the area where the target pixel is located to a certain area of the pixel plane of other cameras to obtain the mapping relationship.

In this embodiment, the mapping relationship includes a mapping relationship of a region where the target pixel is located and a neighborhood of the region where the target pixel is located.

Specifically, a hierarchical region mapping relationship between two phases is obtained, including a mapping relationship of a target region and a neighborhood of the target region.

And carrying out common view space fusion according to the mapping relation to obtain a fusion result.

In this embodiment, the fusion result refers to a result formed after fusion of the multi-camera common view space.

In one embodiment, the steps may include:

a specific transmission transformation relation is calculated for the camera image refinement region to obtain a local transmission transformation relation.

In the present embodiment, the local transmission conversion relationship refers to a transmission conversion relationship for each of the finer areas.

And replacing the global transmission transformation relation between every two camera images by using the local transmission transformation relation.

In this embodiment, according to the hierarchical region mapping, the camera image can be divided into multiple small regions, a coarse co-view space mapping relationship is obtained, a specific transmission transformation relationship is calculated for each matched small region, instead of directly calculating the global transmission transformation relationship between every two camera images, and distortion and ghost generated when the existing method is used for splicing large parallax images can be effectively reduced on the splicing effect.

And carrying out common-view space fusion according to the local transmission transformation relation to obtain a fusion result.

Specifically, through the region where the pixel point is located and the transmission transformation relation of the region, the pixel position of the pixel point in other camera images can be calculated, so that a pixel point pair matched among multiple cameras is obtained, and the fusion of fine multi-camera common view space is realized; firstly, determining a matched common view area, and calculating a local transmission transformation relation of the area; and obtaining a matched point pair, combining camera external parameters obtained by multi-camera calibration, and triangulating the pixel points to recover the 3D coordinates corresponding to the pixel points, thereby recovering and fusing the 3D common view space.

In this embodiment, the pixel positions of the target pixel in other camera images are calculated according to the transmission transformation relationship between the region where the target pixel is located and the region where the target pixel is located, so as to obtain the fusion result.

Specifically, through the region where the pixel point is located and the transmission transformation relation of the region, the pixel position of the pixel point in other camera images can be calculated, so that the fusion of a fine multi-camera co-view space is realized; firstly, determining a matched common view area, and calculating a local transmission transformation relation of the area; and obtaining matched point pairs, and fusing the common view space.

S1142, triangulating the matched characteristic points to obtain three-dimensional coordinates of the characteristic points, wherein the dimensions of the three-dimensional coordinates are consistent with those of the relative pose between cameras.

S115, fitting the water surface of the swimming pool by using the space position;

s116, carrying out multi-view optimization on plane parameters according to the water surface of the swimming pool.

In this embodiment, the optimization problem is constructed by the fact that the plane point-to-plane distance of the swimming pool water surface is zero, and the residual error is determined to optimize the plane parameters.

And obtaining three-dimensional coordinates of the matched feature points through triangulation, wherein the dimensions of the three-dimensional coordinates are consistent with those of the relative pose between cameras, namely the real metric scale. Finally, the horizontal plane can be fitted by three-dimensional points of 3 and more non-collinear points, and the mathematical expression of the horizontal plane is that

. However, because the parallax between every two cameras in the swimming pool is large, the cameras are far away from the water surface, so that errors exist in the space coordinates recovered from the remote water surface points, and errors exist in the final fitted plane parameters.

In order to improve the plane fitting accuracy, an optimization problem is constructed. Since multiple cameras can share a plane, the plane point is passed

The distance to the plane should be 0, the system builds an optimization problem with residual error of 0

Optimizing plane parameters; to->

As a parameter, the pool level calculation is to extract +.>

The matched characteristic points are calculated according to the three-dimensional coordinates of the characteristic points, and the three-dimensional coordinates are minimized

To optimize the horizontal plane parameters.

S120, determining a mapping relation of multiple cameras.

In this embodiment, the mapping relationship of the multiple cameras refers to a transformation matrix between the respective camera poses. Specifically, real data of indoor buildings are collected; constructing a three-dimensional model simulating the indoor building by utilizing the real data; extracting absolute vertical and absolute horizontal line segments from the three-dimensional model; and constructing an external parameter matrix for the line segments, and further obtaining a transformation matrix among the gestures of each camera.

Specifically, the dimensional world model is converted into two-dimensional data.

S130, extracting line segments of absolute vertical and absolute horizontal from the three-dimensional model.

In this embodiment, the segments of absolute vertical and absolute horizontal are screened and filtered from the three-dimensional model.

Specifically, an LSD algorithm is adopted to extract line segments of absolute vertical and absolute horizontal for the three-dimensional model.

And inputting a scene picture, namely a picture of the three-dimensional model, and extracting line segments through an LSD algorithm so as to determine an external parameter matrix according to the line segments.

And constructing an external parameter matrix for the line segments.

In this embodiment, the extrinsic matrix refers to a matrix for calibrating camera extrinsic, including a rotation matrix and a translation vector.

In one embodiment, the steps described above may include the steps of:

and determining a vertical rotation matrix according to the line segments.

In the present embodiment, the vertical rotation matrix refers to a matrix of vertical rotation at the time of camera parameter calibration.

In one embodiment, the steps may include: and obtaining a corresponding vertical vanishing point by multiplying the vertical line segments by two.

In this embodiment, after inputting an image of the three-dimensional model, a vertical 2D line segment of the image is extracted, traversed, and two-by-two cross multiplication of the vertical line segment is performed to obtain a vertical vanishing point.

And measuring the angle error between the vertical vanishing point and other line segments to obtain the vertical vanishing point.

In this embodiment, the vertical vanishing point refers to a vanishing point formed by measuring an angle error between the vertical vanishing point and other line segments.

The vertical direction of the 2D image is defined and the vertical axis of the camera and the vertical vanishing point of the 2D image are aligned.

A vertical rotation matrix is calculated.

In this embodiment, a rotation matrix is defined

It is a rotation in the vertical direction. The LSD algorithm is used to find the vertical and horizontal line segments in the image. Obtaining corresponding vertical vanishing points by multiplying the vertical line segments in a two-by-two manner, and measuring angle errors between the corresponding vertical vanishing points and other line segments, specifically +.>

Wherein->

For each vertical vanishing point +.>

For all vertical line segments, the angle is calculated with an inverse cosine function, an angle error threshold is set as long as +.>

Is smaller than the angle error threshold, +.>

The point with the largest number of interior points, i.e. the dominant vertical vanishing point, is derived from one interior point of (2)>

。

Setting the vertical direction of the 2D image as

Alignment of the vertical axis of the camera and the main vertical vanishing point of the 2D image +.>

Then calculate rotation +.>

. Use->

Lie algebra to lie group to construct rotation +.>

Such as

，/>

For the rotation angle +. >

，/>

Is a rotation axis->

And (5) cross multiplying to obtain the conversion relation from the real world to the image world.

And determining a horizontal rotation matrix according to the line segments.

In the present embodiment, the horizontal direction rotation matrix refers to a matrix of horizontal direction rotation at the time of camera parameter calibration.

In one embodiment, the steps described above may include the steps of:

and determining a horizontal vanishing point according to the three-dimensional model and the line segment.

In this embodiment, the horizontal vanishing point refers to the vanishing point of the horizontal line.

And transforming the horizontal line segments in the line segments into horizontal line segments under a world coordinate system, and determining a horizontal direction rotation matrix.

Specifically, a horizontal vanishing point is determined from the three-dimensional model and the line segment.

In this embodiment, the horizontal vanishing point refers to the vanishing point of the horizontal line. By extracting the vertical face of the indoor 3D model

Three-dimensional horizontal line->

And three-dimensional vertical line->

To get the elevation +.>

Normal of->

First, will->

Normalization processing, and normalizing

Cross-multiplying with vertical axis z gives elevation +.>

Horizontal vanishing point->

。

In the present embodiment, the horizontal vanishing point is obtained by inputting the indoor three-dimensional model

Transforming the extracted horizontal line segment into world coordinates, solving the root of q, substituting +.>

Solving->

The method comprises the steps of carrying out a first treatment on the surface of the Last link rotation->

And->

Calculating the final absolute rotation +.>

I.e. +.>

。

And determining a rotation matrix according to the vertical rotation matrix and the horizontal rotation matrix.

In the present embodiment, the rotation matrix means absolute rotation

。

Specifically, the vertical direction rotation matrix and the horizontal direction rotation matrix are multiplied to obtain a rotation matrix.

And determining a translation vector according to the line segment.

In this embodiment, the translation vector refers to the direction and distance that the camera external parameters need to translate.

In one embodiment, the steps described above may include the steps of:

determining a camera height prior from the three-dimensional model; determining an angular point from the line segment; and determining a translation vector according to the camera height priori and the corner point.

Specifically, obtaining a camera height priori from an indoor three-dimensional model, and obtaining the camera height priori according to the changed inversion matrix

With this a priori, only 2 degrees of freedom remain for the translation to be estimated. Then find one corner from each of the measured line segments, the camera translation can be calculated by solving the following linear system >

，

。

In the present embodiment, two vertical lines are extracted for an image of a three-dimensional model

And->

Construction Point->

And->

To be in straight line->

And->

Go up to restrict->

Further solving for translation vector +.>

。

And integrating the rotation matrix and the translation vector to obtain an extrinsic matrix.

S130, triangulating the human head point pairs to obtain the spatial positions of the human heads.

In this embodiment, the spatial position of the head refers to a specific position of the head under the plurality of cameras.

In one embodiment, referring to fig. 5, the step S130 may include steps S131 to S135.

S131, determining the position of the head of the person based on the YOLO to obtain a target pixel;

and S132, mapping the target pixel to the corresponding region of the other camera pixel planes to obtain a target region.

In this embodiment, the target region refers to a corresponding region in which the target pixel is mapped to the other camera pixel plane.

S133, judging the confidence coefficient of the head of the person detected by each camera under the target area according to the region confidence coefficient priori of the plurality of cameras, and filtering out the cameras with the region confidence coefficient meeting the requirement to obtain candidate cameras.

In this embodiment, the candidate cameras refer to the first three cameras with the region confidence from high to low, and of course, the number of filtered cameras can be determined according to the actual requirement.

And S134, finely dividing the target area and the neighborhood of the target area according to the orientation priori of the candidate camera, and searching the matched head to obtain a search result.

In this embodiment, the search result refers to the matched head.

S135, triangulating the pixel points in the search result, and determining the three-dimensional coordinates of the head in a unified space coordinate system to obtain the space position of the head.

Specifically, single-camera swimming target detection is achieved based on YOLO, and pixel coordinates of the target can be obtained. Due to the wide camera view and large parallax between multiple cameras, multi-camera pixel space mapping is difficult, and in combination with the multi-camera space fusion technique of the super-large parallax set forth above, the target pixel can be mapped to a certain area of the pixel plane of other cameras. And then judging the confidence that each camera detects the head of a person under the area according to the area confidence priors a-h of the cameras with the numbers of 1-n, and filtering out at most three cameras (such as 1, 3 and 4 cameras) with the highest area confidence (three confidence are a, b and c respectively). Further finely dividing the region and the neighborhood thereof according to the orientation priori of the selected camera, searching the matched human head, and then triangulating the human head pixel points

、/>

And->

Three-dimensional coordinates of the head in a unified space coordinate system can be obtained>

。

And S140, determining the distance from the head to the water surface according to the space position.

In the embodiment, the head position of the person can be judged to be underwater or above water according to the water surface space expression of the swimming pool and the multi-camera space mapping relation.

Specifically, according to the spatial position of the head and the spatial expression of the swimming pool water surface, the distance from the head to the horizontal plane and the direction of the normal vector of the horizontal plane are calculated.

S150, judging whether the head is on water or under water according to the direction of the normal vector of the water surface and the distance from the head to the water surface, so as to obtain a judgment result.

In this embodiment, the determination result refers to a determination result that the head is on water or the head is under water.

And S160, generating corresponding information according to the judging result.

In particular, according to the spatial position of the head

And a spatial expression of the horizontal plane, capable of calculating the distance of the head to the horizontal plane +.>

And normal vector at horizontal plane->

The direction of (2) is marked as->

. Association->

And->

The state of the head of the person, namely the water, the water and the water surface, can be judged, so that the early warning effect is achieved.

According to the drowning-prevention three-dimensional positioning method for the swimming pool, the three-dimensional positioning of the head is realized by integrating the multi-view geometry and the bundle set optimization space positioning technology, combining the optimization of the water surface of the swimming pool, the fusion of the multi-camera mapping relation and the common view space, determining the spatial position of the head, and determining the relation between the head and the water surface by adding the water surface of the swimming pool, so that the three-dimensional positioning of the head is realized, and the head of a swimming guest is accurately judged to be on or under water.

Fig. 6 is a schematic block diagram of a three-dimensional positioning device 300 for preventing drowning of a human head in a swimming pool according to an embodiment of the present invention. As shown in fig. 6, the present invention further provides a three-dimensional positioning device 300 for preventing drowning of a swimming pool, corresponding to the above three-dimensional positioning method for preventing drowning of a head of a swimming pool. The swimming pool drowning prevention human head three-dimensional positioning device 300 comprises a unit for executing the swimming pool drowning prevention human head three-dimensional positioning method, and the device can be configured in a server. Specifically, referring to fig. 6, the three-dimensional positioning device 300 for preventing drowning of a human head in a swimming pool includes a parameter recovery unit 301, a relationship determination unit 302, a position acquisition unit 303, a distance determination unit 304, a judgment unit 305, and an information generation unit 306.

A parameter recovery unit 301, configured to recover a plane parameter of the water surface of the swimming pool; a relationship determining unit 302, configured to determine a mapping relationship of multiple cameras; a position obtaining unit 303, configured to triangulate the head point pair to obtain a spatial position of the head; a distance determining unit 304, configured to determine a distance from the head to the water surface according to the spatial position; a judging unit 305, configured to judge that the head is on water or the head is under water by combining the direction of the normal vector of the water surface with the distance from the head to the water surface, so as to obtain a judging result; and the information generating unit 306 is configured to generate corresponding information according to the determination result.

In an embodiment, as shown in fig. 7, the parameter recovery unit 301 includes an image acquisition subunit 3011, a processing subunit 3012, a feature point extraction subunit 3013, a matching subunit 3014, a fitting subunit 3015, and an optimization subunit 3016.

An image acquisition subunit 3011 for acquiring images of a plurality of cameras; a processing subunit 3012, configured to perform de-distortion processing on the image to obtain a processing result; a feature point extracting subunit 3013, configured to extract a swimming pool water surface feature point from the processing result; a matching subunit 3014, configured to match the swimming pool water surface feature points, and calculate a spatial position; a fitting subunit 3015 for fitting the swimming pool water surface with the spatial location; and the optimizing subunit 3016 is used for optimizing plane parameters according to multiple view angles of the swimming pool water surface.

In an embodiment, as shown in fig. 8, the matching subunit 3014 includes a feature point matching module 30141 and a triangularization module 30142.

The feature point matching module 30141 is configured to perform matching of the swimming pool water surface feature points by using a multi-camera spatial fusion technology, so as to obtain matched feature points; and the triangularization module 30142 is used for triangulating the matched characteristic points to obtain three-dimensional coordinates of the characteristic points, and the dimensions of the three-dimensional coordinates are consistent with those of the relative pose between cameras.

In an embodiment, the optimizing subunit 3016 is configured to construct an optimization problem by setting a plane-to-plane distance of the swimming pool water surface to zero, and determine a residual error to optimize a plane parameter.

In one embodiment, as shown in fig. 9, the location obtaining unit 303 includes a location determining subunit 3031, a mapping subunit 3032, a filtering subunit 3033, a head retrieving subunit 3034, and a pixel triangulating subunit 3035.

A position determining subunit 3031, configured to determine, based on YOLO, a position of the head of the person to obtain a target pixel; a mapping subunit 3032, configured to map the target pixel to a corresponding region of the other camera pixel plane, so as to obtain a target region; a filtering subunit 3033, configured to determine, according to the region confidence priors of the multiple cameras, that each camera detects the confidence level of the head under the target region, and filter out the cameras whose region confidence levels meet the requirements, so as to obtain candidate cameras; the head retrieval subunit 3034 is configured to finely divide a target area and a neighborhood of the target area according to the orientation prior of the candidate camera, and retrieve the matched head to obtain a retrieval result; and the pixel point triangularization subunit 3035 is used for triangulating the pixel points in the search result and determining the three-dimensional coordinates of the head in the unified space coordinate system so as to obtain the space position of the head.

In an embodiment, the distance determining unit 304 is configured to calculate the distance from the head to the horizontal plane and the direction of the normal vector of the horizontal plane according to the spatial position of the head and the spatial expression of the water surface of the swimming pool.

It should be noted that, as those skilled in the art can clearly understand, the specific implementation process of the above-mentioned three-dimensional positioning device 300 for preventing drowning of a human head in a swimming pool and each unit may refer to the corresponding description in the foregoing method embodiments, and for convenience and brevity of description, the description is omitted here.

The above-described swimming pool anti-drowning head three-dimensional positioning device 300 may be implemented in the form of a computer program which can be run on a computer device as shown in fig. 10.

Referring to fig. 10, fig. 10 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, where the server may be a stand-alone server or may be a server cluster formed by a plurality of servers.

With reference to FIG. 10, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a method for three-dimensional positioning of a swimming pool anti-drowning head.

The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.

The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform a method for three-dimensional positioning of a swimming pool anti-drowning head.

The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of a portion of the architecture in connection with the present application and is not intended to limit the computer device 500 to which the present application is applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

Wherein the processor 502 is configured to execute a computer program 5032 stored in a memory to implement the steps of:

Restoring the plane parameters of the water surface of the swimming pool; determining a mapping relation of multiple cameras; triangulating the human head point pairs to obtain the spatial position of the human head; determining the distance from the head to the water surface according to the space position; judging whether the head is positioned on water or underwater by combining the direction of the normal vector of the water surface with the distance from the head to the water surface so as to obtain a judgment result; and generating corresponding information according to the judging result.

In one embodiment, the processor 502, when implementing the step of restoring the plane parameters of the water surface of the swimming pool, specifically implements the following steps:

acquiring images of a plurality of cameras; performing de-distortion treatment on the image to obtain a treatment result; extracting swimming pool water surface characteristic points from the processing results; matching the swimming pool water surface characteristic points and calculating the space position; fitting the swimming pool water surface by using the space position; and carrying out multi-view optimization on plane parameters according to the water surface of the swimming pool.

In one embodiment, when the step of matching the characteristic points of the water surface of the swimming pool and calculating the spatial position is implemented by the processor 502, the following steps are specifically implemented:

matching the swimming pool water surface characteristic points by utilizing a multi-camera space fusion technology to obtain matched characteristic points; and triangulating the matched characteristic points to obtain three-dimensional coordinates of the characteristic points, wherein the dimensions of the three-dimensional coordinates are consistent with those of the relative pose between cameras.

In one embodiment, the processor 502 performs the following steps when performing the step of optimizing the plane parameters according to the water surface of the swimming pool:

In one embodiment, when implementing the step of triangulating the head point pairs to obtain the spatial position of the head, the processor 502 specifically implements the following steps:

determining the position of the head of the person based on the YOLO to obtain a target pixel; mapping the target pixel to the corresponding area of the pixel plane of other cameras to obtain a target area; judging the confidence coefficient of the head of a person detected by each camera under a target area according to the area confidence coefficient priori of the plurality of cameras, and filtering out the cameras with the area confidence coefficient meeting the requirement so as to obtain candidate cameras; according to the orientation priori of the candidate camera, finely dividing a target area and the neighborhood of the target area, and searching the matched head to obtain a search result; and triangulating the pixel points in the search result, and determining the three-dimensional coordinates of the human head in a unified space coordinate system to obtain the space position of the human head.

In an embodiment, when the step of determining the distance from the head to the water surface according to the spatial position is implemented by the processor 502, the following steps are specifically implemented:

It should be appreciated that in embodiments of the present application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.

Accordingly, the present invention also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program which, when executed by a processor, causes the processor to perform the steps of:

In one embodiment, the processor, when executing the computer program to implement the step of restoring the plane parameters of the pool water surface, specifically implements the steps of:

In one embodiment, when the processor executes the computer program to implement the step of matching the swimming pool water surface feature points and calculating the spatial position, the following steps are specifically implemented:

In one embodiment, when the processor executes the computer program to implement the step of optimizing plane parameters according to the swimming pool water surface from multiple angles, the processor specifically implements the following steps:

In one embodiment, when the processor executes the computer program to implement the step of triangulating the head pairs to obtain the spatial positions of the heads, the processor specifically implements the following steps:

In one embodiment, when the processor executes the computer program to implement the step of determining the distance from the head to the water surface according to the spatial position, the processor specifically implements the following steps:

The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.

The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention.

While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. The three-dimensional positioning method for the drowning prevention of the head of a swimming pool is characterized by comprising the following steps:

restoring the plane parameters of the water surface of the swimming pool;

determining a mapping relation of multiple cameras;

and generating corresponding information according to the judging result.

2. The method for three-dimensional positioning of a swimming pool anti-drowning head according to claim 1, wherein the restoration of the plane parameters of the swimming pool water surface comprises:

acquiring images of a plurality of cameras;

performing de-distortion treatment on the image to obtain a treatment result;

fitting the swimming pool water surface by using the space position;

3. The method for three-dimensional positioning of a swimming pool anti-drowning head according to claim 2, wherein the matching the characteristic points of the water surface of the swimming pool and calculating the spatial position comprises:

4. The method for three-dimensional positioning of a swimming pool anti-drowning head according to claim 2, wherein the multi-view optimization of plane parameters according to the swimming pool water surface comprises:

5. The method of claim 1, wherein triangulating the pairs of human head points to obtain the spatial position of the human head comprises:

6. The method for three-dimensional positioning of a swimming pool anti-drowning head according to claim 1, wherein the determining the distance from the head to the water surface according to the spatial position comprises:

7. Swimming pool prevents drowned human head three-dimensional positioner, a serial communication port, include:

8. The swimming pool anti-drowning head three-dimensional positioning device according to claim 7, wherein the parameter recovery unit comprises:

an image acquisition subunit for acquiring images of a plurality of cameras;

9. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-6.

10. A storage medium storing a computer program which, when executed by a processor, implements the method of any one of claims 1 to 6.