CN114463663A - Method and device for calculating height of person, electronic equipment and storage medium - Google Patents

Method and device for calculating height of person, electronic equipment and storage medium Download PDF

Info

Publication number
CN114463663A
CN114463663A CN202111584584.8A CN202111584584A CN114463663A CN 114463663 A CN114463663 A CN 114463663A CN 202111584584 A CN202111584584 A CN 202111584584A CN 114463663 A CN114463663 A CN 114463663A
Authority
CN
China
Prior art keywords
dimensional
coordinate
coordinates
central point
head
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111584584.8A
Other languages
Chinese (zh)
Inventor
冯歌
吕旭涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Intellifusion Technologies Co Ltd
Original Assignee
Shenzhen Intellifusion Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Intellifusion Technologies Co Ltd filed Critical Shenzhen Intellifusion Technologies Co Ltd
Priority to CN202111584584.8A priority Critical patent/CN114463663A/en
Publication of CN114463663A publication Critical patent/CN114463663A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1075Measuring physical dimensions, e.g. size of the entire body or parts thereof for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer
    • G06T3/08

Abstract

The invention relates to the technical field of passenger flow cameras, in particular to a method and a device for calculating the height of a person, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a personnel image acquired by image acquisition equipment; acquiring two-dimensional head central point coordinates and two-dimensional foot central point coordinates of a person from a person image; converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate into a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system; acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating the estimated distance from the three-dimensional foot central point coordinates to the auxiliary line; and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates and the head frame coordinates of the person and the two-dimensional foot center point coordinates. The method for estimating the height of the person is small in visual difference and high in accuracy of the calculated height of the person.

Description

Method and device for calculating height of person, electronic equipment and storage medium
Technical Field
The invention relates to the technical field of passenger flow cameras, in particular to a method and a device for calculating the height of a person, electronic equipment and a storage medium.
Background
The passenger flow camera is a camera capable of automatically analyzing a shot video stream in real time so as to output customer information entering and exiting a specified area, for example, attributes such as the number of people entering and exiting a certain store, the height of a person, whether a person wears a mask, whether a hat is worn, and the like are analyzed and output. And the height of the person in the calculated passenger flow is an important human body attribute and index of the passenger flow camera.
The existing technology is generally obtained by calculating the distance between a foot point and a head point through points of a 2D image and adding the error between the head top and the head point. However, the 2D image is projected from the 3D world coordinate system to the 2D coordinate system through the camera, and there is a visual error, and the accuracy of the calculation result is low. Therefore, the problem of low accuracy exists when the existing passenger flow camera calculates the height of a person.
Disclosure of Invention
The embodiment of the invention provides a method for calculating the height of a person, and aims to solve the problem of low accuracy in calculation of the height of the person by using an existing passenger flow camera.
In a first aspect, an embodiment of the present invention provides a method for calculating a height of a person, where the method includes the following steps:
acquiring a personnel image acquired by image acquisition equipment;
acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate;
converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate;
acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating an estimated distance from the three-dimensional foot central point coordinates to the auxiliary line;
and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates.
In a second aspect, an embodiment of the present invention provides a computing device for height of a person, including:
the first acquisition module is used for acquiring a personnel image acquired by the image acquisition equipment;
the second acquisition module is used for acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate;
the conversion module is used for converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate;
the first calculation module is used for acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating the estimated distance from the three-dimensional foot central point coordinates to the auxiliary line;
and the second calculation module is used for calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates.
In a third aspect, an electronic device includes: the device comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the computer program to realize the steps in the method for calculating the height of the person provided by the embodiment of the invention.
In a fourth aspect, a computer-readable storage medium has a computer program stored thereon, where the computer program is executed by a processor to implement steps in a method for calculating a height of a person according to an embodiment of the present invention.
In the embodiment of the invention, the personnel image acquired by the image acquisition equipment is acquired; acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate; converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate; acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating an estimated distance from the three-dimensional foot central point coordinates to the auxiliary line; and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates. According to the method, after the image of the person collected by the image collection device is obtained, the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate of the person in the image are obtained, then the three-dimensional head central point coordinate and the three-dimensional foot central point coordinate are obtained in a mode of converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system, the height of the person is calculated after the estimated distance is calculated in a mode of selecting an auxiliary point coordinate, the visual difference is small through the estimation method, and the accuracy of the calculated height of the person is high.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for calculating a height of a person according to an embodiment of the present invention;
FIG. 2 is a flow chart of another method for calculating the height of a person according to an embodiment of the present invention;
FIG. 3 is a flowchart of step S203 in FIG. 2 according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a personal height computing device according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of another personal height computing device provided in an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of another personal height computing device provided in an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of another personal height computing device provided in an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of another personal height computing device provided in an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "comprising" and "having," and any variations thereof, in the description and claims of this application and the description of the figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, fig. 1 is a flowchart of a method for calculating a height of a person according to an embodiment of the present invention, as shown in fig. 1, including the following steps:
and S101, acquiring a personnel image acquired by image acquisition equipment.
In the embodiment of the invention, the implemented personnel height calculation method applies scenes including, but not limited to, various malls, shops, scenic spots and stations, and gradually expands to airports, airport transit halls, parking lots, fitness places and the like. The electronic equipment on which the method for calculating the height of the person operates can acquire the image of the person in a wired connection mode or a wireless connection mode. The Wireless connection mode may include, but is not limited to, a 3G/4G connection, a WiFi (Wireless-Fidelity) connection, a bluetooth connection, a wimax (worldwide Interoperability for Microwave access) connection, a Zigbee (low power local area network protocol), a uwb (ultra wideband) connection, and other Wireless connection modes now known or developed in the future.
The image acquisition equipment can be a passenger flow camera, the passenger flow camera can be arranged at a corresponding position in a scene applied by a market, a shop, a scenic spot, a station and the like after calibrating basic parameters, and a plurality of passenger flow cameras can be arranged in the same scene. During installation, the horizon of the field of view, i.e. the x-axis in the pixel coordinate system of the passenger flow camera and the ground level, is required. Specifically, the calibrating the basic parameters of the passenger flow camera includes calibrating distortion parameters of the camera, and certainly, if the passenger flow camera has no distortion, the calibrating is not needed. The basic parameters of the passenger flow camera further comprise an internal reference matrix of the passenger flow camera and an external reference matrix of the passenger flow camera, and the internal reference matrix and the external reference matrix can be used for converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system.
S102, acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate.
The passenger flow camera can be used for image acquisition, and whether personnel images exist in the images can be automatically identified in the identification range in the image acquisition process. Specifically, the recognition may be performed according to features of a human body, for example, whether features such as a human face and a human figure exist in the image or not may be recognized. If the image of the person is collected, a human body frame is output and marked, and the human body frame is a target frame containing the whole human body of the person. The method is equivalent to inputting a picture, and outputting a human body frame after recognizing a human body. And a head frame can be further framed from the human body frame, wherein the head frame is a target frame containing the whole head of the person. Then, the head and the feet of the person are found through key point detection, and a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate are respectively calculated based on the body frame and the head frame. Wherein, the human body box can be expressed as person (x, y, w, h); the header box coordinates may be represented as headRect (x, y, w, h); the two-dimensional head center point coordinates may be expressed as headCenter (x, y), the two-dimensional foot center point coordinates footCenter (x, y), the parenthesis indicate coordinates, x, y correspond to x and y coordinates, respectively, and w and h correspond to the width of the x coordinate and the height of the y coordinate, respectively, in pixels.
S103, converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting the two-dimensional pixel coordinate system into the three-dimensional world coordinate system to obtain the three-dimensional head central point coordinate and the three-dimensional foot central point coordinate.
Wherein, the conversion of the two-dimensional head center point coordinate headCenter (x, y) to the three-dimensional head center point coordinate 3Dhead (x, y, z) in the 2D pixel coordinate system and the conversion of the two-dimensional foot center point coordinate footCenter (x, y) to the three-dimensional foot center point coordinate 3Dfoot (x, y, z) in the 3D world coordinate system can be realized based on the conversion of the two-dimensional pixel coordinate system (2D (dimensions) pixel coordinate system) to the three-dimensional world coordinate system (3D world coordinate system).
S104, obtaining auxiliary point coordinates of the reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating the estimated distance from the three-dimensional foot central point coordinates to the auxiliary line.
Wherein, can take a reference surface in the camera shooting range of the passenger flow camera. In this embodiment, the reference plane may refer to a ceiling plane above the top of the head of a person, the auxiliary point 3DheadTmp (x +1, y, z) is taken from the ceiling plane, the auxiliary line L is formed by connecting the auxiliary point 3DheadTmp (x +1, y, z) and the three-dimensional head center point coordinate 3Dhead (x, y, z), and the distance from the three-dimensional foot center point coordinate 3Dfoot (x, y, z) to the auxiliary line L is calculated by calculating the distance from a point to a straight line, that is, the estimated distance headfoot from the three-dimensional foot center point coordinate 3Dfoot (x, y, z) to the three-dimensional head center point coordinate 3Dhead (x, y, z).
And S105, calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot center point coordinates.
Wherein, the specific calculation formula of the height of the person is as follows:
height=headFootHeight+headRect.h*fy+(person.y+person.h-footCenter.y)*fy。
wherein: the headFootHeaght represents the estimated distance; head Rect.h represents the height in the coordinates of the head box; fy belongs to the basic parameter of the camera, and means the actual distance of one pixel in the y direction, and the unit is mm; y represents a y coordinate value of a human body frame coordinate corresponding to the human image; h denotes height in body frame coordinates; footcenter. y represents the y coordinate value in the two-dimensional foot center point coordinates.
Specifically, according to the above calculation formula, the height of the person in the person image collected by the passenger flow camera can be calculated, and the height of the person is not necessarily consistent with the actual height of the person, but can be used as the optimal height estimation identified by the passenger flow camera.
In the embodiment of the invention, the personnel image acquired by the image acquisition equipment is acquired; acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate; converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting the two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate; acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating an estimated distance from the three-dimensional foot central point coordinates to the auxiliary line; and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates. Therefore, the three-dimensional head central point coordinate and the three-dimensional foot central point coordinate are obtained through conversion of a 2D pixel coordinate system to a 3D world coordinate system, visual difference can be reduced through conversion of the three-dimensional head central point coordinate and the three-dimensional foot central point coordinate, the height of a person is calculated after the estimated distance is calculated through the auxiliary point coordinate selecting method, and the height of the person calculated through the estimation method is higher in accuracy.
Referring to fig. 2, fig. 2 is a flowchart of another method for calculating a height of a person according to an embodiment of the present invention, as shown in fig. 2, including the steps of:
s201, acquiring a person image acquired by image acquisition equipment.
S202, detecting a human body frame and a head frame of a person contained in the person image based on the human body detection model, wherein the human body frame is provided with corresponding human body frame coordinates, and the head frame is provided with corresponding head frame coordinates.
The passenger flow camera is installed and started to acquire images, image tracking can be performed by combining a target tracking algorithm, human body detection is performed through a human body detection model PersonDetect of the passenger flow camera, a picture is input and output as a human body frame, a human body is framed, and the human body frame is a rectangular frame and is beneficial to coordinate positioning. The human body detection model personndetect can detect whether human body features exist in the picture, for example: whether features of the head, body, legs, shoes, arms, clothing, etc. are recognized. After the human body is identified, the human body frame is marked, and human body frame coordinates person (x, y, w, h) are obtained.
S203, performing two-dimensional human key point detection on the human body frame and the head frame through the human key point detection model, and calculating to obtain a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate.
The human body key point detection model is a human body key point model KeyPoint obtained in advance, the input of the key point detection model is a human body frame, and the output of the key point detection model is two-dimensional foot center point coordinates footCenter (x, y) of a human body. In the process of detecting the two-dimensional human body key points, people can be tracked through a target tracking algorithm. After the human body frame is calibrated, human body key point detection can be carried out, coordinates of the two feet are obtained, and the middle point between the two feet, namely the coordinate of the two-dimensional foot center point, is calculated based on the coordinates of the two feet. In addition, after the human body frame is calibrated, human head detection can be performed from the human body frame, a head frame header Rect (x, y, w, h) is obtained and calibrated, the center position of the head frame is calculated, and a two-dimensional head center point coordinate header center (x, y) is obtained.
The human body key point detection model can detect the head, the left ankle, the right ankle, the nose, the left eye, the right eye, the left ear, the right ear, the left wrist, the right wrist, the left elbow, the right elbow, the left shoulder, the right shoulder, the left crotch, the right crotch, the left knee, the right knee and the like of a person. And human body estimation can be carried out according to the acquired two-dimensional key point coordinates, namely the posture of the target person in the current state is estimated. It should be noted that, in the detection process, when it is detected that the human body is in the squatting state, the height detection is not performed, and when it is detected that the human body is in the standing state, the detection is performed, that is, the estimation of the posture of the human body is performed.
And S204, converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate under the two-dimensional pixel coordinate system into the two-dimensional image coordinate system based on the internal reference matrix of the image acquisition equipment.
The basic parameters of the passenger flow camera are set before the passenger flow camera is installed, and the basic parameters of the passenger flow camera comprise a passenger flow camera internal parameter matrix and a passenger flow camera external parameter matrix. Therefore, the two-dimensional head center point coordinates in the 2D pixel coordinate system can be converted to be under the 2D image coordinate system and the two-dimensional foot center point coordinates in the 2D pixel coordinate system can be converted to be under the 2D image coordinate system according to the conversion relation between the 2D pixel coordinate system and the 2D image coordinate system based on the internal reference matrix of the passenger flow camera. Specifically, the conversion relationship between the 2D pixel coordinate system (u, v) and the 2D image coordinate system (x, y) is as follows:
Figure BDA0003427455800000071
wherein, the left side of the equal sign is the 2D pixel coordinate system (u, v), and the right side of the equal sign is the product of the internal reference matrix and the 2D image coordinate system (x, y). F in the internal reference matrixx、fyThe actual distance of one pixel in the x direction and the y direction is respectively indicated, and the unit is mm; c. CxAnd cyIs the offset distance in mm from the center of the 2D image coordinate system (x, y) to the 2D pixel coordinate system (u, v).
Specifically, in the present embodiment, the 2D pixel coordinate system (u, v) is converted into the 3D world coordinate system (x)c,yc,zc) Therefore, the 2D pixel coordinate system (u, v) is converted into the 2D image coordinate system (x, y) by an inverse matrix. The internal reference matrix of the passenger flow camera is known, the two-dimensional head center point coordinate head center (x, y) under the 2D pixel coordinate system (u, v) is substituted into the two-dimensional head center point coordinate head center (x, y) under the 2D image coordinate system (x, y), and the two-dimensional foot center point coordinate foot center (x, y) under the 2D pixel coordinate system (u, v) is substituted into the two-dimensional foot center point coordinate foot center (x, y) under the 2D image coordinate system (x, y).
S205, converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate in the two-dimensional image coordinate system into a three-dimensional world coordinate system based on the external parameter matrix of the image acquisition equipment to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate.
Wherein, the 2D image coordinate system (x, y) and the 3D world coordinate system (x)c,yc,zc) The conversion relationship between them is as follows:
Figure BDA0003427455800000081
wherein, the left side of the equal sign is a 2D image coordinate system (x, y), and the right side of the equal sign is an external reference matrix and a 3D world coordinate system (x)c,yc,zc) The product of (a). F in the extrinsic matrix represents the focal length of the passenger flow camera, Zc, the depth of the camera.
Since the extrinsic matrix of the passenger flow camera is known data, the two-dimensional foot center point coordinate footCenter (x, y) and the two-dimensional head center point coordinate headCenter (x, y) in the 2D image coordinate system (x, y) are respectively substituted into the conversion formulas of the 2D image coordinate system (x, y) and the 3D world coordinate system (x, y, z), and the three-dimensional head center point coordinate 3Dhead (x, y, z) and the three-dimensional foot center point coordinate 3Dfoot (x, y, z) in the 3D world coordinate system (x, y, z) can be calculated by using an inverse matrix.
S206, determining a reference surface and selecting an auxiliary point from the reference surface.
And S207, connecting the coordinates of the auxiliary points with the coordinates of the three-dimensional head central point to form an auxiliary line, wherein the auxiliary line is vertical to the ground.
The ceiling of the top of the human head is used as a reference surface, an auxiliary point 3Dhead Tmp (x +1, y, z) is taken on the reference surface, and the three-dimensional head center point coordinate 3Dhead (x, y, z) is connected to obtain an auxiliary line L. An auxiliary line L formed by the auxiliary point 3Dhead Tmp (x +1, y, z) and the three-dimensional head center point coordinate 3Dhead (x, y, z) is vertical to the ground, so that in a 3D world coordinate system, the x direction and the y direction have no deviation, and only the z direction is used for assisting.
And S208, calculating the distance from the three-dimensional foot central point coordinate to the auxiliary line, wherein the distance is the estimated distance from the three-dimensional head central point coordinate to the three-dimensional foot central point coordinate.
By the way of calculating the distance from the point to the line, the distance from the three-dimensional foot center point coordinate 3Dfoot (x, y, z) to the auxiliary line L can be calculated, and the distance is used as the estimated distance between the three-dimensional head center point 3 dfead (x, y, z) and the three-dimensional foot center point 3Dfoot (x, y, z).
S209, calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot center point coordinates.
In the embodiment of the invention, the human body detection is realized by a human body detection model PersonDetect, labeling is carried out by a human body frame, labeling is carried out by the human body frame, a two-dimensional foot central point coordinate footCenter (x, y) can be obtained from the human body frame by a human body key point model KeyPoint, a two-dimensional head central point coordinate headCenter (x, y) can be obtained from the head frame by the human body key point model KeyPoint, then the two-dimensional foot central point coordinate footCenter (x, y) and the two-dimensional head central point coordinate headCenter (x, y) are respectively converted into a three-dimensional foot central point 3Dfoot (x, y, z) and a three-dimensional head central point 3Dhead (x, y, z) in an inverse matrix manner, so that the coordinate conversion between the 2D pixel coordinate system and the 3D world coordinate system is realized, the obtained three-dimensional coordinate point is more beneficial to height estimation. In addition, an auxiliary line is made by means of obtaining auxiliary points by means of a reference surface, the estimated distance is calculated, and the height of the person is calculated by combining the estimated distance, the internal reference matrix and the external reference matrix of the passenger flow camera, the body frame coordinates of the person image, the head frame coordinates and the two-dimensional foot central point coordinates.
Referring to fig. 3, fig. 3 is a flowchart of step S203 in fig. 2 according to an embodiment of the present invention, which specifically includes:
s2031, foot key point detection is carried out on the human body frame through the human body key point detection model, and a first foot coordinate and a second foot coordinate are respectively obtained.
After the human body frame is calibrated, the key points of the human body foot are detected, the first foot coordinate and the second foot coordinate respectively represent a left foot coordinate and a right foot coordinate in the human body frame, wherein the left foot coordinate can be represented by a left foot ankle coordinate leftankle (x, y), and the right foot coordinate is represented by a right foot ankle coordinate rightAnkle (x, y).
S2032, calculating a mean coordinate based on the first foot coordinate and the second foot coordinate, and taking the mean coordinate as a two-dimensional foot center point coordinate.
Wherein, the two-dimensional foot center point coordinate footCenter (x, y) can be obtained by performing an average calculation based on the left ankle coordinate leftankle (x, y) and the right ankle coordinate rightAnkle (x, y).
As a possible embodiment, a compensation value may be preset, and when coordinates of both feet are obtained and only one foot is detected, the detected coordinates of one foot may be combined with the compensation value as coordinates of a center point of the two-dimensional foot, for example: and only detecting the left foot/right foot, after positioning the coordinates of the left foot/right foot, combining a compensation value on the x axis in the horizontal direction to increase or decrease the coordinates of the left foot/right foot, and taking the calculated result as the coordinates of the center point of the two-dimensional foot.
S2033, head key point detection is carried out on the human body frame through the human body key point detection model, coordinates of the head frame are marked, the center position of the head frame is calculated according to the coordinates of the head frame, and the center position is used as coordinates of a two-dimensional head center point.
In the process of obtaining the two-dimensional foot central point coordinate footCenter (x, y), the head frame in the human body frame can be marked, meanwhile, the head frame coordinate headRect (x, y, w, h) is obtained, the head frame can be a rectangular frame, the central point of the rectangular frame can be calculated based on the width w and the height h of the head frame coordinate headRect (x, y, w, h), and the coordinate of the central point is the two-dimensional head central point coordinate headCenter (x, y).
In this embodiment, the human body key point detection model detects the human body key points, the coordinate mean of the left foot ankle coordinate leftankle (x, y) and the right foot ankle coordinate rightAnkle (x, y) is used as the dimension foot center point coordinate footCenter (x, y), and the head frame calibration is performed to calculate the two-dimensional head center point coordinate headCenter (x, y) according to the head frame coordinate, so that the obtained data is more accurate.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a personal height computing device according to an embodiment of the present invention, and as shown in fig. 4, the device 400 includes:
a first obtaining module 401, configured to obtain a person image collected by an image collecting device;
a second obtaining module 402, configured to obtain two-dimensional key point coordinates of people included in the person image from the person image, where the two-dimensional key point coordinates include a two-dimensional head center point coordinate and a two-dimensional foot center point coordinate;
the conversion module 403 is configured to convert the two-dimensional head center point coordinate and the two-dimensional foot center point coordinate by converting the two-dimensional pixel coordinate system into a three-dimensional world coordinate system, so as to obtain a three-dimensional head center point coordinate and a three-dimensional foot center point coordinate;
a first calculating module 404, configured to obtain an auxiliary point coordinate of the reference plane, form an auxiliary line based on the auxiliary point coordinate and the three-dimensional head center point coordinate, and calculate an estimated distance from the three-dimensional foot center point coordinate to the auxiliary line;
the second calculating module 405 is configured to calculate a height of a person included in the person image based on the estimated distance, the basic parameter of the image capturing device, the body frame coordinate of the person included in the person image, the head frame coordinate of the person included in the person image, and the two-dimensional foot center point coordinate.
Optionally, as shown in fig. 5, fig. 5 is a schematic structural diagram of another computing apparatus for height of a person according to an embodiment of the present invention, where the second obtaining module 402 includes:
the first detection unit 4021 is configured to detect a human body frame and a head frame of a person included in the person image based on the human body detection model, where the human body frame has corresponding human body frame coordinates and the head frame has corresponding head frame coordinates;
the second detection unit 4022 is configured to perform two-dimensional human body key point detection on the human body frame and the head frame through the human body key point detection model, and calculate to obtain a two-dimensional head center point coordinate and a two-dimensional foot center point coordinate.
Optionally, as shown in fig. 6, fig. 6 is a schematic structural diagram of another calculation apparatus for height of a person according to an embodiment of the present invention, where the second detection unit 4022 includes:
the first detection subunit 40221 is configured to perform foot key point detection on the human body frame through the human body key point detection model, and obtain a first foot coordinate and a second foot coordinate respectively;
the first calculating subunit 40222 is configured to calculate a mean coordinate based on the first foot coordinate and the second foot coordinate, and use the mean coordinate as a two-dimensional foot center point coordinate;
the second calculating subunit 40223 is configured to perform head keypoint detection on the human body frame through the human body keypoint detection model, mark coordinates of the head frame, calculate a center position of the head frame according to the coordinates of the head frame, and use the center position as a two-dimensional head center point coordinate.
Optionally, as shown in fig. 7, fig. 7 is a schematic structural diagram of another computing apparatus for calculating a height of a person according to an embodiment of the present invention, where the apparatus 400 further includes:
a setting module 406, configured to set basic parameters of the image acquisition device, where the basic parameters of the image acquisition device include an internal parameter matrix and an external parameter matrix of the image acquisition device;
and the conversion module 403 includes:
the first calculation unit 4031 is used for converting two-dimensional head central point coordinates and two-dimensional foot central point coordinates in a two-dimensional pixel coordinate system into a two-dimensional image coordinate system based on an internal reference matrix of the image acquisition equipment;
and a second calculation unit 4031, configured to convert the two-dimensional head center point coordinate and the two-dimensional foot center point coordinate in the two-dimensional image coordinate system to the three-dimensional world coordinate system based on the external reference matrix of the image acquisition device, so as to obtain the three-dimensional head center point coordinate and the three-dimensional foot center point coordinate.
As shown in fig. 8, fig. 8 is a schematic structural diagram of another personal height computing device according to an embodiment of the present invention, in which the first computing module 404 includes:
a selecting unit 4041, configured to determine a reference plane and select an auxiliary point from the reference plane;
the connecting unit 4042 is used for connecting the coordinates of the auxiliary points and the coordinates of the three-dimensional head center point to form an auxiliary line, and the auxiliary line is vertical to the ground;
the third calculating unit 4043 is configured to calculate a distance from the three-dimensional foot center point coordinate to the auxiliary line, where the distance is an estimated distance from the three-dimensional head center point coordinate to the three-dimensional foot center point coordinate.
The device for calculating the height of the person provided by the embodiment of the invention can realize each implementation mode in the method for calculating the height of the person in fig. 1-3 and corresponding beneficial effects, and is not described again to avoid repetition.
As shown in fig. 9, fig. 9 is a structural diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 9, includes: memory 902, processor 901, network interface 903, and a computer program stored on memory 902 and executable on processor 901, wherein:
the processor 901 is used for calling the computer program stored in the memory 902 and executing the following steps:
acquiring a personnel image acquired by image acquisition equipment;
acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate;
converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting the two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate;
acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating an estimated distance from the three-dimensional foot central point coordinates to the auxiliary line;
and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates.
Optionally, the obtaining, by the processor 901, two-dimensional key point coordinates of a person included in the person image from the person image includes:
detecting a human body frame and a head frame of a person contained in the person image based on the human body detection model, wherein the human body frame is provided with corresponding human body frame coordinates, and the head frame is provided with corresponding head frame coordinates;
and performing two-dimensional human body key point detection on the human body frame and the head frame through the human body key point detection model, and calculating to obtain a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate.
Optionally, the processor 901 performs two-dimensional human key point detection on the human body frame and the head frame through the human key point detection model, and calculates to obtain a two-dimensional head center point coordinate and a two-dimensional foot center point coordinate, including:
performing foot key point detection on the human body frame through the human body key point detection model, and respectively acquiring a first foot coordinate and a second foot coordinate;
calculating a mean coordinate based on the first foot coordinate and the second foot coordinate, and taking the mean coordinate as a two-dimensional foot center point coordinate;
and performing head key point detection on the human body frame through the human body key point detection model, marking the coordinates of the head frame, calculating the central position of the head frame according to the coordinates of the head frame, and taking the central position as the coordinates of a two-dimensional head central point.
Optionally, the processor 901 is further configured to perform:
setting basic parameters of the image acquisition equipment, wherein the basic parameters of the image acquisition equipment comprise an internal parameter matrix and an external parameter matrix of the image acquisition equipment.
Optionally, the conversion, performed by the processor 901, of the two-dimensional head center point coordinate and the two-dimensional foot center point coordinate through the conversion from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system includes:
converting a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate under a two-dimensional pixel coordinate system into a two-dimensional image coordinate system based on an internal reference matrix of the image acquisition equipment;
and converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate under the two-dimensional image coordinate system into a three-dimensional world coordinate system based on the external parameter matrix of the image acquisition equipment to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate.
Optionally, the obtaining of the auxiliary point coordinates of the reference surface executed by the processor 901, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculating the estimated distance from the three-dimensional foot center point coordinates to the auxiliary line includes:
determining a reference surface, and selecting an auxiliary point from the reference surface;
connecting the coordinates of the auxiliary points with the coordinates of the three-dimensional head central point to form an auxiliary line, wherein the auxiliary line is vertical to the ground;
and calculating the distance from the three-dimensional foot central point coordinate to the auxiliary line, wherein the distance is the estimated distance from the three-dimensional head central point coordinate to the three-dimensional foot central point coordinate.
The electronic device provided by the embodiment of the invention can realize each implementation mode in the embodiment of the calculation method based on the height of the person and corresponding beneficial effects, and in order to avoid repetition, the details are not repeated.
It is noted that only 901-903 with components are shown, but it is understood that not all of the illustrated components are required and that more or fewer components may alternatively be implemented. As will be understood by those skilled in the art, the electronic device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable gate array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device 900 may be a desktop computer, a notebook, a palm top computer, a cloud server, or other computing device. The electronic device 900 may interact with a client via a keyboard, mouse, remote control, touch pad, or voice control device.
The memory 902 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 901 may be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. In other embodiments, the memory 901 may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device. Of course, the memory 901 may also include both internal and external memory units of the electronic device. In this embodiment, the memory 901 is generally used for storing an operating system installed in the electronic device and various types of application software, such as program codes of a calculation method for the height of a person. In addition, the memory 901 can also be used to temporarily store various types of data that have been output or are to be output.
Processor 901 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 902 is typically used to control the overall operation of the electronic device. In this embodiment, the processor 901 is configured to run program code stored in the memory 901 or program code for processing data, such as a calculation method of the height of the operator.
The network interface 903 may comprise a wireless network interface or a wired network interface, and the network interface 903 is typically used to establish communication connections between the electronic device 900 and other electronic devices.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by the processor 901, the computer program implements each process of the embodiment of the method for calculating the height of a person provided by the embodiment of the present invention, and can achieve the same technical effect, and is not described herein again to avoid repetition.
It will be understood by those skilled in the art that all or part of the processes in the computing method for implementing the embodiments may be implemented by a computer program instructing associated hardware, and the program may be stored in a computer-readable storage medium, and when executed, may include processes such as the embodiments of the methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (10)

1. A method for calculating the height of a person, said method comprising the steps of:
acquiring a personnel image acquired by image acquisition equipment;
acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate;
converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate;
acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating an estimated distance from the three-dimensional foot central point coordinates to the auxiliary line;
and calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates.
2. The method of claim 1, wherein the obtaining two-dimensional key point coordinates of the person included in the person image from the person image comprises:
detecting a human body frame and a head frame of a person contained in a person image based on a human body detection model, wherein the human body frame is provided with corresponding human body frame coordinates, and the head frame is provided with corresponding head frame coordinates;
and performing two-dimensional human key point detection on the human body frame and the head frame through a human key point detection model, and calculating to obtain a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate.
3. The method of claim 2, wherein performing two-dimensional human keypoint detection on the human frame and the head frame through a human keypoint detection model, and calculating the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates comprises:
performing foot key point detection on the human body frame through the human body key point detection model, and respectively acquiring a first foot coordinate and a second foot coordinate;
calculating a mean coordinate based on the first foot coordinate and the second foot coordinate, and taking the mean coordinate as the two-dimensional foot center point coordinate;
and performing head key point detection on the human body frame through the human body key point detection model, marking the coordinates of the head frame, calculating the central position of the head frame according to the coordinates of the head frame, and taking the central position as the coordinates of the two-dimensional head central point.
4. The method of claim 1, wherein the method further comprises:
setting basic parameters of image acquisition equipment, wherein the basic parameters of the image acquisition equipment comprise an internal parameter matrix and an external parameter matrix of the image acquisition equipment;
the converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting the two-dimensional pixel coordinate system into the three-dimensional world coordinate system comprises the following steps:
converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate under the two-dimensional pixel coordinate system to the two-dimensional image coordinate system based on an internal reference matrix of the image acquisition equipment;
and converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate in the two-dimensional image coordinate system into a three-dimensional world coordinate system based on the external parameter matrix of the image acquisition equipment to obtain the three-dimensional head central point coordinate and the three-dimensional foot central point coordinate.
5. The method of claim 1, wherein the obtaining auxiliary point coordinates of a reference plane, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculating an estimated distance from the three-dimensional foot center point coordinates to the auxiliary line comprises:
determining the reference surface and selecting the auxiliary point from the reference surface;
connecting the coordinates of the auxiliary points with the coordinates of the three-dimensional head center point to form an auxiliary line, wherein the auxiliary line is vertical to the ground;
and calculating the distance from the three-dimensional foot central point coordinate to the auxiliary line, wherein the distance is the estimated distance from the three-dimensional head central point coordinate to the three-dimensional foot central point coordinate.
6. A computing device of person height, comprising:
the first acquisition module is used for acquiring a personnel image acquired by the image acquisition equipment;
the second acquisition module is used for acquiring two-dimensional key point coordinates of personnel contained in the personnel image from the personnel image, wherein the two-dimensional key point coordinates comprise a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate;
the conversion module is used for converting the two-dimensional head central point coordinate and the two-dimensional foot central point coordinate through converting a two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain a three-dimensional head central point coordinate and a three-dimensional foot central point coordinate;
the first calculation module is used for acquiring auxiliary point coordinates of a reference surface, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head central point coordinates, and calculating the estimated distance from the three-dimensional foot central point coordinates to the auxiliary line;
and the second calculation module is used for calculating the height of the person contained in the person image based on the estimated distance, the basic parameters of the image acquisition equipment, the body frame coordinates of the person contained in the person image, the head frame coordinates of the person contained in the person image and the two-dimensional foot central point coordinates.
7. The apparatus of claim 6, wherein the acquisition module comprises:
the first detection unit is used for detecting a human body frame and a head frame of a person contained in a person image based on a human body detection model, wherein the human body frame is provided with corresponding human body frame coordinates, and the head frame is provided with corresponding head frame coordinates;
and the second detection unit is used for carrying out two-dimensional human key point detection on the human body frame and the head frame through a human key point detection model, and calculating to obtain a two-dimensional head central point coordinate and a two-dimensional foot central point coordinate.
8. The apparatus of claim 7, wherein the second detection unit comprises:
the first detection subunit is used for performing foot key point detection on the human body frame through the human body key point detection model and respectively acquiring a first foot coordinate and a second foot coordinate;
the first calculating subunit is configured to calculate a mean coordinate based on the first foot coordinate and the second foot coordinate, and use the mean coordinate as the two-dimensional foot center point coordinate;
and the second calculating subunit is used for performing head key point detection on the human body frame through the human body key point detection model, marking the coordinates of the head frame, calculating the central position of the head frame according to the coordinates of the head frame, and taking the central position as the coordinates of the two-dimensional head central point.
9. An electronic device, comprising: memory, processor and computer program stored on said memory and executable on said processor, said processor implementing the steps of a method of calculating a person's height according to any one of claims 1 to 5 when executing said computer program.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of a method for calculating a height of a person as claimed in any one of claims 1 to 5.
CN202111584584.8A 2021-12-22 2021-12-22 Method and device for calculating height of person, electronic equipment and storage medium Pending CN114463663A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111584584.8A CN114463663A (en) 2021-12-22 2021-12-22 Method and device for calculating height of person, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111584584.8A CN114463663A (en) 2021-12-22 2021-12-22 Method and device for calculating height of person, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114463663A true CN114463663A (en) 2022-05-10

Family

ID=81406526

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111584584.8A Pending CN114463663A (en) 2021-12-22 2021-12-22 Method and device for calculating height of person, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114463663A (en)

Similar Documents

Publication Publication Date Title
JP5873442B2 (en) Object detection apparatus and object detection method
TWI574223B (en) Navigation system using augmented reality technology
KR101791590B1 (en) Object pose recognition apparatus and method using the same
CN107273846B (en) Human body shape parameter determination method and device
US20170337701A1 (en) Method and system for 3d capture based on structure from motion with simplified pose detection
US20120070070A1 (en) Learning-based pose estimation from depth maps
US20160232683A1 (en) Apparatus and method for analyzing motion
CN110599540A (en) Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera
WO2019035155A1 (en) Image processing system, image processing method, and program
WO2012040709A2 (en) Augmenting image data based on related 3d point cloud data
WO2021190321A1 (en) Image processing method and device
CN106264537B (en) System and method for measuring human body posture height in image
JP2019204518A (en) Person tracking method, device, instrument and storage medium
CN109740659A (en) A kind of image matching method and device, electronic equipment, storage medium
US11521411B2 (en) System and method for providing multi-camera 3D body part labeling and performance metrics
CN108234932B (en) Method and device for extracting personnel form in video monitoring image
JP5373128B2 (en) Person identification device, person identification method, and person identification program
WO2020261403A1 (en) Height estimation device, height estimation method, and non-transitory computer-readable medium having program stored thereon
WO2020261404A1 (en) Person state detecting device, person state detecting method, and non-transient computer-readable medium containing program
CN113272864A (en) Information processing apparatus, information processing method, and program
CN114463663A (en) Method and device for calculating height of person, electronic equipment and storage medium
KR100951315B1 (en) Method and device detect face using AAMActive Appearance Model
Aliakbarpour et al. Geometric exploration of virtual planes in a fusion-based 3D data registration framework
KR20230113371A (en) Identify the 3D position of an object within an image or video
Burlacu et al. Ground geometry assessment in complex stereo vision based applications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination