CN115546130A - Height measuring method and device for digital twins and electronic equipment - Google Patents
Height measuring method and device for digital twins and electronic equipment Download PDFInfo
- Publication number
- CN115546130A CN115546130A CN202211174305.5A CN202211174305A CN115546130A CN 115546130 A CN115546130 A CN 115546130A CN 202211174305 A CN202211174305 A CN 202211174305A CN 115546130 A CN115546130 A CN 115546130A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- pedestrian detection
- ground
- detection frame
- region image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The application discloses a height measuring method, a height measuring device and electronic equipment for a digital twin, wherein the method comprises the following steps: acquiring a current frame region image acquired by a roadside camera, and performing pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state; if the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground; determining a pedestrian height measurement result corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, so as to realize visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image. The pedestrian height is measured based on the image collected by the camera, and the measurement result of the pedestrian height is used as a characteristic for visual display of the digital twin platform on the target, so that the visual effect of the digital twin platform is improved.
Description
Technical Field
The application relates to the technical field of digital twins, in particular to a height measuring method and device for the digital twins and electronic equipment.
Background
The digital twin is a simulation process integrating multidisciplinary, multi-physical quantity, multi-scale and multi-probability by fully utilizing data such as a physical model, sensor updating, operation history and the like, and mapping is completed in a virtual space, so that the full life cycle process of corresponding entity equipment is reflected. Digital twinning is an beyond-realistic concept that can be viewed as a digital mapping system of one or more important, interdependent equipment systems.
In scenes such as intelligent traffic, the digital twin technology plays an important role. At present, digital twin platforms applied to roads basically detect targets such as pedestrians and vehicles by using multiple sensors such as cameras and the like, and then display the targets on the digital twin platforms by using simulated standard three-dimensional models, wherein the standard three-dimensional models mean that all detected pedestrians are the same in size, all cars are displayed by using the same car model, and all buses are displayed by using the same bus model, so that the targets are poor in distinguishability and are greatly different from the targets in the real world, and further the visualization effect of the digital twin platforms is poor.
In addition, the camera performs target detection independently for each frame of image, and correlation is needed between targets detected by different frames, otherwise redundant repeated targets are generated on the digital twin platform. The existing association scheme generally predicts the approximate position of a target in a current frame through the time difference between two frames, and then performs cross-comparison calculation with a target frame detected by the current frame, so as to determine whether the target is the same target, or is realized through image feature matching and other modes. However, these schemes still cause mismatching due to the occlusion of the target or the prediction error, and further affect the visualization effect of the digital twin platform.
Disclosure of Invention
The embodiment of the application provides a height measuring method and device for a digital twin and electronic equipment, so that the visualization effect of a digital twin platform is improved.
The embodiment of the application adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides a height measurement method for a digital twin, where the method includes:
acquiring a current frame region image acquired by a roadside camera;
carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground;
and determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
Optionally, the projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground includes:
determining a transformation relation between an image coordinate system and a world geodetic coordinate system;
and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain the projection result of the pedestrian detection frame on the ground.
Optionally, the pedestrian detection frame includes a pixel position of an upper central point and a pixel position of a lower central point of the pedestrian detection frame, and projecting the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground includes:
projecting the pixel position of the upper center point of the pedestrian detection frame to the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground;
and projecting the pixel position of the lower central point of the pedestrian detection frame to the ground to obtain the projection position of the lower central point of the pedestrian detection frame on the ground.
Optionally, the projection result of the pedestrian detection frame on the ground includes projection positions of an upper center point and a lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes a height of the roadside camera and a position of the roadside camera on the ground;
the determining the pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the road side camera comprises:
determining the distance from the lower center point of the pedestrian detection frame to the roadside camera according to the projection position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;
determining the distance from the upper center point of the pedestrian detection frame to the roadside camera according to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;
and determining the height measurement result of the pedestrian by using a similar triangle algorithm based on the distance from the lower central point of the pedestrian detection frame to the roadside camera, the distance from the upper central point of the pedestrian detection frame to the roadside camera and the height of the roadside camera.
Optionally, after determining a measurement result of a height of the pedestrian corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, the method further includes:
and generating a pedestrian three-dimensional model according to the pedestrian height measurement result in a preset proportion so as to display the pedestrian three-dimensional model on the digital twin platform.
Optionally, after determining a measurement result of a height of the pedestrian corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, the method further includes:
acquiring a pedestrian detection result of the previous frame of region image and a corresponding pedestrian height measurement result;
performing feature matching on the pedestrian detection result of the previous frame region image and the pedestrian detection result of the current frame region image based on the pedestrian height measurement result corresponding to the previous frame region image and the pedestrian height measurement result corresponding to the current frame region image to obtain a pedestrian feature matching result;
and generating a pedestrian movement track according to the pedestrian feature matching result so as to display on the digital twin platform.
In a second aspect, embodiments of the present application further provide a height measuring device for a digital twin, wherein the device comprises:
the first acquisition unit is used for acquiring a current frame region image acquired by the road side camera;
the target detection unit is used for carrying out pedestrian detection on the current frame region image by utilizing a preset target detection model to obtain a pedestrian detection result of the current frame region image, and the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
the projection unit is used for projecting the pedestrian detection frame to the ground under the condition that the pedestrian state is a standing state, so that a projection result of the pedestrian detection frame on the ground is obtained;
and the determining unit is used for determining a pedestrian height measuring result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so that the pedestrian height measuring result corresponding to the current frame region image is visually displayed on the digital twin platform based on the current frame region image.
Optionally, the projection unit is specifically configured to:
determining a transformation relation between an image coordinate system and a world geodetic coordinate system;
and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain a projection result of the pedestrian detection frame on the ground.
In a third aspect, an embodiment of the present application further provides an electronic device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform any of the methods described above.
In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform any of the methods described above.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects: according to the height measuring method for the digital twin, a current frame region image collected by a road side camera is obtained firstly; then, carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state; then, under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground; and finally, determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image. The height measuring method for the digital twin platform measures the height of a pedestrian based on the image acquired by the camera, and uses the height measuring result of the pedestrian as a characteristic for the visual display of the digital twin platform on a target, so that the visual effect of the digital twin platform is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic flow chart of a height measurement method for a digital twin according to an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a principle of measuring a height of a pedestrian based on a triangle-like algorithm in an embodiment of the present application;
FIG. 3 is a schematic view of a height measuring device for a digital twin according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
The embodiment of the present application provides a height measuring method for a digital twin, and as shown in fig. 1, a schematic flow chart of the height measuring method for the digital twin in the embodiment of the present application is provided, where the method at least includes the following steps S110 to S140:
step S110, a current frame region image collected by the road side camera is obtained.
The height measuring method for the digital twin according to the embodiment of the application can be applied to scenes with large people flow, such as scenic spots and intersections, of course, specifically applied to which scenes, and can be flexibly selected by technicians in the field according to actual requirements without specific limitation. When measuring the height of a pedestrian, a current frame region image acquired by a road side camera needs to be acquired first.
And step S120, carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state.
After the current frame region image is obtained, a preset target detection model may be used to perform pedestrian detection, that is, a pedestrian target is detected from the current frame region image, and the specific detection result may include a pedestrian detection frame and a pedestrian state, for example, a standing state or a non-standing state.
The preset target detection model may be obtained based on network training such as a YOLO network, an SSD (Single Shot multi box Detector), or an RCNN (Region-Convolutional Neural network), and when a training sample is constructed, pedestrian states at various angles and in different postures need to be provided as much as possible to ensure accuracy and robustness of the target detection model. Of course, which detection model is specifically adopted for pedestrian detection can be flexibly selected by those skilled in the art according to actual requirements, and is not specifically limited herein.
Step S130, projecting the pedestrian detection frame to the ground under the condition that the pedestrian state is a standing state, and obtaining a projection result of the pedestrian detection frame on the ground.
Because the measurement of the height of the pedestrian can be highly accurate under the condition that the pedestrian is in the standing state, after the pedestrian detection result is obtained, whether the detected pedestrian state is the standing state needs to be judged firstly, if so, the detected pedestrian detection frame can be further projected onto the ground, and therefore the projection result of the pedestrian detection frame on the ground is obtained.
Step S140, determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
The projection result of the pedestrian detection frame on the ground refers to the position of the pedestrian detection frame under the world geodetic coordinate system, and the height of the pedestrian can be calculated by further combining the position of the roadside camera under the world geodetic coordinate system and the actual height of the roadside camera which are measured in advance. The pedestrian height display method has the advantages that the pedestrian height display method can be applied to visual display of the digital twin platform, for example, the height attribute of the pedestrian can be increased when the digital twin platform displays a pedestrian three-dimensional model, so that the displayed pedestrian effect is closer to a real scene, additional reference information can be provided for tracking and matching of the pedestrian based on the height of the pedestrian, and accuracy of tracking and matching is improved.
In some embodiments of the present application, the projecting the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground includes: determining a transformation relation between an image coordinate system and a world geodetic coordinate system; and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain the projection result of the pedestrian detection frame on the ground.
Because the pedestrian detection frame is located under the image coordinate system, when the pedestrian detection frame is projected onto the ground, the transformation relation between the image coordinate system and the world geodetic coordinate system needs to be determined first, and specifically, the transformation relation between the image coordinate system and the world geodetic coordinate system can be calibrated by adopting the existing calibration mode. For example, a trolley marked with a two-dimensional code can be driven in scenes such as scenic spots, and the trolley can provide longitude and latitude coordinates with centimeter-level precision based on RTK (Real-time kinematic) equipment, so that image pixel positions corresponding to the current longitude and latitude of the trolley can be obtained from an image according to the two-dimensional code, and a transformation relation between an image coordinate system and a world geodetic coordinate system is established. For another example, the point pairs of the position points of the high-precision map on the ground and the corresponding ground pixel points on the image can be manually selected to establish an equation set, and the transformation matrix between the image coordinate system and the world geodetic coordinate system can be solved through the simultaneous equation set.
After the transformation relation between the image coordinate system and the world geodetic coordinate system is obtained, the pedestrian detection frame under the image coordinate system can be transformed into the world geodetic coordinate system based on the transformation relation, and therefore the projection result of the pedestrian detection frame on the ground is obtained.
In some embodiments of the application, the pedestrian detection frame includes a pixel position of an upper central point and a pixel position of a lower central point of the pedestrian detection frame, and projecting the pedestrian detection frame onto the ground, and obtaining a projection result of the pedestrian detection frame on the ground includes: projecting the pixel position of the upper center point of the pedestrian detection frame to the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground; and projecting the pixel position of the lower central point of the pedestrian detection frame to the ground to obtain the projection position of the lower central point of the pedestrian detection frame on the ground.
The pedestrian detection frame is generally a rectangular frame, and when a pedestrian is in a standing state, the vertical distance between the upper boundary and the lower boundary of the pedestrian detection frame can be regarded as the height of the pedestrian, so that when the pedestrian detection frame is projected onto the ground, the upper boundary position and the lower boundary position of the pedestrian detection frame can be projected onto the ground. For convenience of processing, the upper boundary center point position and the lower boundary center point position may be projected onto the ground respectively, so as to obtain the projected positions of the upper/lower center points of the pedestrian detection frame on the ground respectively.
In some embodiments of the present application, the projection result of the pedestrian detection frame on the ground includes projection positions of an upper center point and a lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes a height of the roadside camera and a position of the roadside camera on the ground; the determining the pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the road side camera comprises: determining the distance from the lower center point of the pedestrian detection frame to the roadside camera according to the projection position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground; determining the distance from the upper center point of the pedestrian detection frame to the roadside camera according to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground; and determining the height measurement result of the pedestrian by using a similar triangle algorithm based on the distance from the lower central point of the pedestrian detection frame to the roadside camera, the distance from the upper central point of the pedestrian detection frame to the roadside camera and the height of the roadside camera.
In order to facilitate understanding of the embodiment of the present application, as shown in fig. 2, a schematic diagram for measuring the height of a pedestrian based on a similar triangle algorithm in the embodiment of the present application is provided. Suppose that the pixel position of the upper central point of the pedestrian detection frame is P3, the projection position of the upper central point of the pedestrian detection frame on the ground is P2, the pixel position of the lower central point of the pedestrian detection frame and the projection position corresponding to the lower central point on the ground are the same point P1, the height of the camera is measured to be H in advance, the position of the camera on the ground is O, and the installation position of the camera is C, then the transverse distance from the P1 to the camera can be calculated according to the projection position of the P1 to be S1, the transverse distance from the P2 to the camera can be calculated according to the projection position of the P2 to be S2, and the height of the pedestrian needing to be measured is H finally.
Based on this, according to the principle of similar triangles, the triangle formed by P1P2P3 and the triangle formed by O P2C are similar triangles, so that in the triangle formed by P1P2P3 and the triangle formed by O P2C, the ratio of P1P3 to OC is equal to the ratio of P1P2 to OP2, that is, H/H = (S2-S1)/S2, and then H = (S2-S1)/S2 can be obtained.
In some embodiments of the application, after determining a measurement result of a height of a pedestrian corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, the method further includes: and generating a pedestrian three-dimensional model according to the measurement result of the height of the pedestrian and a preset proportion so as to display the pedestrian three-dimensional model on the digital twin platform.
After the pedestrian height measurement result corresponding to the current frame region image is obtained, richer information can be provided for the visual display of the pedestrian on the digital twin platform based on the pedestrian height measurement result, namely, when the pedestrian three-dimensional model is generated and displayed on the digital twin platform, the measured pedestrian height characteristics can be increased, so that the generated pedestrian three-dimensional model has higher distinctiveness and is closer to the real world, and the visual effect of the digital twin platform is more vivid.
In some embodiments of the present application, after determining a measurement result of a height of a pedestrian corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, the method further includes: acquiring a pedestrian detection result of the previous frame of area image and a corresponding pedestrian height measurement result; performing feature matching on the pedestrian detection result of the previous frame region image and the pedestrian detection result of the current frame region image based on the pedestrian height measurement result corresponding to the previous frame region image and the pedestrian height measurement result corresponding to the current frame region image to obtain a pedestrian feature matching result; and generating a pedestrian movement track according to the pedestrian feature matching result so as to display on the digital twin platform.
After obtaining the pedestrian height measurement result corresponding to the current frame region image, the pedestrian height measurement result can be applied to pedestrian tracking matching, for example, N pedestrians can be detected in the current frame region image, M pedestrians can be detected in the previous frame region image, the pedestrian detection frames in the two frame images need to be paired one by one, the successfully paired detection frame pairs continue to the same pedestrian ID, the moving track of the pedestrian can be displayed on the digital twin platform in real time, and the pedestrian detection frames without successful pairing generate a new pedestrian ID to indicate that the pedestrian just appears in the camera view range. And because the scene such as scenic spot has the condition that a large amount of crowds shelter from each other, and then lead to pairing mistake easily, and to the pedestrian, the feature similarity is also very high, consequently the accuracy of pairing can be increased through introducing pedestrian height attribute to this application embodiment, can confirm that two pedestrian frames do not correspond same pedestrian through the height nonconformity promptly.
Specifically, the pedestrian detection result and the corresponding pedestrian height measurement result of the previous frame of region image may be obtained first, then the pedestrian detection results of the two frames of images are subjected to feature matching by using the existing feature matching algorithm, and meanwhile the pedestrian height measurement results corresponding to the two frames of images are used as additional Features for matching, the feature matching algorithm may be implemented by using algorithms such as SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features), and the like, and of course, which feature matching algorithm is specifically used may be flexibly selected by a person skilled in the art according to actual needs, and is not specifically limited herein.
If the pedestrian features corresponding to the two adjacent images are successfully matched, and the heights of the corresponding pedestrians are the same or the deviation is smaller than a certain threshold value, then the pedestrians detected in the two images can be considered to correspond to the same pedestrian, and if the matching of the pedestrian features corresponding to the two adjacent images fails, or the heights of the corresponding pedestrians are not the same or the deviation is large, then the pedestrians detected in the two images can be considered not to be the same pedestrian, and the accuracy of pedestrian tracking matching is improved by taking the height measurement result of the pedestrian as additional matching information.
For the pedestrians successfully tracked, the pedestrian moving track can be further generated according to the positions of the pedestrians tracked in the multi-frame images, so that the pedestrians can be displayed on the digital twin platform, and the accuracy of the digital twin platform for displaying the pedestrian moving track is guaranteed.
The embodiment of the present application further provides a height measuring device 300 for digital twins, as shown in fig. 3, which provides a schematic structural diagram of the height measuring device for digital twins in the embodiment of the present application, and the device 300 includes: a first acquisition unit 310, an object detection unit 320, a projection unit 330, and a determination unit 340, wherein:
a first obtaining unit 310, configured to obtain a current frame region image collected by a roadside camera;
the target detection unit 320 is configured to perform pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, where the pedestrian detection result includes a pedestrian detection frame and a pedestrian state;
a projection unit 330, configured to project the pedestrian detection frame onto the ground when the pedestrian state is a standing state, so as to obtain a projection result of the pedestrian detection frame on the ground;
the determining unit 340 is configured to determine a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to perform visual display on the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
In some embodiments of the present application, the projection unit 330 is specifically configured to: determining a transformation relation between an image coordinate system and a world geodetic coordinate system; and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain the projection result of the pedestrian detection frame on the ground.
In some embodiments of the present application, the pedestrian detection frame includes a pixel position of an upper center point and a pixel position of a lower center point of the pedestrian detection frame, and the projection unit 330 is specifically configured to: projecting the pixel position of the upper center point of the pedestrian detection frame to the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground; and projecting the pixel position of the lower central point of the pedestrian detection frame to the ground to obtain the projection position of the lower central point of the pedestrian detection frame on the ground.
In some embodiments of the application, the projection result of the pedestrian detection frame on the ground includes projection positions of an upper center point and a lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes a height of the roadside camera and a position of the roadside camera on the ground; the determining unit 340 is specifically configured to: determining the distance from the lower center point of the pedestrian detection frame to the roadside camera according to the projection position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground; determining the distance from the upper center point of the pedestrian detection frame to the roadside camera according to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground; and determining the height measurement result of the pedestrian by using a similar triangle algorithm based on the distance from the lower central point of the pedestrian detection frame to the roadside camera, the distance from the upper central point of the pedestrian detection frame to the roadside camera and the height of the roadside camera.
In some embodiments of the present application, the apparatus further comprises: and the rendering unit is used for generating a pedestrian three-dimensional model according to the pedestrian height measurement result and a preset proportion so as to display the pedestrian three-dimensional model on the digital twin platform.
In some embodiments of the present application, the apparatus further comprises: the second acquisition unit is used for acquiring a pedestrian detection result of the previous frame of area image and a corresponding pedestrian height measurement result; the characteristic matching unit is used for carrying out characteristic matching on the pedestrian detection result of the previous frame region image and the pedestrian detection result of the current frame region image based on the pedestrian height measurement result corresponding to the previous frame region image and the pedestrian height measurement result corresponding to the current frame region image to obtain a pedestrian characteristic matching result; and the generating unit is used for generating a pedestrian moving track according to the pedestrian feature matching result so as to display the pedestrian moving track on the digital twin platform.
It can be understood that the height measuring device for the digital twin described above can realize the steps of the height measuring method for the digital twin provided in the foregoing embodiments, and the related explanations regarding the height measuring method for the digital twin are applicable to the height measuring device for the digital twin, and will not be described herein again.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 4, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the non-volatile memory into the memory and runs it, forming a height measuring device for the digital twin on a logical level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:
acquiring a current frame region image acquired by a roadside camera;
carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground;
and determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
The method performed by the embodiment disclosed in FIG. 1 of the present application for a digital twin height measuring device can be implemented in or by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may further execute the method executed by the height measuring apparatus for digital twin shown in fig. 1, and implement the functions of the height measuring apparatus for digital twin shown in the embodiment shown in fig. 1, which are not described herein again in this application.
Embodiments of the present application also propose a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device comprising a plurality of application programs, enable the electronic device to perform the method for the digital twin height measuring apparatus in the embodiment shown in fig. 1, and in particular to perform:
acquiring a current frame region image acquired by a roadside camera;
carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground;
and determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus comprising the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.
Claims (10)
1. A height measurement method for a digital twin, wherein the method comprises:
acquiring a current frame region image acquired by a roadside camera;
carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground;
and determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.
2. The method of claim 1, wherein the projecting the pedestrian detection frame onto the ground to obtain the projection result of the pedestrian detection frame onto the ground comprises:
determining a transformation relation between an image coordinate system and a world geodetic coordinate system;
and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain the projection result of the pedestrian detection frame on the ground.
3. The method of claim 1, wherein the pedestrian detection frame comprises a pixel position of an upper central point and a pixel position of a lower central point of the pedestrian detection frame, and the projecting the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame onto the ground comprises:
projecting the pixel position of the upper center point of the pedestrian detection frame to the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground;
and projecting the pixel position of the lower central point of the pedestrian detection frame to the ground to obtain the projection position of the lower central point of the pedestrian detection frame on the ground.
4. The method of claim 1, wherein the projection result of the pedestrian detection frame on the ground comprises projection positions of an upper central point and a lower central point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera comprises the height of the roadside camera and the position of the roadside camera on the ground;
the determining the pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the road side camera comprises:
determining the distance from the lower center point of the pedestrian detection frame to the roadside camera according to the projection position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;
determining the distance from the upper center point of the pedestrian detection frame to the roadside camera according to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;
and determining the height measurement result of the pedestrian by using a similar triangle algorithm based on the distance from the lower central point of the pedestrian detection frame to the roadside camera, the distance from the upper central point of the pedestrian detection frame to the roadside camera and the height of the roadside camera.
5. The method of claim 1, wherein after determining the measurement result of the height of the pedestrian corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further comprises:
and generating a pedestrian three-dimensional model according to the pedestrian height measurement result in a preset proportion so as to display the pedestrian three-dimensional model on the digital twin platform.
6. The method of claim 1, wherein after determining the measurement result of the height of the pedestrian corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further comprises:
acquiring a pedestrian detection result of the previous frame of region image and a corresponding pedestrian height measurement result;
performing feature matching on the pedestrian detection result of the previous frame region image and the pedestrian detection result of the current frame region image based on the pedestrian height measurement result corresponding to the previous frame region image and the pedestrian height measurement result corresponding to the current frame region image to obtain a pedestrian feature matching result;
and generating a pedestrian movement track according to the pedestrian feature matching result so as to display on the digital twin platform.
7. A height measuring device for a digital twin, wherein the device comprises:
the first acquisition unit is used for acquiring a current frame region image acquired by the road side camera;
the target detection unit is used for carrying out pedestrian detection on the current frame region image by utilizing a preset target detection model to obtain a pedestrian detection result of the current frame region image, and the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;
the projection unit is used for projecting the pedestrian detection frame to the ground under the condition that the pedestrian state is a standing state, so that a projection result of the pedestrian detection frame on the ground is obtained;
and the determining unit is used for determining a pedestrian height measuring result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so that the pedestrian height measuring result corresponding to the current frame region image is visually displayed on the digital twin platform based on the current frame region image.
8. The apparatus of claim 7, wherein the projection unit is specifically configured to:
determining a transformation relation between an image coordinate system and a world geodetic coordinate system;
and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain a projection result of the pedestrian detection frame on the ground.
9. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any one of claims 1 to 7.
10. A computer readable storage medium storing one or more programs which, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211174305.5A CN115546130A (en) | 2022-09-26 | 2022-09-26 | Height measuring method and device for digital twins and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211174305.5A CN115546130A (en) | 2022-09-26 | 2022-09-26 | Height measuring method and device for digital twins and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115546130A true CN115546130A (en) | 2022-12-30 |
Family
ID=84730051
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211174305.5A Pending CN115546130A (en) | 2022-09-26 | 2022-09-26 | Height measuring method and device for digital twins and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115546130A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984970A (en) * | 2023-03-13 | 2023-04-18 | 浙江宇视科技有限公司 | Pedestrian height determining method and device, electronic equipment and storage medium |
-
2022
- 2022-09-26 CN CN202211174305.5A patent/CN115546130A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984970A (en) * | 2023-03-13 | 2023-04-18 | 浙江宇视科技有限公司 | Pedestrian height determining method and device, electronic equipment and storage medium |
| CN115984970B (en) * | 2023-03-13 | 2023-08-18 | 浙江宇视科技有限公司 | Pedestrian height determining method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322702B (en) | Intelligent vehicle speed measuring method based on binocular stereo vision system | |
| JP7052663B2 (en) | Object detection device, object detection method and computer program for object detection | |
| CN110969592B (en) | Image fusion method, automatic driving control method, device and equipment | |
| CN113255578B (en) | Traffic identification recognition method and device, electronic equipment and storage medium | |
| CN115526990A (en) | Target visualization method and device for digital twins and electronic equipment | |
| CN114705121B (en) | Vehicle pose measurement method and device, electronic equipment and storage medium | |
| WO2020156923A2 (en) | Map and method for creating a map | |
| CN115164918B (en) | Semantic point cloud map construction method and device and electronic equipment | |
| CN112001453B (en) | Method and device for calculating accuracy of video event detection algorithm | |
| CN111340877A (en) | Vehicle positioning method and device | |
| CN112989877A (en) | Method and device for labeling object in point cloud data | |
| WO2020199057A1 (en) | Self-piloting simulation system, method and device, and storage medium | |
| KR20220109537A (en) | Apparatus and method for calibration of sensor system of autonomous vehicle | |
| CN115143952A (en) | Automatic driving vehicle positioning method and device based on visual assistance | |
| CN115546130A (en) | Height measuring method and device for digital twins and electronic equipment | |
| CN115457084A (en) | Multi-camera target detection tracking method and device | |
| CN112017241A (en) | Data processing method and device | |
| Leu et al. | High speed stereo vision based automotive collision warning system | |
| CN117315024A (en) | Remote target positioning method and device and electronic equipment | |
| CN117853904A (en) | Road disease detection method, device, equipment, medium and system | |
| CN116740680A (en) | Vehicle positioning method and device and electronic equipment | |
| CN115830073A (en) | Map element reconstruction method, map element reconstruction device, computer equipment and storage medium | |
| CN115471819A (en) | Target detection positioning method and device for roadside scene, electronic equipment and storage medium | |
| CN112016366A (en) | Obstacle positioning method and device | |
| CN114755663A (en) | External reference calibration method and device for vehicle sensor and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |