CN115546130A - Height measurement method, device and electronic equipment for digital twin - Google Patents

Abstract

The application discloses a height measuring method, a height measuring device and electronic equipment for a digital twin, wherein the method comprises the following steps: acquiring a current frame region image acquired by a roadside camera, and performing pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state; if the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground; determining a pedestrian height measurement result corresponding to the current frame region image according to a projection result of the pedestrian detection frame on the ground and attribute information of the roadside camera, so as to realize visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image. The pedestrian height is measured based on the image collected by the camera, and the measurement result of the pedestrian height is used as a characteristic for visual display of the digital twin platform on the target, so that the visual effect of the digital twin platform is improved.

Description

Height measurement method, device and electronic equipment for digital twin

technical field

The present application relates to the technical field of digital twins, and in particular to a height measurement method, device and electronic equipment for digital twins.

Background technique

Digital twin is a simulation process that makes full use of physical models, sensor updates, operation history, etc., integrates multi-disciplinary, multi-physical quantities, multi-scale, and multi-probability simulation processes, and completes the mapping in the virtual space to reflect the entire life cycle of the corresponding physical equipment. process. A digital twin is a concept beyond reality that can be viewed as a digital twin of one or more important, interdependent equipment systems.

In scenarios such as smart transportation, digital twin technology plays an important role. At present, the digital twin platform applied to the road basically uses multiple sensors such as cameras to detect objects such as pedestrians and vehicles, and then displays them on the digital twin platform with a simulated standard 3D model. The so-called "standard 3D model" refers to all detected The pedestrians are all the same size, all the cars are displayed by the same car model, and all the buses are also displayed by the same bus model, so the distinction between objects is poor, and it is different from real-world objects The difference is large, which leads to the poor visualization effect of the digital twin platform.

In addition, the camera performs target detection independently for each frame of image, and the targets detected in different frames need to be correlated, otherwise redundant and repeated targets will be generated on the digital twin platform. However, the existing association schemes generally use the time difference between two frames to predict the approximate location of the target in the current frame, and then calculate the intersection and union ratio with the target frame detected in the current frame to confirm whether it is the same target. Or it can be realized by means of image feature matching or the like. However, these solutions still cause mismatches due to occluded targets or wrong predictions, which in turn affects the visualization of the digital twin platform.

Contents of the invention

Embodiments of the present application provide a height measurement method, device, and electronic equipment for digital twins, so as to improve the visualization effect of the digital twin platform.

The embodiment of the application adopts the following technical solutions:

In the first aspect, the embodiment of the present application provides a height measurement method for a digital twin, wherein the method includes:

Obtain the current frame area image collected by the roadside camera;

Using a preset target detection model to perform pedestrian detection on the current frame area image to obtain a pedestrian detection result of the current frame area image, the pedestrian detection result includes a pedestrian detection frame and a pedestrian state;

When the pedestrian state is a standing state, project the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground;

According to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determine the pedestrian height measurement result corresponding to the current frame area image, so as to realize the digital twin platform based on the pedestrian height measurement result corresponding to the current frame area image visual display of .

Optionally, the projecting the pedestrian detection frame onto the ground, and obtaining the projection result of the pedestrian detection frame on the ground includes:

Determine the transformation relationship between the image coordinate system and the world geodetic coordinate system;

According to the transformation relationship between the image coordinate system and the world earth coordinate system, the pedestrian detection frame is projected onto the ground to obtain a projection result of the pedestrian detection frame on the ground.

Optionally, the pedestrian detection frame includes the pixel position of the upper center point and the pixel position of the lower center point of the pedestrian detection frame, and the pedestrian detection frame is projected onto the ground to obtain the pedestrian detection frame on the ground. Projection results include:

Projecting the pixel position of the upper center point of the pedestrian detection frame onto the ground to obtain the projected position of the upper center point of the pedestrian detection frame on the ground;

Projecting the pixel position of the lower central point of the pedestrian detection frame onto the ground to obtain the projected position of the lower central point of the pedestrian detection frame on the ground.

Optionally, the projection result of the pedestrian detection frame on the ground includes the projection positions of the upper center point and the lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes the height of the roadside camera and the height of the roadside camera. The position of the side camera on the ground;

According to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determining the pedestrian height measurement result corresponding to the current frame area image includes:

According to the projected position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground, determine the distance from the lower center point of the pedestrian detection frame to the roadside camera;

According to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground, determine the distance from the upper center point of the pedestrian detection frame to the roadside camera;

Based on the distance from the center point of the lower side of the pedestrian detection frame to the roadside camera, the distance from the center point of the upper side of the pedestrian detection frame to the roadside camera, and the height of the roadside camera, the similar triangle algorithm is used to determine The pedestrian height measurements.

Optionally, after determining the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further includes:

A three-dimensional pedestrian model is generated according to a preset ratio according to the pedestrian height measurement result for display on a digital twin platform.

Optionally, after determining the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further includes:

Obtain the pedestrian detection result of the previous frame area image and the corresponding pedestrian height measurement result;

Based on the pedestrian height measurement result corresponding to the previous frame area image and the pedestrian height measurement result corresponding to the current frame area image, the pedestrian detection result of the previous frame area image is compared with the pedestrian detection result of the current frame area image. Feature matching to obtain pedestrian feature matching results;

According to the pedestrian feature matching results, pedestrian movement tracks are generated to be displayed on the digital twin platform.

In the second aspect, the embodiment of the present application also provides a height measurement device for digital twins, wherein the device includes:

The first acquisition unit is used to acquire the current frame area image collected by the roadside camera;

A target detection unit, configured to use a preset target detection model to perform pedestrian detection on the current frame area image to obtain a pedestrian detection result of the current frame area image, the pedestrian detection result including a pedestrian detection frame and a pedestrian state;

A projection unit, configured to project the pedestrian detection frame onto the ground when the pedestrian is in a standing state, to obtain a projection result of the pedestrian detection frame on the ground;

The determination unit is configured to determine the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to determine the pedestrian height measurement result corresponding to the current frame area image. Results are visualized on a digital twin platform.

Optionally, the projection unit is specifically used for:

Determine the transformation relationship between the image coordinate system and the world geodetic coordinate system;

According to the transformation relationship between the image coordinate system and the world earth coordinate system, the pedestrian detection frame is projected onto the ground to obtain a projection result of the pedestrian detection frame on the ground.

In a third aspect, the embodiment of the present application further provides an electronic device, including:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform any one of the preceding methods.

In a fourth aspect, the embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs , causing the electronic device to execute any one of the aforementioned methods.

The above-mentioned at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects: the height measurement method for digital twins in the embodiment of the present application first obtains the current frame area image collected by the roadside camera; then uses the preset target detection model Pedestrian detection is performed on the current frame area image to obtain the pedestrian detection result of the current frame area image. The pedestrian detection result includes the pedestrian detection frame and the pedestrian state; then, when the pedestrian state is standing, the pedestrian detection frame is projected on the ground to obtain The projection result of the pedestrian detection frame on the ground; finally, according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the measurement result of the pedestrian height corresponding to the current frame area image is determined, and the pedestrian height measurement result corresponding to the current frame area image is used. The height measurement results are visualized on the digital twin platform. The height measurement method for digital twins in the embodiment of the present application measures the height of pedestrians based on the images collected by the camera, and uses the pedestrian height measurement results as a feature for the visual display of the digital twin platform for the target, which improves the digital twin platform. Visualization.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

Fig. 1 is a schematic flow chart of a height measurement method for digital twins in the embodiment of the present application;

Fig. 2 is a schematic diagram of the principle of measuring the height of pedestrians based on the similar triangle algorithm in the embodiment of the present application;

FIG. 3 is a schematic structural diagram of a height measuring device for digital twins in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device in an embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

The technical solutions provided by various embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

The embodiment of the present application provides a height measurement method for digital twins, as shown in Figure 1, provides a schematic flow chart of a height measurement method for digital twins in the embodiment of the present application, the method at least includes the following Step S110 to Step S140 of:

Step S110, acquiring the current frame area image collected by the roadside camera.

The height measurement method for digital twins in the embodiment of the present application can be applied to scenes with a large flow of people such as scenic spots and intersections. Of course, those skilled in the art can flexibly choose which scenes are specifically applied according to actual needs. Specific limits. When measuring the height of pedestrians, it is necessary to obtain the current frame area image collected by the roadside camera.

Step S120 , using a preset object detection model to perform pedestrian detection on the current frame area image to obtain a pedestrian detection result of the current frame area image, and the pedestrian detection result includes a pedestrian detection frame and a pedestrian state.

After the current frame area image is obtained, the preset target detection model can be used for pedestrian detection, that is, the pedestrian target is detected from the current frame area image. The specific detection results can include the pedestrian detection frame and the pedestrian status, such as standing or not. standing state.

The preset target detection model can be obtained based on network training such as YOLO network, SSD (Single Shot MultiBoxDetector, single-shot multi-box detector) or RCNN (Region Proposal-Convolutional Neural Networks, regional convolutional neural network), when constructing training samples, It is necessary to provide as many pedestrian states as possible from different angles and poses to ensure the accuracy and robustness of the target detection model. Of course, which detection model to use for pedestrian detection can be flexibly selected by those skilled in the art according to actual needs, and is not specifically limited here.

Step S130 , if the state of the pedestrian is standing, project the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground.

Since the measurement of the pedestrian's height needs to have a high measurement accuracy when the pedestrian is in a standing state, after obtaining the pedestrian detection result, it is necessary to first determine whether the detected pedestrian is in a standing state, and if so, you can The detected pedestrian detection frame is further projected onto the ground to obtain the projection result of the pedestrian detection frame on the ground.

Step S140, according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determine the pedestrian height measurement result corresponding to the current frame area image, so as to realize based on the pedestrian height measurement result corresponding to the current frame area image Visualization of the digital twin platform.

The projection result of the pedestrian detection frame on the ground refers to the position of the pedestrian detection frame in the world geodetic coordinate system, further combined with the previously measured position of the roadside camera in the world geodetic coordinate system and the actual height of the roadside camera. Calculate pedestrian height. With the feature of pedestrian height, it can be applied to the visual display of the digital twin platform. For example, the pedestrian height attribute can be added when the digital twin platform displays the pedestrian stereo model, so that the displayed pedestrian effect is closer to the real scene. Additional reference information can be provided for pedestrian tracking and matching based on pedestrian height, thereby improving the accuracy of tracking and matching.

In some embodiments of the present application, the projecting the pedestrian detection frame onto the ground and obtaining the projection result of the pedestrian detection frame on the ground includes: determining the transformation relationship between the image coordinate system and the world geodetic coordinate system; according to The transformation relationship between the image coordinate system and the world earth coordinate system is to project the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground.

Since the pedestrian detection frame is located in the image coordinate system, when projecting the pedestrian detection frame onto the ground, it is necessary to determine the transformation relationship between the image coordinate system and the world geodetic coordinate system. Specifically, the existing calibration method can be used for The transformation relationship between the image coordinate system and the world geodetic coordinate system is calibrated. For example, a car marked with a QR code can be used to drive in scenic spots and other scenes. The car can provide centimeter-level precision latitude and longitude coordinates based on RTK (Real-time kinematic, real-time differential positioning) equipment, so according to the QR code, it can be obtained from the image. The image pixel position corresponding to the current latitude and longitude of the car, thus establishing the transformation relationship between the image coordinate system and the world geodetic coordinate system. For another example, you can manually select the position points of the high-precision map on the ground and the corresponding ground pixel points on the image to form a point pair and establish a system of equations, and solve the relationship between the image coordinate system and the world geodetic coordinate system through the simultaneous equation system. transformation matrix.

After obtaining the transformation relationship between the image coordinate system and the world geodetic coordinate system, the pedestrian detection frame in the image coordinate system can be transformed to the world geodetic coordinate system based on the transformation relationship, so as to obtain the projection of the pedestrian detection frame on the ground result.

In some embodiments of the present application, the pedestrian detection frame includes the pixel position of the upper center point of the pedestrian detection frame and the pixel position of the lower center point of the pedestrian detection frame, and the pedestrian detection frame is projected onto the ground to obtain the pedestrian detection frame The projection result of the frame on the ground includes: projecting the pixel position of the upper center point of the pedestrian detection frame onto the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground; The pixel position of the lower central point is projected onto the ground to obtain the projected position of the lower central point of the pedestrian detection frame on the ground.

The pedestrian detection frame is usually a rectangular frame. When the pedestrian is standing, the vertical distance between the upper boundary and the lower boundary of the pedestrian detection frame can be regarded as the height of the pedestrian. Therefore, when the pedestrian detection frame is projected on the ground , the upper boundary position and the lower boundary position of the pedestrian detection frame can be projected onto the ground. For the convenience of processing, the position of the center point of the upper boundary and the position of the center point of the lower boundary can be respectively projected onto the ground, so as to obtain the projection positions of the center point of the upper/lower edge of the pedestrian detection frame on the ground respectively.

In some embodiments of the present application, the projection result of the pedestrian detection frame on the ground includes the projection positions of the upper center point and the lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes roadside The height of the camera and the position of the roadside camera on the ground; according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determining the pedestrian height measurement result corresponding to the current frame area image includes: according to the Describe the projected position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground, determine the distance from the lower center point of the pedestrian detection frame to the roadside camera; according to the pedestrian detection The projected position of the upper center point of the frame on the ground and the position of the roadside camera on the ground determine the distance from the upper center point of the pedestrian detection frame to the roadside camera; based on the lower edge of the pedestrian detection frame The distance from the center point to the roadside camera, the distance from the upper center point of the pedestrian detection frame to the roadside camera, and the height of the roadside camera are determined using a similar triangle algorithm to determine the pedestrian height measurement result.

In order to facilitate the understanding of the embodiment of the present application, as shown in FIG. 2 , a schematic diagram of the principle of measuring pedestrian height based on the similar triangle algorithm in the embodiment of the present application is provided. Suppose the pixel position of the upper center point of the pedestrian detection frame is P3, the projection position of the upper center point of the pedestrian detection frame on the ground is P2, and the pixel position of the lower center point of the pedestrian detection frame and the corresponding projection position on the ground are the same Point P1, the height of the camera has been measured as H, the position of the camera on the ground is O, and the installation position of the camera is C, then according to the projection position of P1, the lateral distance between P1 and the camera can be calculated as S1, and according to the projection position of P2 It can be calculated that the lateral distance between P2 and the camera is S2, and the final pedestrian height to be measured is h.

Based on this, according to the principle of similar triangles, the triangle formed by P1P2P3 and the triangle formed by O P2C are similar triangles, so in the triangle formed by P1P2P3 and the triangle formed by O P2C, the ratio of P1P3 to OC is equal to the ratio of P1P2 to OP2, that is h/H=(S2-S1)/S2, and then it can be obtained that h=H*(S2-S1)/S2.

In some embodiments of the present application, after determining the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further includes: according to The pedestrian height measurement result generates a pedestrian three-dimensional model according to a preset ratio for display on a digital twin platform.

After obtaining the pedestrian height measurement results corresponding to the current frame area image, based on the pedestrian height measurement results, more information can be provided for the visual display of pedestrians on the digital twin platform, that is, when the pedestrian stereo model is generated and displayed on the digital twin platform , can increase the measured pedestrian height features, so that the generated pedestrian stereo model is more distinguishable, closer to the real world, and makes the visualization effect of the digital twin platform more vivid.

In some embodiments of the present application, after determining the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further includes: obtaining The pedestrian detection result of the previous frame area image and the corresponding pedestrian height measurement result; based on the pedestrian height measurement result corresponding to the previous frame area image and the pedestrian height measurement result corresponding to the current frame area image, the previous The pedestrian detection result of the frame area image is matched with the pedestrian detection result of the current frame area image to obtain the pedestrian feature matching result; the pedestrian moving track is generated according to the pedestrian feature matching result to be displayed on the digital twin platform.

After obtaining the pedestrian height measurement results corresponding to the current frame area image, the pedestrian height measurement results can also be applied to pedestrian tracking and matching. For example, N pedestrians will be detected in the current frame area image, while in the previous frame area image there are When M pedestrians are detected, the pedestrian detection frames in the two frames of images need to be paired one by one, and the pair of detection frames that are successfully paired will continue with the same pedestrian ID, and the pedestrian’s movement track can be displayed in real time on the digital twin platform. The pedestrian detection frame that has not been paired successfully generates a new pedestrian ID to indicate that the pedestrian has just appeared in the camera's field of view. And because there will be a large number of people blocking each other in scenic spots and other scenes, it is easy to cause matching errors, and for pedestrians, the feature similarity is also high, so the embodiment of the present application can increase the accuracy of matching by introducing the pedestrian height attribute , that is, it can be determined that the two pedestrian frames do not correspond to the same pedestrian through the inconsistent height.

Specifically, the pedestrian detection results of the previous frame of area images and the corresponding pedestrian height measurement results can be obtained first, and then the existing feature matching algorithm is used to perform feature matching on the pedestrian detection results of the two frames of images, and the two frames of images The corresponding pedestrian height measurement results are matched as additional features. The above-mentioned feature matching algorithm can be implemented by using algorithms such as SIFT (Scale-invariant feature transform, scale-invariant feature transform), SURF (Speeded UpRobust Features, accelerated robust features) and so on. , of course, which feature matching algorithm is specifically used can be flexibly selected by those skilled in the art according to actual needs, and is not specifically limited here.

If the pedestrian features corresponding to two adjacent frames of images are successfully matched, and the height of the corresponding pedestrians is the same or the deviation is less than a certain threshold, then it can be considered that the pedestrians detected in the two frames of images correspond to the same pedestrian, and if the two adjacent frames of images If the matching of the corresponding pedestrian features fails, or the height of the corresponding pedestrians is inconsistent or has a large deviation, then it can be considered that the pedestrians detected in the two frames of images are not the same pedestrian. By using the pedestrian height measurement results as additional matching information, pedestrian tracking is improved. matching accuracy.

For successfully tracked pedestrians, the pedestrian trajectory can be further generated according to the tracked pedestrian position in multiple frames of images, so that it can be displayed on the digital twin platform, and the accuracy of the digital twin platform for pedestrian trajectory display is guaranteed.

The embodiment of the present application also provides a height measuring device 300 for digital twins. As shown in FIG. 3 , it provides a schematic structural diagram of a height measuring device for digital twins in the embodiment of the present application. Including: a first acquisition unit 310, a target detection unit 320, a projection unit 330, and a determination unit 340, wherein:

The first acquisition unit 310 is configured to acquire the current frame area image collected by the roadside camera;

The target detection unit 320 is configured to perform pedestrian detection on the current frame area image by using a preset target detection model, to obtain a pedestrian detection result of the current frame area image, and the pedestrian detection result includes a pedestrian detection frame and a pedestrian state;

A projection unit 330, configured to project the pedestrian detection frame onto the ground when the pedestrian state is a standing state, to obtain a projection result of the pedestrian detection frame on the ground;

The determination unit 340 is configured to determine the pedestrian height measurement result corresponding to the current frame area image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to determine the pedestrian height measurement result corresponding to the current frame area image based on the pedestrian height measurement result corresponding to the current frame area image. Measurement results are visualized on a digital twin platform.

In some embodiments of the present application, the projection unit 330 is specifically configured to: determine the transformation relationship between the image coordinate system and the world coordinate system; according to the transformation relationship between the image coordinate system and the world coordinate system, Projecting the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground.

In some embodiments of the present application, the pedestrian detection frame includes the pixel position of the center point of the upper side and the pixel position of the center point of the lower side of the pedestrian detection frame, and the projection unit 330 is specifically configured to: transform the upper side of the pedestrian detection frame The pixel position of the central point is projected onto the ground to obtain the projected position of the upper central point of the pedestrian detection frame on the ground; the pixel position of the lower central point of the pedestrian detection frame is projected onto the ground to obtain the pedestrian detection frame. The projection position of the lower center point on the ground.

In some embodiments of the present application, the projection result of the pedestrian detection frame on the ground includes the projection positions of the upper center point and the lower center point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera includes roadside The height of the camera and the position of the roadside camera on the ground; the determination unit 340 is specifically configured to: according to the projected position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground, Determine the distance from the center point of the lower side of the pedestrian detection frame to the roadside camera; according to the projected position of the center point of the upper side of the pedestrian detection frame on the ground and the position of the roadside camera on the ground, determine the The distance from the upper center point of the pedestrian detection frame to the roadside camera; based on the distance from the lower center point of the pedestrian detection frame to the roadside camera, the upper center point of the pedestrian detection frame to the roadside camera and the height of the roadside camera, and determine the pedestrian height measurement result using a similar triangle algorithm.

In some embodiments of the present application, the device further includes: a rendering unit, configured to generate a three-dimensional model of a pedestrian according to a preset ratio according to the measurement result of the pedestrian's height, for display on a digital twin platform.

In some embodiments of the present application, the device further includes: a second acquiring unit, configured to acquire the pedestrian detection result and the corresponding pedestrian height measurement result of the previous frame area image; a feature matching unit, configured to The pedestrian height measurement result corresponding to a frame area image and the pedestrian height measurement result corresponding to the current frame area image, perform feature matching on the pedestrian detection result of the previous frame area image and the pedestrian detection result of the current frame area image, to obtain Pedestrian feature matching results; a generating unit configured to generate pedestrian movement tracks according to the pedestrian feature matching results for display on the digital twin platform.

It can be understood that the above-mentioned height measurement device for digital twins can implement the various steps of the height measurement method for digital twins provided in the foregoing embodiments, and relevant explanations about the height measurement method for digital twins are applicable to The height measurement device of the digital twin will not be repeated here.

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to FIG. 4 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Wherein, the memory may include a memory, such as a high-speed random-access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. Of course, the electronic device may also include hardware required by other services.

The processor, the network interface and the memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnection standard) bus or an EISA (Extended Industry StandardArchitecture, extended industry standard architecture) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one double-headed arrow is used in FIG. 4 , but it does not mean that there is only one bus or one type of bus.

Memory for storing programs. Specifically, the program may include program code, and the program code includes computer operation instructions. Storage, which can include internal memory and nonvolatile storage, provides instructions and data to the processor.

The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a height measurement device for digital twins on a logical level. The processor executes the program stored in the memory, and is specifically used to perform the following operations:

Obtain the current frame area image collected by the roadside camera;

Using a preset target detection model to perform pedestrian detection on the current frame area image to obtain a pedestrian detection result of the current frame area image, the pedestrian detection result includes a pedestrian detection frame and a pedestrian state;

When the pedestrian state is a standing state, project the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground;

According to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determine the pedestrian height measurement result corresponding to the current frame area image, so as to realize the digital twin platform based on the pedestrian height measurement result corresponding to the current frame area image visual display of .

The method performed by the height measuring device for digital twin disclosed in the above embodiment shown in FIG. 1 of the present application may be applied to a processor or implemented by the processor. A processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. Above-mentioned processor can be general-purpose processor, comprises central processing unit (Central Processing Unit, CPU), network processor (Network Processor, NP) etc.; It can also be Digital Signal Processor (Digital Signal Processor, DSP), ASIC (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.

The electronic device can also execute the method performed by the height measurement device for digital twins in FIG. 1, and realize the functions of the height measurement device for digital twins in the embodiment shown in FIG. 1, and the embodiments of the present application will not repeat them here. .

The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs include instructions, and when the instructions are executed by an electronic device including a plurality of application programs , the electronic device can be made to execute the method implemented by the height measuring device for the digital twin in the embodiment shown in FIG. 1, and is specifically used to execute:

Obtain the current frame area image collected by the roadside camera;

Using a preset target detection model to perform pedestrian detection on the current frame area image to obtain a pedestrian detection result of the current frame area image, the pedestrian detection result includes a pedestrian detection frame and a pedestrian state;

When the pedestrian state is a standing state, project the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame on the ground;

According to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, determine the pedestrian height measurement result corresponding to the current frame area image, so as to realize the digital twin platform based on the pedestrian height measurement result corresponding to the current frame area image visual display of .

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (10)

1. A height measurement method for a digital twin, wherein the method comprises:

acquiring a current frame region image acquired by a roadside camera;

carrying out pedestrian detection on the current frame region image by using a preset target detection model to obtain a pedestrian detection result of the current frame region image, wherein the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;

under the condition that the pedestrian state is a standing state, projecting the pedestrian detection frame to the ground to obtain a projection result of the pedestrian detection frame on the ground;

and determining a pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so as to realize the visual display of the digital twin platform based on the pedestrian height measurement result corresponding to the current frame region image.

2. The method of claim 1, wherein the projecting the pedestrian detection frame onto the ground to obtain the projection result of the pedestrian detection frame onto the ground comprises:

determining a transformation relation between an image coordinate system and a world geodetic coordinate system;

and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain the projection result of the pedestrian detection frame on the ground.

3. The method of claim 1, wherein the pedestrian detection frame comprises a pixel position of an upper central point and a pixel position of a lower central point of the pedestrian detection frame, and the projecting the pedestrian detection frame onto the ground to obtain a projection result of the pedestrian detection frame onto the ground comprises:

projecting the pixel position of the upper center point of the pedestrian detection frame to the ground to obtain the projection position of the upper center point of the pedestrian detection frame on the ground;

and projecting the pixel position of the lower central point of the pedestrian detection frame to the ground to obtain the projection position of the lower central point of the pedestrian detection frame on the ground.

4. The method of claim 1, wherein the projection result of the pedestrian detection frame on the ground comprises projection positions of an upper central point and a lower central point of the pedestrian detection frame on the ground, and the attribute information of the roadside camera comprises the height of the roadside camera and the position of the roadside camera on the ground;

the determining the pedestrian height measurement result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the road side camera comprises:

determining the distance from the lower center point of the pedestrian detection frame to the roadside camera according to the projection position of the lower center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;

determining the distance from the upper center point of the pedestrian detection frame to the roadside camera according to the projection position of the upper center point of the pedestrian detection frame on the ground and the position of the roadside camera on the ground;

and determining the height measurement result of the pedestrian by using a similar triangle algorithm based on the distance from the lower central point of the pedestrian detection frame to the roadside camera, the distance from the upper central point of the pedestrian detection frame to the roadside camera and the height of the roadside camera.

5. The method of claim 1, wherein after determining the measurement result of the height of the pedestrian corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further comprises:

and generating a pedestrian three-dimensional model according to the pedestrian height measurement result in a preset proportion so as to display the pedestrian three-dimensional model on the digital twin platform.

6. The method of claim 1, wherein after determining the measurement result of the height of the pedestrian corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, the method further comprises:

acquiring a pedestrian detection result of the previous frame of region image and a corresponding pedestrian height measurement result;

performing feature matching on the pedestrian detection result of the previous frame region image and the pedestrian detection result of the current frame region image based on the pedestrian height measurement result corresponding to the previous frame region image and the pedestrian height measurement result corresponding to the current frame region image to obtain a pedestrian feature matching result;

and generating a pedestrian movement track according to the pedestrian feature matching result so as to display on the digital twin platform.

7. A height measuring device for a digital twin, wherein the device comprises:

the first acquisition unit is used for acquiring a current frame region image acquired by the road side camera;

the target detection unit is used for carrying out pedestrian detection on the current frame region image by utilizing a preset target detection model to obtain a pedestrian detection result of the current frame region image, and the pedestrian detection result comprises a pedestrian detection frame and a pedestrian state;

the projection unit is used for projecting the pedestrian detection frame to the ground under the condition that the pedestrian state is a standing state, so that a projection result of the pedestrian detection frame on the ground is obtained;

and the determining unit is used for determining a pedestrian height measuring result corresponding to the current frame region image according to the projection result of the pedestrian detection frame on the ground and the attribute information of the roadside camera, so that the pedestrian height measuring result corresponding to the current frame region image is visually displayed on the digital twin platform based on the current frame region image.

8. The apparatus of claim 7, wherein the projection unit is specifically configured to:

determining a transformation relation between an image coordinate system and a world geodetic coordinate system;

and projecting the pedestrian detection frame to the ground according to the transformation relation between the image coordinate system and the world geodetic coordinate system to obtain a projection result of the pedestrian detection frame on the ground.

9. An electronic device, comprising:

a processor; and

a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer readable storage medium storing one or more programs which, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of claims 1-7.

Images