WO2022001193A1

WO2022001193A1 - Method and apparatus for remote setting-out based on machine vision, and terminal device and storage medium

Info

Publication number: WO2022001193A1
Application number: PCT/CN2021/081145
Authority: WO
Inventors: 郑文
Original assignee: 福建汇川物联网技术科技股份有限公司
Priority date: 2020-06-30
Filing date: 2021-03-16
Publication date: 2022-01-06
Also published as: CN111783659B; CN111783659A

Abstract

A method and an apparatus for remote setting-out based on machine vision, and a terminal device and a storage medium, the method for remote setting-out based on machine vision comprising: automatically tracking easily detectable markers by means of a machine vision algorithm and, when receiving a precise positioning command, implementing precise positioning on the basis of a target object to finally complete setting-out; the present method has the advantages of high setting-out efficiency, high setting-out precision, and low costs.

Description

Machine vision-based remote stakeout method, device and terminal equipment, storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application with the application number 202010622000.0 and the title of "Machine Vision-Based Remote Stakeout Method, Device and Terminal Equipment, Storage Medium" filed with the China Patent Office on June 30, 2020, the entire content of which is approved by Reference is incorporated in this application.

technical field

The present application relates to the field of measurement, and in particular, to a machine vision-based remote stakeout method, device, terminal device, and storage medium.

Background technique

In the process of engineering construction, it is necessary to accurately measure the spatial coordinates of each key point to guide the subsequent construction work, which is an important part of engineering construction. The current mainstream stakeout method uses a total station to operate, which requires the close cooperation of two surveyors. Among them, the pole runner is responsible for moving the prism rod to the key point under the guidance of the operator, and the operator is responsible for placing the laser on the total station. Accurately align the center of the prism rod for measurement and obtain the spatial coordinates of key points. However, this measurement method requires the close cooperation of two measurement personnel to implement, and the measurement accuracy is limited by the operator's own professional skill level, resulting in low measurement efficiency, Disadvantages such as low precision.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a machine vision-based remote stakeout method, device, terminal device, and storage medium, so as to improve the efficiency and accuracy of stakeout.

The embodiment of the present application discloses a remote stakeout method based on machine vision, the method is applied to a terminal device, and the method includes the steps:

Receive a first activation instruction, and convert from a target detection state to a target tracking state, in which the terminal device executes:

Obtain video images from ranging equipment;

Detecting whether the video picture contains a marker according to a machine vision algorithm;

When it is detected that the video frame includes a marker, calculating the imaging width and imaging height of the marker in the video frame and the imaging coordinates of the first center point of the marker in the video frame;

Determine the first current angle of the gimbal according to the imaging coordinates of the first center point;

Determine the imaging size of the marker according to the imaging width and the imaging height;

Calculate the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera of the ranging device and the imaging size of the marker; receive a second activation instruction, and track the target from the target The state transitions to a precise alignment state. In the precise alignment state, the terminal device executes:

Obtaining a video image from the ranging device again;

Calculate the imaging coordinates of the center point of the target in the video frame according to the machine vision algorithm;

The rotation angle of the gimbal is calculated according to the imaging coordinates of the center point and preset device parameters, so that the gimbal rotates according to the rotation angle and the laser landing point of the ranging device falls on the target on the center of the object;

Acquiring the second current angle after the rotation of the gimbal and the laser distance between the ranging device and the target;

The spatial coordinates of the center point of the target object are calculated according to the second current angle and the laser distance.

The machine vision-based remote stakeout method of the present application receives a first activation instruction and acquires a video image from a ranging device, and then detects whether the video image contains a marker according to a machine vision algorithm. When it is detected that the video image contains a marker, calculate The imaging width and imaging height of the marker in the video screen and the imaging coordinates of the first center point of the marker in the video screen, and finally determine the first current angle of the pan/tilt according to the imaging coordinates of the first center point, according to the imaging width and imaging height Determine the imaging size of the marker, calculate the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera of the ranging device and the imaging size of the marker, and finally complete the tracking of the marker.

In addition, when the tracking of the marker is completed, the terminal device can enter the precise alignment state. At this time, the terminal device obtains the video image from the ranging device again, and calculates the image of the center point of the target in the video image according to the machine vision algorithm. Coordinates, calculate the rotation angle of the gimbal according to the imaging coordinates of the center point and the preset device parameters, so that the gimbal rotates according to the rotation angle, and the laser landing point of the ranging device falls on the center point of the target object, and the rotation of the gimbal is obtained. After the second current angle and the laser distance between the ranging device and the target object, the spatial coordinates of the center point of the target object are calculated according to the second current angle and the laser distance, so that the terminal device can use the target object to complete accurate locating.

Compared with manual stakeout in the field, the embodiment of the present application adopts machine vision algorithm and ranging equipment to automatically track the marker and complete accurate positioning according to the target object, so that the measurement personnel do not need to run repeatedly, and the measurement personnel's accuracy is not affected. The measurement technology dependence is low, which can reduce the influence of the wrong operation of the measurement personnel on the measurement accuracy. Therefore, the embodiment of the present application has the advantages of high measurement efficiency and high measurement accuracy. At the same time, since the target object is not very conspicuous in the machine vision algorithm, which is not conducive to tracking, the embodiment of the present application can accurately track the target object through an easily recognizable identification object. In addition, compared with the automatic stakeout equipment in the art, the embodiment of the present application has the advantage of low cost.

In the embodiment of the present application, as an optional implementation manner, the identifier is one of a measurement operator, a reflective vest, and a balloon.

In this optional embodiment, measurement operators, reflective vests, and balloons can be used as markers due to their large size or eye-catching colors.

Optionally, the identifier can also be the object to be measured itself, or can be a specific gesture or human posture of the measuring operator.

Optionally, a specific instruction sent by the mobile smart terminal or other devices can be used as the activation instruction.

Optionally, the power-on of the terminal device, or the establishment of a connection between the computing unit and the ranging device through the wireless network can also be used as an activation instruction.

Optionally, the terminal device recognizes a preset specific object, gesture, human body posture, and illumination change (such as strobe) in the video picture, which can also be used as an activation instruction.

Optionally, when the standby time of the terminal device reaches a preset time threshold, it can also be used as an activation instruction.

In the embodiment of the present application, as an optional implementation manner, the determining of the first current angle of the pan/tilt head according to the imaging coordinates of the first center point includes sub-steps:

Comparing and calculating the imaging coordinates of the first center point with the coordinates of the center point of the video picture and obtaining the pixel difference between the imaging coordinates of the first center point and the coordinates of the center point of the video picture;

Calculate the horizontal angle and vertical angle that the gimbal needs to rotate according to the pixel difference;

The pan/tilt is driven to rotate according to the horizontal angle and the vertical angle, so that the center point of the video image is aligned with the center point of the marker, and the rotated angle of the pan/tilt is taken as the first point of the pan/tilt. a current angle.

In this optional embodiment, the pixel difference between the imaging coordinates of the first center point and the center point coordinates of the video image is calculated and obtained by comparing the imaging coordinates of the first center point with the center point coordinates of the video image, and then according to The pixel difference calculates the horizontal and vertical angles that the gimbal needs to rotate, and finally drives the gimbal to rotate according to the horizontal and vertical angles, so that the center point of the video image can be aligned with the center point of the marker, and the first current angle of the gimbal can be obtained. .

In the embodiment of the present application, as an optional implementation manner, the determining the imaging size of the marker according to the imaging width and the imaging height includes:

comparing the imaging width and the imaging height with a preset width interval and a preset height interval, respectively, and obtaining a comparison result;

Adjust the camera magnification of the distance measuring device according to the comparison result, so that the imaging width and the imaging height satisfy preset conditions by adjusting the video picture;

The imaging size of the marker is determined according to the imaging width and the imaging height satisfying preset conditions.

In this optional implementation manner, the imaging width and imaging height are compared with the preset width interval and the preset height interval respectively, and the comparison result is obtained, and then the camera magnification of the ranging device is adjusted according to the comparison result, so as to adjust the video The image makes the imaging width and imaging height meet the preset conditions, and finally the imaging size of the marker can be determined according to the imaging width and imaging height that satisfy the preset conditions.

In the embodiment of the present application, as an optional implementation manner, the camera magnification of the ranging device is adjusted according to the comparison result, so that the imaging width and the imaging height are adjusted by adjusting the video picture. Meet pre-set conditions, including:

When the imaging width and the imaging height are respectively smaller than the preset width interval and the preset height interval, calculate the camera magnification and control the camera zoom of the ranging device to enlarge the video screen ;

When the imaging width and the imaging height are respectively larger than the preset width interval and the preset height interval, the camera magnification is calculated and the camera is controlled to zoom in order to reduce the video picture.

In this optional implementation manner, by scaling the video picture, the video picture can be realized so that the imaging width and the imaging height satisfy the preset conditions.

In the embodiment of the present application, as an optional implementation manner, after calculating the spatial coordinates of the marker according to the angle of the pan/tilt head, the current magnification of the camera, and the imaging size of the marker, the The method also includes:

Receive a second activation instruction, and transition from the target tracking state to a precise alignment state, in which the terminal device executes:

Obtaining a video image from the ranging device again;

In this optional embodiment, by entering the precise alignment state, the spatial coordinates of the center point of the target can be calculated according to the second current angle and the laser distance, thereby further improving the spatial coordinates of the center of the target. Measurement accuracy.

In the embodiment of the present application, as an optional implementation manner, in the calculation method according to the first current angle of the pan/tilt head, the current magnification of the camera of the ranging device, and the imaging size of the marker, the obtained After describing the spatial coordinates of the marker, the method further includes:

Calculate the difference between the spatial coordinates of the marker and the spatial coordinates of the target point;

The travel direction information is generated according to the difference between the space coordinates of the marker and the space coordinates of the target point, so as to prompt the travel direction information to the user.

In the embodiment of the present application, as an optional implementation manner, before the receiving the first activation instruction and the transition from the target detection state to the target tracking state, the method further includes the steps:

Receive a third activation instruction, and switch from the standby state to the target detection state. In the target detection state, the terminal device executes:

Acquire a video image from the ranging device, detect whether the marker exists in the video image, and if so, enter the target tracking state.

In this optional implementation manner, the travel direction information may be generated according to the difference between the spatial coordinates of the marker and the spatial coordinates of the target point, and then the travel direction information may be prompted to the user.

The embodiment of the present application discloses a remote stakeout device based on machine vision, the device is applied to terminal equipment, and the device includes:

a receiving module for receiving the first activation instruction;

The state switching module is used to switch from the target detection state to the target tracking state;

The acquisition module is used to acquire the video image from the ranging device;

an identification module, configured to detect whether the video picture contains a marker according to a machine vision algorithm;

The first calculation module is configured to calculate the imaging width and imaging height of the landmark in the video picture and the imaging height of the landmark in the video when the identification module detects that the video picture contains a marker. The imaging coordinates of the first center point in the screen;

a first determining module, configured to determine the first current angle of the pan-tilt head according to the imaging coordinates of the first center point;

a second determining module, configured to determine the imaging size of the marker according to the imaging width and the imaging height;

a second calculation module, configured to calculate the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker;

The receiving module is further configured to receive a second activation instruction;

The state switching module is further configured to switch from the target tracking state to a precise alignment state, in the precise alignment state;

The acquiring module is further configured to acquire video images from the ranging device again;

The first calculation module is further configured to calculate the imaging coordinates of the center point of the target in the video frame according to a machine vision algorithm;

The first determining module is further configured to calculate the rotation angle of the gimbal according to the imaging coordinates of the center point and preset device parameters, so that the gimbal rotates according to the rotation angle and makes the distance measurement The laser landing point of the device falls on the center point of the target object;

The acquiring module is further configured to acquire the second current angle after the pan/tilt is rotated and the laser distance between the ranging device and the target;

The second calculation module is further configured to calculate the spatial coordinates of the center point of the target object according to the second current angle and the laser distance.

The machine vision-based remote stakeout device of the present application can obtain a video image from the ranging device by receiving the first activation instruction by executing the machine vision-based remote stakeout method, and then detect whether the video image contains a marker according to the machine vision algorithm. When a marker is included in the video image, the imaging width and imaging height of the marker in the video image and the imaging coordinates of the first center point of the marker in the video image are calculated, and finally the first center point of the gimbal is determined according to the imaging coordinates of the first center point. The current angle, the imaging size of the marker is determined according to the imaging width and the imaging height, the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt, the current magnification of the camera of the ranging device, and the imaging size of the marker, and the marker is finally completed. tracking of things.

Compared with manual stakeout in the field, the embodiment of the present application adopts machine vision algorithm and ranging equipment to automatically track the marker and complete accurate positioning according to the target object, so that the measurement personnel do not need to run repeatedly, and the measurement personnel's accuracy is not affected. The measurement technology dependence is low, which can reduce the influence of the wrong operation of the measurement personnel on the measurement accuracy. Therefore, the embodiment of the present application has the advantages of high measurement efficiency and high measurement accuracy. At the same time, since the target object is not very conspicuous in the machine vision algorithm, which is not conducive to tracking, therefore, the embodiment of the present application can track stably through the easily recognizable marking object. In addition, compared with the automatic stakeout equipment in the art, the embodiment of the present application has the advantage of low cost.

An embodiment of the present application discloses a terminal device, where the terminal device includes:

a memory in which executable program code is stored;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to execute the machine vision-based remote stakeout method disclosed in the embodiment of the present application.

By executing the remote stakeout method based on machine vision, the terminal device of the present application can obtain a video image from a ranging device by receiving a first activation instruction, and then detect whether the video image contains a marker according to a machine vision algorithm. When a marker is used, the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture are calculated, and finally the first current angle of the gimbal is determined according to the imaging coordinates of the first center point. The imaging width and imaging height determine the imaging size of the marker, and the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker, and finally the tracking of the marker is completed.

Compared with manual stakeout in the field, the embodiment of the present application adopts machine vision algorithm and ranging equipment to automatically track the marker and complete accurate positioning according to the target object, so that the measurement personnel do not need to run repeatedly, and the measurement personnel's accuracy is not affected. The measurement technology dependence is low, which can reduce the influence of the wrong operation of the measurement personnel on the measurement accuracy. Therefore, the embodiment of the present application has the advantages of high measurement efficiency and high measurement accuracy. At the same time, since the target object is not very conspicuous in the machine vision algorithm, which is not conducive to tracking, therefore, the embodiment of the present application can track stably through the easily identifiable marking object. In addition, compared with the automatic stakeout equipment in the art, the embodiment of the present application has the advantage of low cost.

The embodiments of the present application disclose a storage medium, where the storage medium stores computer instructions, and when the computer instructions are invoked, the computer instructions are used to execute the machine vision-based remote stakeout method disclosed in the embodiments of the present application.

The storage medium of the present application executes the remote stakeout method based on machine vision, and can obtain a video picture from a ranging device by receiving a first activation instruction, and then detect whether the video picture contains a marker according to a machine vision algorithm. When a marker is used, the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture are calculated, and finally the first current angle of the gimbal is determined according to the imaging coordinates of the first center point. The imaging width and imaging height determine the imaging size of the marker, and the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt, the current magnification of the camera of the ranging device, and the imaging size of the marker, and finally the tracking of the marker is completed.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, therefore It should not be regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a machine vision-based remote stakeout method disclosed in an embodiment of the present application;

2 is a schematic structural diagram of a machine vision-based remote stakeout device disclosed in an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a terminal device disclosed in an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a machine vision-based remote stakeout method disclosed in an embodiment of the present application, and the method is applied to a terminal device. As shown in Figure 1, the method of the embodiment of the present application includes the steps:

101. Receive the first activation instruction, and convert from the target detection state to the target tracking state. In the target tracking state, the terminal device performs the following steps:

102. Acquire a video image from a ranging device;

103. Detecting whether the video image contains a marker according to a machine vision algorithm;

104. When detecting that a marker is included in the video picture, calculate the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture;

105. Determine the first current angle of the gimbal according to the imaging coordinates of the first center point;

106. Determine the imaging size of the marker according to the imaging width and imaging height;

107. Calculate the spatial coordinates of the marker according to the first current angle of the PTZ, the current magnification of the camera of the ranging device, and the imaging size of the marker;

108. Receive the second activation instruction, and convert from the target tracking state to the precise alignment state. In the precise alignment state, the terminal device executes:

109. Obtain the video image from the ranging device again;

110. Calculate the imaging coordinates of the center point of the target in the video image according to the machine vision algorithm;

111. Calculate the rotation angle of the gimbal according to the imaging coordinates of the center point and the preset device parameters, so that the gimbal rotates according to the rotation angle, and the laser landing point of the ranging device falls on the center point of the target object;

112. Obtain the second current angle after the gimbal is rotated and the laser distance between the ranging device and the target;

113. Calculate the spatial coordinates of the center point of the target according to the second current angle and the laser distance.

In the embodiment of the present application, optionally, the marker is one of a measurement operator, a reflective vest, and a balloon.

In this embodiment of the present application, optionally, the identifier may also be the object to be measured itself, or may be a specific gesture or human body posture of the measurement operator.

In this embodiment of the present application, a specific instruction sent by a mobile smart terminal or other device may be used as an activation instruction.

In this embodiment of the present application, the power-on of the terminal device or the establishment of a connection between the computing unit and the ranging device through the wireless network can also be used as an activation instruction.

In the embodiment of the present application, the terminal device recognizes a preset specific object, gesture, human body posture, and illumination change (such as strobe) in the video picture, which can also be used as an activation instruction.

In this embodiment of the present application, optionally, when the standby time of the terminal device reaches a preset time threshold, it can also be used as an activation instruction.

In the embodiments of the present application, it should be noted that a computing unit is installed in the terminal device, and the computing unit is used to execute the machine vision-based remote stakeout method disclosed in the embodiments of the present application.

In the embodiment of the present application, the ranging device is installed on the pan-tilt, and the ranging device can rotate as the pan-tilt rotates.

In this embodiment of the present application, the mobile intelligent terminal may be a mobile phone, or may be other mobile communication terminals such as a PAD and a notebook, which are not limited in this embodiment of the present application.

In this embodiment of the present application, the terminal device may perform remote communication with the mobile intelligent terminal through a wireless network. Optionally, the surveyor can send an activation instruction to the terminal device through the mobile smart terminal, and at the same time, the mobile smart terminal can display the measurement results to the surveyor or feed back the interactive results to the surveyor in the form of voice prompts.

In this embodiment of the present application, the marker may be the object to be measured itself, or other objects, such as a vest decorated with a specific pattern, and optionally, the marker may also be a specific gesture or human posture of the measuring person.

In the embodiments of the present application, the naming differences among the first activation instruction, the second activation instruction, and the third activation instruction in the embodiments of the present application are for the convenience of describing the instructions input by the surveyor to the terminal device at different stages.

In the embodiment of the present application, the terminal device can be divided into a standby state, a target detection state, a target tracking state, and a precise alignment state according to the execution content of the terminal device, and transitions between the states can be performed according to specified conditions. It should be noted that the division of states is to facilitate the measurement personnel to intuitively understand the use state of the terminal device, rather than to absolutely limit a certain step of the terminal device to a certain state.

It can be seen that the machine vision-based remote stakeout method of the embodiment of the present application obtains the video picture from the ranging device by receiving the first activation instruction, and then detects whether the video picture contains a marker according to the machine vision algorithm. When the object is detected, the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture are calculated, and finally the first current angle of the pan/tilt is determined according to the imaging coordinates of the first center point, and the imaging coordinates are based on the imaging coordinates. The width and imaging height determine the imaging size of the marker, and the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker, and finally the tracking of the marker is completed.

In the embodiment of the present application, as an optional implementation manner, the first current angle of the gimbal is determined according to the imaging coordinates of the first center point;

Comparing the imaging coordinates of the first center point with the coordinates of the center point of the video screen, and calculating the pixel difference between the imaging coordinates of the first center point and the coordinates of the center point of the video screen;

Calculate the horizontal and vertical angles that the gimbal needs to rotate according to the pixel difference;

Drive the gimbal to rotate according to the horizontal and vertical angles, so that the center point of the video image is aligned with the center point of the marker, and the rotated angle of the gimbal is taken as the first current angle of the gimbal.

In the embodiment of the present application, as an optional implementation manner, the imaging size of the marker is determined according to the imaging width and imaging height, including sub-steps:

Comparing the imaging width and imaging height with the preset width interval and the preset height interval respectively, and obtaining the comparison result;

Adjust the camera magnification of the ranging device according to the comparison result, so that the imaging width and imaging height can meet the preset conditions by adjusting the video screen;

The imaging size of the marker is determined according to the imaging width and imaging height satisfying the preset conditions.

In the embodiment of the present application, as an optional implementation, adjusting the camera magnification of the ranging device according to the comparison result, so that the imaging width and imaging height meet the preset conditions by adjusting the video picture, including the sub-steps:

When the imaging width and imaging height are respectively smaller than the preset width interval and the preset height interval, calculate the camera magnification and control the camera zoom of the ranging device to enlarge the video screen;

When the imaging width and the imaging height are respectively greater than the preset width interval and the preset height interval, the camera magnification is calculated and the camera zoom is controlled to reduce the video image.

In the embodiment of this application, as an optional implementation, in step 107: after calculating the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera of the ranging device, and the imaging size of the marker , the method of the embodiment of the present application also includes the steps:

In the embodiment of the present application, as an optional implementation manner, before receiving the first activation instruction and transitioning from the target detection state to the target tracking state, the method further includes the steps:

Receive the third activation instruction, and switch from the standby state to the target detection state. In the target detection state, the terminal device executes:

Obtain the video picture from the ranging device, detect whether there is a marker in the video picture, and if so, enter the target tracking state.

In this optional embodiment, it is preliminarily detected whether there is a marker in the video image. If there is, the terminal device can enter the target tracking state, and if not, it can enter the standby state, so that the power of the terminal device can be reduced. consume.

Please refer to FIG. 2 . FIG. 2 is a schematic structural diagram of a machine vision-based remote stakeout device disclosed in an embodiment of the present application, and the device is applied to a terminal device. As shown in Figure 2, the device includes:

a receiving module 201, configured to receive a first activation instruction;

a state switching module 202, configured to switch from a target detection state to a target tracking state;

an acquisition module 203, configured to acquire a video picture from a ranging device;

The identification module 204 is used for detecting whether the video picture contains a marker according to the machine vision algorithm;

The first calculation module 205 is used to calculate the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture when the identification module detects that the marker is included in the video picture;

a first determining module 206, configured to determine the first current angle of the pan/tilt head according to the imaging coordinates of the first center point;

The second determining module 207 is configured to determine the imaging size of the marker according to the imaging width and imaging height;

The second calculation module 208 is configured to calculate and obtain the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker;

The receiving module 201 is further configured to receive a second activation instruction;

The state switching module 202 is also used for converting from the target tracking state to the precise alignment state, under the precise alignment state;

The acquiring module 203 is further configured to acquire the video picture from the ranging device again;

The first calculation module 205 is further configured to calculate the imaging coordinates of the center point of the target in the video frame according to the machine vision algorithm;

The first determination module 206 is also used to calculate the rotation angle of the pan/tilt according to the imaging coordinates of the center point and the preset device parameters, so that the pan/tilt rotates according to the rotation angle, and the laser landing point of the ranging device falls on the center of the target object Point;

The acquiring module 203 is also used to acquire the second current angle after the pan/tilt is rotated and the laser distance between the ranging device and the target;

The second calculation module 208 is further configured to calculate the spatial coordinates of the center point of the target object according to the second current angle and the laser distance.

It can be seen that the machine vision-based remote stakeout device of the embodiment of the present application can obtain a video image from the ranging device by receiving the first activation instruction by executing the machine vision-based remote stakeout method, and then detect whether the video image contains a marker according to a machine vision algorithm. , when it is detected that the video picture contains a marker, calculate the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture, and finally determine the cloud according to the imaging coordinates of the first center point The first current angle of the stage, the imaging size of the marker is determined according to the imaging width and imaging height, and the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt, the current magnification of the camera of the ranging device, and the imaging size of the marker. , and finally complete the tracking of the marker.

In the embodiment of the present application, as an optional implementation manner, the specific manner in which the first determination module 206 determines the first current angle of the pan/tilt head according to the imaging coordinates of the first center point is as follows:

In this embodiment of the present application, as an optional implementation manner, the specific manner in which the second determining module 207 determines the imaging size of the marker according to the imaging width and imaging height is as follows:

In this embodiment of the present application, as an optional implementation, the second determining module 207 adjusts the camera magnification of the ranging device according to the comparison result, so as to adjust the video image so that the imaging width and imaging height meet the specific conditions of the preset conditions. The way is:

In the embodiment of the present application, as an optional implementation manner, the apparatus of the embodiment of the present application further includes a third calculation module and a generation module, wherein:

The third calculation module calculates the difference between the spatial coordinates of the marker and the spatial coordinates of the target point;

The generating module is used for generating travel direction information according to the difference between the space coordinates of the marker and the space coordinates of the target point, so as to prompt the travel direction information to the user.

In this embodiment of the present application, as an optional implementation manner, the receiving module 201 is further configured to receive a third activation instruction, and the state switching module 202 is further configured to switch from the standby state to the target detection state. In the target detection state, the terminal Device execution:

The acquisition module 203 is also used for acquiring a video picture from the ranging device, and the identification module 204 is also used for detecting whether there is a marker in the video picture, and if the state switching module 202 is used, it controls to enter the target tracking state.

Please refer to FIG. 3, which is a schematic structural diagram of a terminal device disclosed in an embodiment of the present application. As shown in Figure 3, the terminal equipment includes:

a memory 301 storing executable program code;

a processor 302 coupled to the memory 301;

The processor 302 invokes the executable program code stored in the memory 301 to execute the machine vision-based remote stakeout method disclosed in the embodiments of the present application.

The embodiments of the present application disclose a storage medium, where computer instructions are stored in the storage medium, and when the computer instructions are invoked, they are used to execute the machine vision-based remote stakeout method disclosed in the embodiments of the present application.

By executing the machine vision-based remote stakeout method, the storage medium of the present application can acquire a video image from a ranging device by receiving a first activation instruction, and then detect whether the video image contains a marker according to a machine vision algorithm. When a marker is used, the imaging width and imaging height of the marker in the video picture and the imaging coordinates of the first center point of the marker in the video picture are calculated, and finally the first current angle of the gimbal is determined according to the imaging coordinates of the first center point. The imaging width and imaging height determine the imaging size of the marker, and the spatial coordinates of the marker can be calculated according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker, and finally the tracking of the marker is completed.

In the embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

In addition, units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

Furthermore, each functional module in each embodiment of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

It should be noted that, if the functions are implemented in the form of software function modules and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the technology in the field or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM) random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

In this document, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such existence between these entities or operations. The actual relationship or sequence.

The above descriptions are merely examples of the present application, and are not intended to limit the protection scope of the present application. For those skilled in the art, various modifications and changes may be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Industrial Applicability

The present application provides a machine vision-based remote stakeout method, device, terminal equipment, and storage medium, which use machine vision algorithms and ranging equipment to automatically track markers and complete accurate positioning according to the target, which can reduce the mistakes of measuring personnel. The influence of operation on measurement accuracy has the advantages of high measurement efficiency, high measurement accuracy and low cost.

Claims

A method for remote stakeout based on machine vision, characterized in that the method is applied to terminal equipment, and the method includes:

Receive a first activation instruction, and convert from a target detection state to a target tracking state, in which the terminal device executes:

Obtain video images from ranging equipment;

Detecting whether the video picture contains a marker according to a machine vision algorithm;

When it is detected that the video frame includes a marker, calculating the imaging width and imaging height of the marker in the video frame and the imaging coordinates of the first center point of the marker in the video frame;

Determine the first current angle of the gimbal according to the imaging coordinates of the first center point;

Determine the imaging size of the marker according to the imaging width and the imaging height;

Calculate the spatial coordinates of the marker according to the first current angle of the pan/tilt, the current magnification of the camera of the ranging device, and the imaging size of the marker;

Receive a second activation instruction, and transition from the target tracking state to a precise alignment state, in which the terminal device executes:

Obtaining a video image from the ranging device again;

Calculate the imaging coordinates of the center point of the target in the video frame according to the machine vision algorithm;

The rotation angle of the gimbal is calculated according to the imaging coordinates of the center point and preset device parameters, so that the gimbal rotates according to the rotation angle and the laser landing point of the ranging device falls on the target on the center of the object;

Acquiring the second current angle after the rotation of the gimbal and the laser distance between the ranging device and the target;

The spatial coordinates of the center point of the target object are calculated according to the second current angle and the laser distance.
The method of claim 1, wherein the marker is one of a measurement operator, a reflective vest, and a balloon.
The method according to claim 1, characterized in that, the identifier is the object to be measured itself, or a specific gesture or human body posture of the measuring operator.
The method according to any one of claims 1-3, wherein the activation instruction is a specific instruction sent by a mobile smart terminal or other device.
The method according to any one of claims 1-3, wherein the activation instruction is to power on the terminal device, or to establish a connection between the computing unit and the ranging device through a wireless network.
The method according to any one of claims 1-3, wherein the activation instruction is that the terminal device recognizes a preset specific object, gesture, human posture, and illumination change in the video picture.
The method according to any one of claims 1-3, wherein the activation instruction is when the standby time of the terminal device reaches a preset time threshold.
The method according to any one of claims 1-7, wherein, determining the first current angle of the pan/tilt head according to the first center point imaging coordinates, comprising:

Comparing and calculating the imaging coordinates of the first center point with the coordinates of the center point of the video picture and obtaining the pixel difference between the imaging coordinates of the first center point and the coordinates of the center point of the video picture;

Calculate the horizontal angle and vertical angle that the gimbal needs to rotate according to the pixel difference;

The pan/tilt is driven to rotate according to the horizontal angle and the vertical angle, so that the center point of the video image is aligned with the center point of the marker, and the rotated angle of the pan/tilt is taken as the first point of the pan/tilt. a current angle.
The method according to any one of claims 1-7, wherein the determining the imaging size of the marker according to the imaging width and the imaging height comprises:

comparing the imaging width and the imaging height with a preset width interval and a preset height interval, respectively, and obtaining a comparison result;

Adjust the camera magnification of the distance measuring device according to the comparison result, so that the imaging width of the marker and the imaging height of the marker meet preset conditions by adjusting the video image;

The imaging size of the marker is determined according to the imaging width and the imaging height satisfying preset conditions.
The method according to claim 9, characterized in that, adjusting the camera magnification of the ranging device according to the comparison result, so that the imaging width and the imaging height satisfy a preset by adjusting the video picture. conditions, including:

When the imaging width and the imaging height are respectively smaller than the preset width interval and the preset height interval, calculate the camera magnification and control the camera of the ranging device to zoom in, so as to enlarge the video screen ;

When the imaging width and the imaging height are respectively larger than the preset width interval and the preset height interval, the camera magnification is calculated and the camera is controlled to zoom in order to reduce the video picture.
The method according to any one of claims 1-7, characterized in that, in the calculation, the marker is obtained according to the first current angle of the pan/tilt head, the current magnification of the camera, and the imaging size of the marker. After the spatial coordinates of the object are determined, the method further includes:

Calculate the difference between the spatial coordinates of the marker and the spatial coordinates of the target point;

The travel direction information is generated according to the difference between the space coordinates of the marker and the space coordinates of the target point, so as to prompt the travel direction information to the user.
The method according to any one of claims 1-7, characterized in that, before the first activation instruction is received, and before the target detection state is converted to the target tracking state, the method further comprises:

Receive a third activation instruction, and switch from the standby state to the target detection state. In the target detection state, the terminal device executes:

Acquire a video image from the ranging device, detect whether the marker exists in the video image, and if so, enter the target tracking state.
A machine vision-based remote stakeout device, characterized in that the device is applied to terminal equipment, and the device includes:

a receiving module for receiving the first activation instruction;

The state switching module is used to switch from the target detection state to the target tracking state;

The acquisition module is used to acquire the video image from the ranging device;

an identification module, used for detecting whether the video picture contains a marker according to a machine vision algorithm;

The first calculation module is configured to calculate the imaging width and imaging height of the landmark in the video picture and the imaging height of the landmark in the video when the identification module detects that the video picture contains a marker. The imaging coordinates of the first center point in the screen;

a first determining module, configured to determine the first current angle of the pan-tilt head according to the imaging coordinates of the first center point;

a second determining module, configured to determine the imaging size of the marker according to the imaging width and the imaging height;

a second calculation module, configured to calculate the spatial coordinates of the marker according to the first current angle of the pan/tilt head, the current magnification of the camera and the imaging size of the marker;

The receiving module is further configured to receive a second activation instruction;

The state switching module is further configured to switch from the target tracking state to a precise alignment state, in the precise alignment state;

The acquiring module is further configured to acquire video images from the ranging device again;

The first calculation module is further configured to calculate the imaging coordinates of the center point of the target in the video frame according to a machine vision algorithm;

The first determining module is further configured to calculate the rotation angle of the pan/tilt according to the imaging coordinates of the center point and preset device parameters, so that the pan/tilt rotates according to the rotation angle and makes the ranging The laser landing point of the device falls on the center point of the target object;

The acquiring module is further configured to acquire the second current angle after the pan/tilt is rotated and the laser distance between the ranging device and the target;

The second calculation module is further configured to calculate the spatial coordinates of the center point of the target object according to the second current angle and the laser distance.
A terminal device, characterized in that the terminal device includes:

a memory in which executable program code is stored;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to execute the machine vision-based remote stakeout method according to any one of claims 1-12.
A storage medium, characterized in that, the storage medium stores computer instructions, and when the computer instructions are invoked, they are used to execute the machine vision-based remote stakeout method according to any one of claims 1-12.