WO2022120533A1

WO2022120533A1 - Motion trajectory display system and method, and storage medium

Info

Publication number: WO2022120533A1
Application number: PCT/CN2020/134334
Authority: WO
Inventors: 侯弘毅; 陈旭
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-12-07
Filing date: 2020-12-07
Publication date: 2022-06-16

Abstract

A motion trajectory display system and method, and a storage medium. The system (100) comprises multiple positioning camera devices (10), a livestreaming channel camera (20) and a first server (30). The positioning camera devices (10) are arranged on a sports field at different first postures and synchronously photograph a target to obtain first images; the livestreaming channel camera (20) comprises an augmented reality lens, and photographs the target at a second posture to obtain a second image; the first server (30) determines a three-dimensional position of the target on the sports field according to the multiple first images and corresponding first postures; and an AR picture of the motion trajectory of the target on the second image is generated according to the second posture, the three-dimensional position of the target on the sports field and the second image, and is displayed on a display device of a user.

Description

Display system, method and storage medium for motion trajectory

technical field

The present application relates to the technical field of live broadcasting, and in particular, to a system, method and storage medium for displaying motion trajectories.

Background technique

The Robomaser Mecha Master Competition is a large-scale event that integrates technology, education, and mass entertainment, and AR live broadcast is an indispensable part of it. In order to improve the observability and entertainment of the live broadcast, it is necessary to render the trajectory special effects of the projectile launched by the robot in real time during the live broadcast, so as to make up for the shortcomings of the inconspicuous ballistic trajectory in the live broadcast screen.

The commercial solutions of the multi-objective positioning technology used in the above-mentioned live broadcasts include vicon, nokov, etc., and the positioning accuracy can reach the sub-millimeter level. However, vicon and nokov require a large number of cameras (for example, vikon requires at least 24 cameras), and the cost is high; the motion compensation system for reflective processing of objects to be captured on the market is very expensive, and the market cannot reflect objects to be captured. The processing motion compensation system cannot well adapt to the needs of trajectory special effects.

SUMMARY OF THE INVENTION

Based on this, the present application provides a system, method and storage medium for displaying a motion trajectory.

In a first aspect, the present application provides a system for displaying motion trajectories, the system comprising:

A plurality of positioning camera devices, each of which is arranged on the sports field in a different first posture, for synchronously photographing a target and obtaining a first image, the first image carrying a camera to obtain the first image. At the first shooting time, the target moves on the sports field;

A live broadcast channel machine, including an augmented reality lens, the live broadcast channel machine is used to shoot the target in a second attitude and obtain a second image for live broadcast, the second image carries the second image obtained by shooting the second shooting time;

a first server, configured to determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images; According to the second posture corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, generate the image of the target on the second image The AR picture of the motion track, the AR picture is used for displaying on the user's display device.

In a second aspect, the present application provides a method for displaying a motion trajectory, the method comprising:

A plurality of positioning camera devices are used to simultaneously capture a target and obtain a plurality of corresponding first images. The first images carry the first shooting time when the first image is captured, and the plurality of positioning camera devices use different first images. A gesture is arranged on the sports field on which the target moves;

The target is photographed in a second attitude by a live broadcast channel machine to obtain a second image for live broadcast, the second image carries the second shooting time when the second image is obtained by shooting, and the live broadcast channel machine includes Augmented reality lenses;

Determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images; according to the second images The corresponding second posture, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, to generate an AR picture of the movement track of the target on the second image, The AR picture is used for displaying on the user's display device.

In a third aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to display the above-mentioned motion trajectory method.

Embodiments of the present application provide a system, method, and storage medium for displaying motion trajectories. A plurality of positioning camera devices arranged on a sports field in different first postures synchronously photograph a target moving on the sports field to obtain a plurality of corresponding The first image of the shooting time; the live broadcast channel machine including the augmented reality lens shoots the target in the second posture to obtain the second image for live broadcast and carrying the second shooting time; the first server is based on the multiple first images and multiple The first posture corresponding to the first image is used to determine the three-dimensional position of the target on the sports field at each first shooting time, and the movement trajectory of the target on the second image is generated in combination with the second posture corresponding to the second image and the second image. The AR picture is used for displaying on the user's display device. Due to the multiple different first images obtained by synchronously photographing the moving target on the sports field through a plurality of positioning camera devices with different positions and different attitudes, the three-dimensional positions of the target on the sports field at different times can be recovered, which makes more than two positioning cameras The device can cover the common moving path of the target on the sports field by adjusting the arrangement angle and arrangement method to meet the requirements of determining the three-dimensional position of the target on the sports field, so it does not need too many positioning camera devices. This way, it can save costs. At the same time, the second image for the live broadcast can be obtained by shooting the target in the second attitude through the live broadcast channel machine including the augmented reality lens, and the AR picture of the movement trajectory of the target on the second image can be generated in real time during the live broadcast, which can further be used for rendering. Provide technical support for the picture of the motion track, enhance the live broadcast effect, and improve the observability of the game. In the follow-up, it can also render the picture of the motion trajectory, which can enhance the live broadcast effect and improve the observability of the game.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic structural diagram of an embodiment of a display system for a motion trajectory of the present application;

2 is a schematic structural diagram of another embodiment of the display system of the motion trajectory of the present application;

3 is a schematic structural diagram of another embodiment of the display system of the motion trajectory of the present application;

4 is a schematic structural diagram of another embodiment of the display system of the motion trajectory of the present application;

FIG. 5 is a schematic diagram of the competition field of an embodiment of the display system of the motion trajectory of the present application in the Robomaster competition application scenario;

6 is a schematic diagram of an embodiment of a display system applying the motion trajectory of the present application in the competition venue of FIG. 5;

Fig. 7 is the display effect diagram of Fig. 6;

FIG. 8 is a schematic flowchart of an embodiment of a method for displaying a motion trajectory of the present application.

Description of main components and symbols:

100. Display system of motion trajectory;

10. Positioning camera device; 20. Live broadcast channel machine; 30. First server; 31. Positioning server; 32. AR rendering server; 40. Detection server; 50. Clock synchronization protocol clock module; 60. Tracking PTZ; 70 , Director station.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

The flowcharts shown in the figures are for illustration only, and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to the actual situation.

The AR live broadcast of the Robomaser Mecha Master Competition is an integral part of it. In order to improve the observability and entertainment of the live broadcast, it is necessary to render the trajectory special effects of the projectile launched by the robot in real time during the live broadcast, so as to make up for the shortcomings of the inconspicuous ballistic trajectory in the live broadcast screen. To this end, the solutions adopted are vicon, nokov, etc., but vicon and nokov require a large number of cameras (for example, vikon requires at least 24 cameras), and the cost is high; motion compensation for reflective processing of objects to be captured on the market The system is very expensive, and the motion compensation system that cannot perform reflection processing cannot well meet the needs of trajectory special effects.

Embodiments of the present application provide a system, method, and storage medium for displaying motion trajectories. A plurality of positioning camera devices arranged on a sports field in different first postures synchronously photograph a target moving on the sports field to obtain a plurality of corresponding The first image of the shooting time; the live broadcast channel machine including the augmented reality lens shoots the target in the second posture to obtain the second image for live broadcast and carrying the second shooting time; the first server is based on the multiple first images and multiple The first posture corresponding to the first image is used to determine the three-dimensional position of the target on the sports field at each first shooting time, and the movement trajectory of the target on the second image is generated in combination with the second posture corresponding to the second image and the second image. The AR picture is used for displaying on the user's display device. Due to the multiple different first images obtained by synchronously photographing the moving target on the sports field through a plurality of positioning camera devices with different positions and different attitudes, the three-dimensional positions of the target on the sports field at different times can be recovered, which makes more than two positioning cameras The device can cover the common moving path of the target on the sports field by adjusting the arrangement angle and arrangement method to meet the requirements of determining the three-dimensional position of the target on the sports field, so it does not need too many positioning camera devices. This way, it can save costs. At the same time, the second image for the live broadcast can be obtained by shooting the target in the second posture through the live broadcast channel machine including the augmented reality lens, and the AR picture of the movement trajectory of the target on the second image can be generated in real time during the live broadcast, and further follow-up can also be used. Render the picture of the motion track, which can enhance the live broadcast effect and improve the observability of the game.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.

Referring to FIG. 1 , FIG. 1 is a schematic structural diagram of an embodiment of a motion trajectory display system of the present application. The display system 100 includes a plurality of positioning camera devices 10 , a live broadcast channel machine 20 and a first server 30 .

Each of the plurality of positioning camera devices 10 is arranged on the sports field in a different first posture, and is used for synchronously photographing a target and obtaining a first image. The first image carries the first image obtained by shooting. At the first capture time of the image, the target is moving on the sports field.

The live broadcast channel machine 20 includes an augmented reality lens, and the live broadcast channel machine 20 is used to shoot the target in a second attitude and obtain a second image for live broadcast, the second image carries the second image obtained by shooting. The second capture time of the image.

The first server 30 is configured to determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first gestures corresponding to the plurality of first images; According to the second posture corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, generate the image of the target on the second image The AR picture of the motion track, the AR picture is used for displaying on the user's display device.

The motion trajectory display system of this embodiment can be used in a sports venue to live broadcast the moving target to the user and provide technical support for visually displaying the motion trajectory of the target.

Under normal circumstances, the sports field is square, and a plurality of the positioning camera devices 10 can be evenly arranged around the diagonal line of the sports field. In this way, a plurality of the positioning camera devices 10 can be made to cover the common moving path of the target on the sports field as much as possible, so as to meet the requirement of determining the three-dimensional position of the target on the sports field.

Typically, the targets include spherical targets. In an application scenario, the sports field includes a Robomaser game field. The target includes a golf ball.

In this embodiment, a plurality of positioning camera devices 10 are respectively arranged on the sports field in different first postures, and the objects moving on the sports field are synchronously photographed to obtain the first image. Since the first image carries the corresponding first shooting time, Through two or more first images of the same first shooting time, the three-dimensional position of the target on the sports field at the first shooting time can be obtained, and then the target can be obtained at multiple consecutive and different first shooting times. , The three-dimensional position on the sports field, that is, the three-dimensional position of the target on the sports field at different times, at these positions from the user's viewing angle, the movement trajectory of the target moving on the sports field can be obtained.

This embodiment uses the multi-eye positioning technology. The multi-eye positioning technology is based on the principle of different coordinates of 2D images projected by objects in 3D space on multiple camera devices with different positions and different attitudes. By calibrating the camera device attitude and obtaining A technique for recovering the 3D position of an object from the position of the object in the 2D image of different cameras. The simplest is the binocular positioning technology, which only needs two cameras to recover the 3D position of the object. Therefore, in this embodiment, two or more positioning camera devices 10 can meet the requirements of determining the three-dimensional position of the target on the sports field by adjusting the arrangement angle and arrangement to cover the common moving path of the target on the sports field. There are many positioning camera devices 10, in this way, the cost can be saved.

The live broadcast channel machine 20 shoots the target in the second attitude and obtains the second image for live broadcast. Since the second image is used for live broadcast for the user to watch, the second image when the live broadcast channel machine captures the second image. Gesture can be thought of as a user perspective.

Because the live broadcast channel machine 20 includes an augmented reality lens (AR, Augmented Reality), the AR lens has a built-in rotary encoder in the traditional lens, and can obtain the real-time focal length and focus of the current lens in real time, so it can display the real-time movement trajectory in the live broadcast. screen, enhance the live broadcast effect, and improve the observability of the game. The second image carries the second shooting time at which the second image is obtained. Therefore, the target can be projected on the projection plane corresponding to the user's perspective, and the multiple three-dimensional positions of the target at the second shooting time and before the second shooting time can be projected. The projection position of the projection plane corresponding to the user's perspective is fused with the second image to generate an AR picture of the movement track of the target on the second image.

In the embodiment of the present application, multiple positioning camera devices arranged on the sports field in different first postures synchronously shoot a target moving on the sports field to obtain a plurality of corresponding first images with the first shooting time; live channels including augmented reality lenses The camera captures the target in the second posture to obtain a second image for live broadcast and carries the second shooting time; the first server determines each first image according to the first postures corresponding to the plurality of first images and the plurality of first images The three-dimensional position of the target on the sports field at the time of shooting is combined with the second posture corresponding to the second image and the second image to generate an AR picture of the movement trajectory of the target on the second image, and the AR picture is used for displaying on the user's display device. Show on. Due to the multiple different first images obtained by synchronously photographing the moving target on the sports field through a plurality of positioning camera devices with different positions and different attitudes, the three-dimensional positions of the target on the sports field at different times can be recovered, which makes more than two positioning cameras The device can cover the common moving path of the target on the sports field by adjusting the arrangement angle and arrangement method to meet the requirements of determining the three-dimensional position of the target on the sports field, so it does not need too many positioning camera devices. This way, it can save costs. At the same time, the second image for the live broadcast can be obtained by shooting the target in the second posture through the live broadcast channel machine including the augmented reality lens, and the AR picture of the movement trajectory of the target on the second image can be generated in real time during the live broadcast, and further follow-up can also be used. Render the picture of the motion track, which can enhance the live broadcast effect and improve the observability of the game.

In one embodiment, in order to further clarify the division of labor, improve processing speed, and reduce delay, the first server 30 is divided into two different servers, and the functions of the first server 30 are implemented by the two different servers. That is, the first server 30 includes: a positioning server 31 and an AR rendering server 32, as shown in FIG. 2 .

The positioning server 31 is configured to determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first gestures corresponding to the plurality of first images.

The AR rendering server 32 is configured to generate, according to the second pose corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, the The AR picture of the motion trajectory on the second image is used for displaying on the user's display device.

In this embodiment, the positioning server 31 is connected to the positioning camera device 10, obtains a plurality of the first images captured by the positioning camera device 10 and the first postures corresponding to the first images, and then determines each The three-dimensional position of the target on the sports field at the first shooting time.

In this embodiment, the AR rendering server 32 is connected to the positioning server 31, and is connected to the live broadcast channel machine 20, and the positioning server 31 sends the determined three-dimensional position of the target on the sports field at each of the first shooting times to The AR rendering server 32, the AR rendering server 32 also obtains the second image and the second posture corresponding to the second image from the live broadcast channel machine 20, and then according to the second posture corresponding to the second image, each of the first The three-dimensional position of the target on the sports field and the second image at a shooting time generate an AR picture of the movement track of the target on the second image.

In one embodiment, the AR rendering server 32 is further configured to determine the current projection plane according to the second posture corresponding to the second image obtained by the live broadcast channel machine 20; according to the current projection plane and each each of the three-dimensional positions of the target on the sports field at the first shooting time, and determine each projection position of each of the three-dimensional positions on the current projection plane; The two images are fused and rendered to generate an AR picture of the movement track of the target on the second image.

In an embodiment, in order to further improve the speed of determining the three-dimensional position of the target and further reduce the delay, a server is also added to detect the approximate position of the target in the first image in advance, so as to facilitate the subsequent rapid determination of the three-dimensional position of the target. Location. That is, the system 100 further includes: the detection server 40 , as shown in FIG. 3 .

The detection server 40 is configured to detect the first image, and extract the candidate location regions of the target in the first image; at this time, the positioning server 31 is also configured to locate a plurality of first locations according to the target. The three-dimensional position of the target on the sports field at each of the first shooting times is determined from the plurality of candidate position regions in the image and the first postures corresponding to the plurality of the first images.

In this embodiment, the detection server 40 is connected to the positioning camera device 10, acquires the first image captured by the positioning camera device 10, detects the first image, and extracts the candidate position area of the target in the first image . Since the approximate position of the target in the first image is pre-detected by the detection server 40, the positioning server 31 determines the position of the target according to the plurality of candidate position regions in the plurality of first images and the plurality of first images. The corresponding first posture is to determine the three-dimensional position of the target on the sports field at each of the first shooting times. On the one hand, the speed at which the positioning server 31 determines the three-dimensional position of the target on the sports field can be improved, and the subsequent time can be reduced. On the other hand, interference caused by non-targets can be effectively eliminated in advance, thereby further improving the determination speed of the positioning server 31 and reducing the time delay.

In order to further improve the speed of determining the three-dimensional position of the positioning server, the detection server 40 makes the approximate position of the target in the first image more detailed. That is, the detection server 40 is further configured to detect the first image, and extract the center position of the target in the candidate position area in the first image; at this time, the positioning server is also configured to According to the plurality of center positions of the candidate position regions in the plurality of first images and the first postures corresponding to the plurality of first images, it is determined that the target is on the sports field at each of the first shooting times three-dimensional position.

In an embodiment, if the number of positioning camera devices 10 is relatively large, a plurality of detection servers 40 may be arranged correspondingly to ensure the detection speed. That is, a plurality of the positioning camera devices 10 are divided into groups, the number of the detection servers 40 is equal to the number of groups, and each of the detection servers 40 detects the first images of the positioning camera devices 10 of the corresponding group.

Wherein, when the number of the positioning camera devices 10 is four or more, the number of the detection servers 40 is two or more. Generally speaking, under the condition of taking into account the time delay and cost, one detection server 40 can detect the first images of the three positioning camera devices 10 correspondingly.

Wherein, taking into account cost, time delay and visibility, the number of the positioning camera 10 is six, the number of the detection servers 40 is two, and each of the detection servers 40 detects three of the The first image of the camera 10 is positioned. For example, in the actual application of a Robomaser mecha master's playing field, 6 ordinary industrial cameras can be used, and by adjusting the arrangement angle and arrangement method to cover the lob shot path commonly used in sports field competitions, the track display of golf balls can be completed.

Referring to FIG. 4 , in an embodiment, the system 100 further includes: a clock synchronization protocol clock module 50 . The clock synchronization protocol clock module 50 is used to control a plurality of the positioning camera devices to complete time synchronization through the network clock synchronization protocol PTP timing. In this way, it is possible to ensure that a plurality of the positioning camera devices are photographed synchronously as much as possible, thereby increasing the accuracy of the positioning target. In one application, the clock synchronization protocol clock module 50 may be integrated into the detection server 40 .

In one embodiment, the system 100 further includes: a tracking pan/tilt 60 . The tracking pan/tilt 60 is used to track and acquire the second posture of the live channel machine 20 and the time corresponding to the acquisition of the second posture, and send the second posture and the time corresponding to the acquisition to the second posture. the first server 30 .

The tracking pan/tilt 60 is installed with a high-precision rotary encoder on the two rotating shafts of the conventional general channel pan/tilt, which can obtain the rotation angles of the two axes of the pan/tilt Yaw and Pich at the same time, and the rotation resolution of the pan/tilt is different. Affected by outside temperature and attitude. Through the readings of the tracking platform 60, the posture of the camera mounted on it relative to the ground can be obtained.

It should be noted that the posture of the live broadcast channel machine 20 can be obtained by not only using the tracking pan/tilt 60, but also by setting markers outside the live broadcast channel machine 20 and using an external camera to solve the channel machine using monocular PnP. How to get the attitude.

In one embodiment, the system 100 further includes: a director station 70 . The director station 70 is used to display the AR picture in real time.

The system of the embodiment of the present application is described in detail by taking the specific application of the system of the present embodiment in the Robomaster competition as an example.

Referring to Figure 5, Figure 5 shows the Robomaster2020 competition venue, in which the center on the left and the center on the right are the base robots of both sides, and the areas on the four corners are the selected lob shooting locations commonly used in the competition. The hero robot (one of the participating robots) can launch a 42mm projectile along the trajectory pointed to by the opponent's base robot at the hoisting location to complete the strike against the enemy's base robot across the half court. The projectile is a green glowing golf ball.

In the hitting process, in order to broadcast the effect, it is necessary to complete the display of the entire golf ball flight process. In the previous broadcast scheme, the trajectory and landing point of the golf ball could not be clearly seen, and the audience could not intuitively feel the success of the golf lob, and the presentation effect was greatly reduced. To this end, the system of the embodiment of the present application is used to realize the broadcast solution of AR track display.

As shown in Figure 6, several positioning camera devices (such as positioning cameras) with different postures are arranged around the competition field to detect the position of the golf ball on the field. For processing the first image of each positioning camera, a golf ball image candidate region in the first image captured by the positioning camera is extracted. Each detection server completes time synchronization through network PTP timing, and sends the location of the golf ball image in the currently processed first image to the positioning server. The positioning server summarizes the positions of all golf ball images sent by the detection server, and completes the 3D position recovery of the golf ball in combination with the first posture of the corresponding positioning camera. The positioning server time stamps the 3D position information of the golf ball and sends it to the AR rendering server. The AR rendering server directly receives the second image carrying the time code from the live channel machine, tracks the second posture of the gimbal carrying the time code, and the 3D position information of the golf ball from the positioning server, and completes the information alignment through the time code. The projection transformation of the 3D coordinates of the golf ball on the 2D coordinates of the live broadcast channel machine is completed according to the second posture fed back by the tracking gimbal, and the rendering of the golf ball trajectory is completed.

The display effect after fusion rendering is shown in Fig. 7, the effect is that the live channel machine collects the second image, and fuses the trajectory of the golf ball on the second image and superimposes it. On the left side of the figure, the trajectory line from right to left is the trajectory effect of the golf ball. It can be seen that the trajectory of the golf ball has not changed compared with the position of the field as the camera shakes, and the flight trajectory of the golf ball can be clearly seen.

The detection equipment of golf balls is based on the basic principle of multi-eye ranging. The workflow of each detection server includes: collecting first images obtained by different positioning cameras, performing frame difference algorithm to obtain moving objects in the positioning cameras; performing morphological filtering and screening on the first images to obtain candidates that may be golf balls Region; fit the center point of the region and send it with timestamp encoding. The positioning server receives the candidate regions of the golf ball sent from different detection servers, and completes the matching and exclusion of the candidate regions of the golf ball on each image in combination with the first posture of each positioning camera. Location information, fetched and timestamped and sent to the AR rendering server.

The AR rendering server completes the rendering of the live image by aligning the second image, the second posture corresponding to the second image, and the 3D position of the golf ball, and outputs the result to the director station to complete the rendering output.

Due to the use of AR projectile trajectory rendering technology, it can intuitively display the trajectory of the lob, and improve the sense of technology and substitution of the game. At the same time, because the projectile trajectory can be restored, the rendering of the projectile's "First Person View" (FPV, First Person View) effect can be completed during the tactical review process, and the 360-degree surround view effect map of simulating the projectile launch to hit can improve the viewing experience of the game. and fun.

Referring to FIG. 8, FIG. 8 is a schematic flowchart of an embodiment of a method for displaying a motion trajectory of the present application. Any of the above systems can implement the method of this embodiment. For a detailed description of the relevant content, please refer to the above-mentioned motion trajectory display system content section, I won't go into details here.

The method includes: step S101, step S102 and step S103.

Step S101 : synchronously photographing a target through a plurality of positioning camera devices to obtain a plurality of corresponding first images, the first images carry the first shooting time of the first image obtained by shooting, and the plurality of positioning camera devices are in the form of The different first poses are arranged on the playing field on which the target moves.

Step S102: The target is photographed in a second attitude by the live broadcast channel machine and a second image for live broadcast is obtained, the second image carries the second shooting time when the second image is obtained by shooting, and the live broadcast information is obtained. The Dao machine includes augmented reality footage.

It should be noted that, step S101 and step S102 have no sequence relationship.

Step S103: Determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images; The second pose corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, generate the motion trajectory of the target on the second image. AR picture, the AR picture is used for displaying on the user's display device.

Wherein, the method further includes: detecting the first image, and extracting candidate position regions of the target in the first image; Corresponding to the first posture, determining the three-dimensional position of the target on the sports field at each of the first shooting times includes: according to a plurality of candidate position areas of the target in the plurality of first images and The first poses corresponding to the plurality of first images determine the three-dimensional position of the target on the sports field at each of the first shooting times.

Wherein, the detecting the first image and extracting the candidate position area of the target in the first image includes: detecting the first image, and extracting the target in the first image the center position of the candidate position area; determining each of the first positions according to the plurality of candidate position areas of the target in the plurality of first images and the corresponding first poses of the plurality of first images The three-dimensional position of the target on the sports field at the shooting time includes: according to the plurality of center positions of the candidate position regions of the target in the plurality of first images and the first images corresponding to the plurality of first images. a gesture, determining the three-dimensional position of the target on the sports field at each of the first shooting times.

Wherein, according to the second posture corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, generating the The AR picture of the motion trajectory on the two images includes: determining the current projection plane according to the second posture corresponding to the second image; The three-dimensional position on the sports field, determine each projection position of each of the three-dimensional positions on the current projection plane; fuse and render each of the projected positions and the second image to generate the target on the first The AR screen of the motion trajectory on the two images.

Wherein, before the synchronizing the shooting of the target through the multiple positioning camera devices and obtaining the corresponding multiple first images, the method further includes: controlling the multiple positioning camera devices to complete the time synchronization through the network clock synchronization protocol PTP timing.

Wherein, the method further includes: tracking and acquiring the second posture of the live broadcast channel machine and acquiring the time corresponding to the second posture by using the tracking pan/tilt.

Wherein, the method further includes: displaying the AR picture in real time through a broadcasting station.

The present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to display the motion trajectory described in any one of the above method. For a detailed description of the relevant content, please refer to the above-mentioned relevant content section, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit, such as a hard disk or a memory, of the relevant components in the above-mentioned motion trajectory display system. The computer-readable storage medium may also be an external storage device, such as an equipped plug-in hard disk, smart memory card, secure digital card, flash memory card, and the like.

It should be understood that the terms used in the specification of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A system for displaying motion trajectory, characterized in that the system comprises:

A plurality of positioning camera devices, each of which is arranged on the sports field in a different first posture, for synchronously photographing a target and obtaining a first image, the first image carrying a camera to obtain the first image. At the first shooting time, the target moves on the sports field;

A live broadcast channel machine, including an augmented reality lens, the live broadcast channel machine is used to shoot the target in a second attitude and obtain a second image for live broadcast, the second image carries the second image obtained by shooting the second shooting time;

a first server, configured to determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images; According to the second posture corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, generate the image of the target on the second image The AR picture of the motion track, the AR picture is used for displaying on the user's display device.
The system of claim 1, wherein the first server comprises:

a positioning server, configured to determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images;

The AR rendering server is configured to generate, according to the second pose corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image, the The AR picture of the motion trajectory on the second image is used for displaying on the user's display device.
The system of claim 2, wherein the system further comprises:

a detection server, configured to detect the first image and extract the candidate location area of the target in the first image;

The positioning server is further configured to determine each of the first shots according to a plurality of candidate position regions of the target in the plurality of first images and a first posture corresponding to the plurality of first images time the three-dimensional position of the target on the playing field.
The system of claim 3, wherein:

The detection server is also used to detect the first image, and extract the center position of the candidate position area of the target in the first image;

The positioning server is further configured to, according to multiple center positions of the candidate position regions of the target in the multiple first images and the first postures corresponding to the multiple first images, determine each of the The three-dimensional position of the target on the sports field at the first shooting time.
The system according to claim 3, wherein a plurality of the positioning camera devices are divided into groups, the number of the detection servers is equal to the number of groups, and each detection server detects the first positioning camera device of the corresponding group. image.
The system according to claim 5, wherein when the number of the positioning camera devices is four or more, the number of the detection servers is two or more.
The system according to claim 5, wherein the number of the positioning camera devices is six, the number of the detection servers is two, and each detection server detects the number of the three positioning camera devices. an image.
The system of claim 2, wherein:

The AR rendering server is further configured to determine the current projection plane according to the second posture corresponding to the second image obtained by the live broadcast channel machine; according to the current projection plane and each of the first shooting times the three-dimensional position of the target on the sports field, determine each projection position of each of the three-dimensional positions on the current projection plane; fuse and render each of the projected positions and the second image to generate the The AR picture of the movement track of the target on the second image.
The system of claim 1, wherein the system further comprises:

The clock synchronization protocol clock module is used to control a plurality of the positioning camera devices to complete time synchronization through the network clock synchronization protocol PTP timing.
The system of claim 1, wherein the system further comprises:

A tracking pan/tilt is used to track and acquire the second posture of the live broadcast channel machine and the time corresponding to the acquisition of the second posture, and send the second posture and the time corresponding to the acquisition of the second posture to the the first server.
The system of claim 1, wherein the system further comprises:

The director station is used to display the AR picture in real time.
The system according to claim 1, wherein the sports field is square, and a plurality of the positioning camera devices are evenly arranged around a diagonal line of the sports field.
The system of claim 1, wherein the target comprises a spherical target.
The system of claim 1, wherein the sports field comprises a Robomaser's game field.
15. The system of claim 14, wherein the target comprises a golf ball.
A method for displaying motion trajectory, characterized in that the method comprises:

A plurality of positioning camera devices are used to simultaneously capture a target and obtain a plurality of corresponding first images. The first images carry the first shooting time when the first image is captured, and the plurality of positioning camera devices use different first images. A gesture is arranged on the sports field on which the target moves;

The target is photographed in a second attitude by a live broadcast channel machine, and a second image for live broadcast is obtained, the second image carries the second shooting time for obtaining the second image, and the live broadcast channel machine includes: augmented reality footage;

Determine the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images; according to the second images the corresponding second posture, the three-dimensional position of the target on the sports field at each of the first shooting times, and the second image to generate an AR picture of the movement track of the target on the second image, The AR picture is used for displaying on the user's display device.
The method of claim 16, wherein the method further comprises:

Detecting the first image, and extracting candidate location regions of the target in the first image;

The determining the three-dimensional position of the target on the sports field at each of the first shooting times according to the plurality of first images and the first postures corresponding to the plurality of first images includes:

According to a plurality of candidate position areas of the target in the plurality of first images and the corresponding first poses of the plurality of first images, it is determined that the target is in the sports field at each of the first shooting times 3D position on the ground.
The method according to claim 17, wherein the detecting the first image and extracting the candidate position area of the target in the first image comprises:

Detecting the first image, and extracting the center position of the target in the candidate position area of the first image;

determining the location of the target at each of the first shooting times according to the plurality of candidate position regions of the target in the plurality of first images and the corresponding first postures of the plurality of first images; three-dimensional positions on the sports field described above, including:

According to a plurality of center positions of the candidate position regions of the target in the plurality of first images and the corresponding first postures of the plurality of first images, it is determined that the target is at each of the first shooting times three-dimensional position on the sports field.
The method according to claim 16, wherein the second pose corresponding to the second image, the three-dimensional position of the target on the sports field at each of the first shooting times, and the first Two images, generating an AR picture of the movement trajectory of the target on the second image, including:

determining the current projection plane according to the second posture corresponding to the second image;

According to the current projection plane and the three-dimensional position of the target on the sports field at each of the first shooting times, determine each projection position of each of the three-dimensional positions on the current projection plane;

Each of the projection positions and the second image are fused and rendered to generate an AR picture of the movement track of the target on the second image.
The method according to claim 16, wherein before the synchronously photographing the target through multiple positioning camera devices and obtaining the corresponding multiple first images, the method further comprises:

Control a plurality of the positioning camera devices to complete time synchronization through the network clock synchronization protocol PTP timing.
The method of claim 16, wherein the method further comprises:

The second posture of the live broadcast channel machine and the time corresponding to the second posture are obtained by tracking and acquiring the tracking PTZ.
The method of claim 16, wherein the method further comprises:

The AR picture is displayed in real time through the director station.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the process according to any one of claims 16-22 The display method of the motion trajectory.