CN115100344B - XR space positioning method and device, computer equipment and storage medium - Google Patents

XR space positioning method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN115100344B
CN115100344B CN202211014438.6A CN202211014438A CN115100344B CN 115100344 B CN115100344 B CN 115100344B CN 202211014438 A CN202211014438 A CN 202211014438A CN 115100344 B CN115100344 B CN 115100344B
Authority
CN
China
Prior art keywords
image
server
camera
display
frequency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211014438.6A
Other languages
Chinese (zh)
Other versions
CN115100344A (en
Inventor
殷元江
刘耀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing 7d Vision Technology Co ltd
Original Assignee
Beijing 7d Vision Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing 7d Vision Technology Co ltd filed Critical Beijing 7d Vision Technology Co ltd
Priority to CN202211014438.6A priority Critical patent/CN115100344B/en
Publication of CN115100344A publication Critical patent/CN115100344A/en
Application granted granted Critical
Publication of CN115100344B publication Critical patent/CN115100344B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The application relates to an XR space positioning method, an XR space positioning device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring first image data which is acquired by a camera and displayed by a display screen, wherein the first image data comprises a fusion image, the fusion image is a rendered image and a marker image which are displayed by inserting frames, the marker image is generated by a second server, and the fusion image is formed by fusing the second server; extracting a marker image from the first image data at a frequency of the frame interpolation display; calculating a real three-dimensional space position parameter of the camera according to the marked image based on an image recognition algorithm; and sending the real three-dimensional space position parameters of the camera to a three-dimensional rendering server to be used as tracking data input of the rendered image. Compared with traditional expensive hardware equipment, the method has the advantages that the tracking and the positioning of the spatial position are realized by adopting software, the cost is low, and the popularization and the application of the XR technology are facilitated.

Description

XR space positioning method and device, computer equipment and storage medium
Technical Field
The present application relates to the field of XR technologies, and in particular, to an XR spatial positioning method, apparatus, computer device, and readable storage medium.
Background
XR (Extended reality) refers to a real and virtual combined, human-machine interactive environment created by computer technology and wearable devices. XR technology can be viewed as a covering term encompassing VR (virtual reality), AR (augmented reality), MR (mixed reality), and other new immersive technologies that may emerge as technology advances. XR is a general term for immersive virtual and reality fusion technologies, and different technical branches such as VR, AR and MR are deduced from XR. VR enables people to be completely immersed in a virtual environment; the AR can create a world of overlaid virtual content, but cannot interact with the real environment; MR is a hybrid of virtual and real, creating virtual objects that can interact with real environments. XR has particular implications in the production of film, radio and live entertainment, and has become a mainstream in the virtual presentation industry. Compared with a green screen, the XR system has a more real shooting effect, and an infinite space can be formed in a limited space by the XR technology.
The XR realizes the logic operation, the graphic rendering and the like of the localized XR application display, the position tracking and positioning, multiple operation modes and the localized operation through various sensors, cameras, display screens and other components. In the XR system, the infrared positioning technology is usually adopted to realize position tracking and positioning, the infrared positioning technology is used for calculating the position of the equipment through the infrared camera, the camera needs the support of the space positioning and tracking system, however, the purchase of a set of space positioning system of the infrared camera has a selling price of dozens of ten thousand yuan, if the application range is enlarged, the cost can reach millions of yuan, and the pressure is great for the user, therefore, the position tracking and positioning cost in the XR is higher, and the popularization and the application of the XR technology are not facilitated.
Disclosure of Invention
In view of the foregoing, there is a need to provide an XR space positioning method, apparatus, computer device and readable storage medium with low position tracking and positioning cost.
In a first aspect, the present application provides an XR spatial localization method applied to a first server, the method including:
acquiring first image data which is acquired by a camera and displayed by a display screen, wherein the first image data comprises a fusion image, the fusion image is a rendered image and a marker image which are displayed by inserting frames, the marker image is generated by a second server, and the fusion image is formed by fusing the second server;
extracting the marker image from the first image data according to the frequency of the frame interpolation display;
calculating real three-dimensional space position parameters of the camera according to the marked image based on an image recognition algorithm;
and sending the real three-dimensional space position parameter of the camera to a three-dimensional rendering server to be used as the tracking data input of the rendered image.
In one embodiment, the method further comprises:
and acquiring a reference clock synchronization signal to synchronize the first server with the reference clock of the camera and the reference clock of the second server.
In one embodiment, the extracting the marker image from the first image data according to the frequency of the frame insertion display includes:
acquiring the frequency of the inter-frame display, which is the frequency of displaying the marker image on the display screen set by the second server;
and separating the fused image according to the frequency of the frame interpolation display, and extracting the marked image.
In a second aspect, the present application provides an XR spatial localization method applied to a second server, the method including:
receiving a rendering image, wherein the rendering image is an image rendered by a three-dimensional rendering server;
generating a marker image;
fusing the rendered image and the marked image into a fused image in a frame insertion display mode, outputting the fused image to a display screen for display, acquiring the fused image by a camera to form first image data, and inputting the first image data into a first server;
and recording the display frequency of the interpolation frame, inputting the frequency to the first server, extracting a marked image from the first image data through the first server according to the display frequency of the interpolation frame in a separating manner, wherein the marked image is used for calculating the real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is used for being sent to the three-dimensional rendering server to be used as the tracking data of the rendered image.
In one embodiment, the method further comprises:
and acquiring a reference clock synchronization signal to synchronize the second server with the reference clock of the camera and the reference clock of the first server.
In a third aspect, the present application provides an XR spatial locator device applied to a first server, the XR spatial locator device including:
the system comprises an acquisition module, a display module and a processing module, wherein the acquisition module is used for acquiring first image data which is acquired by a camera and displayed by a display screen, the first image data comprises a fusion image, the fusion image is a rendered image and a marker image which are displayed by inserting frames, the marker image is generated by a second server, and the fusion image is formed by fusing the second server;
an extraction module, configured to extract the marker image from the first image data according to the frequency of the frame insertion display;
the calculation module is used for calculating the real three-dimensional space position parameter of the camera according to the marked image based on an image recognition algorithm; and
and the data output module is used for sending the real three-dimensional space position parameters of the camera to a three-dimensional rendering server for input and use as the tracking data of the rendered image.
In a fourth aspect, the present application provides an XR spatial locator device for use with a second server, the device comprising:
the receiving module is used for receiving a rendering image, and the rendering image is an image rendered by a three-dimensional rendering server;
a generation module for generating a marker image;
the fusion module is used for fusing the rendered image and the marked image into a fused image in a frame insertion display mode, outputting the fused image to a display screen for display, acquiring the fused image by a camera to form first image data, and inputting the first image data into a first server; and with
And the recording module is used for recording the display frequency of the interpolation frame and inputting the frequency to the first server, the first image data is separated and extracted into a marked image through the first server according to the display frequency of the interpolation frame, the marked image is used for calculating a real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is used for being sent to the three-dimensional rendering server and used as tracking data of a rendered image to be input.
In a fifth aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method according to any one of the above when executing the computer program.
In a sixth aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to implement the steps of any one of the methods described above when executed by a processor.
In a seventh aspect, the present application provides a computer program product comprising a computer program, characterized in that the computer program is adapted to perform the steps of any of the methods described above when executed by a processor.
According to the XR space positioning method, the XR space positioning device, the computer equipment and the readable storage medium, the first image data which is acquired by the camera and is displayed by the display screen and contains the rendering image and the marking image of the insertion frame display is obtained, the marking image is extracted from the first image data according to the frequency of the insertion frame display, and the real three-dimensional space position parameter of the camera is calculated according to the marking image, so that the position of the camera is determined, and the positioning of the position of the camera is realized. Meanwhile, the real three-dimensional space position parameters of the camera are used as tracking data of a rendered image and input into a three-dimensional rendering server, so that space tracking and positioning are achieved. Compared with traditional expensive hardware equipment, the method has the advantages that the tracking and the positioning of the spatial position are realized by adopting software, the cost is lower, and the popularization and the application of the XR technology are facilitated.
Drawings
FIG. 1 is a diagram of an exemplary XR spatial localization method;
FIG. 2 is a schematic flow chart diagram of an XR spatial localization method according to one embodiment;
FIG. 3 is a detailed flow chart of the extraction of the marker image in FIG. 2;
FIG. 4 is a schematic flow chart diagram illustrating an XR spatial localization method according to another embodiment;
FIG. 5 is a schematic view of an XR spatial positioning device, according to one embodiment;
FIG. 6 is a schematic view of another embodiment of an XR spatial positioning device;
FIG. 7 is an internal block diagram of a computer device of an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
SDI: the SDI interface is a kind of "digital component serial interface", and the HD-SDI interface is a broadcasting-grade high-definition digital input and output port, where HD denotes a high-definition signal. Because the SDI interface cannot directly transmit the compressed digital signal, the compressed signal recorded by the digital video recorder, the hard disk, and other devices must be decompressed and output through the SDI interface before entering the SDI system. If decompression and compression are repeated, image quality is necessarily reduced and time delay is necessarily increased, so digital video recorders and nonlinear editing systems in various formats have been provided with interfaces for directly transmitting compressed digital signals.
Hz: hertz is the unit of frequency in the international system of units, which is a measure of the number of repetitions of a periodic variation per second. Hertz is simply known as Hertz. The oscillations (or oscillations, undulations) are 1 hz once per second or can be written as times per second, cycles per second. Named after hertz, german scientist.
HDMI: a High Definition Multimedia Interface (HDMI) is a fully digital video and audio transmission Interface, which can transmit uncompressed audio and video signals. The HDMI can be used for set-top boxes, DVD players, personal computers, televisions, game hosts, comprehensive amplifiers, digital stereos, televisions and other equipment. HDMI can send audio frequency and video signal simultaneously, because audio frequency and video signal adopt same wire rod, simplifies the installation degree of difficulty of system's circuit greatly.
PTP: PTP (Precision Time Protocol) is a Protocol for synchronizing clocks through a network. When the hardware supports, the precision of PTP can reach sub-microsecond, which is higher than the precision of NTP (Network Time Protocol).
The ArUco mark is a composite square mark consisting of a wide black border and an internal binary matrix that determines its identifier (id). The black border facilitates its fast detection in the image and the binary coding allows its recognition and application of error detection and correction techniques. The size of the tag determines the size of the internal matrix. For example, the mark size 4 × 4 is composed of 16 bits.
BNC interface, bayonet Nut Connector. The BNC interface is a 10Base2 connector, namely a coaxial thin cable connector. The video input signal can be isolated, the mutual interference of the signals is reduced, the signal bandwidth is larger than that of a common 15-pin D-type interface, and a better signal response effect can be achieved.
The XR spatial localization method provided by the embodiment of the present application may be applied to the application environment shown in fig. 1. The camera 110 is connected with a first server 120, the PTP server 140, and the time service board 150 communicate with an image processor through a network switch 130, and two notebook computers are connected with the image processor, wherein one notebook computer is used as a three-dimensional rendering server 170 for outputting a rendered image, the other notebook computer is used for outputting an ArUco marker image and forms a second server 160 with the image processor, the first server 120 is a graphic calculation processing unit, and the first server 120 outputs the image to a content recording device 190. The image outputted from the image processor is displayed on the display screen 180, and the camera 110 captures the image at the display screen 180 and is coaxially connected to the display screen 180 through the BNC.
Illustratively, a notebook computer serving as the three-dimensional rendering server 170 outputs 60Hz rendered image content, the resolution is 1920 × 1080, a computer screen supports a maximum refresh rate of 240Hz, signals of two computers are connected to an image processor through HDMI interfaces, the refresh rate of a display screen can be as high as 2840Hz, a video camera for shooting is sony FS7, the shutter is adjusted to 120/1 second, the recording format is adjusted to 120 frames per second, the video camera and the image processor are connected by a BNC coaxial line, signals are transmitted by using a 12G-SDI mode, and four HDMI ports are arranged at the rear end of the image processor and can output four independent image signals. The method comprises the steps of opening two notebooks, opening an image processor, connecting all devices by using network cables, initializing clock time of a synchronous system by using PTP, setting interface time sequence setting by opening a frequency controller, setting total output frequency to be 120Hz, setting image output frequency to be 60Hz, displaying image ghosting of the two notebooks on a display screen after setting is finished, setting time reference of a rendering server and the display screen to be the same, setting time sequence delay difference value of two output ports to be 8.33 milliseconds, displaying a marker image in 8.33 milliseconds after setting is finished, displaying a rendering image at interval of 8.33 milliseconds, alternately displaying the images, shooting a fusion image displayed on the display screen by using a camera, setting a shutter of the camera to be 120/1 second, adjusting a recording format to be 120 frames per second, directly outputting a camera signal to a graph calculation processing unit, and finding two independent image signals by watching the marker image and the rendering image output by the graph calculation processing unit at the moment, wherein the image refreshing rate of the two independent image signals is 60Hz. The marked image is output to the image calculation processing unit, the current position of the camera is calculated through a general image recognition algorithm, the rendered image is output to the content recording equipment for recording and using the shot content, and the purpose of realizing space tracking positioning and outputting and using the displayed content at the same time is achieved.
In one embodiment, as shown in fig. 2, an XR spatial localization method is applied to a first server, and includes the following steps:
step S210, acquiring first image data displayed by a display screen and collected by a camera, where the first image data includes a fusion image, the fusion image is a rendered image and a tagged image displayed by an insertion frame, the tagged image is generated by a second server, and the fusion image is formed by fusing the second server.
Specifically, the three-dimensional rendering server outputs the rendering image to the second server in real time, the second server generates a self-defined marking image, the rendering image and the marking image are fused into a fusion image according to a frame insertion display mode and are output to a display screen for display, and meanwhile, the second server records the frame insertion display frequency. The camera shoots an image at the position of the display screen, a fused image is collected through a BNC coaxial line, and the image at the position of the display screen shot by the camera and the fused image form first image data. The camera inputs the first image data into the first server. The marker image is an ArUco marker image.
Step S220 extracts a marker image from the first image data according to the frequency of the frame interpolation display.
Specifically, the first server separates the first image data, extracts the marker image according to the frequency of the frame insertion display, and then analyzes the marker image.
And step S230, calculating the real three-dimensional space position parameter of the camera according to the marked image based on an image recognition algorithm.
Specifically, after the first server completes image separation, the spatial position of the camera is calculated according to the extracted marked image, and the real three-dimensional spatial position parameter of the camera is obtained through an image recognition algorithm. The method comprises the steps of adopting the existing aroco identification scheme, utilizing a correlation function in an aroco library to realize identification of an aroco code under a camera and inserting a video in a designated place, obtaining information such as the distance and the angle of an Aruco code plane relative to the camera, obtaining the coordinate of the center of the Aruco code in the space, and further determining the real three-dimensional space position parameter of the camera.
Step S240, sending the real three-dimensional spatial position parameter of the camera to the three-dimensional rendering server to be input as the tracking data of the rendered image.
Specifically, before the marked image is used for calculating the real three-dimensional space position parameters, the camera lens used needs to be subjected to internal reference calibration operation, a general Opencv algorithm is used, the USB camera is turned on by OpenCV, the camera parameter adjustment is realized, and the normal use process is started after the camera lens calibration is completed.
According to the XR space positioning method, the first image data which is acquired by the camera and is displayed by the display screen and contains the rendering image and the marking image displayed by the interpolation frame is obtained, the marking image is extracted from the first image data according to the frequency of the interpolation frame display, and the real three-dimensional space position parameter of the camera is calculated according to the marking image, so that the position of the camera is determined, and the positioning of the position of the camera is realized. Meanwhile, the real three-dimensional space position parameters of the camera are used as tracking data of a rendered image and input into a three-dimensional rendering server, so that space tracking and positioning are achieved. Compared with traditional expensive hardware equipment, the method has the advantages that the tracking and positioning of the spatial position are realized by adopting software, the cost is low, and the popularization and the application of the XR technology are facilitated. According to the method and the device, the expense for purchasing an external third-party space positioning and tracking system is saved, the system construction cost and the use difficulty are greatly reduced, and the emerging technology can be rapidly applied and served to various industries.
In this embodiment, the XR spatial localization method further includes:
and acquiring a reference clock synchronization signal to synchronize the first server with the reference clock of the camera and the reference clock of the second server.
Specifically, a PTP server is used for ensuring the synchronization of reference clocks of the first server, the second server and the camera, and a time service board card is used for carrying out time service. The fused image displayed by the display screen is output to the camera through the BNC coaxial line, in order to accurately and synchronously acquire image signals of the display screen, the time synchronization of the PTP server needs to be received in the camera, the first server is responsible for acquiring the image output by the camera, and the first server can acquire a synchronous reference clock as time reference according to the PTP server.
As shown in fig. 3, in the present embodiment, the step of extracting the marker image from the first image data at the frequency of the frame insertion display includes the steps of:
in step S224, a frequency of the inter-frame display is acquired, which is a frequency of displaying the marker image on the display screen set by the second server.
Step S226, the fused image is separated according to the frequency of frame interpolation display, and a marker image is extracted.
In one embodiment, as shown in fig. 4, an XR spatial localization method applied to a second server includes the following steps:
step S410, receiving a rendering image, wherein the rendering image is an image rendered by a three-dimensional rendering server.
In step S420, a marker image is generated. The marker image is an ArUco marker image.
And step S430, fusing the rendered image and the marked image into a fused image according to a frame insertion display mode, outputting the fused image to a display screen for displaying, acquiring the fused image by a camera to form first image data, and inputting the first image data into a first server.
Specifically, the three-dimensional rendering server outputs the rendering image to the second server in real time, the second server generates a self-defined marking image, the rendering image and the marking image are fused into a fusion image according to a frame insertion display mode and output to a display screen for display, and meanwhile, the second server records the frame insertion display frequency. The camera shoots an image at the position of the display screen, a fused image is collected through a BNC coaxial line, and the image at the position of the display screen shot by the camera and the fused image form first image data. The camera inputs the first image data into the first server.
Step S440, recording the frequency of the frame insertion display, inputting the frequency to a first server, separating and extracting a marked image from the first image data according to the frequency of the frame insertion display through the first server, wherein the marked image is used for calculating a real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is sent to a three-dimensional rendering server to be used as the tracking data input of the rendered image.
Specifically, the first server separates the first image data, extracts the marker image according to the frequency of the frame insertion display, and then analyzes the marker image. And after the first server finishes image separation, calculating the spatial position of the camera according to the extracted marked image, and obtaining the real three-dimensional spatial position parameter of the camera through an image recognition algorithm. Before the marked image is used for calculating the real three-dimensional space position parameters, internal reference calibration operation needs to be carried out on the used camera lens, a general Opencv algorithm is used, and the normal use process is started after the lens calibration is finished.
In this embodiment, the XR spatial localization method further includes:
and acquiring a reference clock synchronization signal to synchronize the reference clock of the first server with the reference clock of the camera.
Specifically, a PTP server is used for ensuring the synchronization of reference clocks of the first server, the second server and the camera, and a time service board card is used for time service. The fused image displayed by the display screen is output to the camera through the BNC coaxial line, in order to accurately and synchronously acquire image signals of the display screen, the time synchronization of the PTP server needs to be received in the camera, the first server is responsible for acquiring the image output by the camera, and the first server can acquire a synchronous reference clock as time reference according to the PTP server.
As shown in fig. 5, in one embodiment, an XR spatial locator is applied to a first server, and the XR spatial locator includes an obtaining module 510, an extracting module 520, a calculating module 530, and a data outputting module 540.
The acquiring module 510 is configured to acquire first image data, which is acquired by the camera and displayed by the display screen, where the first image data includes a fusion image, the fusion image is a rendered image and a tagged image displayed by frame insertion, the tagged image is generated by a second server, and the fusion image is formed by fusing the second server.
An extracting module 520, configured to extract the marker image from the first image data according to the frequency of the frame insertion display.
And the calculating module 530 is used for calculating the real three-dimensional space position parameter of the camera according to the marked image based on an image recognition algorithm.
And the data output module 540 is configured to send the real three-dimensional spatial position parameter of the camera to the three-dimensional rendering server, so as to be input as the tracking data of the rendered image.
In this embodiment, the XR spatial locator further includes a first synchronization module, where the first synchronization module is configured to obtain a reference clock synchronization signal, so that the first server and the reference clocks of the camera and the second server are synchronized.
In this embodiment, the extracting module is specifically configured to: acquiring the frequency of the frame insertion display, wherein the frequency of the frame insertion display is the frequency of displaying the mark image on the display screen set by the second server; and separating the fused image according to the frequency of frame interpolation display, and extracting the marked image.
As shown in fig. 6, in one embodiment, an XR spatial locator apparatus, applied to a second server, includes a receiving module 610, a generating module 620, a fusing module 630, and a recording module 640.
The receiving module 610 is configured to receive a rendered image, where the rendered image is an image rendered by a three-dimensional rendering server.
A generating module 620 for generating a marker image.
And the fusion module 630 is configured to fuse the rendered image and the marked image into a fused image in a frame insertion display manner, output the fused image to a display screen for display, form first image data after the fused image is collected by a camera, and input the first image data to the first server.
The recording module 640 is configured to record the frequency of the frame insertion display, and input the frequency to the first server, where the first image data is separated and extracted by the first server according to the frequency of the frame insertion display, the marker image is used to calculate a real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is used to be sent to the three-dimensional rendering server to be input as tracking data of a rendered image.
In this embodiment, the XR spatial locator further includes a second synchronization module, where the second synchronization module is configured to obtain a reference clock synchronization signal, so that the second server is synchronized with the reference clock of the camera and the reference clock of the first server.
In one embodiment, a computer device is provided, which may be an intelligent terminal, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by a processor implements an XR spatial localization method.
It will be appreciated by those skilled in the art that the configuration shown in fig. 7 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, a computer device comprises a memory and a processor, the memory storing a computer program, the processor implementing the steps of the above method embodiments when executing the computer program.
In one embodiment, a computer storage medium stores a computer program that, when executed by a processor, performs the steps in the above-described method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. An XR space positioning method applied to a first server, the method comprising:
acquiring first image data which is acquired by a camera and displayed by a display screen, wherein the first image data comprises a fusion image, the fusion image is a rendered image and a marker image which are displayed by inserting frames, the marker image is generated by a second server, and the fusion image is formed by fusing the second server;
extracting the marker image from the first image data according to the frequency of the frame interpolation display;
calculating real three-dimensional space position parameters of the camera according to the marked image based on an image recognition algorithm;
and sending the real three-dimensional space position parameter of the camera to a three-dimensional rendering server to be used as the tracking data input of the rendered image.
2. The XR spatial localization method of claim 1, further comprising:
and acquiring a reference clock synchronization signal to synchronize the first server with the reference clock of the camera and the reference clock of the second server.
3. The XR spatial localization method of claim 1, wherein the extracting the marker image from the first image data at a frequency of the inter-frame display comprises:
acquiring the frequency of the frame insertion display, which is the frequency of displaying the marker image on the display screen set by the second server;
and separating the fused image according to the frequency of the frame insertion display, and extracting the marked image.
4. An XR space positioning method applied to a second server, the method comprising:
receiving a rendering image, wherein the rendering image is an image rendered by a three-dimensional rendering server;
generating a marker image;
fusing the rendered image and the marked image into a fused image in a frame insertion display mode, outputting the fused image to a display screen for display, acquiring the fused image by a camera to form first image data, and inputting the first image data into a first server;
and recording the display frequency of the interpolation frame, inputting the frequency to the first server, extracting a marked image from the first image data through the first server according to the display frequency of the interpolation frame in a separating manner, wherein the marked image is used for calculating the real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is used for being sent to the three-dimensional rendering server to be used as the tracking data of the rendered image.
5. The XR spatial localization method of claim 4, further comprising:
and acquiring a reference clock synchronization signal to synchronize the second server with the reference clock of the camera and the reference clock of the first server.
6. An XR spatial locator, for use with a first server, the XR spatial locator comprising:
the system comprises an acquisition module, a display module and a processing module, wherein the acquisition module is used for acquiring first image data which is acquired by a camera and displayed by a display screen, the first image data comprises a fusion image, the fusion image is a rendered image and a marker image which are displayed by inserting frames, the marker image is generated by a second server, and the fusion image is formed by fusing the second server;
an extraction module, configured to extract the marker image from the first image data according to the frequency of the frame insertion display;
the calculation module is used for calculating real three-dimensional space position parameters of the camera according to the marked image based on an image recognition algorithm; and
and the data output module is used for sending the real three-dimensional space position parameters of the camera to a three-dimensional rendering server for input and use as the tracking data of the rendered image.
7. An XR spatial locator device, for use with a second server, the device comprising:
the receiving module is used for receiving a rendering image, and the rendering image is an image rendered by a three-dimensional rendering server;
a generation module for generating a marker image;
the fusion module is used for fusing the rendered image and the marked image into a fused image in a frame insertion display mode, outputting the fused image to a display screen for display, acquiring the fused image by a camera to form first image data, and inputting the first image data into a first server; and with
And the recording module is used for recording the display frequency of the inserted frame and inputting the frequency to the first server, the first image data is separated and extracted into a marked image through the first server according to the display frequency of the inserted frame, the marked image is used for calculating the real three-dimensional space position parameter of the camera, and the real three-dimensional space position parameter of the camera is used for being sent to the three-dimensional rendering server to be used as the tracking data input of the rendered image.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 3 or 4 to 5.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 3 or 4 to 5.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 3 or 4 to 5 when executed by a processor.
CN202211014438.6A 2022-08-23 2022-08-23 XR space positioning method and device, computer equipment and storage medium Active CN115100344B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211014438.6A CN115100344B (en) 2022-08-23 2022-08-23 XR space positioning method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211014438.6A CN115100344B (en) 2022-08-23 2022-08-23 XR space positioning method and device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115100344A CN115100344A (en) 2022-09-23
CN115100344B true CN115100344B (en) 2022-10-25

Family

ID=83300226

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211014438.6A Active CN115100344B (en) 2022-08-23 2022-08-23 XR space positioning method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115100344B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10497180B1 (en) * 2018-07-03 2019-12-03 Ooo “Ai-Eksp” System and method for display of augmented reality
CN111459267A (en) * 2020-03-02 2020-07-28 杭州嘉澜创新科技有限公司 Data processing method, first server, second server and storage medium
CN111741266A (en) * 2020-06-24 2020-10-02 北京梧桐车联科技有限责任公司 Image display method and device, vehicle-mounted equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030227542A1 (en) * 2001-12-20 2003-12-11 Xiang Zhang Single-computer real-time stereo augmented reality system
CA2915582A1 (en) * 2013-06-17 2014-12-24 Square Enix Holdings Co., Ltd. Image processing apparatus, image processing system, image processing method and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10497180B1 (en) * 2018-07-03 2019-12-03 Ooo “Ai-Eksp” System and method for display of augmented reality
CN111459267A (en) * 2020-03-02 2020-07-28 杭州嘉澜创新科技有限公司 Data processing method, first server, second server and storage medium
CN111741266A (en) * 2020-06-24 2020-10-02 北京梧桐车联科技有限责任公司 Image display method and device, vehicle-mounted equipment and storage medium

Also Published As

Publication number Publication date
CN115100344A (en) 2022-09-23

Similar Documents

Publication Publication Date Title
CN106210861B (en) Method and system for displaying bullet screen
CN107230236B (en) System and method for encoding and decoding light field image files
US8780173B2 (en) Method and apparatus for reducing fatigue resulting from viewing three-dimensional image display, and method and apparatus for generating data stream of low visual fatigue three-dimensional image
CN106303289B (en) Method, device and system for fusion display of real object and virtual scene
CN101589621B (en) Reproduction apparatus, display apparatus, reproduction method, and display method
KR101210315B1 (en) Recommended depth value for overlaying a graphics object on three-dimensional video
CN109089064B (en) Apparatus and method for processing media signal
CN101873507A (en) Transmitting device, receiving system, stereoscopic image data transmission method and method of reseptance
CN104618803A (en) Information push method, information push device, terminal and server
KR100576544B1 (en) Apparatus and Method for Processing of 3D Video using MPEG-4 Object Descriptor Information
CN108243318B (en) Method and device for realizing live broadcast of multiple image acquisition devices through single interface
WO2023279793A1 (en) Video playing method and apparatus
CN111919451A (en) Live broadcasting method, live broadcasting device and terminal
CN110913278B (en) Video playing method, display terminal and storage medium
CN115100344B (en) XR space positioning method and device, computer equipment and storage medium
US9338429B2 (en) Video processing apparatus capable of reproducing video content including a plurality of videos and control method therefor
CN113923354A (en) Video processing method and device based on multi-frame image and virtual background shooting system
CN109862385B (en) Live broadcast method and device, computer readable storage medium and terminal equipment
US10783670B2 (en) Method for compression of 360 degree content and electronic device thereof
CN107995538B (en) Video annotation method and system
JP6934052B2 (en) Display control device, display control method and program
CN116193182A (en) Screen projection method and system of AR content, electronic equipment and storage medium
CN114915798A (en) Real-time video generation method, multi-camera live broadcast method and device
EP3407297B1 (en) Method and device for determining a characteristic of a display device
US20220247891A1 (en) Processing method and processing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant