WO2020063172A1

WO2020063172A1 - Data processing method, terminal, server, and storage medium

Info

Publication number: WO2020063172A1
Application number: PCT/CN2019/100650
Authority: WO
Inventors: 夏炀; 李虎
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-09-30
Filing date: 2019-08-14
Publication date: 2020-04-02
Also published as: CN109274953A

Abstract

Disclosed in embodiments of the present application is a data processing method. The method comprises: emitting N pieces of different coded light, wherein N is a positive integer greater than 1; obtaining N depth images of one three-dimensional (3D) video data frame on the basis of the N pieces of different coded light; and sending the N depth images to a mobile edge computing (MEC) server, wherein the N depth images are used for the MEC server to determine, from the N depth images, a depth image that matches a two-dimensional (2D) image in the 3D video data frame. Also disclosed in the embodiments of the present application are a server and a storage medium.

Description

Data processing method, terminal, server and storage medium

Cross-reference to related applications

The embodiment of the present application is based on a Chinese patent application with an application number of 201811162631.8 and an application date of September 30, 2018, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference Application examples.

Technical field

The present application relates to the field of image processing technology, and relates to, but is not limited to, a data processing method, a terminal, a server, and a storage medium.

Background technique

With the continuous development of mobile communication networks, the transmission rate of mobile communication networks has increased rapidly, thus providing strong technical support for the generation and development of 3D video services. The three-dimensional video data includes two-dimensional image data (for example, Red Green Blue (RGB) data) and depth data (Depth data), and the three-dimensional video data is transmitted by transmitting two-dimensional video data and depth data, respectively. At present, the relationship between the speckle image signal-to-noise ratio, depth accuracy, and speckle density is used to design and encode a multi-password structured light pattern. Finally, the projected structured light pattern and the modulated image are used to obtain corresponding matching points. The offset of the points before and after the projection calculates the depth value of the matching point to achieve the depth data acquisition of the object to be measured. In this way, the accuracy of the obtained depth information is insufficient.

Summary of the Invention

The embodiments of the present application provide a data processing method, a terminal, a server, and a storage medium.

The technical solution of the embodiment of the present application is implemented as follows:

An embodiment of the present application provides a data processing method. The method includes:

Emit N different coded lights; where N is a positive integer greater than 1;

Obtaining N depth images of one frame of three-dimensional video data based on the N different coded lights;

Send the N depth images to a Mobile Edge Computing (MEC) server, where the N depth images are used by the MEC server to determine the N depth images from the N depth images. Depth images that match two-dimensional (Second dimension) images in three-dimensional video data.

In the above solution, the method further includes:

Obtaining N different coded light sequences;

Correspondingly, the transmitting N different coded lights includes:

According to N different coded light sequences, N coded lights of different shapes and / or textures are emitted.

In the above solution, the obtaining N coded light sequences includes:

N different coded light sequences are generated based on different coding modes.

In the above solution, the sending N depth images to a mobile edge computing MEC server includes:

Sending the N depth images to the MEC server through time division multiplexing.

An embodiment of the present application further provides a data processing method, an application and a MEC server, the method includes:

A frame of three-dimensional video data sent by a receiving terminal, where the frame of three-dimensional video data includes a two-dimensional 2D image and N depth images; N is a positive integer greater than 1;

Determining a depth image matching the 2D image according to the N depth images;

Based on the 2D image and the determined depth image, a three-dimensional video is established.

In the above solution, the determining the depth information matching the 2D image according to the N depth images includes at least one of the following:

Selecting one of the depth images matching the 2D image from the N depth images;

Based on the N depth images, one depth image matching the 2D image is generated.

An embodiment of the present application further provides a terminal, where the terminal includes: a transmitting module, a first acquiring module, and a first sending module;

The transmitting module is configured to transmit N different coded lights; wherein N is a positive integer greater than 1;

The first obtaining module is configured to obtain N depth images of one frame of three-dimensional video data based on the N different coded lights;

The first sending module is configured to send N depth images to a mobile edge computing MEC server, wherein the N depth images are configured for the MEC server to determine from the N depth images. A depth image that matches a two-dimensional 2D image in the one-dimensional three-dimensional video data.

In the above solution, the terminal further includes:

A second acquisition module configured to obtain N different coded light sequences;

The transmitting module includes:

The first transmitting unit is configured to transmit N coded lights of different shapes and / or textures according to the N different coded light sequences.

In the above solution, the second acquisition module includes:

The first generating unit is configured to generate N different coded light sequences based on different coding modes.

In the above solution, the first sending module includes:

The first sending unit is configured to send the N depth images to the MEC server through time division multiplexing.

An embodiment of the present application further provides a MEC server, where the server includes: a first receiving module, a first determining module, and a first establishing module;

The first receiving module is configured to receive a frame of three-dimensional video data sent by a terminal, where the frame of three-dimensional video data includes a two-dimensional 2D image and N depth images; N is a positive integer greater than 1;

The first determining module is configured to determine a depth image matching the 2D image according to the N depth images;

The first establishing module is configured to establish a three-dimensional video based on the 2D image and the determined depth image.

In the above solution, the first determining module includes at least one of the following:

A first selection unit configured to select one of the depth images that matches the 2D image from the N depth images;

A second generating unit is configured to generate one depth image matching the 2D image based on the N depth images.

An embodiment of the present application further provides a computer storage medium that stores computer instructions that, when executed by a processor, implement the steps of the data processing method applied to a terminal according to the embodiments of the present application; or the instructions are When the processor executes, the steps of the data processing method applied to the MEC server according to the embodiments of the present application are implemented.

An embodiment of the present application further provides a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the application described in the embodiment of the present application is implemented. Steps of the terminal-based data processing method.

An embodiment of the present application further provides a MEC server including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the processor implements the program described in the embodiment of the present application. Steps of a data processing method applied to a MEC server.

The embodiments of the present application provide a data processing method, terminal, server, and storage medium. First, N different coded lights are transmitted; where N is a positive integer greater than 1, and then based on the N different codes Light to obtain N depth images of one frame of three-dimensional video data; finally, send the N depth images to a mobile edge computing MEC server, where the N depth images are used by the MEC server to The N depth images determine a depth image that matches the two-dimensional 2D image in the three-dimensional video data. In this way, when a depth image is acquired through multiple different coded lights, and the depth value of this frame of three-dimensional video is determined, the server passes Combine the depth values in multiple depth images to make the obtained depth values more accurate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a system architecture applied to a data processing method according to an embodiment of the present application; FIG.

2 is a schematic flowchart of an implementation process of a data processing method according to an embodiment of the present application;

3 is an implementation interaction diagram of a data processing method according to an embodiment of the present application;

4A is a schematic structural diagram of a terminal according to an embodiment of the present application;

FIG. 4B is another schematic structural diagram of a terminal according to an embodiment of the present application; FIG.

4C is another schematic structural diagram of a terminal according to an embodiment of the present application;

4D is a schematic structural diagram of still another composition of a terminal according to an embodiment of the present application;

5A is a schematic structural diagram of a server according to an embodiment of the present application;

FIG. 5B is another schematic structural diagram of a server according to an embodiment of the present application; FIG.

FIG. 6 is a schematic diagram of a hardware composition and structure of a data processing device according to an embodiment of the present application.

detailed description

Before the technical solution of the embodiment of the present application is described in detail, the system architecture of the data processing method application of the embodiment of the present application is briefly described first. The data processing method in the embodiment of the present application is applied to a service related to three-dimensional video data. The service is, for example, a service for sharing three-dimensional video data, or a live broadcast service based on three-dimensional video data. In this case, due to the large amount of data of the three-dimensional video data, the transmitted depth values and two-dimensional video data require higher technical support in the data processing process, so the mobile communication network needs a faster data processing rate. , And a more stable data processing environment.

FIG. 1 is a schematic diagram of a system architecture applied to a data processing method according to an embodiment of the present application. As shown in FIG. 1, the system may include a terminal, a base station, a MEC server, a service processing server, a core network, and the Internet, etc .; a MEC server and services High-speed channels are established between the processing servers through the core network to achieve data synchronization.

Taking the application scenario where two terminals interact as shown in FIG. 1 as an example, MEC server A is a MEC server deployed near terminal A (sender), and core network A is the core network in the area where terminal A is located; B is a MEC server deployed near terminal B (receiving end), and core network B is the core network in the area where terminal B is located; MEC server A and MEC server B can communicate with the service processing server through core network A and core network B, respectively. Establish high-speed channels for data synchronization.

Among them, after the 3D video data sent by the terminal A is processed to the MEC server A, the MEC server A synchronizes the data to the service processing server through the core network A; and the MEC server B obtains the 3D video data sent by the terminal A from the service processing server. And send it to terminal B for presentation.

Here, if terminal B and terminal A use the same MEC server to implement transmission, then terminal B and terminal A directly implement three-dimensional video data transmission through one MEC server without the participation of a service processing server. This method is called local Way back. Specifically, it is assumed that the terminal B and the terminal A realize the transmission of three-dimensional video data through the MEC server A. After the three-dimensional video data sent by the terminal A is processed to the MEC server A, the three-dimensional video data is sent by the MEC server A to the terminal B for presentation.

Here, the terminal may select an evolved base station (eNB) that accesses a 4G network or a next-generation evolved base station (gNB) that accesses a 5G network based on the network situation, or the configuration of the terminal itself, or an algorithm configured by itself. The eNB is connected to the MEC server through a Long Term Evolution (LTE) access network, so that the gNB is connected to the MEC server through the Next Generation Access Network (NG-RAN).

Here, the MEC server is deployed on the edge of the network near the terminal or the source of the data. The so-called near the terminal or the source of the data is not only at the logical location, but also geographically close to the terminal or the source of the data. Different from the main service processing servers in existing mobile communication networks, which are deployed in several large cities, multiple MEC servers can be deployed in one city. For example, in an office building with many users, a MEC server can be deployed near the office building.

Among them, the MEC server, as an edge computing gateway with core capabilities of converged networks, computing, storage, and applications, provides platform support for edge computing including device domains, network domains, data domains, and application domains. It connects various types of smart devices and sensors, and provides smart connection and data processing services nearby, allowing different types of applications and data to be processed in the MEC server, realizing business real-time, business intelligence, data aggregation and interoperation, security and privacy protection, etc. Key intelligent services effectively improve the intelligent decision-making efficiency of the business.

The following describes the present application in detail with reference to the drawings and specific embodiments.

The embodiment of the present application provides a data processing method, which is applied to a terminal. The terminal may be a mobile terminal such as a mobile phone or a tablet computer, or a terminal such as a computer. FIG. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application. As shown in FIG. 2, the method includes the following steps:

In step S201, N different coded lights are emitted.

Here, N is a positive integer greater than 1. The step S201 can be understood as transmitting N coded lights of different shapes and / or textures according to the N different coded light sequences. It is also possible to generate N different coded light sequences based on different coding methods, for example, encoding using M-ary coding, two-gray-level coding, phase-shifting coding, etc., to generate different N consecutive sequences. Coded light; M is greater than or equal to 2. The different coded light may be: structured light of different shapes and / or textures projected by the depth camera toward the collection object, and the structured light may be non-visible light, so as not to interfere with the collection of two-dimensional images based on visible light imaging. In this embodiment, the different coded lights may be coded lights of different wavelengths. At this time, at least two depth images corresponding to the two-dimensional image may be acquired at the same time or may not be acquired at the same time.

In step S202, N depth images of one frame of three-dimensional video data are obtained based on the N different coded lights.

Here, the terminal uses N different coded lights, that is, structured light of different shapes and / or textures, which are projected onto the collection object. Due to the unevenness of the surface of the collection object and the distance from the depth camera, the depth camera is based on structured light collection. The depth image is different in combination with the shape and / or texture of the projected structured light, and the comparison of the shape and / or texture presented in the image of the collected structured light can determine the depth value of each structured light projection point And constructing a depth image based on the depth value. Since at least two depth images are collected using different structured light, the terminal can obtain at least two depth values at the same position of the collection target according to combining at least two depth images, thereby improving the accuracy of the depth value.

Step S203: Send the N depth images to a MEC server.

Here, the N depth images are used for the MEC server to determine, from the N depth images, a depth image that matches a two-dimensional 2D image in the one-dimensional three-dimensional video data. After the terminal sends the depth image to the MEC server, the MEC server can obtain at least two depth images corresponding to the two-dimensional image from the terminal, and can combine the depth values in the two depth images to obtain a more accurate depth value, or reduce the A phenomenon in which some objects in a two-dimensional image lack a depth value in one frame of the three-dimensional video data.

In this embodiment, by using N different coded lights, N depth images of one frame of three-dimensional video data are collected, so that one frame to be collected corresponds to multiple depth images. In this way, according to multiple depth images, it can be more Determining the depth value of the 3D video data of the frame accurately.

In this embodiment, as an implementation manner, the obtaining N depth images of one frame of three-dimensional video data includes that the terminal obtains N depth images from an acquisition component capable of acquiring depth data at least, and the acquisition component can A communication link is established with at least one terminal so that a corresponding terminal obtains the three-dimensional video data.

In this embodiment, since the acquisition component capable of acquiring depth data is relatively expensive, the terminal does not have the function of acquiring three-dimensional video data, but collects three-dimensional video data through an acquisition component independent of the terminal, and then acquires the data through the acquisition component and the terminal. The communication component establishes a communication link, so that the terminal obtains the three-dimensional video data collected by the acquisition component. The acquisition component may be implemented by at least one of the following: a depth camera, a binocular camera, a 3D structured light camera module, and a time of flight (TOF) camera module.

Here, the acquisition component can establish a communication link with at least one terminal to process the acquired three-dimensional video data to the at least one terminal, so that the corresponding terminal obtains the three-dimensional video data, so that the three-dimensional video data collected by one acquisition component can be shared. To at least one terminal, so as to realize the sharing of the collection components.

As another implementation mode, the terminal itself has a function of acquiring three-dimensional video data. It can be understood that the terminal is provided with an acquisition component capable of acquiring at least depth data, for example, at least one of the following components: a depth camera, a binocular camera, and a 3D structure. Optical camera module and TOF camera module to collect three-dimensional video data.

The obtained three-dimensional video data includes two-dimensional video data and depth data. The two-dimensional video data is used to represent a planar image, and may be RGB data, for example. The depth data represents the surface of the acquisition object targeted by the acquisition component and the acquisition component. Distance.

An embodiment of the present application further provides a data processing method. FIG. 3 is an implementation interaction diagram of the data processing method of the embodiment of the present application. As shown in FIG. 3, the method includes the following steps:

Step S301: The terminal transmits N coded lights of different shapes and / or textures according to the N different coded light sequences.

In some embodiments, the N different coded light sequences may include: N different coded light sequences generated based on a same coding mode.

In other embodiments, the step S301 may be that the terminal generates N different coded light sequences based on different coding modes.

Step S302: The terminal obtains N depth images of one frame of three-dimensional video data based on the N different coded lights.

Here, the step S302 can also be understood as: projecting different encoded light of the N consecutive sequences onto the same acquisition target in the three-dimensional video data to be acquired to obtain N for describing the same acquisition target. Continuous depth images. That is, the same acquisition target corresponds to one two-dimensional image and N depth images. In this way, since at least two depth images are collected using different structured light, the terminal can obtain at least two depth values at the same position of the collection target according to combining at least two depth images, thereby improving the accuracy of the depth value. The depth information contained in the N depth images is different, so the terminal can obtain more accurate depth information according to the N depth images.

Step S303: The terminal sends the N depth images to the MEC server through time division multiplexing.

Here, the step S303 can be understood as sending the N depth images to the MEC server at different time points to avoid situations such as network congestion.

Step S304: The server receives one frame of three-dimensional video data sent by the terminal.

Here, the one-dimensional three-dimensional video data includes a two-dimensional image and N depth images; N is a positive integer greater than 1.

Step S305: The server determines a depth image that matches the 2D image according to the N depth images.

Here, the step S305 may be performed in the following two ways, way one:

The server selects one of the depth images that matches the 2D image from the N depth images. That is, the server selects one or more depth images containing the most comprehensive or accurate depth information from the N depth images as one of the depth images matching the 2D image. In this embodiment, the server selects a depth image containing the most comprehensive or accurate depth information from the N depth images, which can be implemented by the following two methods:

The first method is: The server judges whether the depth values corresponding to the N depth images are missing one by one, or whether the depth value contains enough feature points for matching with the 2D image. If there is a depth image, the depth value contains Complete, then the depth image can be used as one of the depth images matching the 2D image; for example, there are a total of 10 depth images. If the depth value contained in the fourth depth image of the 10 depth images is not missing, Then, the fourth depth image is used as a depth image matching the 2D image.

The second method is: the server judges whether the depth values corresponding to the N depth images have abnormal values one by one, such as being too large, too small, or far from the preset depth value, etc., then there will be depth images with abnormal values It is excluded that only depth images in which there are no outliers are retained, and a depth image containing enough feature points for matching the 2D image is selected from these depth images as one of the depth images matching the 2D image.

Manner 2: According to the N depth images, one depth image matching the 2D image is generated according to the N depth images. That is, the service device combines the N depth images to obtain a depth image containing depth values corresponding to the N depth images, as a depth image matching the 2D image. For example, the above 10 depth images are combined into one depth image, so that the depth information obtained by this combination is very rich in depth information, so the accuracy of the depth information is also improved.

Step S306: The server establishes a three-dimensional video based on the 2D image and the determined depth image.

Here, the MEC server may obtain at least two depth images corresponding to the two-dimensional image from the terminal, and may combine the depth values in the two depth images to obtain a more accurate depth value, or reduce some objects in the two-dimensional image in one frame. The phenomenon of missing depth values in three-dimensional video data is described.

In this embodiment, the terminal sends multiple depth images of one frame of three-dimensional video data to the MEC server, so that the MEC server can obtain more accurate depth values by combining the depth values in the multiple depth images.

An embodiment of the present application provides a data processing method. Projects are projected onto three-dimensional video data to be acquired using N consecutive serially-coded different lights to obtain N consecutive depth images corresponding to the three-dimensional video data. The receiving end then identifies each coded point based on the N consecutive depth images received. Then, the depth values corresponding to the N consecutive depth images are determined according to these coding points. Among them, the encoding methods of the projected N consecutive sequences of different encoded light may be a binary code (most commonly used), an N-ary code (N = 5, 8), a two-gray level encoding, and a phase-shifting encoding scheme.

In still other embodiments, the different coded lights may be: coded lights of the same wavelength; at this time, at least two depth images corresponding to the two-dimensional image may be time-division multiplexed in the same frame of three-dimensional video. The data is collected at different times in the data collection process; thereby realizing time-division multiplexing of depth images.

In other embodiments, different coded light is used to emit differently. The coded light is reflected to form reflected light after encountering the collection target. The depth can be calculated based on the emission time of the emitted light, the reflection time of the reflected light, and the light propagation speed. value. At this time, the coded light of different shapes and / or textures may make the light transmission path inconsistent, so that when one coded light is blocked, the reflected light may be formed after the other coded light is emitted, so as to collect the depth value; or The two coded lights are reflected to form the reflected light and reach the receiver of the depth camera, and two depth images can also be generated separately.

If coded light of different wavelengths is used, two depth images can be collected simultaneously; if coded light of the same wavelength is used, time division multiplexing can be used for processing.

In this embodiment, the depth information obtained under different coded light is transmitted to the MEC in a time-division manner through time division multiplexing for the MEC to obtain accurate depth information.

In order to implement the terminal-side method in the embodiment of the present application, the embodiment of the present application further provides a terminal. 4A is a schematic structural diagram of a structure of a terminal according to an embodiment of the present application; as shown in FIG. 4A, the terminal 40 includes: a transmitting module 41, a first obtaining module 42, and a first sending module 43;

The transmitting module 41 is configured to transmit N different coded lights; wherein N is a positive integer greater than 1;

The first obtaining module 42 is configured to obtain N depth images of one frame of three-dimensional video data based on the N different coded lights;

The first sending module 43 is configured to send N depth images to a mobile edge computing MEC server, wherein the N depth images are configured for the MEC server to send N depth images from the N depth images. A depth image matching a two-dimensional 2D image in the one-dimensional three-dimensional video data is determined.

In the above solution, as shown in FIG. 4B, the terminal 40 further includes:

A second obtaining module 44 configured to obtain N different coded light sequences;

The transmitting module 41 includes:

The first transmitting unit 401 is configured to transmit N coded lights of different shapes and / or textures according to the N different coded light sequences.

In the above solution, as shown in FIG. 4C, the second obtaining module 44 includes:

The first generating unit 402 is configured to generate N different coded light sequences based on different coding modes.

In the above solution, as shown in FIG. 4D, the first sending module 43 includes:

The first sending unit 403 is configured to send N depth images to the MEC server through time division multiplexing.

In the embodiment of the present application, the first sending module 43 in the terminal may be implemented by a processor in the terminal, such as a central processing unit (CPU, Central Processing Unit), and a digital signal processor (DSP, Digital (Signal Processor), Microcontroller Unit (MCU, Microcontroller Unit) or FPGA (Field-Programmable Gate Array), etc .; the first sending module 43 in the terminal can be used in the actual application through the communication module (Including: basic communication suite, operating system, communication module, standardized interface and protocol, etc.) and transceiver antenna implementation; the transmitting module 41 in the terminal can be implemented by a stereo camera, a binocular camera, or a structured light camera in practical applications Or, it can be implemented through a communication module (including: basic communication suite, operating system, communication module, standardized interface and protocol, etc.) and a transmitting and receiving antenna; the first acquisition module 42 in the terminal can be implemented by a processor such as CPU, DSP, MCU or FPGA are combined with communication modules.

It should be noted that the terminal provided in the foregoing embodiment only uses the division of the foregoing program modules as an example when performing data processing. In actual applications, the above processing can be allocated by different program modules according to needs, that is, the terminal The internal structure is divided into different program modules to complete all or part of the processing described above. In addition, the terminal and the data processing method embodiments provided in the foregoing embodiments belong to the same concept. For specific implementation processes, refer to the method embodiments, and details are not described herein again.

Accordingly, in order to implement the server-side method in the embodiment of the present application, the embodiment of the present application further provides a server, specifically a MEC server. FIG. 5A is a schematic structural diagram of a server according to an embodiment of the present application; as shown in FIG. 5A, the server 50 includes a first receiving module 51, a first determining module 52, and a first establishing module 53;

The first receiving module 51 is configured to receive a frame of three-dimensional video data sent by a terminal, where the frame of three-dimensional video data includes a two-dimensional 2D image and N depth images; N is a positive integer greater than 1;

The first determining module 52 is configured to determine a depth image matching the 2D image according to the N depth images;

The first establishing module 53 is configured to establish a three-dimensional video based on the 2D image and the determined depth image.

In the above solution, as shown in FIG. 5B, the first determining module 52 includes at least one of the following:

A first selection unit 501 configured to select one of the depth images matching the 2D image from the N depth images;

The second generating unit 502 is configured to generate one depth image matching the 2D image according to the N depth images.

In the embodiment of the present application, the second data processing unit 52 in the server may be implemented by a processor in the server, such as a CPU, a DSP, an MCU, or an FPGA, in actual applications; the second communication in the server The unit 51 may be implemented by a communication module (including: a basic communication suite, an operating system, a communication module, a standardized interface and a protocol, etc.) and a transmitting and receiving antenna in practical applications.

It should be noted that the server provided by the foregoing embodiment only uses the division of the foregoing program modules as an example for data processing. In practical applications, the above-mentioned transmission and distribution can be completed by different program modules according to needs, that is, the server The internal structure is divided into different program modules to complete all or part of the processing described above. In addition, the server and the data processing method embodiments provided in the foregoing embodiments belong to the same concept. For specific implementation processes, refer to the method embodiments, and details are not described herein again.

Based on the hardware implementation of the above device, an embodiment of the present application further provides a data processing device. FIG. 6 is a schematic diagram of a hardware composition structure of the data processing device according to the embodiment of the present application. As shown in FIG. 6, the data processing device 60 includes a memory. 61. The processor 62 and a computer program stored in the memory and executable on the processor. As a first implementation manner, when the data processing device is a terminal, the processor located at the terminal executes the program to implement: transmitting N different coded lights; wherein N is a positive integer greater than 1; based on the N different Coded light to obtain N depth images of one frame of three-dimensional video data; send N said depth images to a mobile edge computing MEC server, wherein said N said depth images are used by said MEC server The N depth images determine a depth image that matches a two-dimensional 2D image in the one-dimensional three-dimensional video data.

In an embodiment, when the processor at the terminal executes the program, it is implemented: obtaining N different coded light sequences; and transmitting N coded lights of different shapes and / or textures according to the N different coded light sequences. .

In an embodiment, when the processor at the terminal executes the program, it is implemented that: N different coded light sequences are generated based on different coding modes.

In an embodiment, when the processor located at the terminal executes the program, it is realized that: the N depth images are sent to the MEC server through time division multiplexing.

As a second implementation manner, when the data processing device is a server, the processor located on the server executes the program when the processor executes the program: receiving a frame of three-dimensional video data sent by the terminal, where the frame of three-dimensional video data includes two-dimensional 2D Images and N depth images; N is a positive integer greater than 1; a depth image matching the 2D image is determined according to the N depth images; and a three-dimensional video is established based on the 2D image and the determined depth image.

In an embodiment, when the processor located on the server executes the program, it is implemented: selecting one of the depth images matching the 2D image from the N depth images; and generating the depth images based on the N depth images. One of the depth images matching the 2D image.

It can be understood that the data processing device (terminal or server) further includes a communication interface 63; various components in the data processing device (terminal or server) are coupled together through a bus system. Understandably, the bus system is configured to enable connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

In the several embodiments provided in this application, it should be understood that the disclosed method and smart device may be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, such as multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed components are coupled, or directly coupled, or communicated with each other through some interfaces. The indirect coupling or communication connection of the device or unit may be electrical, mechanical, or other forms. of.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into a second processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into a unit; The above integrated unit may be implemented in the form of hardware, or in the form of hardware plus software functional units.

A person of ordinary skill in the art may understand that all or part of the steps of the foregoing method embodiments may be completed by a program instructing related hardware. The foregoing program may be stored in a computer-readable storage medium. When the program is executed, the program is executed. The method includes the steps of the foregoing method embodiment. The foregoing storage medium includes: various types of media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disc.

Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application that are essentially or contribute to the existing technology can be embodied in the form of software products. The computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a personal computer, a server, or a mobile phone) is caused to execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes: various types of media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disc.

It should be noted that the technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.

The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the protection scope of this application.

Industrial applicability

In the embodiment of the present application, first, N different coded lights are emitted; wherein N is a positive integer greater than 1, and then, based on the N different coded lights, N depth images of one frame of three-dimensional video data are obtained; Finally, the N depth images are sent to a mobile edge computing MEC server, where the N depth images are configured for the MEC server to determine from the N depth images and the 3D video data. Two-dimensional 2D image matching depth image; thus, when the depth image is collected through multiple different coded lights to determine the depth value of this frame of three-dimensional video, the server combines the depth values in the multiple depth images to make the obtained The depth value is more accurate.

Claims

A data processing method, which is applied to a terminal, and the method includes:

Emit N different coded lights; where N is a positive integer greater than 1;

Obtaining N depth images of one frame of three-dimensional video data based on the N different coded lights;

Sending N said depth images to a mobile edge computing MEC server, wherein said N said depth images are used by said MEC server to determine from said N depth images and said one frame of three-dimensional video data 2D 2D image matching depth image.
The method according to claim 1, further comprising:

Obtaining N different coded light sequences;

Correspondingly, the transmitting N different coded lights includes:

According to N different coded light sequences, N coded lights of different shapes and / or textures are emitted.
The method according to claim 2, wherein the obtaining N coded light sequences comprises:

N different coded light sequences are generated based on different coding modes.
The method according to claim 1, wherein said sending N said depth images to a mobile edge computing MEC server comprises:

Sending the N depth images to the MEC server through time division multiplexing.
A data processing method, which is applied to a MEC server, the method includes:

A frame of three-dimensional video data sent by a receiving terminal, where the frame of three-dimensional video data includes a two-dimensional 2D image and N depth images; N is a positive integer greater than 1;

Determining a depth image matching the 2D image according to the N depth images;

Based on the 2D image and the determined depth image, a three-dimensional video is established.
The method according to claim 5, wherein the determining depth information that matches the 2D image based on the N depth images comprises at least one of the following:

Selecting one of the depth images matching the 2D image from the N depth images;

According to the N depth images, one depth image matching the 2D image is generated.
A terminal, wherein the terminal includes: a transmitting module, a first acquiring module, and a first transmitting module; wherein,

The transmitting module is configured to transmit N different coded lights; wherein N is a positive integer greater than 1;

The first obtaining module is configured to obtain N depth images of one frame of three-dimensional video data based on the N different coded lights;

The first sending module is configured to send N depth images to a mobile edge computing MEC server, wherein the N depth images are configured for the MEC server to determine from the N depth images. A depth image that matches a two-dimensional 2D image in the one-dimensional three-dimensional video data.
The terminal according to claim 7, wherein the terminal further comprises:

A second acquisition module configured to obtain N different coded light sequences;

The transmitting module includes:

The first transmitting unit is configured to transmit N coded lights of different shapes and / or textures according to the N different coded light sequences.
The terminal according to claim 8, wherein the second acquisition module comprises:

The first generating unit is configured to generate N different coded light sequences based on different coding modes.
The terminal according to claim 7, wherein the first sending module comprises:

The first sending unit is configured to send the N depth images to the MEC server through time division multiplexing.
A MEC server, wherein the server includes: a first receiving module, a first determining module, and a first establishing module; wherein,

The first receiving module is configured to receive a frame of three-dimensional video data sent by a terminal, where the frame of three-dimensional video data includes a two-dimensional 2D image and N depth images; N is a positive integer greater than 1;

The first determining module is configured to determine a depth image matching the 2D image according to the N depth images;

The first establishing module is configured to establish a three-dimensional video based on the 2D image and the determined depth image.
The server according to claim 11, wherein the first determining module comprises at least one of the following:

A first selection unit configured to select one of the depth images that matches the 2D image from the N depth images;

A second generating unit is configured to generate one depth image matching the 2D image based on the N depth images.
A computer storage medium having computer instructions stored therein, wherein when the instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 4 are implemented; or when the instructions are executed by a processor, the right is implemented The steps of the data processing method according to claim 5 or 6.
A terminal includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the data processing method according to any one of claims 1 to 4 when executing the program. A step of.
A MEC server includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the steps of the data processing method according to claim 5 or 6 are implemented. .