WO2020062919A1 - Data processing method, mec server and terminal device - Google Patents

Data processing method, mec server and terminal device Download PDF

Info

Publication number
WO2020062919A1
WO2020062919A1 PCT/CN2019/090328 CN2019090328W WO2020062919A1 WO 2020062919 A1 WO2020062919 A1 WO 2020062919A1 CN 2019090328 W CN2019090328 W CN 2019090328W WO 2020062919 A1 WO2020062919 A1 WO 2020062919A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal device
information
dimensional image
shooting angle
computer program
Prior art date
Application number
PCT/CN2019/090328
Other languages
French (fr)
Chinese (zh)
Inventor
夏炀
谭正鹏
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2020062919A1 publication Critical patent/WO2020062919A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194Transmission of image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/261Image signal generators with monoscopic-to-stereoscopic image conversion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/275Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals

Definitions

  • the embodiments of the present application relate to the field of information processing technologies, and in particular, to a data processing method, a terminal device, a MEC server, a chip, a computer-readable storage medium, a computer program product, and a computer program.
  • the three-dimensional data includes two-dimensional image data (for example, RGB data, etc.) and depth data (Depth data), and three-dimensional data is transmitted by transmitting two-dimensional video data and depth data, respectively.
  • two-dimensional image data for example, RGB data, etc.
  • Depth data depth data
  • three-dimensional data is transmitted by transmitting two-dimensional video data and depth data, respectively.
  • embodiments of the present invention provide a data processing method, a terminal device, a MEC server, a chip, a computer-readable storage medium, a computer program product, and a computer program.
  • the data processing method provided in the embodiment of the present application which is applied to a MEC server, includes:
  • Three-dimensional image modeling is performed based on the shooting angle of the terminal device and the spatial relative position information.
  • a data processing method provided in an embodiment of the present application, which is applied to a terminal device includes:
  • the network side determines the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and based on the shooting angle and location of the terminal device
  • the spatial relative position information is described, and three-dimensional image modeling is performed.
  • a first communication unit configured to receive attitude information of a terminal device
  • a first processing unit configured to determine a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device, and perform three-dimensional image construction based on the shooting angle of the terminal device and the spatial relative position information mold.
  • the second communication unit is configured to send attitude information of the terminal device to the network side.
  • the MEC server provided in the embodiment of the present application includes a processor and a memory.
  • the memory is used to store a computer program
  • the processor is used to call and run the computer program stored in the memory to execute the data processing method described above.
  • the terminal device provided in the embodiment of the present application includes a processor and a memory.
  • the memory is used to store a computer program
  • the processor is used to call and run the computer program stored in the memory to execute the data processing method described above.
  • the chip provided in the embodiment of the present application is used to implement the foregoing data processing method.
  • the chip includes: a processor, configured to call and run a computer program from a memory, so that a device installed with the chip executes the foregoing data processing method.
  • the computer-readable storage medium provided in the embodiments of the present application is used to store a computer program, and the computer program causes a computer to execute the foregoing data processing method.
  • the computer program product provided in the embodiment of the present application includes computer program instructions, and the computer program instructions cause a computer to execute the foregoing data processing method.
  • the computer program provided in the embodiment of the present application when run on a computer, causes the computer to execute the above-mentioned data processing method.
  • FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present application.
  • FIG. 2 is a first schematic flowchart of a data processing method according to an embodiment of the present application
  • FIG. 3a is a first schematic scenario diagram provided by an embodiment of the present application.
  • FIG. 3b is a second schematic view of a scenario provided by an embodiment of the present application.
  • FIG. 4 is a second schematic flowchart of a data processing method according to an embodiment of the present application.
  • FIG. 5 is a schematic structural composition diagram of a MEC server according to an embodiment of the present application.
  • FIG. 6 is a schematic structural composition diagram of a terminal device according to an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a communication device according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a chip according to an embodiment of the present application.
  • FIG. 9 is a schematic block diagram of a communication system according to an embodiment of the present application.
  • the data processing method of the embodiment of the present invention is applied to a service related to three-dimensional video data, such as a service for sharing three-dimensional video data, or a live broadcast service based on three-dimensional video data.
  • a service related to three-dimensional video data such as a service for sharing three-dimensional video data, or a live broadcast service based on three-dimensional video data.
  • the transmitted depth data and 2D video data require higher technical support in the data transmission process, so the mobile communication network needs a faster data transmission rate. , And a more stable data transmission environment.
  • FIG. 1 is a schematic diagram of a system architecture to which a data processing method is applied according to an embodiment of the present invention
  • the system may include a terminal, a base station, a MEC server, a service processing server, a core network, and the Internet, etc .; High-speed channels are established between the processing servers through the core network to achieve data synchronization.
  • MEC server A is a MEC server deployed near terminal A (sender), and core network A is the core network in the area where terminal A is located;
  • the MEC server B is a MEC server deployed near terminal B (receiving end), and core network B is the core network in the area where terminal B is located;
  • MEC server A and MEC server B can communicate with the service processing server through core network A and core network B, respectively.
  • the MEC server A synchronizes the data to the service processing server through the core network A; and the MEC server B obtains the three-dimensional video data sent by the terminal A from the service processing server And send it to terminal B for presentation.
  • terminal B and terminal A use the same MEC server to implement transmission, then terminal B and terminal A directly implement three-dimensional video data transmission through one MEC server without the participation of a service processing server.
  • This method is called local Way back. Specifically, it is assumed that the terminal B and the terminal A realize the transmission of three-dimensional video data through the MEC server A. After the three-dimensional video data sent by the terminal A is transmitted to the MEC server A, the three-dimensional video data sent by the MEC server A to the terminal B is presented.
  • the terminal may select an evolved base station (eNB) that accesses a 4G network or a next-generation evolved base station (gNB) that accesses a 5G network based on the network situation, or the configuration of the terminal itself, or an algorithm configured by itself.
  • eNB evolved base station
  • gNB next-generation evolved base station
  • the eNB is connected to the MEC server through a Long Term Evolution (LTE) access network, so that the gNB is connected to the MEC server through the Next Generation Access Network (NG-RAN).
  • LTE Long Term Evolution
  • NG-RAN Next Generation Access Network
  • the MEC server is deployed on the edge of the network near the terminal or the source of the data.
  • the so-called near the terminal or the source of the data is not only at the logical location, but also geographically close to the terminal or the source of the data.
  • multiple MEC servers can be deployed in one city. For example, in an office building with many users, a MEC server can be deployed near the office building.
  • the MEC server as an edge computing gateway with core capabilities of converged networks, computing, storage, and applications, provides platform support for edge computing including device domains, network domains, data domains, and application domains. It connects various types of smart devices and sensors, and provides smart connection and data processing services nearby, allowing different types of applications and data to be processed in the MEC server, realizing business real-time, business intelligence, data aggregation and interoperation, security and privacy protection, etc. Key intelligent services effectively improve the intelligent decision-making efficiency of the business.
  • An embodiment of the present invention provides a data processing method, which is applied to a MEC server. As shown in FIG. 2, the method includes:
  • Step 201 Receive attitude information of a terminal device
  • Step 202 Determine a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device.
  • Step 203 Perform three-dimensional image modeling based on the shooting angle of the terminal device and the spatial relative position information.
  • the terminal device is a device provided with a camera.
  • this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
  • the attitude information of the terminal device may be collected by using a gyroscope; the obtained attitude information may be angle information of the three axes of the terminal device when it is deflected and tilted; it needs to be understood here that the The angle information of each axis can be referred to the world coordinate system.
  • determining the shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device may include: determining, based on the positional relationship between the terminal device and its acquisition component, the terminal device's A shooting angle; determining spatial relative position information of the terminal device and a target object based on the shooting angle, depth information corresponding to the two-dimensional image information, and posture information of the terminal device.
  • the shooting angle has a corresponding relationship with the positional relationship between the terminal device and its collection component; specifically, the method further includes: obtaining at least one preset shooting angle, each shooting angle described in The positional relationship between the terminal device and its acquisition component.
  • the collection component of the terminal device can be understood as the positional relationship between the terminal device and its two-dimensional camera; the terminal device can use the plane on which the rear shell is located to measure the positional relationship, and the two-dimensional camera can use its central axis for positional relationship
  • the two-dimensional camera can use its central axis for positional relationship
  • the depth information corresponding to the two-dimensional image information, and the attitude information of the terminal device determine the spatial relative position information of the terminal device and the target object. See FIG. 3b.
  • the depth information corresponding to the image information can determine the spatial relative position information between the terminal device and the target object based on the posture information of the terminal device, that is, based on the relative position relationship between the terminal device and the target object based on world coordinates, that is, It enables the network device to obtain the relative position relationship between the terminal device and the target object in the world coordinate system.
  • the three-dimensional image modeling is performed, and finally a three-dimensional image consistent with the current shooting scene can be obtained.
  • An embodiment of the present invention provides a data processing method, which is applied to a terminal device, as shown in FIG. 4, and includes:
  • Step 401 Collect attitude information of the terminal device
  • Step 402 Send the attitude information of the terminal device to the network side; so that the network side determines the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and the shooting based on the terminal device.
  • the angle and the spatial relative position information are used to perform three-dimensional image modeling.
  • the terminal device is a device provided with a camera.
  • this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
  • the attitude information of the terminal device may be collected by using a gyroscope to detect the attitude information of the terminal device.
  • the attitude information collected by the gyroscope can be the angle information of the three axes when the terminal device is deflected and tilted. It should be understood here that the angle information of the three axis directions of the terminal device can be used as a reference for the world coordinate system. .
  • the acquiring the two-dimensional image information and the depth information corresponding to the two-dimensional image information may be: acquiring a two-dimensional image of the target object through the first camera unit to obtain N frames of two-dimensional image information for the target object; N Is an integer greater than or equal to 1; depth information corresponding to each frame of the two-dimensional image information in the N-frame two-dimensional image information is synchronously collected by the second camera unit and the first camera unit;
  • the first camera unit may be a 2D camera
  • the second camera unit may be a depth camera
  • the 2D camera and the depth camera are synchronized to shoot, that is, a two-dimensional image acquired by the two cameras
  • Information and depth information can correspond in the time domain; for example, two-dimensional image 1 is collected at time 1 and depth information 1 is collected at time 1, that is, there is a one-to-one correspondence between the two-dimensional image 1 and the depth information 1, Further, when two-dimensional images and depth information are collected at other times, they can also correspond one-to-one through the time information.
  • the terminal device can send two-dimensional image information and depth information corresponding to the subject area based on the TCP protocol.
  • the posture information of the terminal device can be sent to the network side, so that the network side can determine the shooting angle and spatial relative position information of the terminal device, and then perform 3D image modeling.
  • the environmental factors of the shooting scene are increased, so the accuracy of 3D modeling is guaranteed.
  • An embodiment of the present invention provides a MEC server, as shown in FIG. 5, including:
  • a first communication unit 51 configured to receive posture information of a terminal device
  • a first processing unit 52 configured to determine a shooting angle of the terminal device and spatial relative position information based on the posture information of the terminal device; and perform a three-dimensional image based on the shooting angle of the terminal device and the spatial relative position information Modeling.
  • the terminal device is a device provided with a camera.
  • this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
  • a first processing unit 52 configured to determine a shooting angle of the terminal device for a target object according to a positional relationship between the terminal device and its collecting component; based on the shooting angle and a depth corresponding to the two-dimensional image information Information and posture information of the terminal device, determine spatial relative position information of the terminal device and a target object.
  • the first processing unit 52 is configured to obtain each shooting at a preset at least one shooting angle.
  • the positional relationship between the terminal device and its acquisition component in the angle is a corresponding relationship between the shooting angle and the positional relationship between the terminal device and its collection component; specifically, the first processing unit 52 is configured to obtain each shooting at a preset at least one shooting angle. The positional relationship between the terminal device and its acquisition component in the angle.
  • the collection component of the terminal device can be understood as the positional relationship between the terminal device and its two-dimensional camera; the terminal device can use the plane on which the rear shell is located to measure the positional relationship, and the two-dimensional camera can use its central axis to perform the positional relationship.
  • the terminal device can use the A tangent to slice the terminal equipment to obtain the schematic diagram of the vertical section on the right side; the center axis of the cutting section on the right side can be relative to the plane of the housing of the terminal device. Relationship; further, determining a shooting angle of the terminal device according to a preset shooting angle and a relative relationship between the terminal device and the collecting component.
  • the depth information corresponding to the two-dimensional image information, and the attitude information of the terminal device determine the spatial relative position information of the terminal device and the target object. See FIG. 3b.
  • the depth information corresponding to the image information can determine the spatial relative position information between the terminal device and the target object based on the posture information of the terminal device, that is, based on the relative position relationship between the terminal device and the target object based on world coordinates, that is, It enables the network device to obtain the relative position relationship between the terminal device and the target object in the world coordinate system.
  • the three-dimensional image modeling is performed, and finally a three-dimensional image consistent with the current shooting scene can be obtained.
  • An embodiment of the present invention provides a terminal device, as shown in FIG. 6, including:
  • An acquisition unit 61 configured to acquire attitude information of a terminal device
  • the second communication unit 62 is configured to send attitude information of the terminal device to the network side.
  • the terminal device is a device provided with a camera.
  • this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
  • the foregoing collecting unit 61 is configured to detect a posture information of the terminal device by using a gyroscope.
  • the attitude information collected by the gyroscope can be the angle information of the three axes when the terminal device is deflected and tilted. It should be understood here that the angle information of the three axis directions of the terminal device can be used as a reference for the world coordinate system. .
  • An acquisition unit 61 configured to acquire two-dimensional image information and corresponding depth information of a target object
  • the second communication unit 62 is configured to send two-dimensional image information and depth information corresponding to the two-dimensional image information to the network side.
  • the acquiring the two-dimensional image information and the depth information corresponding to the two-dimensional image information may be: acquiring a two-dimensional image of the target object through the first camera unit to obtain N frames of two-dimensional image information for the target object; N Is an integer greater than or equal to 1; depth information corresponding to each frame of the two-dimensional image information in the N-frame two-dimensional image information is synchronously collected by the second camera unit and the first camera unit;
  • the first camera unit may be a 2D camera
  • the second camera unit may be a depth camera
  • the 2D camera and the depth camera are synchronized to shoot, that is, a two-dimensional image acquired by the two cameras
  • Information and depth information can correspond in the time domain; for example, two-dimensional image 1 is collected at time 1 and depth information 1 is collected at time 1, that is, there is a one-to-one correspondence between the two-dimensional image 1 and the depth information 1, Further, when two-dimensional images and depth information are collected at other times, they can also correspond one-to-one through the time information.
  • the terminal device can send two-dimensional image information and depth information corresponding to the subject area based on the TCP protocol.
  • the posture information of the terminal device can be sent to the network side, so that the network side can determine the shooting angle and spatial relative position information of the terminal device, and then perform 3D image modeling.
  • the environmental factors of the shooting scene are increased, so the accuracy of 3D modeling is guaranteed.
  • FIG. 7 is a schematic structural diagram of a communication device 700 according to an embodiment of the present application. It should be understood that the communication device shown in FIG. 7 may be a terminal device or a MEC server in this embodiment.
  • the communication device 700 shown in FIG. 7 includes a processor 710, and the processor 710 may call and run a computer program from a memory to implement the method in the embodiment of the present application.
  • the communication device 700 may further include a memory 720.
  • the processor 710 may call and run a computer program from the memory 720 to implement the method in the embodiment of the present application.
  • the memory 720 may be a separate device independent of the processor 710, or may be integrated in the processor 710.
  • the communication device 700 may further include a transceiver 730, and the processor 710 may control the transceiver 730 to communicate with other devices. Specifically, it may send information or data to other devices, or receive other information. Information or data sent by the device.
  • the transceiver 730 may include a transmitter and a receiver.
  • the transceiver 730 may further include antennas, and the number of antennas may be one or more.
  • the communication device 700 may specifically be the MEC server according to the embodiment of the present application, and the communication device 700 may implement the corresponding process implemented by the MEC server in each method of the embodiment of the present application. For brevity, details are not described herein again. .
  • the communication device 700 may specifically be a mobile terminal device / terminal device in the embodiment of the present application, and the communication device 700 may implement a corresponding process implemented by the mobile terminal device / terminal device in each method in the embodiments of the present application, For brevity, I will not repeat them here.
  • FIG. 8 is a schematic structural diagram of a chip according to an embodiment of the present application.
  • the chip 800 shown in FIG. 8 includes a processor 810, and the processor 810 can call and run a computer program from a memory to implement the method in the embodiment of the present application.
  • the chip 800 may further include a memory 820.
  • the processor 810 may call and run a computer program from the memory 820 to implement the method in the embodiment of the present application.
  • the memory 820 may be a separate device independent of the processor 810, or may be integrated in the processor 810.
  • the chip 800 may further include an input interface 830.
  • the processor 810 may control the input interface 830 to communicate with other devices or chips. Specifically, the processor 810 may obtain information or data sent by other devices or chips.
  • the chip 800 may further include an output interface 840.
  • the processor 810 may control the output interface 840 to communicate with other devices or chips. Specifically, the processor 810 may output information or data to the other devices or chips.
  • the chip may be applied to the MEC server in the embodiments of the present application, and the chip may implement the corresponding processes implemented by the MEC server in the methods of the embodiments of the present application.
  • the chip may implement the corresponding processes implemented by the MEC server in the methods of the embodiments of the present application.
  • the chip may be applied to the mobile terminal device / terminal device in the embodiments of the present application, and the chip may implement the corresponding process implemented by the mobile terminal device / terminal device in each method of the embodiments of the present application. For simplicity, I will not repeat them here.
  • the chip mentioned in the embodiments of the present application may also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip.
  • FIG. 9 is a schematic block diagram of a communication system 900 according to an embodiment of the present application.
  • the communication system 900 includes a terminal device 910 and a MEC server 920.
  • the terminal device 910 may be used to implement the corresponding functions implemented by the terminal device in the foregoing method
  • the MEC server 920 may be used to implement the corresponding functions implemented by the MEC server in the foregoing method.
  • details are not described herein again. .
  • the processor in the embodiment of the present application may be an integrated circuit chip and has a signal processing capability.
  • each step of the foregoing method embodiment may be completed by using an integrated logic circuit of hardware in a processor or an instruction in a form of software.
  • the above processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field, Programmable Gate Array, FPGA), or other Programming logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • Various methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware decoding processor, or may be performed by using a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like.
  • the storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the foregoing method in combination with its hardware.
  • the memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), and an electronic memory. Erase programmable read-only memory (EPROM, EEPROM) or flash memory.
  • the volatile memory may be Random Access Memory (RAM), which is used as an external cache.
  • RAM Static Random Access Memory
  • DRAM Dynamic Random Access Memory
  • Synchronous Dynamic Random Access Memory Synchronous Dynamic Random Access Memory
  • SDRAM double data rate synchronous dynamic random access memory
  • Double SDRAM, DDR SDRAM enhanced synchronous dynamic random access memory
  • Enhanced SDRAM, ESDRAM synchronous connection dynamic random access memory
  • Synchronous DRAM Synchronous Dynamic Random Access Memory
  • Enhanced SDRAM Enhanced SDRAM, ESDRAM
  • synchronous connection dynamic random access memory Synchrobus RAM, SLDRAM
  • Direct Rambus RAM Direct Rambus RAM
  • the memory in the embodiment of the present application may also be a static random access memory (static RAM, SRAM), a dynamic random access memory (dynamic RAM, DRAM), Synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (Double SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection Dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct RAMbus RAM, DR RAM) and so on. That is, the memories in the embodiments of the present application are intended to include, but not limited to, these and any other suitable types of memories.
  • An embodiment of the present application further provides a computer-readable storage medium for storing a computer program.
  • the computer-readable storage medium may be applied to the MEC server in the embodiment of the present application, and the computer program causes the computer to execute the corresponding process implemented by the MEC server in each method in the embodiment of the present application. No longer.
  • the computer-readable storage medium may be applied to the mobile terminal device / terminal device in the embodiments of the present application, and the computer program causes the computer to execute the mobile terminal device / terminal device implemented in each method in the embodiments of the present application.
  • the corresponding process is omitted here for brevity.
  • An embodiment of the present application further provides a computer program product, including computer program instructions.
  • the computer program product can be applied to the MEC server in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the MEC server in each method of the embodiments of the present application. More details.
  • the computer program product can be applied to a mobile terminal device / terminal device in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding method implemented by the mobile terminal device / terminal device in each method in the embodiments of the present application.
  • the computer program instructions cause the computer to execute the corresponding method implemented by the mobile terminal device / terminal device in each method in the embodiments of the present application.
  • the embodiment of the present application also provides a computer program.
  • the computer program may be applied to the MEC server in the embodiment of the present application.
  • the computer program When the computer program is run on the computer, the computer is caused to execute the corresponding processes implemented by the MEC server in each method in the embodiment of the present application. , Will not repeat them here.
  • the computer program may be applied to a mobile terminal device / terminal device in the embodiments of the present application.
  • the computer program When the computer program is run on a computer, the computer is caused to execute the methods in the embodiments of the application by the mobile terminal device / terminal.
  • the corresponding process implemented by the device is not repeated here for brevity.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a MEC server, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory) ROM, random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes .

Abstract

Provided by the embodiments of the present invention are a data processing method, terminal device, MEC server, chip, computer readable storage medium, computer program product and computer program, the method comprising: receiving orientation information of a terminal device; determining a photographing angle and spatial relative position information of the terminal device on the basis of the orientation information of the terminal device; and carrying out three-dimensional image modeling on the basis of the photographing angle and spatial relative position information of the terminal device.

Description

一种数据处理方法、MEC服务器、终端设备Data processing method, MEC server and terminal equipment 技术领域Technical field
本申请实施例涉及信息处理技术领域,具体涉及一种数据处理方法、终端设备、MEC服务器、芯片、计算机可读存储介质、计算机程序产品以及计算机程序。The embodiments of the present application relate to the field of information processing technologies, and in particular, to a data processing method, a terminal device, a MEC server, a chip, a computer-readable storage medium, a computer program product, and a computer program.
背景技术Background technique
随着移动通信网络的不断发展,移动通信网络的传输速率飞速提高,从而给三维视频业务的产生和发展提供了有力的技术支持。三维数据包括二维图像数据(例如RGB数据等)和深度数据(Depth数据),而三维数据的传输是分别传输二维视频数据和深度数据。但是,在进行三维建模的时候由于仅能够获得二维图像信息的上下左右的信息,无法保证三维建模的准确性。With the continuous development of mobile communication networks, the transmission rate of mobile communication networks has increased rapidly, thus providing strong technical support for the generation and development of 3D video services. The three-dimensional data includes two-dimensional image data (for example, RGB data, etc.) and depth data (Depth data), and three-dimensional data is transmitted by transmitting two-dimensional video data and depth data, respectively. However, when performing three-dimensional modeling, since only the upper, lower, left, and right information of the two-dimensional image information can be obtained, the accuracy of the three-dimensional modeling cannot be guaranteed.
发明内容Summary of the Invention
为解决上述技术问题,本发明实施例提供了一种数据处理方法、终端设备、MEC服务器、芯片、计算机可读存储介质、计算机程序产品以及计算机程序。To solve the above technical problems, embodiments of the present invention provide a data processing method, a terminal device, a MEC server, a chip, a computer-readable storage medium, a computer program product, and a computer program.
本申请实施例提供的数据处理方法,应用于MEC服务器,包括:The data processing method provided in the embodiment of the present application, which is applied to a MEC server, includes:
接收终端设备的姿态信息;Receiving attitude information of a terminal device;
基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;Determining a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device;
基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Three-dimensional image modeling is performed based on the shooting angle of the terminal device and the spatial relative position information.
本申请实施例提供的一种数据处理方法,应用于终端设备,包括:A data processing method provided in an embodiment of the present application, which is applied to a terminal device, includes:
采集终端设备的姿态信息;Collect attitude information of the terminal equipment;
向网络侧发送终端设备的姿态信息;以使得所述网络侧基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息,以及基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Sending the attitude information of the terminal device to the network side; so that the network side determines the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and based on the shooting angle and location of the terminal device The spatial relative position information is described, and three-dimensional image modeling is performed.
本申请实施例提供的一种MEC服务器,包括:A MEC server provided in an embodiment of the present application includes:
第一通信单元,用于接收终端设备的姿态信息;A first communication unit, configured to receive attitude information of a terminal device;
第一处理单元,用于基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。A first processing unit, configured to determine a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device, and perform three-dimensional image construction based on the shooting angle of the terminal device and the spatial relative position information mold.
本申请实施例提供的一种终端设备,包括:A terminal device provided in an embodiment of the present application includes:
采集单元,用于采集终端设备的姿态信息;An acquisition unit for acquiring attitude information of the terminal device;
第二通信单元,用于向网络侧发送终端设备的姿态信息。The second communication unit is configured to send attitude information of the terminal device to the network side.
本申请实施例提供的MEC服务器,包括处理器和存储器。该存储器用于存储计算机程序,该处理器用于调用并运行该存储器中存储的计算机程序,执行上述的数据处理方法。The MEC server provided in the embodiment of the present application includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data processing method described above.
本申请实施例提供的终端设备,包括处理器和存储器。该存储器用于存储计算机程序,该处理器用于调用并运行该存储器中存储的计算机程序,执行上述的数据处理方法。The terminal device provided in the embodiment of the present application includes a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data processing method described above.
本申请实施例提供的芯片,用于实现上述的数据处理方法。The chip provided in the embodiment of the present application is used to implement the foregoing data processing method.
具体地,该芯片包括:处理器,用于从存储器中调用并运行计算机程序,使得安装有该芯片的设备执行上述的数据处理方法。Specifically, the chip includes: a processor, configured to call and run a computer program from a memory, so that a device installed with the chip executes the foregoing data processing method.
本申请实施例提供的计算机可读存储介质,用于存储计算机程序,该计算机程序使得计算机执行上述的数据处理方法。The computer-readable storage medium provided in the embodiments of the present application is used to store a computer program, and the computer program causes a computer to execute the foregoing data processing method.
本申请实施例提供的计算机程序产品,包括计算机程序指令,该计算机程序指令使得计算机执行上述的数据处理方法。The computer program product provided in the embodiment of the present application includes computer program instructions, and the computer program instructions cause a computer to execute the foregoing data processing method.
本申请实施例提供的计算机程序,当其在计算机上运行时,使得计算机执行上述的数据处理方法。The computer program provided in the embodiment of the present application, when run on a computer, causes the computer to execute the above-mentioned data processing method.
通过上述技术方案,就能够基于终端设备的姿态信息,确定终端设备的拍摄角度以及空间相对位置信息,进而进行三维图像建模,由于在三维建模的时候增加了拍摄场景的环境因素,因此保证了三维建模的准确性。Through the above technical solution, it is possible to determine the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and then perform 3D image modeling. Since the environmental factors of the shooting scene are added during the 3D modeling, it is guaranteed This improves the accuracy of 3D modeling.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本申请实施例提供的一种系统架构示意图;FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present application; FIG.
图2为本申请实施例提供的数据处理方法的流程示意图一;FIG. 2 is a first schematic flowchart of a data processing method according to an embodiment of the present application; FIG.
图3a为本申请实施例提供的场景示意图一;FIG. 3a is a first schematic scenario diagram provided by an embodiment of the present application; FIG.
图3b为本申请实施例提供的场景示意图二;FIG. 3b is a second schematic view of a scenario provided by an embodiment of the present application; FIG.
图4为本申请实施例提供的数据处理方法的流程示意图二;4 is a second schematic flowchart of a data processing method according to an embodiment of the present application;
图5为本申请实施例提供的MEC服务器的结构组成示意图;FIG. 5 is a schematic structural composition diagram of a MEC server according to an embodiment of the present application; FIG.
图6为本申请实施例提供的终端设备的结构组成示意图;6 is a schematic structural composition diagram of a terminal device according to an embodiment of the present application;
图7为本申请实施例提供的一种通信设备示意性结构图;7 is a schematic structural diagram of a communication device according to an embodiment of the present application;
图8为本申请实施例的芯片的示意性结构图;8 is a schematic structural diagram of a chip according to an embodiment of the present application;
图9为本申请实施例提供的一种通信系统的示意性框图。FIG. 9 is a schematic block diagram of a communication system according to an embodiment of the present application.
具体实施方式detailed description
在对本发明实施例的技术方案进行详细说明之前,首先对本发明实施例的数据处理方法应用的系统架构进行简单说明。本发明实施例的数据处理方法应用于三维视频数据的相关业务,该业务例如是三维视频数据分享的业务,或者基于三维视频数据的直播业务等等。在这种情况下,由于三维视频数据的数据量较大,分别传输的深度数据和二维视频数据在数据传输过程中需要较高的技术支持,因此需要移动通信网络具有较快的数据传输速率,以及较稳定的数据传输环境。Before the technical solution of the embodiment of the present invention is described in detail, a system architecture to which the data processing method of the embodiment of the present invention is applied is briefly described. The data processing method of the embodiment of the present invention is applied to a service related to three-dimensional video data, such as a service for sharing three-dimensional video data, or a live broadcast service based on three-dimensional video data. In this case, due to the large amount of 3D video data, the transmitted depth data and 2D video data require higher technical support in the data transmission process, so the mobile communication network needs a faster data transmission rate. , And a more stable data transmission environment.
图1为本发明实施例的数据处理方法应用的系统架构示意图;如图1所示,系统可包括终端、基站、MEC服务器、业务处理服务器、核心网和互联网(Internet)等;MEC服务器与业务处理服务器之间通过核心网构建高速通道以实现数据同步。FIG. 1 is a schematic diagram of a system architecture to which a data processing method is applied according to an embodiment of the present invention; as shown in FIG. 1, the system may include a terminal, a base station, a MEC server, a service processing server, a core network, and the Internet, etc .; High-speed channels are established between the processing servers through the core network to achieve data synchronization.
以图1所示的两个终端交互的应用场景为例,MEC服务器A为部署于靠近终端A(发送端)的MEC服务器,核心网A为终端A所在区域的核心网;相应的,MEC服务器B为部署于靠近终端B(接收端)的MEC服务器,核心网B为终端B所在区域的核心网;MEC服务器A和MEC服务器B可与业务处理服务器之间分别通过核心网A和核心网B构建高速通道以实现数据同步。Taking the application scenario where two terminals interact as shown in Figure 1 as an example, MEC server A is a MEC server deployed near terminal A (sender), and core network A is the core network in the area where terminal A is located; correspondingly, the MEC server B is a MEC server deployed near terminal B (receiving end), and core network B is the core network in the area where terminal B is located; MEC server A and MEC server B can communicate with the service processing server through core network A and core network B, respectively. Build high-speed channels for data synchronization.
其中,终端A发送的三维视频数据传输到MEC服务器A后,由MEC服务器A通过核心网A将数据同步至业务处理服务器;再由MEC服务器B从业务处理服务器获取终端A发送的三维视频数据,并发送至终端B进行呈现。Among them, after the three-dimensional video data sent by the terminal A is transmitted to the MEC server A, the MEC server A synchronizes the data to the service processing server through the core network A; and the MEC server B obtains the three-dimensional video data sent by the terminal A from the service processing server And send it to terminal B for presentation.
这里,如果终端B与终端A通过同一个MEC服务器来实现传输,此时终端B和终端A直接通过一个MEC服务器实现三维视频数据的传输,不需要业务处理服务器的参与,这种方式称为本地回传方式。具体地,假设终端B与终端A通过MEC服务器A实现三维视频数据的传输,终端A发送的三维视频数据传输到MEC服务器A后,由MEC 服务器A发送三维视频数据至终端B进行呈现。Here, if terminal B and terminal A use the same MEC server to implement transmission, then terminal B and terminal A directly implement three-dimensional video data transmission through one MEC server without the participation of a service processing server. This method is called local Way back. Specifically, it is assumed that the terminal B and the terminal A realize the transmission of three-dimensional video data through the MEC server A. After the three-dimensional video data sent by the terminal A is transmitted to the MEC server A, the three-dimensional video data sent by the MEC server A to the terminal B is presented.
这里,终端可基于网络情况、或者终端自身的配置情况、或者自身配置的算法选择接入4G网络的演进型基站(eNB),或者接入5G网络的下一代演进型基站(gNB),从而使得eNB通过长期演进(Long Term Evolution,LTE)接入网与MEC服务器连接,使得gNB通过下一代接入网(NG-RAN)与MEC服务器连接。Here, the terminal may select an evolved base station (eNB) that accesses a 4G network or a next-generation evolved base station (gNB) that accesses a 5G network based on the network situation, or the configuration of the terminal itself, or an algorithm configured by itself. The eNB is connected to the MEC server through a Long Term Evolution (LTE) access network, so that the gNB is connected to the MEC server through the Next Generation Access Network (NG-RAN).
这里,MEC服务器部署于靠近终端或数据源头的网络边缘侧,所谓靠近终端或者靠近数据源头,不仅是逻辑位置上,还在地理位置上靠近终端或者靠近数据源头。区别于现有的移动通信网络中主要的业务处理服务器部署于几个大城市中,MEC服务器可在一个城市中部署多个。例如在某写字楼中,用户较多,则可在该写字楼附近部署一个MEC服务器。Here, the MEC server is deployed on the edge of the network near the terminal or the source of the data. The so-called near the terminal or the source of the data is not only at the logical location, but also geographically close to the terminal or the source of the data. Different from the main service processing servers in existing mobile communication networks, which are deployed in several large cities, multiple MEC servers can be deployed in one city. For example, in an office building with many users, a MEC server can be deployed near the office building.
其中,MEC服务器作为具有融合网络、计算、存储、应用核心能力的边缘计算网关,为边缘计算提供包括设备域、网络域、数据域和应用域的平台支撑。其联接各类智能设备和传感器,就近提供智能联接和数据处理业务,让不同类型的应用和数据在MEC服务器中进行处理,实现业务实时、业务智能、数据聚合与互操作、安全与隐私保护等关键智能服务,有效提升业务的智能决策效率。Among them, the MEC server, as an edge computing gateway with core capabilities of converged networks, computing, storage, and applications, provides platform support for edge computing including device domains, network domains, data domains, and application domains. It connects various types of smart devices and sensors, and provides smart connection and data processing services nearby, allowing different types of applications and data to be processed in the MEC server, realizing business real-time, business intelligence, data aggregation and interoperation, security and privacy protection, etc. Key intelligent services effectively improve the intelligent decision-making efficiency of the business.
本发明实施例提供一种数据处理方法,应用于MEC服务器,如图2所示,包括:An embodiment of the present invention provides a data processing method, which is applied to a MEC server. As shown in FIG. 2, the method includes:
步骤201:接收终端设备的姿态信息;Step 201: Receive attitude information of a terminal device;
步骤202:基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;Step 202: Determine a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device.
步骤203:基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Step 203: Perform three-dimensional image modeling based on the shooting angle of the terminal device and the spatial relative position information.
这里,所述终端设备为具备摄像头的设备,具体的,本实施例中可以包括有能够采集二维图像信息的摄像头、以及采集深度信息的摄像头。Here, the terminal device is a device provided with a camera. Specifically, this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
前述步骤201中,所述终端设备的姿态信息可以采用陀螺仪采集;得到的姿态信息可以为终端设备的偏转、倾斜时的三个轴向的角度信息;这里需要理解的是,终端设备的三个轴向的角度信息可以世界坐标系为参考。In the foregoing step 201, the attitude information of the terminal device may be collected by using a gyroscope; the obtained attitude information may be angle information of the three axes of the terminal device when it is deflected and tilted; it needs to be understood here that the The angle information of each axis can be referred to the world coordinate system.
前述步骤202中,基于终端设备的姿态信息,确定终端设备的拍摄角度以及空间相对位置信息,可以包括:根据所述终端设备与其采集部件之间的位置关系,确定所述终端设备针对目标对象的拍摄角度;基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信 息。In the foregoing step 202, determining the shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device may include: determining, based on the positional relationship between the terminal device and its acquisition component, the terminal device's A shooting angle; determining spatial relative position information of the terminal device and a target object based on the shooting angle, depth information corresponding to the two-dimensional image information, and posture information of the terminal device.
其中,所述拍摄角度与终端设备及其采集部件之间的位置关系具备对应关系;具体的,所述方法还包括:获取预设的至少一种拍摄角度下,每一种拍摄角度中所述终端设备与其采集部件之间的位置关系。Wherein, the shooting angle has a corresponding relationship with the positional relationship between the terminal device and its collection component; specifically, the method further includes: obtaining at least one preset shooting angle, each shooting angle described in The positional relationship between the terminal device and its acquisition component.
进一步地,终端设备的采集部件可以理解为终端设备与其二维摄像头之间的位置关系;终端设备可以采用后外壳所在的平面进行位置关系的衡量,所述二维摄像头可以用其中轴进行位置关系的衡量,比如,参见图3a,可以使用A切线对终端设备刨切后可以从可以得到右侧的纵切面示意图;右侧的切面采集部件的中轴线可以和终端设备的外壳平面可以具备相对位置关系;进一步地,根据预设的拍摄角度和终端设备及采集部件之间的相对关系,确定终端设备的拍摄角度。Further, the collection component of the terminal device can be understood as the positional relationship between the terminal device and its two-dimensional camera; the terminal device can use the plane on which the rear shell is located to measure the positional relationship, and the two-dimensional camera can use its central axis for positional relationship For example, referring to Figure 3a, you can use the A tangent to slice the terminal equipment to obtain the schematic diagram of the vertical section on the right side; the center axis of the cutting section on the right side can be relative to the plane of the housing of the terminal device. Relationship; further, determining a shooting angle of the terminal device according to a preset shooting angle and a relative relationship between the terminal device and the collecting component.
基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信息,参见图3b,通过拍摄角度以及二维图像信息所对应的深度信息,在基于终端设备的姿态信息,能够确定终端设备与目标对象之间的空间相对位置信息,也就是,基于世界坐标终端设备与目标对象之间的相对位置关系,即能够使得网络设备获取在世界坐标系下的终端设备与目标对象之间的相对位置关系。Based on the shooting angle, the depth information corresponding to the two-dimensional image information, and the attitude information of the terminal device, determine the spatial relative position information of the terminal device and the target object. See FIG. 3b. The depth information corresponding to the image information can determine the spatial relative position information between the terminal device and the target object based on the posture information of the terminal device, that is, based on the relative position relationship between the terminal device and the target object based on world coordinates, that is, It enables the network device to obtain the relative position relationship between the terminal device and the target object in the world coordinate system.
进而,根据终端设备的拍摄角度以及空间相对位置关系,再结合二维图像信息、以及二维图像信息的深度信息,进行三维图像建模,最终能够得到符合当前拍摄场景下的三维图像。Furthermore, according to the shooting angle and the spatial relative position relationship of the terminal device, and combined with the two-dimensional image information and the depth information of the two-dimensional image information, the three-dimensional image modeling is performed, and finally a three-dimensional image consistent with the current shooting scene can be obtained.
可见,通过采用上述方案,就能够基于终端设备的姿态信息,确定终端设备的拍摄角度以及空间相对位置信息,进而进行三维图像建模,由于在三维建模的时候增加了拍摄场景的环境因素,因此保证了三维建模的准确性。It can be seen that by adopting the above solution, it is possible to determine the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and then perform three-dimensional image modeling. Since the environmental factors of the shooting scene are added during the three-dimensional modeling, Therefore, the accuracy of 3D modeling is guaranteed.
本发明实施例提供一种数据处理方法,应用于终端设备,如图4所示,包括:An embodiment of the present invention provides a data processing method, which is applied to a terminal device, as shown in FIG. 4, and includes:
步骤401:采集终端设备的姿态信息;Step 401: Collect attitude information of the terminal device;
步骤402:向网络侧发送终端设备的姿态信息;以使得所述网络侧基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息,以及基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Step 402: Send the attitude information of the terminal device to the network side; so that the network side determines the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and the shooting based on the terminal device. The angle and the spatial relative position information are used to perform three-dimensional image modeling.
这里,所述终端设备为具备摄像头的设备,具体的,本实施例中可以包括有能够采集二维图像信息的摄像头、以及采集深度信息的摄像头。Here, the terminal device is a device provided with a camera. Specifically, this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
前述步骤401中,采集终端设备的姿态信息可以利用陀螺仪检测所述终端设备的姿 态信息。其中,采用陀螺仪采集得到的姿态信息可以为终端设备的偏转、倾斜时的三个轴向的角度信息;这里需要理解的是,终端设备的三个轴向的角度信息可以世界坐标系为参考。In the foregoing step 401, the attitude information of the terminal device may be collected by using a gyroscope to detect the attitude information of the terminal device. Among them, the attitude information collected by the gyroscope can be the angle information of the three axes when the terminal device is deflected and tilted. It should be understood here that the angle information of the three axis directions of the terminal device can be used as a reference for the world coordinate system. .
执行前述方案的同时,还可以包括:While implementing the foregoing scheme, it may also include:
采集针对目标对象的二维图像信息以及其对应的深度信息;Collect two-dimensional image information and corresponding depth information for the target object;
向网络侧发送二维图像信息、以及二维图像信息所对应的深度信息。Send two-dimensional image information and depth information corresponding to the two-dimensional image information to the network side.
所述采集二维图像信息以及所述二维图像信息所对应的深度信息,可以为:通过第一摄像单元对目标对象进行二维图像采集,得到针对目标对象的N帧二维图像信息;N为大于等于1的整数;通过第二摄像单元与第一摄像单元同步采集所述N帧二维图像信息中的每一帧二维图像信息所对应的深度信息;The acquiring the two-dimensional image information and the depth information corresponding to the two-dimensional image information may be: acquiring a two-dimensional image of the target object through the first camera unit to obtain N frames of two-dimensional image information for the target object; N Is an integer greater than or equal to 1; depth information corresponding to each frame of the two-dimensional image information in the N-frame two-dimensional image information is synchronously collected by the second camera unit and the first camera unit;
需要指出的是,第一摄像单元可以为2D摄像头,第二摄像单元可以为深度摄像头;并且,2D摄像头与深度摄像头为同步进行拍摄的,也就是说,通过这两个摄像头采集的二维图像信息以及深度信息可以在时域上相对应;比如,在时刻1采集了二维图像1、并在时刻1采集了深度信息1,即二维图像1以及深度信息1为一一对应的关系,进一步地,当在其他时刻采集了二维图像以及深度信息,也均通过时刻信息能够一一对应。It should be noted that the first camera unit may be a 2D camera, and the second camera unit may be a depth camera; and the 2D camera and the depth camera are synchronized to shoot, that is, a two-dimensional image acquired by the two cameras Information and depth information can correspond in the time domain; for example, two-dimensional image 1 is collected at time 1 and depth information 1 is collected at time 1, that is, there is a one-to-one correspondence between the two-dimensional image 1 and the depth information 1, Further, when two-dimensional images and depth information are collected at other times, they can also correspond one-to-one through the time information.
还需要指出的是,终端设备可以基于TCP协议发送二维图像信息以及主体区域对应的深度信息。It should also be noted that the terminal device can send two-dimensional image information and depth information corresponding to the subject area based on the TCP protocol.
可见,通过采用上述方案,就能够向网络侧发送终端设备的姿态信息,以使得网络侧能够确定终端设备的拍摄角度以及空间相对位置信息,进而进行三维图像建模,由于在三维建模的时候增加了拍摄场景的环境因素,因此保证了三维建模的准确性。It can be seen that by adopting the foregoing solution, the posture information of the terminal device can be sent to the network side, so that the network side can determine the shooting angle and spatial relative position information of the terminal device, and then perform 3D image modeling. The environmental factors of the shooting scene are increased, so the accuracy of 3D modeling is guaranteed.
本发明实施例提供一种MEC服务器,如图5所示,包括:An embodiment of the present invention provides a MEC server, as shown in FIG. 5, including:
第一通信单元51,用于接收终端设备的姿态信息;A first communication unit 51, configured to receive posture information of a terminal device;
第一处理单元52,用于基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。A first processing unit 52, configured to determine a shooting angle of the terminal device and spatial relative position information based on the posture information of the terminal device; and perform a three-dimensional image based on the shooting angle of the terminal device and the spatial relative position information Modeling.
这里,所述终端设备为具备摄像头的设备,具体的,本实施例中可以包括有能够采集二维图像信息的摄像头、以及采集深度信息的摄像头。Here, the terminal device is a device provided with a camera. Specifically, this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
第一处理单元52,用于根据所述终端设备与其采集部件之间的位置关系,确定所述终端设备针对目标对象的拍摄角度;基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信 息。A first processing unit 52, configured to determine a shooting angle of the terminal device for a target object according to a positional relationship between the terminal device and its collecting component; based on the shooting angle and a depth corresponding to the two-dimensional image information Information and posture information of the terminal device, determine spatial relative position information of the terminal device and a target object.
其中,所述拍摄角度与终端设备及其采集部件之间的位置关系具备对应关系;具体的,所述第一处理单元52,用于获取预设的至少一种拍摄角度下,每一种拍摄角度中所述终端设备与其采集部件之间的位置关系。Wherein, there is a corresponding relationship between the shooting angle and the positional relationship between the terminal device and its collection component; specifically, the first processing unit 52 is configured to obtain each shooting at a preset at least one shooting angle. The positional relationship between the terminal device and its acquisition component in the angle.
进一步地,终端设备的采集部件可以理解为终端设备与其二维摄像头之间的位置关系;终端设备可以采用后外壳所在的平面进行位置关系的衡量,所述二维摄像头可以用其中轴进行位置关系的衡量,比如,参见图3a,可以使用A切线对终端设备刨切后可以从可以得到右侧的纵切面示意图;右侧的切面采集部件的中轴线可以和终端设备的外壳平面可以具备相对位置关系;进一步地,根据预设的拍摄角度和终端设备及采集部件之间的相对关系,确定终端设备的拍摄角度。Further, the collection component of the terminal device can be understood as the positional relationship between the terminal device and its two-dimensional camera; the terminal device can use the plane on which the rear shell is located to measure the positional relationship, and the two-dimensional camera can use its central axis to perform the positional relationship. For example, referring to Figure 3a, you can use the A tangent to slice the terminal equipment to obtain the schematic diagram of the vertical section on the right side; the center axis of the cutting section on the right side can be relative to the plane of the housing of the terminal device. Relationship; further, determining a shooting angle of the terminal device according to a preset shooting angle and a relative relationship between the terminal device and the collecting component.
基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信息,参见图3b,通过拍摄角度以及二维图像信息所对应的深度信息,在基于终端设备的姿态信息,能够确定终端设备与目标对象之间的空间相对位置信息,也就是,基于世界坐标终端设备与目标对象之间的相对位置关系,即能够使得网络设备获取在世界坐标系下的终端设备与目标对象之间的相对位置关系。Based on the shooting angle, the depth information corresponding to the two-dimensional image information, and the attitude information of the terminal device, determine the spatial relative position information of the terminal device and the target object. See FIG. 3b. The depth information corresponding to the image information can determine the spatial relative position information between the terminal device and the target object based on the posture information of the terminal device, that is, based on the relative position relationship between the terminal device and the target object based on world coordinates, that is, It enables the network device to obtain the relative position relationship between the terminal device and the target object in the world coordinate system.
进而,根据终端设备的拍摄角度以及空间相对位置关系,再结合二维图像信息、以及二维图像信息的深度信息,进行三维图像建模,最终能够得到符合当前拍摄场景下的三维图像。Furthermore, according to the shooting angle and the spatial relative position relationship of the terminal device, and combined with the two-dimensional image information and the depth information of the two-dimensional image information, the three-dimensional image modeling is performed, and finally a three-dimensional image consistent with the current shooting scene can be obtained.
可见,通过采用上述方案,就能够基于终端设备的姿态信息,确定终端设备的拍摄角度以及空间相对位置信息,进而进行三维图像建模,由于在三维建模的时候增加了拍摄场景的环境因素,因此保证了三维建模的准确性。It can be seen that by adopting the above solution, it is possible to determine the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and then perform three-dimensional image modeling. Since the environmental factors of the shooting scene are added during the three-dimensional modeling, Therefore, the accuracy of 3D modeling is guaranteed.
本发明实施例提供一种终端设备,如图6所示,包括:An embodiment of the present invention provides a terminal device, as shown in FIG. 6, including:
采集单元61,用于采集终端设备的姿态信息;An acquisition unit 61, configured to acquire attitude information of a terminal device;
第二通信单元62,用于向网络侧发送终端设备的姿态信息。The second communication unit 62 is configured to send attitude information of the terminal device to the network side.
这里,所述终端设备为具备摄像头的设备,具体的,本实施例中可以包括有能够采集二维图像信息的摄像头、以及采集深度信息的摄像头。Here, the terminal device is a device provided with a camera. Specifically, this embodiment may include a camera capable of collecting two-dimensional image information and a camera capable of collecting depth information.
前述采集单元61,用于利用陀螺仪检测所述终端设备的姿态信息。其中,采用陀螺仪采集得到的姿态信息可以为终端设备的偏转、倾斜时的三个轴向的角度信息;这里需要理解的是,终端设备的三个轴向的角度信息可以世界坐标系为参考。The foregoing collecting unit 61 is configured to detect a posture information of the terminal device by using a gyroscope. Among them, the attitude information collected by the gyroscope can be the angle information of the three axes when the terminal device is deflected and tilted. It should be understood here that the angle information of the three axis directions of the terminal device can be used as a reference for the world coordinate system. .
执行前述方案的同时,还可以包括:While implementing the foregoing scheme, it may also include:
采集单元61,用于采集针对目标对象的二维图像信息以及其对应的深度信息;An acquisition unit 61, configured to acquire two-dimensional image information and corresponding depth information of a target object;
第二通信单元62,用于向网络侧发送二维图像信息、以及二维图像信息所对应的深度信息。The second communication unit 62 is configured to send two-dimensional image information and depth information corresponding to the two-dimensional image information to the network side.
所述采集二维图像信息以及所述二维图像信息所对应的深度信息,可以为:通过第一摄像单元对目标对象进行二维图像采集,得到针对目标对象的N帧二维图像信息;N为大于等于1的整数;通过第二摄像单元与第一摄像单元同步采集所述N帧二维图像信息中的每一帧二维图像信息所对应的深度信息;The acquiring the two-dimensional image information and the depth information corresponding to the two-dimensional image information may be: acquiring a two-dimensional image of the target object through the first camera unit to obtain N frames of two-dimensional image information for the target object; N Is an integer greater than or equal to 1; depth information corresponding to each frame of the two-dimensional image information in the N-frame two-dimensional image information is synchronously collected by the second camera unit and the first camera unit;
需要指出的是,第一摄像单元可以为2D摄像头,第二摄像单元可以为深度摄像头;并且,2D摄像头与深度摄像头为同步进行拍摄的,也就是说,通过这两个摄像头采集的二维图像信息以及深度信息可以在时域上相对应;比如,在时刻1采集了二维图像1、并在时刻1采集了深度信息1,即二维图像1以及深度信息1为一一对应的关系,进一步地,当在其他时刻采集了二维图像以及深度信息,也均通过时刻信息能够一一对应。It should be noted that the first camera unit may be a 2D camera, and the second camera unit may be a depth camera; and the 2D camera and the depth camera are synchronized to shoot, that is, a two-dimensional image acquired by the two cameras Information and depth information can correspond in the time domain; for example, two-dimensional image 1 is collected at time 1 and depth information 1 is collected at time 1, that is, there is a one-to-one correspondence between the two-dimensional image 1 and the depth information 1, Further, when two-dimensional images and depth information are collected at other times, they can also correspond one-to-one through the time information.
还需要指出的是,终端设备可以基于TCP协议发送二维图像信息以及主体区域对应的深度信息。It should also be noted that the terminal device can send two-dimensional image information and depth information corresponding to the subject area based on the TCP protocol.
可见,通过采用上述方案,就能够向网络侧发送终端设备的姿态信息,以使得网络侧能够确定终端设备的拍摄角度以及空间相对位置信息,进而进行三维图像建模,由于在三维建模的时候增加了拍摄场景的环境因素,因此保证了三维建模的准确性。It can be seen that by adopting the foregoing solution, the posture information of the terminal device can be sent to the network side, so that the network side can determine the shooting angle and spatial relative position information of the terminal device, and then perform 3D image modeling. The environmental factors of the shooting scene are increased, so the accuracy of 3D modeling is guaranteed.
图7是本申请实施例提供的一种通信设备700示意性结构图,需要理解的是,图7中示出的通信设备可以为本实施例中的终端设备或者MEC服务器。图7所示的通信设备700包括处理器710,处理器710可以从存储器中调用并运行计算机程序,以实现本申请实施例中的方法。FIG. 7 is a schematic structural diagram of a communication device 700 according to an embodiment of the present application. It should be understood that the communication device shown in FIG. 7 may be a terminal device or a MEC server in this embodiment. The communication device 700 shown in FIG. 7 includes a processor 710, and the processor 710 may call and run a computer program from a memory to implement the method in the embodiment of the present application.
可选地,如图7所示,通信设备700还可以包括存储器720。其中,处理器710可以从存储器720中调用并运行计算机程序,以实现本申请实施例中的方法。Optionally, as shown in FIG. 7, the communication device 700 may further include a memory 720. The processor 710 may call and run a computer program from the memory 720 to implement the method in the embodiment of the present application.
其中,存储器720可以是独立于处理器710的一个单独的器件,也可以集成在处理器710中。The memory 720 may be a separate device independent of the processor 710, or may be integrated in the processor 710.
可选地,如图7所示,通信设备700还可以包括收发器730,处理器710可以控制该收发器730与其他设备进行通信,具体地,可以向其他设备发送信息或数据,或接收其他设备发送的信息或数据。Optionally, as shown in FIG. 7, the communication device 700 may further include a transceiver 730, and the processor 710 may control the transceiver 730 to communicate with other devices. Specifically, it may send information or data to other devices, or receive other information. Information or data sent by the device.
其中,收发器730可以包括发射机和接收机。收发器730还可以进一步包括天线,天线的数量可以为一个或多个。The transceiver 730 may include a transmitter and a receiver. The transceiver 730 may further include antennas, and the number of antennas may be one or more.
可选地,该通信设备700具体可为本申请实施例的MEC服务器,并且该通信设备700可以实现本申请实施例的各个方法中由MEC服务器实现的相应流程,为了简洁,在此不再赘述。Optionally, the communication device 700 may specifically be the MEC server according to the embodiment of the present application, and the communication device 700 may implement the corresponding process implemented by the MEC server in each method of the embodiment of the present application. For brevity, details are not described herein again. .
可选地,该通信设备700具体可为本申请实施例的移动终端设备/终端设备,并且该通信设备700可以实现本申请实施例的各个方法中由移动终端设备/终端设备实现的相应流程,为了简洁,在此不再赘述。Optionally, the communication device 700 may specifically be a mobile terminal device / terminal device in the embodiment of the present application, and the communication device 700 may implement a corresponding process implemented by the mobile terminal device / terminal device in each method in the embodiments of the present application, For brevity, I will not repeat them here.
图8是本申请实施例的芯片的示意性结构图。图8所示的芯片800包括处理器810,处理器810可以从存储器中调用并运行计算机程序,以实现本申请实施例中的方法。FIG. 8 is a schematic structural diagram of a chip according to an embodiment of the present application. The chip 800 shown in FIG. 8 includes a processor 810, and the processor 810 can call and run a computer program from a memory to implement the method in the embodiment of the present application.
可选地,如图8所示,芯片800还可以包括存储器820。其中,处理器810可以从存储器820中调用并运行计算机程序,以实现本申请实施例中的方法。Optionally, as shown in FIG. 8, the chip 800 may further include a memory 820. The processor 810 may call and run a computer program from the memory 820 to implement the method in the embodiment of the present application.
其中,存储器820可以是独立于处理器810的一个单独的器件,也可以集成在处理器810中。The memory 820 may be a separate device independent of the processor 810, or may be integrated in the processor 810.
可选地,该芯片800还可以包括输入接口830。其中,处理器810可以控制该输入接口830与其他设备或芯片进行通信,具体地,可以获取其他设备或芯片发送的信息或数据。Optionally, the chip 800 may further include an input interface 830. The processor 810 may control the input interface 830 to communicate with other devices or chips. Specifically, the processor 810 may obtain information or data sent by other devices or chips.
可选地,该芯片800还可以包括输出接口840。其中,处理器810可以控制该输出接口840与其他设备或芯片进行通信,具体地,可以向其他设备或芯片输出信息或数据。Optionally, the chip 800 may further include an output interface 840. The processor 810 may control the output interface 840 to communicate with other devices or chips. Specifically, the processor 810 may output information or data to the other devices or chips.
可选地,该芯片可应用于本申请实施例中的MEC服务器,并且该芯片可以实现本申请实施例的各个方法中由MEC服务器实现的相应流程,为了简洁,在此不再赘述。Optionally, the chip may be applied to the MEC server in the embodiments of the present application, and the chip may implement the corresponding processes implemented by the MEC server in the methods of the embodiments of the present application. For brevity, details are not described herein again.
可选地,该芯片可应用于本申请实施例中的移动终端设备/终端设备,并且该芯片可以实现本申请实施例的各个方法中由移动终端设备/终端设备实现的相应流程,为了简洁,在此不再赘述。Optionally, the chip may be applied to the mobile terminal device / terminal device in the embodiments of the present application, and the chip may implement the corresponding process implemented by the mobile terminal device / terminal device in each method of the embodiments of the present application. For simplicity, I will not repeat them here.
应理解,本申请实施例提到的芯片还可以称为系统级芯片,系统芯片,芯片系统或片上系统芯片等。It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip.
图9是本申请实施例提供的一种通信系统900的示意性框图。如图9所示,该通信系统900包括终端设备910和MEC服务器920。FIG. 9 is a schematic block diagram of a communication system 900 according to an embodiment of the present application. As shown in FIG. 9, the communication system 900 includes a terminal device 910 and a MEC server 920.
其中,该终端设备910可以用于实现上述方法中由终端设备实现的相应的功能,以及该MEC服务器920可以用于实现上述方法中由MEC服务器实现的相应的功能为了 简洁,在此不再赘述。The terminal device 910 may be used to implement the corresponding functions implemented by the terminal device in the foregoing method, and the MEC server 920 may be used to implement the corresponding functions implemented by the MEC server in the foregoing method. For brevity, details are not described herein again. .
应理解,本申请实施例的处理器可能是一种集成电路芯片,具有信号的处理能力。在实现过程中,上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器,处理器读取存储器中的信息,结合其硬件完成上述方法的步骤。It should be understood that the processor in the embodiment of the present application may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the foregoing method embodiment may be completed by using an integrated logic circuit of hardware in a processor or an instruction in a form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (Field, Programmable Gate Array, FPGA), or other Programming logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware decoding processor, or may be performed by using a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the foregoing method in combination with its hardware.
可以理解,本申请实施例中的存储器可以是易失性存储器或非易失性存储器,或可包括易失性和非易失性存储器两者。其中,非易失性存储器可以是只读存储器(Read-Only Memory,ROM)、可编程只读存储器(Programmable ROM,PROM)、可擦除可编程只读存储器(Erasable PROM,EPROM)、电可擦除可编程只读存储器(Electrically EPROM,EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory,RAM),其用作外部高速缓存。通过示例性但不是限制性说明,许多形式的RAM可用,例如静态随机存取存储器(Static RAM,SRAM)、动态随机存取存储器(Dynamic RAM,DRAM)、同步动态随机存取存储器(Synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM,SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM,DR RAM)。应注意,本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。It can be understood that the memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), and an electronic memory. Erase programmable read-only memory (EPROM, EEPROM) or flash memory. The volatile memory may be Random Access Memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synchlink DRAM, SLDRAM ) And direct memory bus random access memory (Direct Rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
应理解,上述存储器为示例性但不是限制性说明,例如,本申请实施例中的存储器还可以是静态随机存取存储器(static RAM,SRAM)、动态随机存取存储器(dynamic RAM,DRAM)、同步动态随机存取存储器(synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM,DDR SDRAM)、增强型同 步动态随机存取存储器(enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(synch link DRAM,SLDRAM)以及直接内存总线随机存取存储器(Direct Rambus RAM,DR RAM)等等。也就是说,本申请实施例中的存储器旨在包括但不限于这些和任意其它适合类型的存储器。It should be understood that the foregoing memory is exemplary but not restrictive. For example, the memory in the embodiment of the present application may also be a static random access memory (static RAM, SRAM), a dynamic random access memory (dynamic RAM, DRAM), Synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (Double SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection Dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct RAMbus RAM, DR RAM) and so on. That is, the memories in the embodiments of the present application are intended to include, but not limited to, these and any other suitable types of memories.
本申请实施例还提供了一种计算机可读存储介质,用于存储计算机程序。An embodiment of the present application further provides a computer-readable storage medium for storing a computer program.
可选的,该计算机可读存储介质可应用于本申请实施例中的MEC服务器,并且该计算机程序使得计算机执行本申请实施例的各个方法中由MEC服务器实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer-readable storage medium may be applied to the MEC server in the embodiment of the present application, and the computer program causes the computer to execute the corresponding process implemented by the MEC server in each method in the embodiment of the present application. No longer.
可选地,该计算机可读存储介质可应用于本申请实施例中的移动终端设备/终端设备,并且该计算机程序使得计算机执行本申请实施例的各个方法中由移动终端设备/终端设备实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer-readable storage medium may be applied to the mobile terminal device / terminal device in the embodiments of the present application, and the computer program causes the computer to execute the mobile terminal device / terminal device implemented in each method in the embodiments of the present application. The corresponding process is omitted here for brevity.
本申请实施例还提供了一种计算机程序产品,包括计算机程序指令。An embodiment of the present application further provides a computer program product, including computer program instructions.
可选的,该计算机程序产品可应用于本申请实施例中的MEC服务器,并且该计算机程序指令使得计算机执行本申请实施例的各个方法中由MEC服务器实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer program product can be applied to the MEC server in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the MEC server in each method of the embodiments of the present application. More details.
可选地,该计算机程序产品可应用于本申请实施例中的移动终端设备/终端设备,并且该计算机程序指令使得计算机执行本申请实施例的各个方法中由移动终端设备/终端设备实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer program product can be applied to a mobile terminal device / terminal device in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding method implemented by the mobile terminal device / terminal device in each method in the embodiments of the present application. For the sake of brevity, we will not repeat them here.
本申请实施例还提供了一种计算机程序。The embodiment of the present application also provides a computer program.
可选的,该计算机程序可应用于本申请实施例中的MEC服务器,当该计算机程序在计算机上运行时,使得计算机执行本申请实施例的各个方法中由MEC服务器实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer program may be applied to the MEC server in the embodiment of the present application. When the computer program is run on the computer, the computer is caused to execute the corresponding processes implemented by the MEC server in each method in the embodiment of the present application. , Will not repeat them here.
可选地,该计算机程序可应用于本申请实施例中的移动终端设备/终端设备,当该计算机程序在计算机上运行时,使得计算机执行本申请实施例的各个方法中由移动终端设备/终端设备实现的相应流程,为了简洁,在此不再赘述。Optionally, the computer program may be applied to a mobile terminal device / terminal device in the embodiments of the present application. When the computer program is run on a computer, the computer is caused to execute the methods in the embodiments of the application by the mobile terminal device / terminal. The corresponding process implemented by the device is not repeated here for brevity.
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art may realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者MEC服务器等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,)ROM、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a MEC server, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory) ROM, random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes .
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应所述以权利要求的保护范围为准。The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (21)

  1. 一种数据处理方法,应用于MEC服务器,包括:A data processing method applied to a MEC server includes:
    接收终端设备的姿态信息;Receiving attitude information of a terminal device;
    基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;Determining a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device;
    基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Three-dimensional image modeling is performed based on the shooting angle of the terminal device and the spatial relative position information.
  2. 根据权利要求1所述的方法,其中,所述确定所述终端设备的拍摄角度以及空间相对位置信息时,所述方法还包括:The method according to claim 1, wherein when determining the shooting angle and spatial relative position information of the terminal device, the method further comprises:
    接收终端设备发来的二维图像信息、以及二维图像信息所对应的深度信息。Receive two-dimensional image information and depth information corresponding to the two-dimensional image information from the terminal device.
  3. 根据权利要求2所述的方法,其中,所述基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息,包括:The method according to claim 2, wherein determining the shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device comprises:
    根据所述终端设备与其采集部件之间的位置关系,确定所述终端设备针对目标对象的拍摄角度;Determining a shooting angle of the terminal device for a target object according to a position relationship between the terminal device and its collection component;
    基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信息。Based on the shooting angle, the depth information corresponding to the two-dimensional image information, and the attitude information of the terminal device, determine the spatial relative position information of the terminal device and the target object.
  4. 根据权利要求3所述的方法,其中,所述根据所述终端设备与其采集部件之间的位置关系,以及所述终端设备的姿态信息,确定所述终端设备针对目标对象的拍摄角度之前,所述方法还包括:The method according to claim 3, wherein before determining the shooting angle of the terminal device for the target object based on the positional relationship between the terminal device and its acquisition component, and the posture information of the terminal device, all The method also includes:
    获取预设的至少一种拍摄角度下,每一种拍摄角度中所述终端设备与其采集部件之间的位置关系。Under a preset at least one shooting angle, a positional relationship between the terminal device and its collection component in each shooting angle is obtained.
  5. 一种数据处理方法,应用于终端设备,包括:A data processing method applied to a terminal device includes:
    采集终端设备的姿态信息;Collect attitude information of the terminal equipment;
    向网络侧发送终端设备的姿态信息;以使得所述网络侧基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息,以及基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。Sending the attitude information of the terminal device to the network side; so that the network side determines the shooting angle and spatial relative position information of the terminal device based on the attitude information of the terminal device, and based on the shooting angle and location of the terminal device The spatial relative position information is described, and three-dimensional image modeling is performed.
  6. 根据权利要求5所述的方法,其中,所述方法还包括:The method according to claim 5, wherein the method further comprises:
    采集针对目标对象的二维图像信息以及其对应的深度信息;Collect two-dimensional image information and corresponding depth information for the target object;
    向网络侧发送二维图像信息、以及二维图像信息所对应的深度信息。Send two-dimensional image information and depth information corresponding to the two-dimensional image information to the network side.
  7. 根据权利要求5所述的方法,其中,所述采集终端设备的姿态信息,包括:The method according to claim 5, wherein the collecting posture information of the terminal device comprises:
    利用陀螺仪检测所述终端设备的姿态信息。Using a gyroscope to detect attitude information of the terminal device.
  8. 一种MEC服务器,包括:A MEC server includes:
    第一通信单元,接收终端设备的姿态信息;A first communication unit, receiving posture information of a terminal device;
    第一处理单元,基于所述终端设备的姿态信息,确定所述终端设备的拍摄角度以及空间相对位置信息;基于所述终端设备的拍摄角度以及所述空间相对位置信息,进行三维图像建模。The first processing unit determines a shooting angle and spatial relative position information of the terminal device based on the posture information of the terminal device, and performs three-dimensional image modeling based on the shooting angle of the terminal device and the spatial relative position information.
  9. 根据权利要求8所述的MEC服务器,其中,所述第一通信单元,接收终端设备发来的二维图像信息、以及二维图像信息所对应的深度信息。The MEC server according to claim 8, wherein the first communication unit receives two-dimensional image information and depth information corresponding to the two-dimensional image information from the terminal device.
  10. 根据权利要求9所述的MEC服务器,其中,所述第一处理单元,根据所述终端设备与其采集部件之间的位置关系,确定所述终端设备针对目标对象的拍摄角度;基于所述拍摄角度、所述二维图像信息所对应的深度信息以及所述终端设备的姿态信息,确定所述终端设备与目标对象的空间相对位置信息。The MEC server according to claim 9, wherein the first processing unit determines a shooting angle of the terminal device for the target object according to a positional relationship between the terminal device and its acquisition component; based on the shooting angle 2. Depth information corresponding to the two-dimensional image information and posture information of the terminal device determine spatial relative position information of the terminal device and a target object.
  11. 根据权利要求10所述的MEC服务器,其中,所述第一处理单元,获取预设的至少一种拍摄角度下,每一种拍摄角度中所述终端设备与其采集部件之间的位置关系。The MEC server according to claim 10, wherein the first processing unit acquires a positional relationship between the terminal device and its collection component in each of the preset shooting angles in at least one shooting angle.
  12. 一种终端设备,包括:A terminal device includes:
    采集单元,采集终端设备的姿态信息;An acquisition unit, which collects attitude information of the terminal device;
    第二通信单元,向网络侧发送终端设备的姿态信息。The second communication unit sends posture information of the terminal device to the network side.
  13. 根据权利要求12所述的终端设备,其中,所述采集单元,采集针对目标对象的二维图像信息以及其对应的深度信息;The terminal device according to claim 12, wherein the acquisition unit collects two-dimensional image information for a target object and its corresponding depth information;
    所述第二通信单元,向网络侧发送二维图像信息、以及二维图像信息所对应的深度信息。The second communication unit sends two-dimensional image information and depth information corresponding to the two-dimensional image information to the network side.
  14. 根据权利要求12所述的终端设备,其中,所述采集单元,利用陀螺仪检测所述终端设备的姿态信息。The terminal device according to claim 12, wherein the acquisition unit uses a gyroscope to detect attitude information of the terminal device.
  15. 一种MEC服务器,包括:处理器和存储器,该存储器用于存储计算机程序,所述处理器用于调用并运行所述存储器中存储的计算机程序,执行如权利要求1至4中任一项所述的方法。A MEC server includes a processor and a memory, where the memory is configured to store a computer program, and the processor is configured to call and run the computer program stored in the memory, and execute the method according to any one of claims 1 to 4. Methods.
  16. 一种终端设备,包括:处理器和存储器,该存储器用于存储计算机程序,所述处理器用于调用并运行所述存储器中存储的计算机程序,执行如权利要求5至7中任一项所述的方法。A terminal device includes a processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory, and execute the method according to any one of claims 5 to 7. Methods.
  17. 一种芯片,包括:处理器,用于从存储器中调用并运行计算机程序,使得安装 有所述芯片的设备执行如权利要求1至4中任一项所述的方法。A chip includes a processor for calling and running a computer program from a memory, so that a device having the chip installed performs the method according to any one of claims 1 to 4.
  18. 一种芯片,包括:处理器,用于从存储器中调用并运行计算机程序,使得安装有所述芯片的设备执行如权利要求5至7中任一项所述的方法。A chip includes a processor for calling and running a computer program from a memory, so that a device having the chip installed executes the method according to any one of claims 5 to 7.
  19. 一种计算机可读存储介质,用于存储计算机程序,所述计算机程序使得计算机执行如权利要求1至7中任一项所述的方法。A computer-readable storage medium for storing a computer program that causes a computer to perform the method according to any one of claims 1 to 7.
  20. 一种计算机程序产品,包括计算机程序指令,该计算机程序指令使得计算机执行如权利要求1至7中任一项所述的方法。A computer program product comprising computer program instructions that cause a computer to perform the method according to any one of claims 1 to 7.
  21. 一种计算机程序,所述计算机程序使得计算机执行如权利要求1至7中任一项所述的方法。A computer program that causes a computer to perform the method according to any one of claims 1 to 7.
PCT/CN2019/090328 2018-09-30 2019-06-06 Data processing method, mec server and terminal device WO2020062919A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811162079.2A CN109413409B (en) 2018-09-30 2018-09-30 Data processing method, MEC server and terminal equipment
CN201811162079.2 2018-09-30

Publications (1)

Publication Number Publication Date
WO2020062919A1 true WO2020062919A1 (en) 2020-04-02

Family

ID=65466690

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/090328 WO2020062919A1 (en) 2018-09-30 2019-06-06 Data processing method, mec server and terminal device

Country Status (2)

Country Link
CN (1) CN109413409B (en)
WO (1) WO2020062919A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109413409B (en) * 2018-09-30 2020-12-22 Oppo广东移动通信有限公司 Data processing method, MEC server and terminal equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103517061A (en) * 2013-09-03 2014-01-15 展讯通信(上海)有限公司 Method and device for display control of terminal device
CN106296801A (en) * 2015-06-12 2017-01-04 联想(北京)有限公司 A kind of method setting up object three-dimensional image model and electronic equipment
CN108053435A (en) * 2017-11-29 2018-05-18 深圳奥比中光科技有限公司 Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device
US20180253894A1 (en) * 2015-11-04 2018-09-06 Intel Corporation Hybrid foreground-background technique for 3d model reconstruction of dynamic scenes
CN109272576A (en) * 2018-09-30 2019-01-25 Oppo广东移动通信有限公司 A kind of data processing method, MEC server, terminal device and device
CN109413409A (en) * 2018-09-30 2019-03-01 Oppo广东移动通信有限公司 A kind of data processing method, MEC server, terminal device

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104641399B (en) * 2012-02-23 2018-11-23 查尔斯·D·休斯顿 System and method for creating environment and for location-based experience in shared environment
CN103047969B (en) * 2012-12-07 2016-03-16 北京百度网讯科技有限公司 By method and the mobile terminal of mobile terminal generating three-dimensional figures picture
US10291845B2 (en) * 2015-08-17 2019-05-14 Nokia Technologies Oy Method, apparatus, and computer program product for personalized depth of field omnidirectional video
CN105898271A (en) * 2015-12-28 2016-08-24 乐视致新电子科技(天津)有限公司 360-degree panoramic video playing method, playing module and mobile terminal
CN105787988B (en) * 2016-03-21 2021-04-13 联想(北京)有限公司 Information processing method, server and terminal equipment
US10531068B2 (en) * 2016-04-28 2020-01-07 Sony Corporation Information processing device, information processing method, and three-dimensional image data transmission method
CN106023302B (en) * 2016-05-06 2020-06-09 武汉雄楚高晶科技有限公司 Mobile communication terminal, server and method for realizing three-dimensional reconstruction
CN106204630B (en) * 2016-08-19 2019-03-12 浙江宇视科技有限公司 A kind of method and device configuring video camera
CN108230437B (en) * 2017-12-15 2021-11-09 深圳市商汤科技有限公司 Scene reconstruction method and apparatus, electronic device, program, and medium
CN108182726A (en) * 2017-12-29 2018-06-19 努比亚技术有限公司 Three-dimensional rebuilding method, cloud server and computer readable storage medium
CN108495112B (en) * 2018-05-10 2020-12-22 Oppo广东移动通信有限公司 Data transmission method, terminal and computer storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103517061A (en) * 2013-09-03 2014-01-15 展讯通信(上海)有限公司 Method and device for display control of terminal device
CN106296801A (en) * 2015-06-12 2017-01-04 联想(北京)有限公司 A kind of method setting up object three-dimensional image model and electronic equipment
US20180253894A1 (en) * 2015-11-04 2018-09-06 Intel Corporation Hybrid foreground-background technique for 3d model reconstruction of dynamic scenes
CN108053435A (en) * 2017-11-29 2018-05-18 深圳奥比中光科技有限公司 Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device
CN109272576A (en) * 2018-09-30 2019-01-25 Oppo广东移动通信有限公司 A kind of data processing method, MEC server, terminal device and device
CN109413409A (en) * 2018-09-30 2019-03-01 Oppo广东移动通信有限公司 A kind of data processing method, MEC server, terminal device

Also Published As

Publication number Publication date
CN109413409A (en) 2019-03-01
CN109413409B (en) 2020-12-22

Similar Documents

Publication Publication Date Title
US10841537B2 (en) Spatially faithful telepresence supporting varying geometries and moving users
CN108495112B (en) Data transmission method, terminal and computer storage medium
US11861797B2 (en) Method and apparatus for transmitting 3D XR media data
Younis et al. Latency-aware hybrid edge cloud framework for mobile augmented reality applications
EP3629235B1 (en) Method for processing data, server and computer storage medium
CN109272576B (en) Data processing method, MEC server, terminal equipment and device
WO2020063170A1 (en) Data processing method, terminal, server and storage medium
JP2017229067A (en) Method and apparatus for creating pair of stereoscopic images using at least one lightfield camera
WO2020062919A1 (en) Data processing method, mec server and terminal device
WO2020063171A1 (en) Data transmission method, terminal, server and storage medium
WO2020063168A1 (en) Data processing method, terminal, server and computer storage medium
CN109413405B (en) Data processing method, terminal, server and computer storage medium
CN109246408B (en) Data processing method, terminal, server and computer storage medium
CN105933651B (en) Method and apparatus based on target route jumper connection video
CN108632376B (en) Data processing method, terminal, server and computer storage medium
CN109147043B (en) Data processing method, server and computer storage medium
CN109151435B (en) Data processing method, terminal, server and computer storage medium
KR101788005B1 (en) Method for generating multi-view image by using a plurality of mobile terminal
CN109246409B (en) Data processing method, terminal, server and computer storage medium
CN109389674B (en) Data processing method and device, MEC server and storage medium
CN109299323B (en) Data processing method, terminal, server and computer storage medium
WO2021200226A1 (en) Information processing device, information processing method, and program
CN104581122A (en) Three-dimensional image generating method and device, system and mobile terminals
CN108737807B (en) Data processing method, terminal, server and computer storage medium
RU2800627C2 (en) Method of data processing, server and computer storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19866762

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19866762

Country of ref document: EP

Kind code of ref document: A1