WO2018188499A1 - 图像、视频处理方法和装置、虚拟现实装置和存储介质 - Google Patents

图像、视频处理方法和装置、虚拟现实装置和存储介质 Download PDF

Info

Publication number
WO2018188499A1
WO2018188499A1 PCT/CN2018/081699 CN2018081699W WO2018188499A1 WO 2018188499 A1 WO2018188499 A1 WO 2018188499A1 CN 2018081699 W CN2018081699 W CN 2018081699W WO 2018188499 A1 WO2018188499 A1 WO 2018188499A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
background
area
background image
video
Prior art date
Application number
PCT/CN2018/081699
Other languages
English (en)
French (fr)
Inventor
董霙
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2018188499A1 publication Critical patent/WO2018188499A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality

Definitions

  • the present application relates to the field of video images and, more particularly, to image processing methods and apparatus, video processing methods and apparatus, and virtual reality devices.
  • Virtual Reality (VR) technology is a computer simulation system that can create and experience virtual worlds. It uses computer to generate a simulation environment. It is a multi-source information fusion, interactive 3D dynamic vision and entity. System simulation of behaviors that immerse users in the environment.
  • the present application provides a method, device and device for image processing and a method, device and device for video processing, which can reduce the processing load of the VR device.
  • An embodiment of the present application provides an image processing method, including:
  • a background image which is a spherical panoramic image or a cubic panoramic image
  • the image to be played is a spherical panoramic image or a cubic panoramic image
  • the image to be played includes a first area and a second area
  • the first area includes Based on the pixels obtained from the background image
  • the second region includes pixels obtained from the target image.
  • An embodiment of the present application provides an image processing apparatus including: a processor and a memory, the processor executing machine readable instructions in the memory, for
  • the background image being a spherical panoramic image or a cubic panoramic image
  • the target image being a non-panoramic image
  • the image to be played is a spherical panoramic image or a cubic panoramic image
  • the image to be played includes a first area and a second area
  • the first region includes pixels obtained from the background image
  • the second region includes pixels obtained according to the target image.
  • An embodiment of the present application provides a video processing method, where the method includes:
  • Obtaining a background video comprising at least one frame image, each frame image in the background video comprising: a spherical panoramic image or a cube panoramic image;
  • Obtaining a target video where the target video includes at least one frame image, and each frame image in the target video includes: a non-panoramic image;
  • each frame of the to-be-played video includes: a spherical panoramic image or a cubic panoramic image, each frame image of the to-be-played video A first area and a second area are included, the first area including pixels in the background video, the second area including pixels obtained according to the target video.
  • An embodiment of the present application provides a video processing apparatus, including: a processor and a memory, where the processor executes machine readable instructions in the memory, where
  • the background video comprising at least one frame image
  • each frame image in the background video comprising: a spherical panoramic image or a cube panoramic image
  • the target video comprising at least one frame image
  • Each frame of image in the video includes: a non-panoramic image
  • each frame of the to-be-played video includes: a spherical panoramic image or a cubic panoramic image, in the to-be-played video
  • Each frame image includes a first region including pixels obtained from the background video and a second region including pixels obtained from the target video.
  • An embodiment of the present application provides a virtual reality device, including: a processing unit, configured to perform each step in the foregoing method, to obtain an image to be played, or to acquire a video to be played; and a display unit, configured to present the image to be played or The video to be played.
  • An embodiment of the present application provides a non-volatile storage medium for storing machine readable instructions, and when the machine readable instructions are executed, performing the steps in the method provided by the embodiments of the present application.
  • FIG. 1 is a logical configuration diagram of an example of a processing node capable of executing a method of image processing or a method of video processing according to an embodiment of the present invention.
  • FIG. 2 is a logical configuration diagram of another example of a processing node capable of executing the image processing method or the video processing method of the embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of a method for image processing according to an embodiment of the present invention.
  • FIG. 4 is a schematic flow chart of a synthesizing process of an embodiment of the present invention.
  • Fig. 5 is a schematic diagram showing an example of a background image of an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of an example of a first extraction template according to an embodiment of the present invention.
  • Fig. 7 is a schematic diagram showing an example of a target image according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an example of an image to be stitched according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram showing an example of an image to be played back according to an embodiment of the present invention.
  • FIG. 10 is an effect diagram of an image to be played after being played by a VR device according to an embodiment of the present invention.
  • FIG. 11 is a schematic block diagram of an apparatus for image processing according to an embodiment of the present invention.
  • Figure 12 is a schematic block diagram of a processing unit of an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of an apparatus for image processing according to an embodiment of the present invention.
  • FIG. 14 is a schematic flowchart of a method for video processing according to an embodiment of the present invention.
  • FIG. 15 is a schematic block diagram of an apparatus for video processing according to an embodiment of the present invention.
  • FIG. 16 is a schematic structural diagram of an apparatus for video processing according to an embodiment of the present invention.
  • 17 is a schematic configuration diagram of a terminal device to which a method of image processing or a method of video processing according to an embodiment of the present invention is applied.
  • FIG. 18 is a schematic block diagram of a VR device according to an embodiment of the present invention.
  • FIG. 19 is a schematic block diagram of a VR device according to an embodiment of the present invention.
  • the processing node may be a computing device having a graphics processing capability, such as a terminal device, a personal computer, or a server, which is not specifically limited in the embodiment of the present invention.
  • a panoramic image (or a panoramic video composed of multi-frame panoramic images) is one of the main means of implementing a VR scene.
  • the panoramic image refers to the surrounding environment as much as possible by means of, for example, a wide-angle representation and painting, photographs, videos, three-dimensional models, and the like.
  • the panoramic image can be completed by means of a panoramic camera, but the price of the panoramic camera is higher.
  • the panoramic image can also be generated by means of a computer modeling process, but the process has a higher requirement for the graphics processing capability of the computer.
  • VR virtual cinema technology
  • users wear VR devices, they will be placed in a virtual theater.
  • the front and rear of the user's field of vision will show the completed modeling.
  • a playback screen Directly in front is a playback screen, and the video selected by the user will be played on the screen.
  • the method of realizing VR by using virtual cinema technology mainly by making a virtual background image as a panoramic image, for example, a panoramic image of a theater, playing a real video while playing a virtual background image, and overlaying the real video at a fixed position of the virtual background image, A realistic image is viewed in a virtual place by similarly changing the skin of the player.
  • the VR device needs to simultaneously perform display video playback and panoramic image playback. Therefore, the VR device has a large processing load and requires high VR devices.
  • the hardware layer of the processing node includes a central processing unit (CPU) and/or a graphics processing unit (GPU).
  • CPU central processing unit
  • GPU graphics processing unit
  • the hardware layer that can also handle nodes can also include memory, input/output devices, memory, memory controllers, network interfaces, and the like.
  • the input device may include a keyboard, a mouse, a touch screen, and the like.
  • the output device may include a display device such as a liquid crystal display (LCD), a cathode ray tube (Cathode Ray Tube) display, a holographic display (Project), or the like.
  • a display device such as a liquid crystal display (LCD), a cathode ray tube (Cathode Ray Tube) display, a holographic display (Project), or the like.
  • the core library is the core part of the operating system, including input/output services, core services, graphics device interfaces, and graphics engine (Graphics Engine) for CPU and GPU graphics processing.
  • the graphics engine may include a 2D engine, a 3D engine, a composition, a frame buffer, and the like.
  • the terminal further includes a driving layer, a frame layer, and an application layer.
  • the driver layer may include a CPU driver, a GPU driver, a display controller driver, and the like.
  • the framework layer may include a graphic service (Graphic Service), a system service (System service), a web service (Web Service), and a customer service (Customer Service); and the graphic service may include, for example, a widget (widget) or a canvas (Canvas). , Views, Render Script, and more.
  • the application layer may include a desktop, a media player, a browser, and the like.
  • the method for graphics processing provided by the embodiment of the present invention is applied to a computing node, where the hardware layer of the computing node may include a processor (eg, a CPU and/or a GPU), a display controller (Display Controller), Hardware such as memory, memory controller, input device, display device, etc.
  • the Kernel Library can include an Input/Output Service (I/O Service), a Kernel Service, and a Graphic Engine.
  • FIG. 1 is only an exemplary description of the logical structure of the execution body of the method of the embodiment of the present invention, and the present invention is not particularly limited as long as the image processing method of the embodiment of the present invention can be executed. can.
  • the execution subject of the method of the embodiment of the present invention may also be a virtual reality system
  • FIG. 2 shows an example of a virtual reality system according to an embodiment of the present invention.
  • the hardware layer of the virtual reality system includes a central processing unit (CPU), that is, a central processing unit and a graphics processing unit (GPU), that is, a graphics processor, etc., and of course, may also include a memory, such as a memory, an input/ Output device, memory, memory controller, network interface, etc.
  • CPU central processing unit
  • GPU graphics processing unit
  • memory such as a memory, an input/ Output device, memory, memory controller, network interface, etc.
  • the input/output device may also be referred to as an interactive device.
  • the input device may include a keyboard, a mouse, a touch screen, a voice recognition device, a motion sensor, etc.
  • the motion sensor may also be referred to as a motion capture device for detecting a gesture operation of the user, etc.
  • the motion capture device can include an eye tracker, a force feedback device, a data glove, a joystick, a tactile feedback device, and the like.
  • the output device can include a display device, such as a 3D display system, a large projection system, a head-mounted (head-mounted stereoscopic display, etc.), and the like.
  • a display device such as a 3D display system, a large projection system, a head-mounted (head-mounted stereoscopic display, etc.), and the like.
  • An operating system for example, a system such as Android
  • some applications can be run on top of the hardware layer.
  • the display device may be a virtual reality display, wherein the virtual reality display may also be referred to as a head-mounted display, which utilizes the left and right eyes of the person to obtain information differences, and guides the user to generate a kind of A head-mounted stereoscopic display that feels in a virtual environment.
  • the display principle is that the left and right eye screens respectively display the images of the left and right eyes, and the human eye obtains such a difference information and generates a stereoscopic effect in the mind.
  • virtual reality head display has the characteristics of small size and strong sealing, and has wide application in military training, virtual driving, virtual city and other projects.
  • the display device may be a binocular omnidirectional display (BOOM), which is a stereoscopic display device coupled to the head and is a special head display device.
  • BOOM binocular omnidirectional display
  • Using BOOM is similar to using a telescope to bundle two separate CRT displays, supported by two mutually perpendicular robotic arms, which not only allows the user to manipulate the display by hand in a spherical space with a radius of, for example, 2 meters.
  • the position can also be used to balance the weight of the display so that it is always level and unaffected by platform movement. There is a position tracker at each node on the support arm.
  • the display device may be a CRT terminal, for example, a liquid crystal shutter glass.
  • the working principle of the stereo vision system is that two computers respectively generate two images of left and right eyes, and after being synthesized, Displayed on the CRT terminal in a time-sharing manner.
  • the user wears a pair of liquid crystal shutter glasses connected to the computer. Under the action of the driving signal, the glasses will be alternately opened and closed at a synchronous rate with the image display. That is, when the computer displays the left eye image, the right eye lens will be Shield, when the right eye image is displayed, the left eye lens is shielded.
  • the human visual physiology system can automatically combine the two inspection images into one stereo image.
  • the display device may be a cave-type (CAVE) projection system
  • the CAVE projection system is a highly immersive virtual demonstration environment composed of three or more surface (including three sides) rigid back projection walls.
  • the three-dimensional tracker With the three-dimensional tracker, the user can touch the virtual three-dimensional object in close proximity to the system surrounded by the projection wall, or roam the "real" virtual environment at will.
  • CAVE systems are generally used in high standard virtual reality systems.
  • the CAVE projection system is a room-based projection visual collaboration environment based on multi-channel visual synchronization technology and stereoscopic display technology.
  • the system can provide a minimum three-sided or maximum seventy-cube cube projection display space for room size for multiple participants. All participants are completely immersed in an advanced virtual simulation environment surrounded by stereoscopic projections, with corresponding virtual reality interaction devices (such as data gloves, position trackers, etc.) to achieve an immersive high resolution 3D stereoscopic video and 6 degrees of freedom interactive experience.
  • virtual reality interaction devices such as data gloves,
  • the hardware layer of the virtual reality system may further include a modeling device (such as a 3D scanner).
  • 3D scanner also known as 3D scanner, 3D scanner, is a high-tech product that integrates light, machine, electricity and computer technology. It is mainly used to obtain the three-dimensional coordinates of the outer surface of the object and the three-dimensional digital model of the object.
  • the equipment can be used not only in the fields of reverse engineering, rapid prototyping, three-dimensional inspection (machine vision measurement), but also with the continuous development of 3D scanning technology, such as 3D film and television animation, digital exhibition hall, clothing tailoring, computer More and more industries such as virtual reality simulation and visualization have begun to use 3D scanners as a convenient means to create digital models of physical objects.
  • the accurate 3D point cloud data of the physical surface is obtained, and finally the digital model of the real object is generated, which is fast and accurate, and can almost completely replicate any object in the real world. Realistically reproduce the real world in a digital form.
  • the core library is a core part of the operating system, including a appearance manager, a media framework, a relational database, a 2G graphics engine library, a web browser engine, a kernel library, and The virtual machine or the like, wherein the identification of the application scenario and the determination of the power consumption control policy described in the embodiment of the present invention may be implemented at a virtual machine, that is, the virtual machine acquires and analyzes the running application on the terminal device to obtain The feature data of the application; the application scene information corresponding to the feature data of the application is determined from the scene feature data set according to the feature data of the application, and the scene feature data set includes a plurality of application scenario information and Corresponding relationship of the feature data of the plurality of applications, wherein the application scenario information corresponding to the feature data of the application is used to indicate an application scenario in which the terminal device is currently used, and the slave control according to the application scenario information Determining, by the policy, a power consumption control policy corresponding to the
  • the terminal device further includes a driving layer, a frame layer, and an application layer.
  • the driver layer may include a CPU driver, a GPU driver, a display controller driver, and the like.
  • the framework layer may include a browser engine, a typesetting engine, a file parser, etc.; the application layer may include a plurality of applications such as a home, a media player, and a browser.
  • FIG. 3 shows a schematic flow of a method 100 of image processing according to an embodiment of the present invention. As shown in FIG. 3, the method 100 includes:
  • S120 Acquire a target image, where the target image is a non-panoramic image
  • the image to be played is a spherical panoramic image or a cubic panoramic image
  • the image to be played includes a first area and a second area, the first The area includes pixels obtained from the background image, the second area including pixels obtained from the target image.
  • the first area includes a pixel obtained according to the background image, that is, the first area includes pixels acquired from the background image.
  • the second region includes a pixel obtained from the target image, that is, the second region includes pixels obtained from the target image.
  • the processing node may acquire a background image for presenting a virtual background (or a virtual scene).
  • the background image may be a panoramic image that the VR device (eg, VR helmet, VR glasses, VR browser, etc.) can recognize and play (or render).
  • the VR device eg, VR helmet, VR glasses, VR browser, etc.
  • the panoramic image may refer to image information of an entire scene captured by a professional camera or an image obtained by using modeling software, and the target image and the background image are synthesized by using software to obtain an image to be played.
  • the VR device to play the to-be-played image
  • the plane photo or the computer-modeled picture can be converted into a 360-degree view for virtual reality browsing, and the two-dimensional floor plan is simulated into a real three-dimensional space and presented to the viewer.
  • the panoramic image refers to an image whose viewing angle is larger than the human eye (or, in other words, the normal effective viewing angle of the human eye).
  • the range of the angle of view of the panoramic image in the horizontal direction (or the viewing angle) is larger than the normal horizontal angle of view of the human eye (for example, 90 degrees)
  • the range of the angle of view of the panoramic image in the vertical direction is larger than the normal vertical angle of the human eye (for example, 70 degrees).
  • the panoramic image may comprise a spherical panoramic image.
  • a spherical coordinate system may be defined, that is, the spherical coordinate system includes a plurality of coordinates distributed over longitude and latitude.
  • a plane coordinate system (or a planar grid) may be defined, the plane coordinate system including a plurality of coordinates distributed in the horizontal and vertical directions.
  • a spherical panorama may mean that the longitude and latitude coordinates of the sphere are directly to the horizontal and vertical coordinate planes (or grids in a planar grid).
  • the height of the planar mesh may be twice as wide, or the width of the spherical panoramic image (ie, the size of the horizontal row) and the height (ie, the vertical direction)
  • the size of the line can be 2:1.
  • ratios of the spherical panoramic images enumerated above are merely illustrative, and the present invention is not limited thereto, and other ratios capable of forming spherical panoramic images fall within the scope of the present invention.
  • a spherical panoramic image may refer to a panoramic image capable of achieving a 360 degree panorama of the entire horizontal direction and a 360 panorama of the vertical direction.
  • the panoramic view of the spherical panoramic image in the horizontal direction may not be a 360-degree panoramic view, for example,
  • the panorama of the spherical panoramic image in the horizontal direction may also be, for example, a 180 degree panorama.
  • the panoramic view of the spherical panoramic image in the vertical direction may not be a 360-degree panoramic view.
  • the panoramic view of the spherical panoramic image in the vertical direction may also be, for example, a 180-degree panoramic view.
  • the definition of the spherical panorama in the embodiment of the present invention may also be similar to the prior art, and a detailed description thereof will be omitted herein to avoid redundancy.
  • FIG. 5 shows an example of a background image as a spherical panoramic image, and the background image shown in FIG. 5 can make the user feel in the movie theater after being played (or presented) by the VR device.
  • the panoramic image may also include a cube panoramic image.
  • the cube panorama may refer to dividing the panorama into six front, back, left, and right sides.
  • the six faces are combined into one closed space to realize the whole horizontal and vertical.
  • Straight 360 degree panoramic scene may refer to dividing the panorama into six front, back, left, and right sides.
  • the background image may be obtained by photographing by an imaging device for capturing a panoramic image.
  • the background image can also be obtained by three-dimensional virtual modeling by a computer.
  • the background image may also be a VR image, where the VR image refers to a visual, audible or touchable transmission to the user's sensory organ tip after being played by the VR device. Things, they seem to come from three-dimensional space around the user.
  • the VR image may refer to a panoramic image having a length to width ratio (or a ratio in the horizontal direction and the vertical direction) of 2:1.
  • the panoramic image may also be referred to as a VR panoramic image.
  • the background image comprises a background area and a content area.
  • the size and location of the background area may be preset (eg, by a user or an administrator) and is an area other than the content area in the background image.
  • the size and location of the content area may be preset (eg, by a user or an administrator).
  • the background area can present a virtual background after being played by the VR device.
  • the position of the content area in the background image may have a corresponding relationship with the position of the content of the target image after being played by the VR device in the virtual background.
  • the content area can make an area corresponding to the movie screen.
  • the processing node may acquire a target image, where the target image is a non-panoramic image.
  • the non-panoramic image refers to an image that can be presented by a normal playback device (non-VR device, for example, a display screen of a mobile phone or a personal computer) and recognized by the human eye, or a non-panoramic image refers to: the viewing angle is less than or An image equal to the human eye (or, in other words, the normal effective viewing angle of the human eye).
  • the range of viewing angles (or viewing angles) of the non-panoramic image in the horizontal direction is less than or equal to the normal horizontal viewing angle of the human eye (for example, 90 degrees), and the viewing angle range of the non-panoramic image in the vertical direction is smaller than or normal to the human eye.
  • Perspective for example, 70 degrees.
  • Fig. 7 shows an example of a target image of the present invention.
  • the processing node may perform a synthesis process on the target image and the background image to generate an image to be played.
  • the background image includes a background area and a content area
  • the synthesis process is performed on the target image and the background image, including:
  • the spliced image and the background image are subjected to a compositing process, wherein the first region includes pixels in the background region, and the second region includes pixels in the image to be spliced.
  • the processing node may determine the content area from the background image.
  • the processing node may determine the content area according to any one of the following manners.
  • the content area of the background image is determined, including:
  • a content area of the background image is determined according to the first extraction model.
  • the first extraction template may also be referred to as an annotation file.
  • the first extraction model may be a user configuration or a server generated, and the first extraction model may indicate (or mark) a range of the content area of the background image in the background image, specifically, the first extraction.
  • the model may indicate the size and shape of the content area of the background image, as well as the location of the content area in the background image.
  • the obtaining the first extraction model comprises:
  • each of the M image sets includes at least one panoramic image, each panoramic image including a content region, the M extraction models
  • Each of the extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image collection
  • the extraction model corresponding to the first image set is used as the first extraction model.
  • a plurality of extraction models may be saved in the processing node, and a mapping relationship between the plurality of extraction models and the plurality of image collections may be saved in the processing node, where Each image collection may include at least one panoramic image (or VR panoramic image, spherical panoramic image, or cube panoramic image).
  • the processing node can determine the set of images to which the background image belongs (ie, the first set of images).
  • the source of the background image the name of the background image, or the format of the background image.
  • a plurality of panoramic images may be divided into a plurality of image sets according to a source, so that images having the same source are obtained. Belongs to the same collection.
  • the processing node can determine the set of images to which the background image belongs (i.e., the first set of images) based on the source of the background image.
  • source may refer to a domain name or a web address of a website or server that publishes an image, or “source” may refer to a name of an operator or manufacturer that publishes an image. .
  • a plurality of panoramic images may be divided into a plurality of image collections by name to have the same name (or name) Images that include the same string) belong to the same collection.
  • the processing node can determine the image set to which the background image belongs (ie, the first image set) according to the name of the background image.
  • the “name” may be the entire name of the image, or the “name” may also refer to the partial name of the image, which is not particularly limited in the present invention.
  • a plurality of panoramic images may be divided into a plurality of image sets according to a format, so that images of the same format belong to the same Collection.
  • the processing node can determine the set of images to which the background image belongs (ie, the first set of images) according to the format of the background image.
  • the “format” may refer to the encoding mode of the image, or the “format” may also refer to the saving format of the image, which is not particularly limited in the present invention.
  • the processing node may search for the image set corresponding to the first image set according to the mapping relationship, and use the extracted model corresponding to the first image set as the first extraction model.
  • the processing node can determine the content area of the background image according to the first extraction model.
  • FIG. 6 shows an example of a first extraction template according to an embodiment of the present invention.
  • the first extraction template may be, for example, a black and white binary image, and the size of the first extraction template or , the aspect ratio can be the same as the size of the background image, wherein the white portion of the first extraction template (or the region occupied by the pixel with the pixel value of 255) corresponds to the background region of the background image, and the black of the first extraction template The portion (or the area occupied by the pixel whose pixel value is 0) corresponds to the background area of the background image.
  • the relative position of the background area in the background image is the same as the relative position of the white portion in the first extraction template.
  • the processing node can determine the region in the background image corresponding to the position of the white portion in the first extraction template as the background region.
  • the relative position of the content area in the background image is the same as the relative position of the black portion in the first extraction template.
  • the processing node can determine the region in the background image corresponding to the position of the black portion in the first extraction template as the content region.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and
  • the content area of the background image is determined, including:
  • a content region of the background image is determined.
  • the pixel value of the pixel of the background area adjacent to the content area may be greater than or equal to a first threshold, and the content area is The pixel values of the pixels included in the adjacent edges of the background region (ie, the second boundary region) may be less than the first threshold.
  • the server or user can input the first threshold to the processing node.
  • the processing node can identify the first boundary area and the second boundary area according to the first threshold, that is, identify the boundary between the background area and the content area, and further, the processing node can identify the content area.
  • the method for determining the content area of the background image by the processing nodes listed above is only an exemplary description, and the present invention is not limited thereto.
  • the pixel values of all the pixels in the background area may be Greater than or equal to the first threshold, and the pixel values of all pixels in the content region may be less than the first threshold.
  • the processing node can recognize an area composed of pixel points whose pixel value is greater than or equal to the first threshold as the background area, and the processing node can recognize the area constituted by the pixel point whose pixel value is smaller than the first threshold as the content area.
  • the processing node may perform pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched.
  • the processing node may perform pixel reconstruction (or deformation processing) on the target image according to one or more parameters of the determined content region of the background image, and reconstruct the pixel (or The image after the deformation process is used as the image to be stitched.
  • one or more parameters of the content area may include, but is not limited to, at least one of the following:
  • a range of a content area of the background image in the background image, a shape of a content area of the background image, a size of a content area of the background image, a position of a content area of the background image in the background image, and a background image The panoramic view or the way the background image is expanded.
  • the background image is a panoramic image (for example, a spherical panoramic image or a cubic panoramic image)
  • the graphic A that can be normally recognized by the human eye in the normal image is (for example, a graphic corresponding to the content area)
  • the graphic A is placed in the background image, and the human eye can normally recognize the graphic A presentation when the background image is played by the VR device, the human eye is naked (ie, When the background image is viewed without using the VR device, the graphic A is deformed compared to the presentation in the ordinary image.
  • the movie screen shown in FIG. 5 is rendered as a rectangle in a normal image, and the side in the horizontal direction of the rectangle presented in the panoramic image has a preset radian.
  • the above deformation is related to at least one of the following parameters:
  • the position of the graphic A in the background image that is, the position of the graphic A in the background image is different, and the deformation of the image A is also different.
  • the image A is set as a movie.
  • the screen ie, an example of the content area
  • the screen has less distortion when the movie screen is at the center of the background image.
  • the distortion is large.
  • the image A is set to be a movie screen (ie, , an example of the content area), when the movie screen is in the center position of the background image, if the movie screen is large, the deformation thereof is correspondingly large. And, if the movie screen is small, its deformation is correspondingly small.
  • the angle of view of the background image that is, when the angle of view of the background image is different, the deformation of the pattern A also changes accordingly. For example, when the background image has a 360 degree angle of view in the vertical direction, the deformation of the pattern A is large, and the background image is in the vertical direction. The deformation of the pattern A is small at a 90 degree angle of view.
  • the background image is expanded, that is, the deformation of the graphic A changes correspondingly when the background image is expanded.
  • the background image is a spherical panoramic image
  • the deformation of the graphic A is large, when the background image is a cubic panoramic image.
  • the shape A is deformed less.
  • the processing node can calculate the deformation of the content region according to the above parameters, and perform pixel reconstruction on the target image based on the deformation of the content region.
  • the background image is expanded in a spherical manner, that is, the background image may be a spherical panoramic image.
  • the shape of the background image is a first rectangle
  • the content area of the background image is located at a center position of the first rectangle
  • the shape of the content area of the background image is a shape formed by the second rectangle in a first deformation manner.
  • the first modification is to deform the edge in the horizontal direction of the second rectangle into an edge having a preset curvature.
  • the content region may be formed to be approximately a second rectangle, and unlike the second rectangle, the edges in the two horizontal directions of the content region are formed to have a preset curvature.
  • the processing node may perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radians.
  • the processing node may traverse the pixel points of the background image one by one according to the first extraction model, if the position of the pixel point in the background image (for example, the coordinates in the Cartesian coordinate system) and the first In the extraction model, the position of the pixel having the pixel value of 0 (ie, black) corresponds, and the pixel located at the position in the background image is used as each pixel of the content region.
  • the processing node can record the position (eg, coordinates) of each pixel of the content area, and the processing node records the coordinates of the four corner vertices of the content area, and the minimum value of the content area in the height (eg, the Y-axis) direction. And the maximum.
  • the processing node may determine the aspect ratio of the second rectangle according to the coordinates of the four corner vertices determined as shown above, and further, the processing node may determine the scaling of the target image according to the aspect ratio, and determine the target The image is scaled such that the size of the scaled target image is the same or approximately the same as the size of the second rectangle, and the aspect ratio of the scaled target image is the same or approximately the same as the aspect ratio of the second rectangle.
  • the processing node may determine the preset radian according to the change of the position of each pixel point of the content area acquired as described above in the height (for example, the Y-axis) direction, thereby determining the corresponding stretching of the preset radian. And performing a stretching process in a height (for example, Y-axis) direction on the scaled target image based on the stretch ratio.
  • the stretching process may refer to an edge in the horizontal direction of the target image (specifically, the scaled target image) (ie, may also be called Interpolation calculation is performed on pixels in the vicinity of the side in the width direction such that the side in the horizontal direction of the target image has the above-described preset curvature.
  • the pixel can be reconstructed into an image to be stitched, wherein
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the size of the image to be stitched may be the same as or approximately the same as the size of the background image
  • the shape of the image to be stitched may be the same or approximately the same as the size and shape of the content area of the background image.
  • Fig. 8 shows an example of an image to be stitched obtained by the above processing, and as shown in Fig. 8, the shape of the image to be stitched is similar to the shape of the content area of the background image shown in Fig. 5.
  • the processing node may determine the image to be played based on the image to be stitched and the background image (specifically, the background area of the background image).
  • composition image to be stitched and the background image are combined, including:
  • the to-be-played image may be an image having the same size as the background image, that is, in the embodiment of the present invention, the number of pixels included in the image to be played and the background image are included.
  • the number of pixels can be the same.
  • the processing node may determine an area corresponding to the pixel position i in the image to be played, specifically, an area to which the pixel position i' in the background image corresponding to the pixel position i in the image to be played belongs to. (ie, a background area or a content area), where i ⁇ [1, w], and i' ⁇ [1, w], w is the number of pixel points included in the background image.
  • the processing node can determine the pixel value of the pixel point position i' as the pixel value of the pixel point position i.
  • the processing node may determine the pixel point i" corresponding to the pixel point position i' from the image to be stitched, and set the pixel of the determined pixel point i" The value is determined as the pixel value of the pixel position i.
  • the processing node may determine a one-to-one mapping relationship between a plurality of pixel point positions of the image to be stitched and a plurality of pixel point positions of the content area.
  • composition image to be stitched and the background image are synthesized, including:
  • the pixels of the image to be stitched cover part or all of the pixels of the content area of the background image.
  • the processing node may set the image to be stitched to layer 1 and the background image to layer 2, wherein the size of layer 1 is the same or approximately the same as the size of layer 2. And, the relative position of the image to be stitched in the layer 1 is the same or approximately the same as the relative position of the content area in the image 2.
  • the processing node may set the area other than the image to be stitched in the layer 1 to be transparent, and then the processing node may overlay the layer 1 onto the layer 2, thereby completing the combination of the image to be stitched and the background image. deal with.
  • composition image to be stitched and the background image are synthesized, including:
  • the pixels of the image to be stitched are replaced with some or all of the pixels of the content area of the background image.
  • the processing node may determine a one-to-one mapping relationship between a plurality of pixel point positions of the image to be stitched and a plurality of pixel point positions of the content area.
  • the coordinates of the pixel position B in the image to be stitched are ( ⁇ , ⁇ )
  • the coordinates of the pixel position B′ in the content region are ( ⁇ ′, ⁇ ′)
  • the processing node can determine that the pixel point position B corresponds to the pixel point position B'. Thereafter, the processing node can replace the pixel value of the pixel point position B' with the pixel value of the pixel of the pixel point position B.
  • the image to be played can be generated through the above-described merging process, wherein the image to be played includes pixels in the background region in the background image (ie, an example of pixels in the first region), and the image to be played includes a pixel in the image to be stitched (ie, an image determined based on the target image) (ie, an example of a pixel in the second region), that is, the image to be played includes an image for presenting a virtual scene (ie, a background of the background image) An image of pixels in the area), and the image to be played includes an image that needs to be viewed in the present virtual scene (ie, pixels determined based on the target image).
  • the content of both the background image and the target image can be simultaneously presented, that is, the process of viewing the target image in the virtual scene corresponding to the background image can be realized by playing the image to be played, and then Can reduce the processing load of VR devices.
  • FIG. 9 shows an example of an image to be played which is generated through the above processing.
  • 10 is an effect of the to-be-played image being played by the VR device, as shown in FIG. 10, the portion of the image to be played corresponding to the image to be stitched (or the target image) (ie, the second region) is presented.
  • the image is the same or approximately the same as the target image shown in FIG.
  • the above-exemplified process for generating an image to be played is only an exemplary description, and the present invention is not limited thereto.
  • the user may also cause the processing node to determine that the target image needs to be embedded into the splicing area in the background image by using a control command or the like. .
  • the processing node can directly cover the target image on the splicing area in the background image without performing pixel reconstruction on the target image according to the above control instruction.
  • the processing node may directly replace the pixels in the splicing area of the background image with the pixels in the target image without performing pixel reconstruction on the target image according to the above control instruction.
  • the image to be played includes a first area and a second area, where the first area includes a pixel for presenting a virtual background in the background image, the second area includes a pixel corresponding to the target image, so that when the image to be played is
  • the content of both the background image and the target image can be simultaneously presented, that is, the process of viewing the target image in the virtual scene corresponding to the background image can be realized by playing the image to be played, and further, the processing of the VR device can be reduced.
  • one of the to-be-played images generated by the production may be played in multiple VR devices without each VR device playing synchronously by changing the skin of the player. Images, ie virtual background images and realistic images.
  • the background image is saved in the VR device, and the background images saved in different VR devices may also be different, which may result in the user not being able to view the same virtual scene in different VR devices, as opposed to the background image.
  • the content of the content and the target graphic are both carried on the image to be played, and the target image can be viewed in the scene desired by the user in any VR device.
  • the acquiring the target image includes: acquiring the target image from the target video, wherein the target video is a non-panoramic video, the target video includes a multi-frame first image, and the target image is any frame in the multi-frame image.
  • An image acquiring the target image from the target video, wherein the target video is a non-panoramic video, the target video includes a multi-frame first image, and the target image is any frame in the multi-frame image. An image.
  • the processing node may determine each frame image in a non-panoramic video (ie, target video) file, and use each frame image as the target image, and perform the foregoing merge with the background image. Processing, thereby generating a multi-frame image to be played, and the processing node can synthesize the multi-frame image to be played (specifically, synthesizing into a generated image sequence), and synthesizing with the audio file of the target video, thereby being able to generate a panorama Video (or VR video).
  • a non-panoramic video ie, target video
  • the acquiring the background image comprises: acquiring the background image from the background video, the background video being a spherical panoramic video or a cube panoramic video, the background video comprising a multi-frame second image, the background image being in the multi-frame second image Any frame of image.
  • the background image may also be taken from a panoramic video (ie, a background video).
  • each frame image in the target video as a target image and performing the above-described processing, it is possible to acquire a plurality of frames to be played as a star image or a cube panoramic image, and The generated multi-frame images are combined in time series to generate a spherical panoramic video or a cube panoramic video, which can display a virtual background generated based on the background image and a video content of the target video after being played by the VR device.
  • FIG. 11 is a schematic block diagram of an apparatus 200 for image processing according to an embodiment of the present invention. As shown in FIG. 11, the apparatus 200 includes:
  • the acquiring unit 210 is configured to acquire a background image and a target image, where the background image is a spherical panoramic image or a cubic panoramic image, and the target image is a non-panoramic image;
  • the processing unit 220 is configured to perform a synthesis process on the target image and the background image to generate an image to be played, the image to be played is a spherical panoramic image or a cubic panoramic image, and the image to be played includes the first region and the second region.
  • the first area includes pixels obtained from the background image, the second area including pixels obtained from the target image.
  • the background image includes a background area and a content area
  • the processing unit 220 includes: an extraction module 222, configured to determine a content area of the background image,
  • the processing unit 220 includes: a reconstruction module 224, configured to perform pixel reconstruction on the target image according to a content area of the background image, to generate an image to be stitched,
  • the processing unit 220 includes a synthesizing module 226 for performing a synthesizing process on the spliced image and the background image, wherein the first region includes pixels in the background region, and the second region includes pixels in the image to be spliced.
  • the compositing module 226 is specifically configured to determine pixels in the first region according to pixels in the background region, and determine pixels in the second region according to pixels of the image to be stitched; or
  • the compositing module 226 is specifically configured to cover or replace some or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the reconstruction module 224 is specifically configured to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters:
  • a range of a content area of the background image in the background image, a shape of a content area of the background image, a size of a content area of the background image, a position of a content area of the background image in the background image, and a background image The panoramic view or the way the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner.
  • the first mode is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature, and
  • the reconstruction module 224 is specifically configured to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset curvature.
  • the extraction module 222 is specifically configured to acquire a first extraction model, where the first extraction model is used to indicate a range of a content area of the background image in the background image, and configured to determine content of the background image according to the first extraction model. region.
  • the extraction module 222 is specifically configured to acquire a one-to-one mapping relationship between the M image sets and the M extraction models, where each of the M image sets includes at least one panoramic image, and each of the panoramic images includes content. a region, each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set, and is used to determine a first image set to which the background image belongs, for Mapping the relationship, and extracting the corresponding model corresponding to the first image set as the first extraction model.
  • the extraction module 222 is specifically configured to determine, according to at least one of the following information, a first image set to which the background image belongs:
  • the source of the background image the name of the background image, or the format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and
  • the extraction module 222 is specifically configured to determine a content area of the background image according to the first threshold.
  • the apparatus 200 for image processing may correspond to (for example, may be configured or itself) a processing node described in the above method 100, and each module or unit in the apparatus 200 of the image processing is used to perform processing in the method 100 described above, respectively.
  • the details of the operations or processes performed by the nodes are omitted here to avoid redundancy.
  • the image to be played includes a first area and a second area, where the first area includes a pixel for presenting a virtual background in the background image, the second area includes a pixel corresponding to the target image, so that when the image to be played is
  • the content of both the background image and the target image can be simultaneously presented, that is, the process of viewing the target image in the virtual scene corresponding to the background image can be realized by playing the image to be played, and further, the processing load of the VR device can be reduced.
  • the background image is saved in the VR device, and the background images saved in different VR devices may also be different, which may result in the user not being able to view the same virtual scene in different VR devices, as opposed to the background image.
  • the content of the content and the target graphic are both carried on the image to be played, and the target image can be viewed in the scene desired by the user in any VR device.
  • FIG. 13 is a diagram showing the structure of an apparatus 300 for image processing according to an embodiment of the present invention.
  • the image processing apparatus 300 includes: at least one processor 301, at least one network interface 304 or other user interface 303, a memory 305, and at least one communication bus.
  • 302. Communication bus 302 is used to implement connection communication between these components.
  • the terminal device 300 includes a user interface 303, including a display (eg, a touch screen, an LCD, a CRT, a holographic imaging device, or a projection device, etc.), a keyboard or a pointing device (eg, a mouse, a trackball, a touchpad, or a touch screen, etc.) .
  • a display eg, a touch screen, an LCD, a CRT, a holographic imaging device, or a projection device, etc.
  • a keyboard or a pointing device eg, a mouse, a trackball, a touchpad, or a touch screen, etc.
  • Memory 305 can include read only memory and random access memory and provides instructions to processor 301, such as machine readable instructions and data. A portion of the memory 305 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 305 stores the following elements, executable modules or data structures, or a subset thereof, or their extension set:
  • the operating system 3051 includes various system programs, such as the framework layer, the core library layer, the driver layer, and the like shown in FIG. 1, for implementing various basic services and processing hardware-based tasks;
  • the application module 3052 includes various applications, such as a desktop, a media player, a browser, and the like as shown in FIG. 1 for implementing various application services.
  • the processor 301 is configured to: acquire a background image and a target image by calling a program or an instruction stored in the memory 305, where the background image is a spherical panoramic image or a cubic panoramic image, and the target image is a non-panoramic image. And combining the target image and the background image to generate an image to be played, the image to be played is a spherical panoramic image or a cubic panoramic image, and the image to be played includes a first area and a second area, the first The area includes pixels obtained from the background image, the second area including pixels obtained from the target image.
  • the background image includes a background area and a content area
  • the processor 301 is specifically configured to determine a content area of the background image, and perform pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched for synthesizing the stitched image and the background image. Processing, wherein the first area includes pixels in the background area, and the second area includes pixels in the image to be stitched.
  • the processor 301 is specifically configured to determine pixels in the first area according to pixels in the background area, and determine pixels in the second area according to pixels of the image to be stitched; or
  • the processor 301 is specifically configured to cover or replace a part or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the shape of the content area of the background image is in a preset second range Inside.
  • the processor 301 is specifically configured to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters:
  • a range of a content area of the background image in the background image, a shape of a content area of the background image, a size of a content area of the background image, a position of a content area of the background image in the background image, and a background image The panoramic view or the way the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner.
  • the first mode is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature, and
  • the processor 301 is specifically configured to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radian.
  • the processor 301 is specifically configured to acquire a first extraction model, where the first extraction model is used to indicate a range of a content area of the background image in the background image, and configured to determine content of the background image according to the first extraction model. region.
  • the processor 301 is specifically configured to acquire a one-to-one mapping relationship between the M image sets and the M extracted models, where each of the M image sets includes at least one panoramic image, and each of the panoramic images includes content. a region, each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set, and is used to determine a first image set to which the background image belongs, for Mapping the relationship, and extracting the corresponding model corresponding to the first image set as the first extraction model.
  • the processor 301 is specifically configured to determine, according to at least one of the following information, a first image set to which the background image belongs:
  • the source of the background image the name of the background image, or the format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and
  • the processor 301 is specifically configured to determine a content area of the background image according to the first threshold.
  • the image processing device 300 may correspond to (for example, may be configured or be itself) the processing node described in the above method 100, and each module or unit in the image processing device 300 is used to perform the processing in the above method 100, respectively.
  • the details of the operations or processes performed by the nodes are omitted here to avoid redundancy.
  • the embodiment of the present invention further provides a computer program product, comprising: computer program code, when the computer program code is processed by a node (for example, the image processing device or device, specifically, image processing)
  • a node for example, the image processing device or device, specifically, image processing
  • the processing node causes the following actions to be performed:
  • a background image which is a spherical panoramic image or a cubic panoramic image
  • the image to be played is a spherical panoramic image or a cubic panoramic image
  • the image to be played includes a first area and a second area
  • the first area includes Based on the pixels obtained from the background image
  • the second region includes pixels obtained from the target image.
  • the processing node when executed by the processing node, further causing the processing node to determine a content area of the background image; performing pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched;
  • the background image is subjected to a synthesis process, wherein the first area includes pixels in the background area, and the second area includes pixels in the image to be stitched.
  • the processing node When the computer program code is executed by the processing node, further causing the processing node to determine pixels in the first region according to pixels in the background region, and determining pixels in the second region according to pixels of the image to be stitched; or The pixels of the image to be stitched cover or replace some or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the shape of the content area of the background image is in a preset second range Inside.
  • the processing node When the computer program code is executed by the processing node, the processing node further causes the processing node to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters: a range of the content region of the background image in the background image, the The shape of the content area of the background image, the size of the content area of the background image, the position of the content area of the background image in the background image, the panoramic viewing angle of the background image, or the manner in which the background image is expanded.
  • the shape of the background image is a first rectangle, the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner, the first
  • the method is that the edge in the horizontal direction of the second rectangle is deformed to have a preset radian, and when the computer program code is run by the processing node, the processing node is further caused to have a length to width ratio of the second rectangle and the preset radian. And performing pixel reconstruction on the target image.
  • the processing node When the computer program code is executed by the processing node, the processing node further acquires a first extraction model, where the first extraction model is used to indicate a range of the content area of the background image in the background image; according to the first extraction model, The content area of the background image is determined.
  • the processing node When the computer program code is executed by the processing node, the processing node further acquires a one-to-one mapping relationship between the M image sets and the M extraction models, wherein each of the M image sets includes at least one panorama An image, each panoramic image including a content area, each of the M extraction models being used to indicate a range of content regions in the panoramic image in the corresponding image collection; determining a first image to which the background image belongs And according to the mapping relationship, the extraction model corresponding to the first image set is used as the first extraction model.
  • the processing node when executed by the processing node, further causing the processing node to determine, according to at least one of the following information, the first image set to which the background image belongs: the source of the background image, the name of the background image, or the background image format.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is smaller than the first threshold, and when The computer program code, when executed by the processing node, further causes the processing node to determine a content region of the background image based on the first threshold.
  • the embodiment of the present invention further provides a computer readable storage medium storing a program that causes a processing node (for example, the apparatus or device for image processing described above, specifically, an image processing device) Or the processing unit or processor of the device) performs the following actions:
  • a processing node for example, the apparatus or device for image processing described above, specifically, an image processing device
  • the processing unit or processor of the device performs the following actions:
  • a background image which is a spherical panoramic image or a cubic panoramic image
  • the image to be played is a spherical panoramic image or a cubic panoramic image
  • the image to be played includes a first area and a second area
  • the first area includes Based on the pixels obtained from the background image
  • the second region includes pixels obtained from the target image.
  • the program further causes the processing node to determine a content area of the background image; performing pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched; and synthesizing the image to be stitched and the background image, wherein
  • the first area includes pixels in the background area, the second area including pixels in the image to be stitched.
  • the program further causes the processing node to determine pixels in the first region according to pixels in the background region, and determine pixels in the second region according to pixels of the image to be stitched; or cover or replace pixels of the image to be stitched Part or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the shape of the content area of the background image is in a preset second range Inside.
  • the program further causes the processing node to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters: a range of the content region of the background image in the background image, a shape of a content region of the background image, the The size of the content area of the background image, the position of the content area of the background image in the background image, the panoramic viewing angle of the background image, or the manner in which the background image is expanded.
  • the shape of the background image is a first rectangle, the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner, the first
  • the method is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature, and the program further causes the processing node to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset curvature.
  • the program further causes the processing node to acquire a first extraction model for indicating a range of the content region of the background image in the background image; and determining a content region of the background image according to the first extraction model.
  • the program further causes the processing node to acquire a one-to-one mapping relationship between the M image sets and the M extraction models, wherein each of the M image sets includes at least one panoramic image, each panoramic image including a content region
  • Each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set; determining a first image set to which the background image belongs; according to the mapping relationship, The extraction model corresponding to the first image set is taken as the first extraction model.
  • the program further causes the processing node to determine, based on at least one of the following information, a first set of images to which the background image belongs: a source of the background image, a name of the background image, or a format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is smaller than the first threshold, and the program The processing node is further caused to determine a content region of the background image based on the first threshold.
  • the image to be played includes a first area and a second area, where the first area includes a pixel for presenting a virtual background in the background image, the second area includes a pixel corresponding to the target image, so that when the image to be played is
  • the content of both the background image and the target image can be simultaneously presented, that is, the process of viewing the target image in the virtual scene corresponding to the background image can be realized by playing the image to be played, and further, the processing load of the VR device can be reduced.
  • the background image is saved in the VR device, and the background images saved in different VR devices may also be different, which may result in the user not being able to view the same virtual scene in different VR devices, as opposed to the background image.
  • the content of the content and the target graphic are both carried on the image to be played, and the target image can be viewed in the scene desired by the user in any VR device.
  • FIG. 14 is a schematic flowchart of a method 400 of video processing according to an embodiment of the present invention. As shown in FIG. 14, the method 400 of the video processing includes:
  • S410 Acquire a background video, where the background video includes at least one frame image, and each frame image included in the background video is a spherical panoramic image or a cubic panoramic image;
  • the method and method for acquiring the background video may be similar to the prior art, and a detailed description thereof is omitted herein to avoid redundancy.
  • each frame image in the background video may be similar to the specific form of the background image described in the above method 100.
  • detailed description thereof is omitted.
  • S420 Acquire a target video, where the target video includes at least one frame image, and each frame image included in the target video is a non-panoramic image;
  • the method and manner of acquiring the target video may be similar to the prior art, and a detailed description thereof is omitted herein to avoid redundancy.
  • each frame image in the target video may be similar to the specific form of the target image described in the above method 100.
  • detailed description thereof is omitted.
  • each frame image in the to-be-played video is a spherical panoramic image or a cubic panoramic image.
  • Each frame image of the to-be-played video includes a first area including a pixel obtained from the background video, and a second area including pixels obtained according to the target video.
  • the synthesizing the target video and the background video comprises: synthesizing the target image and the background image to generate an image to be played, wherein the target image is the target video.
  • the target image is the target video.
  • the background image is any one of the background images
  • the first region of the image to be played includes pixels obtained according to the background image
  • the second region of the image to be played includes The pixel obtained by the target image.
  • the processing node may synthesize each frame image in the target video with one frame image in the background video to generate a video to be played.
  • the specific process of the synthesis processing of each frame image in the target video and the one frame image in the background video may be similar to the process of synthesizing the target image and the background image described in the above method 100.
  • the detailed description is omitted.
  • the processing node may determine a plurality of image groups, where each image group includes one frame image in the target video and one frame image in the background video, The image in the target video included in any two image groups is different, so that the processing node can combine the image in the target video in each image group with the image in the background video to generate a video to be played.
  • the specific process of the processing node synthesizing one frame image in the target video and one frame image in the background video may be similar to the process of synthesizing the target image and the background image described in the above method 100.
  • the description is omitted. Its detailed description.
  • a method of video processing by acquiring a background video as a panoramic video (specifically, a spherical panoramic video or a cube panoramic video) and a target video as a non-panoramic video, and the background video and the target video Performing a synthesis process to generate a to-be-played video as a panoramic video, the to-be-played video including a first area and a second area, where the first area includes pixels for presenting a virtual background in the background video, and the second area includes a target video corresponding to The pixel, so that when the to-be-played video is played by the VR device, the content of both the background video and the target video can be simultaneously presented, that is, the video to be played can be played, and the target video is viewed in the virtual scene corresponding to the background video.
  • the process in turn, can reduce the processing load of the VR device when playing virtual cinema video.
  • FIG. 15 is a schematic block diagram of an apparatus 500 for image processing according to an embodiment of the present invention. As shown in FIG. 15, the apparatus 500 includes:
  • the acquiring unit 510 is configured to acquire a background video and a target video, where the background video includes at least one frame image, and each frame image included in the background video is a spherical panoramic image or a cubic panoramic image, and the target video includes at least one frame image, where Each frame image included in the target video is a non-panoramic image;
  • the processing unit 520 is configured to perform a synthesis process on the target video and the background video to generate a to-be-played video, where the background video includes at least one frame image, where each frame image in the to-be-played video is a spherical panoramic image or A cube panoramic image, each frame image of the to-be-played video including a first region and a second region, the first region including pixels in the background video, the second region including pixels obtained according to the target video.
  • the processing unit 520 is specifically configured to perform a synthesis process on the target image and the background image to generate an image to be played, wherein the target image is any one of the target videos, and the background image is any one of the background videos.
  • a frame image and the first region of the image to be played includes a pixel obtained from the background image, the second region of the image to be played including a pixel obtained according to the target image.
  • the background image includes a background area and a content area
  • the processing unit 520 is specifically configured to determine a content area of the background image, and perform pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched, for performing the stitched image and the background image. a composition process, wherein the first area includes pixels in the background area, and the second area includes pixels in the image to be stitched.
  • the processing unit 520 is specifically configured to determine pixels in the first area according to pixels in the background area, and determine pixels in the second area according to pixels of the image to be stitched; or
  • the processing unit 520 is specifically configured to cover or replace a part or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the processing unit 520 is specifically configured to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters:
  • a range of a content area of the background image in the background image, a shape of a content area of the background image, a size of a content area of the background image, a position of a content area of the background image in the background image, and a background image The panoramic view or the way the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner.
  • the first mode is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature, and
  • the processing unit 520 is specifically configured to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radiance.
  • the processing unit 520 is specifically configured to acquire a first extraction model, where the first extraction model is used to indicate a range of a content area of the background image in the background image, and configured to determine content of the background image according to the first extraction model. region.
  • the processing unit 520 is specifically configured to acquire a one-to-one mapping relationship between the M image sets and the M extracted models, where each of the M image sets includes at least one panoramic image, and each of the panoramic images includes the content. a region, each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set, and is used to determine a first image set to which the background image belongs, for Mapping the relationship, and extracting the corresponding model corresponding to the first image set as the first extraction model.
  • the processing unit 520 is specifically configured to determine, according to at least one of the following information, the first image set to which the background image belongs:
  • the source of the background image the name of the background image, or the format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and
  • the processing unit 520 is specifically configured to determine a content area of the background image according to the first threshold.
  • the video processing device 500 may correspond to (eg, may be configured or be itself) the processing node described in the method 400 above, and each module or unit in the image processing device 500 is configured to perform processing in the method 400 described above, respectively.
  • the details of the operations or processes performed by the nodes are omitted here to avoid redundancy.
  • FIG. 16 is a diagram showing the structure of a device 600 for video processing according to an embodiment of the present invention.
  • the image processing device 600 includes: at least one processor 601, at least one network interface 604 or other user interface 603, a memory 605, and at least one communication bus. 602. Communication bus 602 is used to implement connection communication between these components.
  • the terminal device 600 includes a user interface 603, including a display (eg, a touch screen, an LCD, a CRT, a holographic imaging device, or a projection device, etc.), a keyboard or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen, etc.) .
  • a display eg, a touch screen, an LCD, a CRT, a holographic imaging device, or a projection device, etc.
  • a keyboard or a pointing device eg, a mouse, a trackball, a touch pad, or a touch screen, etc.
  • Memory 605 can include read only memory and random access memory and provides instructions to processor 601, such as machine readable instructions and data. A portion of the memory 605 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 605 stores the following elements, executable modules or data structures, or a subset thereof, or their extension set:
  • the operating system 6051 includes various system programs, such as the framework layer, the core library layer, the driver layer, and the like shown in FIG. 1, for implementing various basic services and processing hardware-based tasks;
  • the application module 6052 includes various applications, such as a desktop, a media player, a browser, and the like as shown in FIG. 1 for implementing various application services.
  • the processor 601 is configured to acquire a background video and a target video by calling a program or an instruction stored in the memory 605.
  • the background video includes at least one frame image, and each frame image included in the background video is a spherical panorama.
  • An image or a cube panoramic image, the target video including at least one frame image, each frame image included in the target video being a non-panoramic image;
  • the processor 601 is configured to perform a synthesis process on the target video and the background video to generate a to-be-played video, where the background video includes at least one frame image, wherein each frame image in the to-be-played video is a spherical panoramic image or a cube.
  • the panoramic image, each frame image of the to-be-played video includes a first area including pixels in the background video, and a second area including pixels obtained according to the target video.
  • the processor 601 is specifically configured to perform a synthesis process on the target image and the background image to generate an image to be played, wherein the target image is any one of the target videos, and the background image is any one of the background videos. a frame image, and the first region of the image to be played includes a pixel obtained from the background image, the second region of the image to be played including a pixel obtained according to the target image.
  • the background image includes a background area and a content area
  • the processor 601 is specifically configured to determine a content area of the background image, and perform pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched, for performing the stitched image and the background image. a composition process, wherein the first area includes pixels in the background area, and the second area includes pixels in the image to be stitched.
  • the processor 601 is specifically configured to determine pixels in the first area according to pixels in the background area, and determine pixels in the second area according to pixels of the image to be stitched; or
  • the processor 601 is specifically configured to cover or replace a part or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the processor 601 is specifically configured to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters:
  • a range of a content area of the background image in the background image, a shape of a content area of the background image, a size of a content area of the background image, a position of a content area of the background image in the background image, and a background image The panoramic view or the way the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a center position of the first rectangle, and the shape of the content area of the background image is a shape formed by deforming the second rectangle in a first manner.
  • the first mode is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature, and
  • the processor 601 is specifically configured to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radians.
  • the processor 601 is specifically configured to acquire a first extraction model, where the first extraction model is used to indicate a range of a content area of the background image in the background image, and configured to determine content of the background image according to the first extraction model. region.
  • the processor 601 is specifically configured to acquire a one-to-one mapping relationship between the M image sets and the M extracted models, where each of the M image sets includes at least one panoramic image, and each of the panoramic images includes the content. a region, each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set, and is used to determine a first image set to which the background image belongs, for Mapping the relationship, and extracting the corresponding model corresponding to the first image set as the first extraction model.
  • the processor 601 is specifically configured to determine, according to at least one of the following information, a first image set to which the background image belongs:
  • the source of the background image the name of the background image, or the format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and
  • the processor 601 is specifically configured to determine a content area of the background image according to the first threshold.
  • the video processing device 600 may correspond to (eg, may be configured on or in itself) the processing node described in the method 400 above, and each module or unit in the video processing device 600 is configured to perform processing in the method 400 described above, respectively.
  • the details of the operations or processes performed by the nodes are omitted here to avoid redundancy.
  • the embodiment of the present invention further provides a computer program product, comprising: computer program code, when the computer program code is processed by a node (for example, the image processing device or device, specifically, image processing)
  • a node for example, the image processing device or device, specifically, image processing
  • the processing node causes the following actions to be performed:
  • the background video includes at least one frame image, and each frame image included in the background video is a spherical panoramic image or a cubic panoramic image;
  • the background video includes at least one frame image, wherein each frame image of the to-be-played video is a spherical panoramic image or a cube panoramic image, and the Each frame of the video in the playback video includes a first region and a second region, the first region including pixels in the background video, the second region including pixels obtained from the target video.
  • the processing node When the computer program code is executed by the processing node, the processing node further performs a synthesis process on the target image and the background image to generate an image to be played, wherein the target image is any one of the target video, the background image Is any one of the background videos, and the first region of the image to be played includes pixels obtained according to the background image, and the second region of the image to be played includes pixels obtained according to the target image.
  • the background image includes a background area and a content area, and when the computer program code is executed by the processing node, further causing the processing node to determine a content area of the background image; performing pixel reconstruction on the target image according to the content area of the background image And generating a to-be-spliced image; performing a composition processing on the mosaic image and the background image, wherein the first region includes pixels in the background region, and the second region includes pixels in the image to be stitched.
  • the processing node When the computer program code is executed by the processing node, the processing node further determines the pixels in the first region according to the pixels in the background region, and determines the pixels in the second region according to the pixels of the image to be stitched.
  • the processing node When the computer program code is executed by the processing node, the processing node also causes the pixel of the image to be stitched to cover part or all of the pixels of the content area of the background image.
  • the processing node When the computer program code is executed by the processing node, the processing node also causes the processing node to replace the pixels of the image to be stitched with some or all of the pixels of the content area of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the content to be stitched when played by the virtual reality VR device is the same as the content presented by the target image.
  • the processing node When the computer program code is executed by the processing node, the processing node further causes the processing node to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters: a range of the content region of the background image in the background image, the The shape of the content area of the background image, the size of the content area of the background image, the position of the content area of the background image in the background image, the panoramic viewing angle of the background image, or the manner in which the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a central position of the first rectangle, and the shape of the content area of the background image is: a shape formed by deforming the second rectangle in a first manner, the first One way is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature
  • the processing node When the computer program code is executed by the processing node, the processing node further causes the processing node to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radiance.
  • the processing node When the computer program code is executed by the processing node, the processing node further acquires a first extraction model, where the first extraction model is used to indicate a range of the content area of the background image in the background image; according to the first extraction model, The content area of the background image is determined.
  • the processing node When the computer program code is executed by the processing node, the processing node further acquires a one-to-one mapping relationship between the M image sets and the M extraction models, wherein each of the M image sets includes at least one panorama An image, each panoramic image including a content area, each of the M extraction models being used to indicate a range of content regions in the panoramic image in the corresponding image collection; determining a first image to which the background image belongs And according to the mapping relationship, the extraction model corresponding to the first image set is used as the first extraction model.
  • the processing node when executed by the processing node, further causing the processing node to determine, according to at least one of the following information, the first image set to which the background image belongs: the source of the background image, the name of the background image, or the background image format.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is less than the first threshold, and when the computer When the program code is operated by the processing node, the processing node further determines the first boundary area and the second boundary area according to the first threshold, and determines a content area of the background image according to the first boundary area and the second boundary area.
  • the pixel value of any pixel in the background area is greater than or equal to the first threshold, the pixel value of any pixel in the content area is less than the first threshold, and the content area of the background image is determined, including: according to the first A threshold determines the content area of the background image.
  • a difference between a range of the second area in the image to be played and a range of the content area in the background image and a range of the first area in the first panoramic image is preset Within three limits.
  • the embodiment of the present invention further provides a computer readable storage medium storing a program that causes a processing node (for example, the apparatus or device for image processing described above, specifically, an image processing device) Or the processing unit or processor of the device) performs the following actions:
  • a processing node for example, the apparatus or device for image processing described above, specifically, an image processing device
  • the processing unit or processor of the device performs the following actions:
  • the background video includes at least one frame image, and each frame image included in the background video is a spherical panoramic image or a cubic panoramic image;
  • the background video includes at least one frame image, wherein each frame image of the to-be-played video is a spherical panoramic image or a cube panoramic image, and the Each frame of the video in the playback video includes a first region and a second region, the first region including pixels in the background video, the second region including pixels obtained from the target video.
  • the program further causes the processing node to perform a synthesis process on the target image and the background image to generate an image to be played, wherein the target image is any one of the target video, and the background image is any frame in the background video.
  • An image, and the first region of the image to be played includes a pixel obtained from the background image, the second region of the image to be played including a pixel obtained according to the target image.
  • the background image includes a background area and a content area
  • the program further causes the processing node to determine a content area of the background image; performing pixel reconstruction on the target image according to the content area of the background image to generate an image to be stitched;
  • the image and the background image are subjected to a synthesis process, wherein the first area includes pixels in the background area, and the second area includes pixels in the image to be stitched.
  • the program further causes the processing node to determine pixels in the first region according to pixels in the background region, and determines pixels in the second region according to pixels of the image to be stitched.
  • the program also causes the processing node to overlay the pixels of the image to be stitched over some or all of the pixels of the content region of the background image.
  • the program also causes the processing node to replace the pixels of the image to be stitched with some or all of the pixels of the content region of the background image.
  • the difference between the size of the image to be stitched and the size of the content area of the background image is within a preset first range, and the difference between the shape of the image to be stitched and the size and shape of the content area of the background image is at a preset second Within the scope.
  • the content to be stitched when played by the virtual reality VR device is the same as the content presented by the target image.
  • the program further causes the processing node to perform pixel reconstruction on the target image according to the target image and at least one of the following parameters: a range of the content region of the background image in the background image, a shape of a content region of the background image, the The size of the content area of the background image, the position of the content area of the background image in the background image, the panoramic viewing angle of the background image, or the manner in which the background image is expanded.
  • the shape of the background image is a first rectangle, and the content area of the background image is located at a central position of the first rectangle, and the shape of the content area of the background image is: a shape formed by deforming the second rectangle in a first manner, the first One way is that the edge in the horizontal direction of the second rectangle is deformed to have a preset curvature
  • the program further causes the processing node to perform pixel reconstruction on the target image according to the aspect ratio of the second rectangle and the preset radians.
  • the program further causes the processing node to obtain a first extraction model for indicating a range of the content region of the background image in the background image; and determining a content region of the background image based on the first extraction model.
  • the program further causes the processing node to acquire a one-to-one mapping relationship between the M image sets and the M extraction models, wherein each of the M image sets includes at least one panoramic image, each panoramic image including a content region
  • Each of the M extraction models is used to indicate a range of content regions in the panoramic image in the corresponding image set; determining a first image set to which the background image belongs; according to the mapping relationship, The extraction model corresponding to the first image set is taken as the first extraction model.
  • the program further causes the processing node to determine, based on at least one of the following information, a first set of images to which the background image belongs: a source of the background image, a name of the background image, or a format of the background image.
  • the background area includes a first boundary area, where the area of the background area is adjacent to the content area, the content area includes a second boundary area, and the second boundary area is in the content area and the background An area adjacent to the area, a pixel value of any pixel in the first boundary area is greater than or equal to a first threshold, and a pixel value of any pixel in the second boundary area is smaller than the first threshold, and the program further And causing the processing node to determine the first boundary area and the second boundary area according to the first threshold, and determining a content area of the background image according to the first boundary area and the second boundary area.
  • the pixel value of any pixel in the background area is greater than or equal to the first threshold, the pixel value of any pixel in the content area is less than the first threshold, and the content area of the background image is determined, including: according to the first A threshold determines the content area of the background image.
  • a difference between a range of the second area in the image to be played and a range of the content area in the background image and a range of the first area in the first panoramic image is preset Within three limits.
  • the to-be-played video includes a first area and a second area, where the first area includes pixels for presenting a virtual background in the background video, and the second area includes pixels corresponding to the target video, so that when the to-be-played video is
  • the content of both the background video and the target video can be simultaneously presented, that is, the process of watching the target video in the virtual scene corresponding to the background video can be realized by playing the to-be-played video, thereby reducing the VR device playing virtual The processing burden of the cinema video.
  • the method 100 or the method 400 may be used in a terminal device, and the terminal device involved in the embodiments of the present application may include a handheld device, an in-vehicle device, a wearable device, a computing device, or other connected to a wireless modem. Processing equipment. It may also include a subscriber unit, a cellular phone, a smart phone, a wireless data card, a personal digital assistant (PDA) computer, a tablet computer, a wireless modem, and a handheld device.
  • PDA personal digital assistant
  • MTC Machine Type Communication
  • WLAN wireless local area network
  • STAION ST
  • STAION ST
  • STAION ST
  • SIP Session Initiation Protocol
  • WLL Wireless Local Loop
  • next-generation communication systems such as terminals in fifth-generation (5G) networks
  • PLMN Public Land Mobile Network
  • the wearable device may also be referred to as a wearable smart device, and is a general term for applying wearable technology to intelligently design and wear wearable devices such as glasses, gloves, watches, clothing, and shoes.
  • a wearable device is a portable device that is worn directly on the body or integrated into the user's clothing or accessories.
  • Wearable devices are more than just a hardware device, but they also implement powerful functions through software support, data interaction, and cloud interaction.
  • Generalized wearable smart devices include full-featured, large-size, non-reliable smartphones for full or partial functions, such as smart watches or smart glasses, and focus on only one type of application, and need to work with other devices such as smartphones. Use, such as various smart bracelets for smart signs monitoring, smart jewelry, etc.
  • the terminal device may further include a VR device, wherein the VR device may also be referred to as virtual reality hardware, and the virtual reality hardware refers to a hardware product related to the virtual reality technology field, and is a hardware device used in the virtual reality solution.
  • the VR device may also be referred to as virtual reality hardware
  • the virtual reality hardware refers to a hardware product related to the virtual reality technology field, and is a hardware device used in the virtual reality solution.
  • hardware devices commonly used in virtual reality may include, but are not limited to, the following types of devices.
  • Modeling equipment for example, a 3D scanner.
  • a three-dimensional visual display device for example, a 3D display system, a large projection system, a head display, for example, a head mounted stereo display, smart glasses, and the like.
  • Sound equipment for example, a three-dimensional sound system and stereo in non-traditional sense.
  • Interactive devices including: position trackers, data gloves, 3D input devices (eg, three-dimensional mice), motion capture devices, eye trackers, force feedback devices, and other interactive devices.
  • the terminal device 700 may include:
  • a terminal device 700 of the embodiment of the present invention includes a first memory 720, a processor 760, and an input unit 730.
  • the first memory 720 stores interface information of a preset number of applications of the terminal, where the interface information includes An interface element, an interface number, a correspondence between the interface number and the interface element, and location information of the application interface corresponding to the interface element;
  • the input unit 730 is configured to receive a user switching application interface operation, and generate Switching a signal;
  • the processor 760 is configured to determine a target interface number according to the switching signal; determining an interface number adjacent to the target interface number according to a predetermined number adjacent to the target interface number; and storing according to the first memory 720 An interface number and an interface number adjacent to the target interface number, and determining interface information corresponding to the interface number to be loaded; releasing interface information corresponding to at least part of the interface number of the first memory 720 that is not adjacent to the target interface number The occupied storage space; loading the interface letter corresponding to the interface number to be loaded
  • the preset number refers to the number of interface information of an application that can be stored in the first memory.
  • the predetermined number refers to the number of interface numbers adjacent to each side of the target interface number.
  • the processor 760 can release the storage space occupied by the interface information corresponding to at least part of the interface number of the first memory 720 that is not adjacent to the target interface number, and load the interface number adjacent to the target interface number.
  • the interface information is in the first memory 720, so that the interface information can be cyclically loaded, and the contradiction between the limitation of the storage capacity of the terminal device 700 and the increasing number of application interfaces is alleviated.
  • the interface information corresponding to the interface number to be loaded is determined according to the interface number stored in the first memory 720 and the interface number adjacent to the target interface number, specifically, according to the interface number stored in the first memory 720.
  • An interface number adjacent to the target interface number is used to determine an interface number that is not stored in the first memory 720.
  • the interface information corresponding to the unstored interface number is an interface corresponding to the interface number to be loaded in the first memory 720. information.
  • the processor 760 can call the interface element corresponding to the target interface number stored in the first memory 720 and the location information of the application interface corresponding to the interface element displayed by the interface element, thereby The interface element is displayed in the application interface corresponding to the target interface number.
  • the interface element may be an application icon or a widget desktop control.
  • the terminal device 700 may further include a second memory 721, where the second memory 721 may be used to store interface information of all applications of the terminal device 700.
  • the processor 760 loads the interface information corresponding to the interface number to be loaded into the first memory 720. Specifically, the processor 760 calls the interface information corresponding to the interface number to be loaded in the second memory 721, and the The interface information corresponding to the loaded interface number is loaded into the first memory 720.
  • the second memory 721 can be an external storage of the terminal device 700
  • the first memory 720 can be a memory of the terminal 700.
  • the processor 760 can load a preset amount of interface information from the second memory 721 into the first memory 720. Each loaded interface information corresponds to one storage space in the first memory 720, and each storage space may be the same.
  • the first memory 720 can be a non-Volatile Random Access Memory (NVRAM), a Dynamic Random Access Memory (DRAM) dynamic random access memory, or a static random access memory (Static Random).
  • NVRAM non-Volatile Random Access Memory
  • DRAM Dynamic Random Access Memory
  • Static Random static random access memory
  • SRAM static random access memory
  • flash memory any type of non-volatile memory
  • second memory 721 may be a hard disk, an optical disk, a Universal Serial Bus (USB) disk, a floppy disk, or a tape drive.
  • USB Universal Serial Bus
  • all the interface information of the terminal may be stored in the cloud server, and the cloud server may be the second memory 721.
  • the processor 760 loads the interface information corresponding to the interface number to be loaded into the first memory 720. Specifically, the processor 760 obtains interface information corresponding to the interface number to be loaded in the cloud server through the network channel, and the The interface information corresponding to the interface number to be loaded is loaded into the first memory 720.
  • the input unit 730 can be configured to receive input digital or character information and to generate signal inputs related to user settings and function control of the terminal 700.
  • the input unit 730 may include a touch panel 731.
  • the touch panel 731 also referred to as a touch screen, can collect touch operations on or near the user (such as the user's operation on the touch panel 731 or the touch panel 731 using any suitable object or accessory such as a finger, a stylus, or the like. ), and drive the corresponding connection device according to a preset program.
  • the touch panel 731 may include two parts of a touch detection device and a touch controller.
  • the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information.
  • the processor 760 is provided and can receive commands from the processor 760 and execute them.
  • the touch panel 731 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.
  • the input unit 730 may further include other input devices 732, which may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like. One or more of them.
  • the terminal device 700 can also include a display unit 740 that can be used to display information input by the user or information provided to the user and various menu interfaces of the terminal 700.
  • the display unit 740 can include a display panel 741.
  • the display panel 741 can be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).
  • the display unit 740 can also display the above-mentioned image to be played or the video to be played.
  • the touch panel 731 covers the display panel 741 to form a touch display screen.
  • the touch display screen detects a touch operation on or near the touch display screen, the touch display screen transmits to the processor 760 to determine the type of the touch event.
  • the processor 760 then provides a corresponding visual output on the touch display based on the type of touch event.
  • the touch display screen includes an application interface display area and a common control display area.
  • the arrangement manner of the application interface display area and the display area of the common control is not limited, and the arrangement manner of the two display areas can be distinguished by up-and-down arrangement, left-right arrangement, and the like.
  • the application interface display area can be used to display the interface of the application. Each interface can contain interface elements such as at least one application's icon and/or widget desktop control.
  • the application interface display area 443 can also be an empty interface that does not contain any content.
  • the common control display area is used to display controls with high usage, such as setting buttons, interface numbers, scroll bars, phone book icons, and the like.
  • the processor 760 is a control center of the terminal 700, which connects various parts of the entire mobile phone using various interfaces and lines, by running or executing software programs and/or modules stored in the first memory 720, and calling the storage in the first
  • the data in the second memory 721 performs various functions and processing data of the terminal 700, thereby performing overall monitoring of the terminal 700.
  • the processor 760 can include one or more processing units.
  • a preset amount of interface information can be loaded into the first memory 720 from the interface information stored in the second memory 721, and an interface corresponding to the preset number of interface information is recorded.
  • the processor 760 reads interface information of any one of the first memory 720 or a predetermined number, and generates an interface according to the interface information, and controls an application interface display area of the touch display screen to display the generated interface as an initial interface. And controlling the common control display area display interface number, and providing a user selection interface, wherein the interface number displayed by the common control display area may be the interface number corresponding to the loaded interface information in the first memory 720, or may be the first The interface number corresponding to the interface information stored in the second memory 721.
  • the preset number is not greater than the maximum number of interface information that the first memory 720 can store.
  • the processor 760 can control at least part of the interface number displayed by the common control display area to respond to the user input operation.
  • the processor 760 controls the interface number corresponding to the loaded interface information to respond to the user input operation, and the interface number corresponding to the unloaded interface information does not respond to the user input operation. .
  • the processor 760 can perform the steps in the method 400 in FIG. 11. Here, in order to avoid redundancy, detailed description thereof is omitted.
  • FIG. 18 is a schematic block diagram of a VR device 800 according to an embodiment of the present invention. As shown in FIG. 18, the apparatus 800 includes:
  • the processing unit 810 is configured to perform the steps in the foregoing method 100 to obtain an image to be played, or to perform the steps in the method 400 to obtain a video to be played.
  • the display unit 820 is configured to present the image to be played or the video to be played.
  • the VR device 800 can correspond to (eg, can include, belong to, or be itself) the processing node described in the above method 100 or 400, and the processing unit 810 is configured to perform the processing performed by the processing node in the method 100 or 400 described above.
  • the operation or processing procedure is omitted here for avoiding redundancy.
  • the operation of the display unit 820 may be similar to the operation when the VR display device presents a virtual image or a virtual video in the related art.
  • detailed description thereof will be omitted.
  • FIG. 19 illustrates a structure of a VR device 900 according to an embodiment of the present invention.
  • the VR device 900 includes: at least one processor 901, at least one network interface 904 or other user interface 903, a memory 905, a display 906, and at least one communication bus 902. .
  • Communication bus 902 is used to implement connection communication between these components.
  • the display 906 can be, for example, a touch screen, an LCD, a CRT, a holographic imaging device, or a projection device, and the like.
  • Memory 905 can include read only memory and random access memory and provides instructions and data to processor 901. A portion of the memory 905 may also include a non-volatile random access memory.
  • the memory 905 stores the following elements, executable modules or data structures, or a subset thereof, or their extended set:
  • the operating system 9051 includes various system programs, such as the framework layer, the core library layer, the driver layer, and the like shown in FIG. 1, for implementing various basic services and processing hardware-based tasks;
  • the application module 9052 includes various applications, such as a desktop, a media player, a browser, and the like as shown in FIG. 1 for implementing various application services.
  • the processor 901 is configured to execute the steps in the foregoing method 100 to obtain an image to be played, or to perform the steps in the foregoing method 400, by calling a program or an instruction stored in the memory 905.
  • the display unit 820 is configured to present the image to be played or the video to be played.
  • the VR device 900 can correspond to (eg, can include, belong to, or be itself) the processing node described in the above method 100 or 400, and the processor in the VR device 900 can be used to perform the processing in the method 100 or 400 described above.
  • the details of the operations or processes performed by the nodes are omitted here to avoid redundancy.
  • the action of the display 906 can be similar to the action when the VR display presents a virtual image or a virtual video in the prior art. Here, in order to avoid redundancy, detailed description thereof will be omitted.
  • the size of the sequence numbers of the foregoing processes does not mean the order of execution sequence, and the order of execution of each process should be determined by its function and internal logic, and the present invention should not be The implementation of the embodiments constitutes any limitation.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the embodiments of the present invention, or the part contributing to the prior art or the part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • the instructions include a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

本申请提供了一种图像处理的方法和装置、视频处理的方法和装置及虚拟现实装置，该图像处理的方法包括：获取背景图像，该背景图像为球形全景图像或立方体全景图像；获取目标图像，该目标图像为非全景图像；对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。

Description

图像、视频处理方法和装置、虚拟现实装置和存储介质
本申请要求于2017年4月11日提交中国专利局、申请号为201710234566.4、发明名称为“图像处理的方法和装置及视频处理的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频图像领域，并且更具体地，涉及图像处理方法和装置、视频处理方法和装置及虚拟现实装置。
背景技术
虚拟现实(Virtual Reality，VR)技术是一种可以创建和体验虚拟世界的计算机仿真系统，它利用计算机生成一种模拟环境，是一种多源信息融合的、交互式的三维动态视景和实体行为的系统仿真，可以使用户沉浸到该环境中。
发明内容
本申请提供一种图像处理的方法、装置和设备及视频处理的方法、装置和设备，能够降低VR设备的处理负担。
本申请一实施例提供一种图像处理方法，该方法包括：
获取背景图像，该背景图像为球形全景图像或立方体全景图像；
获取目标图像，该目标图像为非全景图像；
对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
本申请一实施例提供一种图像处理装置，包括：处理器和存储器，所述处理器执行所述存储器中的机器可读指令，用于
获取背景图像和目标图像，所述背景图像为球形全景图像或立方体全景图像，所述目标图像为非全景图像；
对所述目标图像和所述背景图像进行合成处理，以生成待播放图像，所述待播放图像为球形全景图像或立方体全景图像，且所述待播放图像包括第一区域和第二区域，所述第一区域包括根据所述背景图像获得的像素，所述第二区域包括根据所述目标图像获得的像素。
本申请一实施例提供一种视频处理方法，该方法包括：
获取背景视频，该背景视频包括至少一帧图像，该背景视频中的每一帧图像包括：球形全景图像或立方体全景图像；
获取目标视频，该目标视频包括至少一帧图像，该目标视频中的每一帧图像包括：非全景图像；
对该目标视频和该背景视频进行合成处理，以生成待播放视频，其中，该待播放视频中的每一帧图像包括：球形全景图像或立方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括该背景视频中的像素，该第二区域包括根据该目标视频获得的像素。
本申请一实施例提供一种视频处理装置，包括：处理器和存储器，所述处理器执行所述存储器中的机器可读指令，用于
获取背景视频和目标视频，所述背景视频包括至少一帧图像，所述背景视频中的每一帧图像包括：球形全景图像或立方体全景图像，所述目标视频包括至少一帧图像，所述目标视频中的每一帧图像包括：非全景图像；
对所述目标视频和所述背景视频进行合成处理，以生成待播放视频，其中，所述待播放视频中的每一帧图像包括：球形全景图像或立方体全景图像，所述待播放视频中的每一帧图像包括第一区域和第二区域，所述第一区域包括根据所述背景视频获得的像素，所述第二区域包括根据所述目标视频获得的像素。
本申请一实施例提供一种虚拟现实装置，包括：处理单元，用于执行上述方法中的各步骤，以获取待播放图像，或获取待播放视频；显示单元，用于呈现该待播放图像或该待播放视频。
本申请一实施例提供一种非易失性存储介质，用于存储机器可读指令，当所述机器可读指令被执行时，执行本申请实施例提供的方法中的步骤。
附图说明
图1是能够执行本发明实施例的图像处理的方法或视频处理的方法的处理节点的一例的逻辑结构图。
图2是能够执行本发明实施例的图像处理的方法或视频处理的方法的处理节点的另一例的逻辑结构图。
图3是本发明实施例的图像处理的方法的示意性流程图。
图4是本发明实施例的合成处理的示意性流程图。
图5是本发明实施例的背景图像的一例的示意图。
图6是本发明实施例的第一提取模板的一例的示意图。
图7是本发明实施例的目标图像的一例的示意图。
图8是本发明实施例的待拼接图像的一例的示意图。
图9是本发明实施例的待播放图像的一例的示意图。
图10是本发明实施例的待播放图像经VR设备播放后的效果图。
图11是本发明实施例的图像处理的装置的示意性框图。
图12是本发明实施例的处理单元的示意性框图。
图13是本发明实施例的图像处理的设备的示意性结构图。
图14是本发明实施例的视频处理的方法的示意性流程图。
图15是本发明实施例的视频处理的装置的示意性框图。
图16是本发明实施例的视频处理的设备的示意性结构图。
图17是适用本发明实施例的图像处理的方法或视频处理的方法的终端设备的示意性结构图。
图18是本发明实施例的VR装置的示意性框图。
图19是本发明实施例的VR设备的示意性框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
首先，介绍本发明实施例提供的图像处理的方法或视频处理的方法所应用于的处理节点的逻辑结构。
作为示例而非限定，在本发明实施例中，该处理节点可以是终端设备、个人计算机或服务器等具有图形处理能力的计算设备，本发明实施例并未特别限定。
全景图像(或者说，由多帧全景图像组成的全景视频)是实现VR场景的主要手段之一。全景图像是指通过例如，广角的表现手段以及绘画、相片、视频、三维模型等形式，尽可能多的表现出周围的环境。其中，全景图像可以借助全景摄像机拍摄完成，但是全景摄像机的价格较高。或者，全景图像也可以借助计算机建模处理生成，但是，该过程对于计算机的图形处理能力的要求较高。
为了为用户提供VR体现，出现了虚拟影院技术，用户在戴上VR设备的瞬间就会置身于一个虚拟的影院当中，例如，在用户视野的前后左右展现的都是完成建模的座椅，正前方是一个播放幕布，用户所选的视频将会在幕布上播放。
利用虚拟影院技术实现VR的方式主要通过制作作为全景图像的虚拟背景图像，例如，影院的全景图像，在播放虚拟背景图像的同时播放现实视频，将现实视频覆盖在虚拟背景图像的固定位置，相当于通过类似更换播放器皮肤的方式实现在虚拟场所观看现实图像。
但是，该虚拟影院技术中，需要VR设备同时完成显示视频的播放和全景图像的播放，因此，VR设备处理负担大，对VR设备的要求较高。
如图1所示，该处理节点的硬件层包括中央处理器(Central Processing Unit，CPU)和/或图形处理器(Graphics Processing Unit，GPU)等。
还可以处理节点的硬件层还可以包括存储器、输入/输出设备、内存、内存控制器、网络接口等。
其中，输入设备可包括键盘、鼠标、触摸屏等。
输出设备可包括显示设备如液晶显示器(Liquid Crystal Display，LCD)、阴极射线管(Cathode Ray Tube)显示器、全息成像(Holographic)显示器或投影(Projector)等。
在硬件层之上可运行有操作系统(如Android等)以及一些应用程序。核心库是操作系统的核心部分，包括输入/输出服务、核心服务、图形设备接口以及实现CPU、GPU图形处理的图形引擎(Graphics Engine)等。图形引擎可包括2D引擎、3D引擎、合成器(Composition)、帧缓冲区(Frame Buffer)等。除此之外，该终端还包括驱动层、框架层和应用层。驱动层可包括CPU驱动、GPU驱动、显示控制 器驱动等。框架层可包括图形服务(Graphic Service)、系统服务(System service)、网页服务(Web Service)和用户服务(Customer Service)等；图形服务中，可包括如微件(Widget)、画布(Canvas)、视图(Views)、渲染脚本(Render Script)等。应用层可包括桌面(launcher)、媒体播放器(Media Player)、浏览器(Browser)等。
以图1为例，本发明实施例提供的图形处理的方法，应用于计算节点，该计算节点的硬件层可包括处理器(例如，CPU和/或GPU)、显示控制器(Display Controller)、内存、内存控制器、输入设备、显示设备等硬件。核心库层(Kernel Library)可包括输入/输出服务(Input/Output Service，I/O Service)、核心服务(Kernel Service)及图形引擎(Graphic Engine)。
应当理解的是，图1所示的逻辑结构仅为本发明实施例方法的执行主体的逻辑结构的示例性说明，本发明并未特别限定，只要能够执行本发明实施例的图像处理的方法即可。
例如，作为示例而非限定，本发明实施例方法的执行主体也可以是虚拟现实系统，图2示出了本发明实施例的虚拟现实系统的一例。
如图2所示，该虚拟现实系统的硬件层包括中央处理器(CPU)，即中央处理单元和图形处理单元(GPU)，即图形处理器等，当然还可以包括存储器，例如内存、输入/输出设备、内存、内存控制器、网络接口等。
其中，输入/输出设备也可以称为交互设备。
作为示例而非限定，输入设备可包括键盘、鼠标、触摸屏、语音识别设备、运动传感器等，其中，该运动传感器也可以称为动作捕捉设备，用于检测用户的手势操作等，作为示例而非限定，该动作捕捉设备可以包括眼动仪、力觉反馈设备、数据手套、操纵杆、触觉反馈装置等。
作为示例而非限定，输出设备可包括显示设备，例如，3D展示系统、大型投影系统、头显(头戴式立体显示器等)等。在硬件层之上可运行有操作系统(例如，安卓(Android)等系统)以及一些应用程序。
例如，在本发明实施例中，该显示设备可以为虚拟现实头显，其中，该虚拟现实头显也可以称为头戴式显示器，是利用人的左右眼获取信息差异，引导用户产生一种身在虚拟环境中的感觉的一种头戴式立体显示器。其显示原理是左右眼屏幕分别显示左右眼的图像，人眼获取这种带有差异的信息后在脑海中产生立体感。虚拟 现实头显作为虚拟现实的显示设备，具有小巧和封闭性强的特点，在军事训练，虚拟驾驶，虚拟城市等项目中具有广泛的应用
再例如，在本发明实施例中，该显示设备可以为双目全方位显示器(BOOM)，BOOM是一种偶联头部的立体显示设备，是一种特殊的头部显示设备。使用BOOM比较类似使用一个望远镜，把两个独立的CRT显示器捆绑在一起，由两个相互垂直的机械臂支撑，这不仅让用户可以在半径为例如2米的球面空间内用手自由操纵显示器的位置，还能将显示器的重量加以巧妙的平衡而使之始终保持水平，不受平台运动的影响。在支撑臂上的每个节点处都有位置跟踪器。
再例如，在本发明实施例中，该显示设备可以为CRT终端，例如，液晶光闸眼镜，其立体视觉系统的工作原理是：有计算机分别产生左右眼的两幅图像，经过合成处理之后，采用分时交替的方式显示在CRT终端上。用户则佩戴一副与计算机相连的液晶光闸眼镜，眼镜片在驱动信号的作用下，将以与图像显示同步的速率交替开和闭，即当计算机显示左眼图像时，右眼透镜将被屏蔽，显示右眼图像时，左眼透镜被屏蔽。根据双目视察与深度距离正比的关系，人的视觉生理系统可以自动的将这两幅视察图像合成一个立体图像。
再例如，在本发明实施例中，该显示设备可以为洞穴式(CAVE)投影系统，CAVE投影系统是由3个面以上(含3面)硬质背投影墙组成的高度沉浸的虚拟演示环境，配合三维跟踪器，用户可以在被投影墙包围的系统近距离接触虚拟三维物体，或者随意漫游“真实”的虚拟环境。CAVE系统一般应用于高标准的虚拟现实系统。CAVE投影系统是一种基于多通道视景同步技术和立体显示技术的房间式投影可视协同环境，该系统可提供一个房间大小的最小三面或最大七十面立方体投影显示空间，供多人参与，所有参与者均完全沉浸在一个被立体投影画面包围的高级虚拟仿真环境中，借助相应虚拟现实交互设备(如数据手套、位置跟踪器等)，从而获得一种身临其境的高分辨率三维立体视听影像和6自由度交互感受。
此外，如图1所示，该虚拟现实系统的硬件层还可以包括建模设备(如3D扫描仪)。3D扫描仪，也称为三维立体扫描仪，3D扫描仪，是融合光、机、电和计算机技术于一体的高新科技产品，主要用于获取物体外表面的三维坐标及物体的三维数字化模型。该设备不但可用于产品的逆向工程、快速原型制造、三维检测(机器视觉测量)等领域，而且随着三维扫描技术的不断深入发展，诸如三维影视动画、 数字化展览馆、服装量身定制、计算机虚拟现实仿真与可视化等越来越多的行业也开始应用三维扫描仪这一便捷的手段来创建实物的数字化模型。通过三维扫描仪非接触扫描实物模型，得到实物表面精确的三维点云(Point Cloud)数据，最终生成实物的数字模型，不仅速度快，而且精度高，几乎可以完美的复制现实世界中的任何物体，以数字化的形式逼真的重现现实世界。
如图2所示，在该虚拟现实系统的软件系统架构中，核心库是操作系统的核心部分，包括外观管理器、媒体框架、关系数据库、2G图形引擎库、Web浏览器引擎，内核库和虚拟机等，其中，本发明实施例所描述的应用场景的识别和功耗控制策略的确定可以是在虚拟机处进行实施的，即虚拟机通过编译分析终端设备上正在运行的应用程序，获取所述应用程序的特征数据；根据所述应用程序的特征数据，从场景特征数据集中确定与所述应用程序的特征数据对应的应用场景信息，所述场景特征数据集包括多种应用场景信息与多种应用程序的特征数据的对应关系，其中所述与所述应用程序的特征数据对应的应用场景信息用于表示所述终端设备当前被使用的应用场景，根据所述应用场景信息，从控制策略集中确定与所述应用场景信息对应的功耗控制策略，所述控制策略集包括所述多种应用场景信息与多种功耗控制策略的对应关系，并根据所述与所述应用场景信息对应的功耗控制策略生成对应的功耗控制指令，将功耗控制指令传递给内核，由内核对终端设备的相应硬件(例如CPU、GPU或显示设备等)实施功耗控制。
除此之外，该终端设备还包括驱动层、框架层和应用层。驱动层可包括CPU驱动、GPU驱动、显示控制器驱动等。框架层可包括浏览器引擎、排版引擎、文件解析器等；应用层可包括主界面(home)、媒体播放器(Media Player)、浏览器(Browser)等多种应用程序。
下面，结合图3，对本发明实施例的图像处理的方法的具体过程进行详细说明。
图3示出了本发明实施例的图像处理的方法100的示意性流程，如图3所示，该方法100包括：
S110，获取背景图像，该背景图像为球形全景图像或立方体全景图像；
S120，获取目标图像，该目标图像为非全景图像；
S130，对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区 域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
在本申请一实施例中，第一区域包括根据该背景图像获得的像素即该第一区域包括从该背景图像中获取的像素。该第二区域包括根据该目标图像获得的像素即该第二区域包括从该目标图像中获得的像素。
具体的说，在S110，处理节点可以获取用于呈现虚拟背景(或者说，虚拟场景)的背景图像。
在本发明实施例中，该背景图像可以为VR设备(例如，VR头盔，VR眼镜或VR浏览器等)能够识别并播放(或者说，呈现)的全景图像。
在本发明实施例中，全景图像可以是指对通过专业相机捕捉的整个场景的图像信息或者使用建模软件进行渲染得到的图像，使用软件对目标图像和背景图像进行合成处理，得到待播放图像，并利用VR设备播放该待播放图像，即可将平面照片或者计算机建模图片转变为360度全观，用于虚拟现实浏览，把二维的平面图模拟成真实的三维空间，呈现给观赏者。
在本发明实施例中，全景图像是指：可视角度大于人眼视度(或者说，人眼正常有效视角)的图像。例如，全景图像在水平方向上的视角范围(或者说，可视角度)大于人眼正常水平视角(例如，90度)，全景图像在垂直方向上的视角范围大于人眼正常垂直视角(例如，70度)。
作为实例而非限定，在本发明实施例中，该全景图像可以包括球形全景图像。
其中，作为示例而非限定，在本发明实施例中，可以定义球形坐标系，即，该球形坐标系包括分布在经度和纬度上的多个坐标。并且，在本发明实施例中，可以定义平面坐标系(或者说，平面网格)，该平面坐标系包括分布在水平和垂直方向上的多个坐标。
从而，作为示例而非限定，在本发明实施例中，球形全景可以是指：将球形的经度和纬度坐标直接为水平和垂直坐标平面(或者说，平面网格中格子)。
作为示例而非限定，在本发明实施例中，该平面网格的高度可以是宽的两倍，或者说，球形全景图像的宽度(即，水平方向行的大小)和高度(即，垂直方向行的大小)的比例可以为2:1。
应理解，以上列举的球形全景图像的比例仅为示例性说明，本发明并未限定于 此，其他能够形成球形全景图像的比例均落入本发明的保护范围内。
因此，在球形全景中，从赤道到两极，横向拉伸不断加剧，南北两个极点被拉伸成了扁平的网格在整个上部和下部边缘。
作为示例而非限定，在本发明实施例中，球形全景图像可以是指能够实现整个水平方向360度全景以及垂直方向360全景的全景图像。
应理解，以上对球形全景的描述仅为示例性说明，本发明并未限定于此，例如，在本发明实施例中，球形全景图像在水平方向上的全景也可以不是360度全景，例如，球形全景图像在水平方向上的全景也可以是例如，180度全景。或者，在本发明实施例中，球形全景图像在垂直方向上的全景也可以不是360度全景，例如，球形全景图像在垂直方向上的全景也可以是例如，180度全景。
本发明实施例中关于球形全景的定义也可以与现有技术相似，这里，为了避免赘述，省略其详细说明。
图5示出了作为球形全景图像的背景图像的一例，图5所示的背景图像在经过VR设备播放(或者说，呈现)后，能够让使用者感觉处于电影院中。
作为实例而非限定，在本发明实施例中，该全景图像还可以包括立方体全景图像。
其中，作为示例而非限定，在本发明实施例中，立方体全景可以是指将全景图分成了前后左右上下六个面，浏览的时候将六个面结合成一个密闭空间来现实整个水平和竖直的360度全景场景。
应理解，以上列举的全景图像的具体形式仅为示例性说明，本发明实施例并未特别限定，现有技术中对于能够应用于VR场景的实现的全景图像的任何定义均落入本发明实施例的保护范围内。
在本发明实施例中，该背景图像可以是通过用于拍摄全景图像的摄像设备拍摄获得。或者，该背景图像也可以通过计算机进行三维虚拟建模而获得。
另外，在本发明实施例中，上述背景图像也可以是VR图像，其中，VR图像是指经过VR设备播放后传送到用户的感觉器官末梢的可视的、可听到的或可触摸到的事物，它们看起来好像来源于围绕用户的三维空间。
作为实例而非限定，在本发明实施例中，该VR图像可以是指长宽比例(或者说，水平方向和垂直方向上的比例)为2：1的全景图像。
即，在本发明实施例中，全景图像也可以称为VR全景图像。
在本发明实施例中，该背景图像包括背景区域和内容区域。
其中，该背景区域的大小和位置可以是(例如，由使用者或管理员)预设的，是背景图像中除内容区域以外的区域。
或者，该内容区域的大小和位置可以是(例如，由使用者或管理员)预设的置。
其中，该背景区域在经过VR设备播放后能够呈现虚拟背景。
该内容区域在背景图像中的位置可以与经过VR设备播放后的目标图像的内容在虚拟背景中的位置具有对应关系。
例如，如图5所示，当该背景图像用于呈现虚拟影院时，该内容区域可以使电影屏幕所对应的区域。
S120，处理节点可以获取目标图像，该目标图像为非全景图像。其中，非全景图像是指能够通过普通播放设备(非VR设备，例如，手机、个人计算机的显示屏)呈现并被人眼识别的图像，或者说，非全景图像是指：可视角度小于或等于人眼视度(或者说，人眼正常有效视角)的图像。例如，非全景图像在水平方向上的视角范围(或者说可视角度)小于或等于人眼正常水平视角(例如，90度)，非全景图像在垂直方向上的视角范围小于或人眼正常垂直视角(例如，70度)。图7示出了本发明的目标图像的一例。
S130，处理节点可以对该目标图像和该背景图像进行合成处理，以生成待播放图像。
下面，对该“合成处理”的具体过程进行示例性说明。
该背景图像包括背景区域和内容区域，以及
该对该目标图像和该背景图像进行合成处理，包括：
确定该背景图像的内容区域；
根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像；
对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
具体地说，如图4所示，在S132，处理节点可以从背景图像中确定上述内容区域。
其中，作为实例而非限定，在本发明实施例中，处理节点可以根据以下任意一 种方式确定内容区域。
方式1
该确定该背景图像的内容区域，包括：
获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围，该范围包括内容区域在背景图像中的位置和大小。
根据该第一提取模型，确定该背景图像的内容区域。
具体地说，在本发明实施例中，在本发明实施例中，该第一提取模板也可以称为标注文件。该第一提取模型可以是使用者配置或者服务器生成的，该第一提取模型可以指示(或者说，标注)该背景图像的内容区域在该背景图像中的范围，具体的说，该第一提取模型可以指示该背景图像的内容区域的大小和形状，以及该内容区域在该背景图像中的位置。
该获取第一提取模型，包括：
获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围；
确定该背景图像所属的第一图像集合；
根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
具体地说，在本发明实施例中，在处理节点中可以保存有多个提取模型，并且，在处理节点中可以保存有该多个提取模型和多个图像集合之间的映射关系，其中，每个图像集合可以包括至少一个全景图像(或者说，VR全景图像、球形全景图像或立方体全景图像)。
从而，处理节点可以确定该背景图像所属的图像集合(即，第一图像集合)。
该确定该背景图像该属于的第一图像集合包括：
根据以下至少一种信息，确定该背景图像该属于的第一图像集合：
该背景图像的来源、该背景图像的名称或该背景图像的格式。
具体地说，在本发明实施例中，可以按照来源，将多个全景图像(或者说，VR全景图像、球形全景图像或立方体全景图像)划分为多个图像集合，以使具有同一来源的图像属于相同的集合。从而，处理节点能够根据背景图像的来源，确定该背 景图像所属于的图像集合(即，第一图像集合)。
并且，作为实例而非限定，在本发明实施例中，“来源”可以是指发布图像的网站或服务器的域名或网址，或者，“来源”可以是指发布图像的运营商或制造商的名称。
或者，在本发明实施例中，可以按照名称，将多个全景图像(或者说，VR全景图像、球形全景图像或立方体全景图像)划分为多个图像集合，以使具有同一名称(或者，名称包括同一字符串)的图像属于相同的集合。从而，处理节点能够根据背景图像的名称，确定该背景图像所属的图像集合(即，第一图像集合)。
并且，作为实例而非限定，在本发明实施例中，“名称”可以是图像的全部名称，或者“名称”也可以是指图像的部分名称，本发明并未特别限定。
再或者，在本发明实施例中，可以按照格式，将多个全景图像(或者说，VR全景图像、球形全景图像或立方体全景图像)划分为多个图像集合，以使格式相同的图像属于相同的集合。从而，处理节点能够根据背景图像的格式，确定该背景图像所属于的图像集合(即，第一图像集合)。
并且，作为实例而非限定，在本发明实施例中，“格式”可以是指图像的编码方式，或者，“格式”也可以是指图像的保存格式，本发明并未特别限定。
其后，处理节点可以根据该映射关系，查找该第一图像集合所对应的图像集合，并将该将该第一图像集合对应的提取模型作为该第一提取模型。
从而，处理节点能够根据该第一提取模型，确定该背景图像的内容区域。
例如，图6示出了本发明实施例的第一提取模板的一例，如图6所示，该第一提取模板可以是例如，黑白二值图，并且，该第一提取模板的大小或者说，长宽比例)可以与背景图像的大小相同，其中，第一提取模板的白色部分(或者说，像素值为255的像素点占据的区域)对应背景图像的背景区域，第一提取模板的黑色部分(或者说，像素值为0的像素点占据的区域)对应背景图像的背景区域。
即，在本发明实施例中，背景区域在背景图像中的相对位置，与白色部分在第一提取模板中的相对位置相同。
从而，处理节点可以将背景图像中位置与白色部分在第一提取模板中的位置相对应的区域确定为背景区域。
类似地，在本发明实施例中，内容区域在背景图像中的相对位置，与黑色部分 在第一提取模板中的相对位置相同。
从而，处理节点可以将背景图像中位置与黑色部分在第一提取模板中的位置相对应的区域确定为内容区域。
方式2
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及
该确定该背景图像的内容区域，包括：
根据该第一阈值，确定该背景图像的内容区域。
具体的说，在本发明实施例中，该背景区域的与内容区域邻接的边缘(即，第一边界区域)包括的像素的像素值可以大于或等于第一阈值，并且，该内容区域的与背景区域邻接的边缘(即，第二边界区域)包括的像素的像素值可以小于第一阈值。
从而，服务器或使用者可以将该第一阈值输入至该处理节点。
处理节点能够根据该第一阈值，识别出该第一边界区域和第二边界区域，即，识别出背景区域和内容区域的边界，进而，处理节点能够识别出该内容区域。
应理解，以上列举的处理节点确定背景图像的内容区域的方法仅为示例性说明，本发明并未限定于此，例如，在本发明实施例中，该背景区域中的所有像素的像素值可以大于或等于第一阈值，并且，内容区域中的所有像素的像素值可以小于第一阈值。
从而，处理节点可以将像素值大于或等于第一阈值的像素点构成的区域识别为背景区域，并且，处理节点可以将像素值小于第一阈值的像素点构成的区域识别为内容区域。
在确定内容区域之后，在S134，处理节点可以根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像。
在本发明实施例中，处理节点可以根据所确定的背景图像的内容区域的一种或多种参数，对目标图像进行像素重构(或者说，变形处理)，并将像素重构(或者说，变形处理)后的图像作为待拼接图像。
作为实例而非限定，内容区域的一种或多种参数可以包括但不限于以下至少一种参数：
该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
具体的说，在本发明实施例中，由于背景图像为全景图像(例如，球形全景图像或立方体全景图像)，因此，对于在正常图像(非全景图像中)中人眼能够正常识别的图形A(例如，内容区域对应的图形)，如果将该图形A置于该背景图像中，并使在该背景图像被VR设备播放时，人眼能够正常识别图形A呈现，则人眼裸视(即，不使用VR设备)观看该背景图像时，该图形A较在普通图像中的呈现相比，会发生变形。例如，图5所示的电影屏幕，在普通图像中呈现为矩形，在全景图像中呈现为该矩形的水平方向上的边具有预设弧度。
并且，在本发明实施例中，上述变形与以下至少一种参数有关：
1.图形A配置在背景图像中的位置，即，图形A在背景图像中的位置不同，该图像A发生的变形也不同，例如，在图5所示的背景图像中，设图像A为电影屏幕(即，内容区域的一例)，则该电影屏幕在背景图像中心位置时，变形较小。并且，例如，该电影屏幕靠近背景图像的南北两极时，变形较大。
2.图形A配置在背景图像中的大小，即，图形A的大小不同时，该图像A发生的变形也不同，例如，在图5所示的背景图像中，设图像A为电影屏幕(即，内容区域的一例)，设该电影屏幕在背景图像中心位置时，如果该电影屏幕较大，则其变形也相应较大。并且，如果该电影屏幕较小，则其变形也相应较小。
3.图形A配置在背景图像中的范围，其中，该“范围”可以是包括上述“位置”和“大小”两层含义，这里，为了避免赘述，省略其详细说明。
4.背景图像的视角，即，当背景图像的视角不同时，图形A的变形也相应变化，例如，背景图像在垂直方向为360度视角时图形A的变形较大，背景图像在垂直方向为90度视角时图形A的变形较小。
5.该背景图像的展开方式，即，当背景图像的展开方式不同时图形A的变形也相应变化，例如，背景图像为球形全景图像是图形A的变形较大，当背景图像为立方体全景图像时，图形A的变形较小。
从而，处理节点可以根据上述参数，计算内容区域发生的变形，并基于内容区域的变形对目标图像进行像素重构。
作为实例而非限定，如图5所示，在本发明实施例中，该背景图像的展开方式为球形展开，即，该背景图像可以为球形全景图像。
此情况下，该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一变形方式后形成的形状，该第一变形方式为将该第二矩形的水平方向上的边变形为具有预设弧度的边。
即，如图5所示，该内容区域可以形成为近似为第二矩形，与第二矩形不同的是，该内容区域的两条水平方向上的边形成为具有预设弧度。
此情况下，处理节点可以根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
具体的说，在本发明实施例中，处理节点可以根据上述第一提取模型，逐一遍历背景图像的像素点，如果背景图像中像素点的位置(例如，在直角坐标系中的坐标)与第一提取模型中像素值为0(即，黑色)的像素点的位置相对应，则将该背景图像中位于该位置的像素点作为内容区域的各像素点。处理节点可以记录内容区域的各像素点的位置(例如，坐标)，并且，处理节点记录内容区域的四个边角顶点的坐标，以及，内容区域在高度(例如，Y轴)方向的最小值和最大值。
其后，处理节点可以根据如上所示确定的四个边角顶点的坐标，确定第二矩形的长宽比，进而，处理节点可以根据该长宽比，确定目标图像的缩放比例，并对目标图像进行缩放，使缩放后的目标图像的大小与第二矩形的大小相同或近似相同，并且，使缩放后的目标图像的长宽比与第二矩形的长宽比相同或近似相同。
其后，处理节点可以根据如上所述获取的内容区域的各像素点的位置在高度(例如，Y轴)方向上的变化，确定该预设弧度，进而确定该预设弧度相对应的拉伸比，并基于该拉伸比对经过缩放处理的目标图像进行高度(例如，Y轴)方向上拉伸处理。作为实例而非限定，在本发明实施例中，该拉伸处理可以是指，对目标图像(具体的说，是经过缩放后的目标图像)的位于水平方向上的边(即，也可以称为宽度方向上的边)附近的像素进行插值计算而使该目标图像的水平方向上的边具有上述预设弧度。
从而，目标图像经过上述处理后，能够像素重构为待拼接图像，其中，
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
具体地说，在本发明实施例中，该待拼接图像的大小可以与该背景图像的大小相同或近似相同，该待拼接图像的形状可以与该背景图像的内容区域的大小形状相同或近似相同。
图8示出了经过上述处理而获得的待拼接图像的一例，如图8所示，该待拼接图像的形状与图5所示的背景图像的内容区域的形状相似。
在确定待拼接图像后，在S136，处理节点可以基于待拼接图像和背景图像(具体的说，是背景图像的背景区域)，确定待播放图像。
例如，该对待拼接图像和该背景图像进行合成处理，包括：
根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素。
具体的说，在本发明实施例中，该待播放图像可以是大小与背景图像的大小相同的图像，即，在本发明实施例中，待播放图像包括的像素点的数量与背景图像包括的像素点的数量可以相同。
此情况下，在本发明实施例中，处理节点可以确定待播放图像的多个像素点位置与背景图像的多个像素点位置之间的一一映射关系。例如，设待播放图像中的像素点位置A的坐标为(α，β)，设背景图像中的像素点位置A’的坐标为(α’，β’)，如果α＝α’，且β＝β’，则处理节点可以确定像素点位置A与像素点位置A’对应。
其中，处理节点可以判定待播放图像中的像素点位置i所对应的区域，具体的说，是待播放图像中的像素点位置i所对应的背景图像中的像素点位置i’所属于的区域(即，背景区域或内容区域)，其中，i∈[1，w]，并且，i’∈[1，w]，w为背景图像包括的像素点的数量。
例如，如果像素点位置i’属于背景区域，则处理节点可以将像素点位置i’的像素值确定为像素点位置i的像素值。
再例如，如果像素点位置i’属于内容区域，则处理节点可以从待拼接图像中确定位置与该像素点位置i’相对应的像素点i”，并将所确定的像素点i”的像素值，确 定为像素点位置i的像素值。具体的说，处理节点可以确定待拼接图像的多个像素点位置与内容区域的多个像素点位置之间的一一映射关系。例如，设待拼接图像中的像素点位置B的坐标为(λ，η)，设内容区域中的像素点位置B’的坐标为(λ’，η’)，如果λ＝λ’，且η＝η’，则处理节点可以确定像素点位置B与像素点位置B’对应。
或者，该对待拼接图像和该背景图像进行合成处理，包括：
将该待拼接图像的像素覆盖该背景图像的内容区域的部分或全部像素。
具体地说，在本发明实施例中，处理节点可以将待拼接图像设置为图层1，将背景图像设置为图层2，其中，图层1的大小与图层2的大小相同或近似相同，并且，待拼接图像在图层1中的相对位置，与内容区域在图像2中的相对位置相同或近似相同。
其后，处理节点可以将图层1中除待拼接图像以外的区域设置为透明，其后，处理节点可以将图层1覆盖至图层2之上，从而完成待拼接图像与背景图像的合并处理。
或者，该对待拼接图像和该背景图像进行合成处理，包括：
将该待拼接图像的像素替换该背景图像的内容区域的部分或全部像素。
具体地说，在本发明实施例中，处理节点可以确定待拼接图像的多个像素点位置与内容区域的多个像素点位置之间的一一映射关系。例如，设待拼接图像中的像素点位置B的坐标为(λ，η)，设内容区域中的像素点位置B’的坐标为(λ’，η’)，如果λ＝λ’，且η＝η’，则处理节点可以确定像素点位置B与像素点位置B’对应。其后，处理节点可以将像素点位置B’的像素值替换为像素点位置B的像素的像素值。
从而，经过上述合并处理，能够生成待播放图像，其中，该待播放图像包括背景图像中的背景区域中的像素(即，第一区域中的像素的一例，)，并且，该待播放图像包括待拼接图像(即，基于目标图像确定的图像)中的像素(即，第二区域中的像素的一例)，即，该待播放图像包括用于呈现虚拟场景的图像(即，背景图像的背景区域中的像素构成的图像)，并且，该待播放图像包括需要在呈现虚拟场景观看的图像(即，基于目标图像确定的像素)。进而，当该待播放图像被VR设备播放时，能够同时呈现背景图像和目标图像双方的内容，即，能够通过播放待播放图像，实 现在于背景图像对应的虚拟场景中观看目标图像的过程，进而，能够降低VR设备的处理负担。
图9示出了经过上述处理而生成的待播放图像的一例。图10是该待播放图像经VR设备播放后的效果，如图10所示，该待播放图像中与待拼接图像(或者说，目标图像)相对应的部分(即，第二区域)所呈现的图像，与图7所示的目标图像相同或近似相同。
应理解，以上列举的生成待播放图像的过程仅为示例性说明，本发明并未限定于此，例如，用户也可以通过控制指令等使处理节点确定目标图像需要嵌入至背景图像中的拼接区域。
从而，处理节点可以根据上述控制指令，在不对目标图像进行像素重构的情况下，直接将目标图像覆盖在背景图像中的拼接区域上。
或者，处理节点可以根据上述控制指令，在不对目标图像进行像素重构的情况下，直接将背景图像的拼接区域中的像素替换为目标图像中的像素。
通过获取作为全景图像(具体的说，是球形全景图像或立方体全景图像)的背景图像和作为非全景图像的目标图像，并将该背景图像和目标图像进行合成处理，能够生产作为全景图像的待播放图像，该待播放图像包括第一区域和第二区域，第一区域包括背景图像中用于呈现虚拟背景的像素，该第二区域包括目标图像对应的像素，从而，当该待播放图像被VR设备播放时，能够同时呈现背景图像和目标图像双方的内容，即，能够通过播放待播放图像，实现在与背景图像对应的虚拟场景中观看目标图像的过程，进而，能够降低VR设备的处理负担。此外，要实现在虚拟场景中观看目标图像，可以将该制作生成的一个待播放图像在多个VR设备中进行播放，而无需每个VR设备均通过类似更换播放器皮肤的方式同步播放两个图像，即虚拟背景图像和现实图像。
并且，在现有技术中，背景图像保存在VR设备中，不同VR设备中保存的背景图像也可能不同，可能导致用户无法在不同VR设备中观看到同一虚拟场景，与此相对，由于背景图像的内容与目标图形的内容均承载于待播放图像，在任意VR设备均能够实现在用户期望的场景中观看目标图像。
该获取目标图像，包括：从目标视频中获取该目标图像，其中，该目标视频为非全景视频，该目标视频包括多帧第一图像，该目标图像是该多帧图像中的任一帧 第一图像。
具体的，在本发明实施例中，处理节点可以确定一个非全景视频(即，目标视频)文件中的每一帧图像，并对每一帧图像作为上述目标图像，并与背景图像进行上述合并处理，从而生成多帧待播放图像，并且，处理节点可以将多帧待播放图像进行合成(具体的说，是合成为生成图像序列)，并与目标视频的音频文件进行合成，从而能够生成全景视频(或者说，VR视频)。
该获取背景图像，包括：从背景视频中获取该背景图像，该背景视频为球形全景视频或立方体全景视频，该背景视频包括多帧第二图像，该背景图像是该多帧第二图像中的任意一帧图像。
具体的说，在本发明实施例中，该背景图像也可以取自一个全景视频(即，背景视频)。
根据本发明实施例的图像处理的方法，通过将目标视频中的每一帧图像作为目标图像，并进行上述处理过程，能够获取多帧作为球星全景图像或立方体全景图像的待播放图像，并将所生成的多帧图像按时序组合，能够生成球形全景视频或立方体全景视频，该全景视频在通过VR设备播放后，能够呈现基于背景图像生成的虚拟背景，以及目标视频的视频内容。
图11是本发明实施例的图像处理的装置200的示意性框图。如图11所示，该装置200包括：
获取单元210，用于获取背景图像和目标图像，该背景图像为球形全景图像或立方体全景图像，该目标图像为非全景图像；
处理单元220，用于对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及
如图12所示，该处理单元220包括：提取模块222，用于确定该背景图像的内容区域，
该处理单元220包括：重构模块224，用于根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像，
该处理单元220包括：合成模块226，用于对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
该合成模块226具体用于根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或
该合成模块226具体用于将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
其中，该重构模块224具体用于根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：
该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
其中，该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及
该重构模块224具体用于根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
该提取模块222具体用于获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围，用于根据该第一提取模型，确定该背景图像的内容区域。
该提取模块222具体用于获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围，用于确定该背景图像该属于的第一图像集合，用于根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该提取模块222具体用于根据以下至少一种信息，确定该背景图像该属于的第 一图像集合：
该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及
该提取模块222具体用于根据该第一阈值，确定该背景图像的内容区域。
该图像处理的装置200可以对应(例如，可以配置于或本身即为)上述方法100中描述的处理节点，并且，该图像处理的装置200中各模块或单元分别用于执行上述方法100中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。
通过获取作为全景图像(具体的说，是球形全景图像或立方体全景图像)的背景图像和作为非全景图像的目标图像，并将该背景图像和目标图像进行合成处理，能够生产作为全景图像的待播放图像，该待播放图像包括第一区域和第二区域，第一区域包括背景图像中用于呈现虚拟背景的像素，该第二区域包括目标图像对应的像素，从而，当该待播放图像被VR设备播放时，能够同时呈现背景图像和目标图像双方的内容，即，能够通过播放待播放图像，实现在于背景图像对应的虚拟场景中观看目标图像的过程，进而，能够降低VR设备的处理负担。
并且，在现有技术中，背景图像保存在VR设备中，不同VR设备中保存的背景图像也可能不同，可能导致用户无法在不同VR设备中观看到同一虚拟场景，与此相对，由于背景图像的内容与目标图形的内容均承载于待播放图像，在任意VR设备均能够实现在用户期望的场景中观看目标图像。
图13描述了本发明实施例提供的图像处理的设备300的结构，该图像处理的设备300包括：至少一个处理器301，至少一个网络接口304或者其他用户接口303，存储器305，至少一个通信总线302。通信总线302用于实现这些组件之间的连接通信。
该终端设备300包含用户接口303，包括显示器(例如，触摸屏、LCD、CRT、全息成像设备或者投影设备等)，键盘或者点击设备(例如，鼠标，轨迹球(trackball)，触感板或者触摸屏等)。
存储器305可以包括只读存储器和随机存取存储器，并向处理器301提供指令，例如机器可读指令和数据。存储器305的一部分还可以包括非易失性随机存取存储器(NVRAM)。
在一些实施方式中，存储器305存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集：
操作系统3051，包含各种系统程序，例如图1所示的框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务；
应用程序模块3052，包含各种应用程序，例如图1所示的桌面(launcher)、媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。
在本发明实施例中，通过调用存储器305存储的程序或指令，处理器301用于：获取背景图像和目标图像，该背景图像为球形全景图像或立方体全景图像，该目标图像为非全景图像，用于对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及
处理器301具体用于确定该背景图像的内容区域，用于根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像，用于对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
处理器301具体用于根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或
该处理器301具体用于将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的形状的差异在预设的第二范围内。
其中，该处理器301具体用于根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：
该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
其中，该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及
该处理器301具体用于根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
该处理器301具体用于获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围，用于根据该第一提取模型，确定该背景图像的内容区域。
该处理器301具体用于获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围，用于确定该背景图像该属于的第一图像集合，用于根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该处理器301具体用于根据以下至少一种信息，确定该背景图像该属于的第一图像集合：
该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及
该处理器301具体用于根据该第一阈值，确定该背景图像的内容区域。
该图像处理的设备300可以对应(例如，可以配置于或本身即为)上述方法100中描述的处理节点，并且，该图像处理的设备300中各模块或单元分别用于执行上述方法100中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。
本发明实施例还提供了一种计算机程序产品，该计算机程序产品包括：计算机 程序代码，当该计算机程序代码被处理节点(例如，上述图像处理的装置或设备，具体地说，是图像处理的装置或设备的处理单元或处理器)运行时，使得处理节点执行以下动作：
获取背景图像，该背景图像为球形全景图像或立方体全景图像；
获取目标图像，该目标图像为非全景图像；
对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
当该计算机程序代码被处理节点运行时，还使得处理节点确定该背景图像的内容区域；根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像；对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
当该计算机程序代码被处理节点运行时，还使得处理节点根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的形状的差异在预设的第二范围内。
当该计算机程序代码被处理节点运行时，还使得处理节点根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及，当该计算机程序代码被处理节点运行时，还使得处理节点根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
当该计算机程序代码被处理节点运行时，还使得处理节点获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围；根据该第一提取模型，确定该背景图像的内容区域。
当该计算机程序代码被处理节点运行时，还使得处理节点获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围；确定该背景图像该属于的第一图像集合；根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
当该计算机程序代码被处理节点运行时，还使得处理节点根据以下至少一种信息，确定该背景图像该属于的第一图像集合：该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及，当该计算机程序代码被处理节点运行时，还使得处理节点根据该第一阈值，确定该背景图像的内容区域。
本发明实施例还提供了一种计算机可读存储介质，该计算机可读存储介质存储有程序，该程序使得处理节点(例如，上述图像处理的装置或设备，具体地说，是图像处理的装置或设备的处理单元或处理器)执行以下动作：
获取背景图像，该背景图像为球形全景图像或立方体全景图像；
获取目标图像，该目标图像为非全景图像；
对该目标图像和该背景图像进行合成处理，以生成待播放图像，该待播放图像为球形全景图像或立方体全景图像，且该待播放图像包括第一区域和第二区域，该第一区域包括根据该背景图像获得的像素，该第二区域包括根据该目标图像获得的像素。
该程序还使得处理节点确定该背景图像的内容区域；根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像；对待拼接图像和该背景图像 进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
该程序还使得处理节点根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的形状的差异在预设的第二范围内。
该程序还使得处理节点根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及，该程序还使得处理节点根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
该程序还使得处理节点获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围；根据该第一提取模型，确定该背景图像的内容区域。
该程序还使得处理节点获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围；确定该背景图像该属于的第一图像集合；根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该程序还使得处理节点根据以下至少一种信息，确定该背景图像该属于的第一图像集合：该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值， 与该第二边界区域中的任一像素的像素值小于该第一阈值，以及，该程序还使得处理节点根据该第一阈值，确定该背景图像的内容区域。
通过获取作为全景图像(具体的说，是球形全景图像或立方体全景图像)的背景图像和作为非全景图像的目标图像，并将该背景图像和目标图像进行合成处理，能够生产作为全景图像的待播放图像，该待播放图像包括第一区域和第二区域，第一区域包括背景图像中用于呈现虚拟背景的像素，该第二区域包括目标图像对应的像素，从而，当该待播放图像被VR设备播放时，能够同时呈现背景图像和目标图像双方的内容，即，能够通过播放待播放图像，实现在于背景图像对应的虚拟场景中观看目标图像的过程，进而，能够降低VR设备的处理负担。
并且，在现有技术中，背景图像保存在VR设备中，不同VR设备中保存的背景图像也可能不同，可能导致用户无法在不同VR设备中观看到同一虚拟场景，与此相对，由于背景图像的内容与目标图形的内容均承载于待播放图像，在任意VR设备均能够实现在用户期望的场景中观看目标图像。
图14示出了本发明实施例的视频处理的方法400的示意性流程图，如图14所示，该视频处理的方法400包括：
S410，获取背景视频，该背景视频包括至少一帧图像，该背景视频包括的每一帧图像为球形全景图像或立方体全景图像；
在本发明实施例中，该背景视频的获取方法和方式可以与现有技术相似，这里，为了避免赘述省略其详细说明。
并且，该背景视频中的每一帧图像的具体形式可以与上述方法100中描述的背景图像的具体形式相似，这里，为了避免赘述，省略其详细说明。
S420，获取目标视频，该目标视频包括至少一帧图像，该目标视频包括的每一帧图像为非全景图像；
在本发明实施例中，该目标视频的获取方法和方式可以与现有技术相似，这里，为了避免赘述省略其详细说明。
并且，该目标视频中的每一帧图像的具体形式可以与上述方法100中描述的目标图像的具体形式相似，这里，为了避免赘述，省略其详细说明。
S430，对该目标视频和该背景视频进行合成处理，以生成待播放视频，该背景视频包括至少一帧图像，其中，该待播放视频中的每一帧图像为球形全景图像或立 方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括根据该背景视频获得的像素，该第二区域包括根据该目标视频获得的像素。
具体的说，作为示例而非限定，该对该目标视频和该背景视频进行合成处理，包括：对目标图像和背景图像进行合成处理，以生成待播放图像，其中，该目标图像是该目标视频中的任意一帧图像，该背景图像是该背景视频中的任意一帧图像，且该待播放图像的第一区域包括根据该背景图像获得的像素，该待播放图像的第二区域包括根据该目标图像获得的像素。
例如，在本发明实施例中，如果背景视频只包括一帧图像，则处理节点可以将目标视频中每一帧图像分别与该背景视频中的一帧图像进行合成，从而生成待播放视频。并且，这里，目标视频中每一帧图像与该背景视频中的一帧图像的合成处理的具体过程可以与上述方法100中描述的目标图像与背景图像的合成过程相似，这里，为了避免赘述，省略其详细说明。
再例如，在本发明实施例中，如果背景视频包括多帧图像，则处理节点可以确定多个图像组，其中每个图像组包括目标视频中的一帧图像以及背景视频中的一帧图像，其中，任意两个图像组包括的目标视频中的图像相异，从而，处理节点可以将每个图像组中的目标视频中的图像与背景视频中的图像进行合成，从而生成待播放视频。并且，这里，处理节点将目标视频中一帧图像与背景视频中的一帧图像合成的具体过程可以与上述方法100中描述的目标图像与背景图像的合成过程相似，这里，为了避免赘述，省略其详细说明。
根据本发明实施例的视频处理的方法，通过获取作为全景视频(具体的说，是球形全景视频或立方体全景视频)的背景视频和作为非全景视频的目标视频，并将该背景视频和目标视频进行合成处理，能够生产作为全景视频的待播放视频，该待播放视频包括第一区域和第二区域，第一区域包括背景视频中用于呈现虚拟背景的像素，该第二区域包括目标视频对应的像素，从而，当该待播放视频被VR设备播放时，能够同时呈现背景视频和目标视频双方的内容，即，能够通过播放待播放视频，实现在于背景视频对应的虚拟场景中观看目标视频的过程，进而，能够降低VR设备在播放虚拟影院视频时的处理负担。
图15是本发明实施例的图像处理的装置500的示意性框图。如图15所示，该装置500包括：
获取单元510，用于获取背景视频和目标视频，该背景视频包括至少一帧图像，该背景视频包括的每一帧图像为球形全景图像或立方体全景图像，该目标视频包括至少一帧图像，该目标视频包括的每一帧图像为非全景图像；
处理单元520，用于对该目标视频和该背景视频进行合成处理，以生成待播放视频，该背景视频包括至少一帧图像，其中，该待播放视频中的每一帧图像为球形全景图像或立方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括该背景视频中的像素，该第二区域包括根据该目标视频获得的像素。
该处理单元520具体用于对目标图像和背景图像进行合成处理，以生成待播放图像，其中，该目标图像是该目标视频中的任意一帧图像，该背景图像是该背景视频中的任意一帧图像，且该待播放图像的第一区域包括根据该背景图像获得的像素，该待播放图像的第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及
该处理单元520具体用于确定该背景图像的内容区域，用于根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像，用于对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
该处理单元520具体用于根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或
该处理单元520具体用于将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
其中，该处理单元520具体用于根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：
该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
其中，该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及
该处理单元520具体用于根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
该处理单元520具体用于获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围，用于根据该第一提取模型，确定该背景图像的内容区域。
该处理单元520具体用于获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围，用于确定该背景图像该属于的第一图像集合，用于根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该处理单元520具体用于根据以下至少一种信息，确定该背景图像该属于的第一图像集合：
该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及
该处理单元520具体用于根据该第一阈值，确定该背景图像的内容区域。
该视频处理的装置500可以对应(例如，可以配置于或本身即为)上述方法400中描述的处理节点，并且，该图像处理的装置500中各模块或单元分别用于执行上述方法400中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。
图16描述了本发明实施例提供的视频处理的设备600的结构，该图像处理的设备600包括：至少一个处理器601，至少一个网络接口604或者其他用户接口603，存储器605，至少一个通信总线602。通信总线602用于实现这些组件之间的连接通信。
该终端设备600包含用户接口603，包括显示器(例如，触摸屏、LCD、CRT、全息成像设备或者投影设备等)，键盘或者点击设备(例如，鼠标，轨迹球(trackball)，触感板或者触摸屏等)。
存储器605可以包括只读存储器和随机存取存储器，并向处理器601提供指令，例如机器可读指令和数据。存储器605的一部分还可以包括非易失性随机存取存储器(NVRAM)。
在一些实施方式中，存储器605存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集：
操作系统6051，包含各种系统程序，例如图1所示的框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务；
应用程序模块6052，包含各种应用程序，例如图1所示的桌面(launcher)、媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。
在本发明实施例中，通过调用存储器605存储的程序或指令，处理器601用于获取背景视频和目标视频，该背景视频包括至少一帧图像，该背景视频包括的每一帧图像为球形全景图像或立方体全景图像，该目标视频包括至少一帧图像，该目标视频包括的每一帧图像为非全景图像；
处理器601用于对该目标视频和该背景视频进行合成处理，以生成待播放视频，该背景视频包括至少一帧图像，其中，该待播放视频中的每一帧图像为球形全景图像或立方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括该背景视频中的像素，该第二区域包括根据该目标视频获得的像素。
该处理器601具体用于对目标图像和背景图像进行合成处理，以生成待播放图像，其中，该目标图像是该目标视频中的任意一帧图像，该背景图像是该背景视频中的任意一帧图像，且该待播放图像的第一区域包括根据该背景图像获得的像素，该待播放图像的第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及
该处理器601具体用于确定该背景图像的内容区域，用于根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像，用于对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
该处理器601具体用于根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素；或
该处理器601具体用于将该待拼接图像的像素覆盖或替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
其中，该处理器601具体用于根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：
该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
其中，该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度，以及
该处理器601具体用于根据该第二矩形的长宽比例和该预设弧度，对该目标图像进行像素重构。
该处理器601具体用于获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围，用于根据该第一提取模型，确定该背景图像的内容区域。
该处理器601具体用于获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围，用于确定该背景图像该属于的第一图像集合，用于根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该处理器601具体用于根据以下至少一种信息，确定该背景图像该属于的第一图像集合：
该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域 相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及
该处理器601具体用于根据该第一阈值，确定该背景图像的内容区域。
该视频处理的设备600可以对应(例如，可以配置于或本身即为)上述方法400中描述的处理节点，并且，该视频处理的设备600中各模块或单元分别用于执行上述方法400中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。
本发明实施例还提供了一种计算机程序产品，该计算机程序产品包括：计算机程序代码，当该计算机程序代码被处理节点(例如，上述图像处理的装置或设备，具体地说，是图像处理的装置或设备的处理单元或处理器)运行时，使得处理节点执行以下动作：
获取背景视频，该背景视频包括至少一帧图像，该背景视频包括的每一帧图像为球形全景图像或立方体全景图像；
获取目标视频，该目标视频包括至少一帧图像，该目标视频包括的每一帧图像为非全景图像；
对该目标视频和该背景视频进行合成处理，以生成待播放视频，该背景视频包括至少一帧图像，其中，该待播放视频中的每一帧图像为球形全景图像或立方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括该背景视频中的像素，该第二区域包括根据该目标视频获得的像素。
当该计算机程序代码被处理节点运行时，还使得处理节点对目标图像和背景图像进行合成处理，以生成待播放图像，其中，该目标图像是该目标视频中的任意一帧图像，该背景图像是该背景视频中的任意一帧图像，且该待播放图像的第一区域包括根据该背景图像获得的像素，该待播放图像的第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及当该计算机程序代码被处理节点运行时，还使得处理节点确定该背景图像的内容区域；根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像；对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接 图像中的像素。
当该计算机程序代码被处理节点运行时，还使得处理节点根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素。
当该计算机程序代码被处理节点运行时，还使得处理节点将该待拼接图像的像素覆盖该背景图像的内容区域的部分或全部像素。
当该计算机程序代码被处理节点运行时，还使得处理节点将该待拼接图像的像素替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
该待拼接图像在通过虚拟现实VR设备播放时呈现的内容为与该目标图像呈现的内容相同。
当该计算机程序代码被处理节点运行时，还使得处理节点根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为：第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度
当该计算机程序代码被处理节点运行时，还使得处理节点根据该第二矩形的长宽比和该预设弧度，对该目标图像进行像素重构。
当该计算机程序代码被处理节点运行时，还使得处理节点获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围；根据该第一提取模型，确定该背景图像的内容区域。
当该计算机程序代码被处理节点运行时，还使得处理节点获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模 型用于指示该对应的图像集合中的全景图像中的内容区域的范围；确定该背景图像该属于的第一图像集合；根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
当该计算机程序代码被处理节点运行时，还使得处理节点根据以下至少一种信息，确定该背景图像该属于的第一图像集合：该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及当该计算机程序代码被处理节点运行时，还使得处理节点根据该第一阈值，确定该第一边界区域和第二边界区域，根据该第一边界区域和第二边界区域，确定该背景图像的内容区域。
该背景区域中的任一像素的像素值大于或等于第一阈值，该内容区域中的任一像素的像素值小于该第一阈值，以及该确定该背景图像的内容区域，包括：根据该第一阈值，确定该背景图像的内容区域。
所述第二区域在所述待播放图像中的范围与所述内容区域在所述背景图像中的范围与所述第一区域在所述第一全景图像中的范围的差异在预设的第三范围内。
本发明实施例还提供了一种计算机可读存储介质，该计算机可读存储介质存储有程序，该程序使得处理节点(例如，上述图像处理的装置或设备，具体地说，是图像处理的装置或设备的处理单元或处理器)执行以下动作：
获取背景视频，该背景视频包括至少一帧图像，该背景视频包括的每一帧图像为球形全景图像或立方体全景图像；
获取目标视频，该目标视频包括至少一帧图像，该目标视频包括的每一帧图像为非全景图像；
对该目标视频和该背景视频进行合成处理，以生成待播放视频，该背景视频包括至少一帧图像，其中，该待播放视频中的每一帧图像为球形全景图像或立方体全景图像，该待播放视频中的每一帧图像包括第一区域和第二区域，该第一区域包括该背景视频中的像素，该第二区域包括根据该目标视频获得的像素。
该程序还使得处理节点对目标图像和背景图像进行合成处理，以生成待播放图 像，其中，该目标图像是该目标视频中的任意一帧图像，该背景图像是该背景视频中的任意一帧图像，且该待播放图像的第一区域包括根据该背景图像获得的像素，该待播放图像的第二区域包括根据该目标图像获得的像素。
该背景图像包括背景区域和内容区域，以及该程序还使得处理节点确定该背景图像的内容区域；根据该背景图像的内容区域，对该目标图像进行像素重构，以生成待拼接图像；对待拼接图像和该背景图像进行合成处理，其中，该第一区域包括该背景区域中的像素，该第二区域包括该待拼接图像中的像素。
该程序还使得处理节点根据该背景区域中的像素确定该第一区域中的像素，并根据该待拼接图像的像素确定该第二区域中的像素。
该程序还使得处理节点将该待拼接图像的像素覆盖该背景图像的内容区域的部分或全部像素。
该程序还使得处理节点将该待拼接图像的像素替换该背景图像的内容区域的部分或全部像素。
该待拼接图像的大小与该背景图像的内容区域的大小的差异在预设的第一范围内，该待拼接图像的形状与该背景图像的内容区域的大小形状的差异在预设的第二范围内。
该待拼接图像在通过虚拟现实VR设备播放时呈现的内容为与该目标图像呈现的内容相同。
该程序还使得处理节点根据该目标图像和以下至少一种参数，对该目标图像进行像素重构：该背景图像的内容区域在该背景图像中的范围、该背景图像的内容区域的形状、该背景图像的内容区域的大小、该背景图像的内容区域在该背景图像中的位置、该背景像的全景视角或该背景图像的展开方式。
该背景图像的形状为第一矩形，该背景图像的内容区域位于该第一矩形的中心位置，该背景图像的内容区域的形状为：第二矩形以第一方式变形后形成的形状，该第一方式为该第二矩形的水平方向上的边变形为具有预设弧度
该程序还使得处理节点根据该第二矩形的长宽比和该预设弧度，对该目标图像进行像素重构。
该程序还使得处理节点获取第一提取模型，该第一提取模型用于指示该背景图像的内容区域在该背景图像中的范围；根据该第一提取模型，确定该背景图像的内 容区域。
该程序还使得处理节点获取M个图像集合和M个提取模型之间的一一映射关系，其中，该M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，该M个提取模型中的每个提取模型用于指示该对应的图像集合中的全景图像中的内容区域的范围；确定该背景图像该属于的第一图像集合；根据该映射关系，将该第一图像集合对应的提取模型作为该第一提取模型。
该程序还使得处理节点根据以下至少一种信息，确定该背景图像该属于的第一图像集合：该背景图像的来源、该背景图像的名称或该背景图像的格式。
该背景区域包括第一边界区域，该第一边界区域是该背景区域中与该内容区域相邻的区域，该内容区域包括第二边界区域，该第二边界区域是该内容区域中与该背景区域相邻的区域，该第一边界区域中的任一像素的像素值大于或等于第一阈值，与该第二边界区域中的任一像素的像素值小于该第一阈值，以及该程序还使得处理节点根据该第一阈值，确定该第一边界区域和第二边界区域，根据该第一边界区域和第二边界区域，确定该背景图像的内容区域。
该背景区域中的任一像素的像素值大于或等于第一阈值，该内容区域中的任一像素的像素值小于该第一阈值，以及该确定该背景图像的内容区域，包括：根据该第一阈值，确定该背景图像的内容区域。
所述第二区域在所述待播放图像中的范围与所述内容区域在所述背景图像中的范围与所述第一区域在所述第一全景图像中的范围的差异在预设的第三范围内。
通过获取作为全景视频(具体的说，是球形全景视频或立方体全景视频)的背景视频和作为非全景视频的目标视频，并将该背景视频和目标视频进行合成处理，能够生产作为全景视频的待播放视频，该待播放视频包括第一区域和第二区域，第一区域包括背景视频中用于呈现虚拟背景的像素，该第二区域包括目标视频对应的像素，从而，当该待播放视频被VR设备播放时，能够同时呈现背景视频和目标视频双方的内容，即，能够通过播放待播放视频，实现在于背景视频对应的虚拟场景中观看目标视频的过程，进而，能够降低VR设备在播放虚拟影院视频时的处理负担。作为示例而非限定，该方法100或方法400可以使用于终端设备，本申请实施例中所涉及到的终端设备可以包括手持设备、车载设备、可穿戴设备、计算设备或连接到无线调制解调器的其它处理设备。还可以包括用户单元、蜂窝电话(cellular  phone)、智能手机(smart phone)、无线数据卡、个人数字助理(Personal Digital Assistant，PDA)电脑、平板型电脑、无线调制解调器(modem)、手持设备(handset)、膝上型电脑(laptop computer)、机器类型通信(Machine Type Communication，MTC)终端、无线局域网(Wireless Local Area Networks，WLAN)中的站点(STAION，ST)，可以是蜂窝电话、无绳电话、会话启动协议(Session Initiation Protocol，SIP)电话、无线本地环路(Wireless Local Loop，WLL)站以及下一代通信系统，例如，第五代通信(fifth-generation，简称“5G”)网络中的终端设备或者未来演进的公共陆地移动网络(Public Land Mobile Network，简称“PLMN”)网络中的终端设备等。
其中，可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。
并且，该终端设备还可以包括VR设备，其中VR设备也可以称为虚拟现实硬件，虚拟现实硬件指的是与虚拟现实技术领域相关的硬件产品，是虚拟现实解决方案中用到的硬件设备。
作为示例而非限定，在本发明实施例中，虚拟现实中常用到的硬件设备，大致可以包括但不限于以下类型的设备。
1、建模设备，例如，3D扫描仪。
2、三维视觉显示设备，例如，3D展示系统、大型投影系统、头显，例如，头戴式立体显示器、智能眼镜等。
3、声音设备，例如，三维的声音系统以及非传统意义的立体声。
4、交互设备，包括：位置追踪仪、数据手套、3D输入设备(例如，三维鼠标)、动作捕捉设备、眼动仪、力反馈设备以及其他交互设备。
图17是适用本发明实施例的图像处理的方法的终端设备的一例的示意性结构图。如图17所示，在本发明实施例中，该终端设备700可以包括：
本发明实施例的一种终端设备700，包括，第一存储器720、处理器760及输入单元730，该第一存储器720存储该终端预设数量的应用程序的界面信息，其中，该界面信息包括界面元素、界面编号、该界面编号与该界面元素的对应关系、以及该界面元素位于该界面编号对应的应用程序界面的位置信息；该输入单元730用于接收用户切换应用程序界面操作，并产生切换信号；该处理器760用于根据该切换信号确定目标界面编号；根据与该目标界面编号相邻的预定数量确定与该目标界面编号相邻的界面编号；根据该第一存储器720中存储的界面编号和与该目标界面编号相邻的界面编号，确定待加载的界面编号对应的界面信息；释放该第一存储器720中与该目标界面编号不相邻的至少部分界面编号对应的界面信息所占用的存储空间；加载该待加载的界面编号对应的界面信息到该第一存储器720中。
其中，该预设数量是指可以存储于该第一存储器的应用程序的界面信息的数量。该预定数量是指与该目标界面编号每一侧相邻的界面编号的数量。
该处理器760可以通过释放该第一存储器720中，与该目标界面编号不相邻的至少部分界面编号对应的界面信息所占用的存储空间，加载与该目标界面编号相邻的界面编号对应的界面信息于该第一存储器720中，从而可以循环加载界面信息，减缓终端设备700的存储容量的限制与日益增长的应用程序界面数量之间的矛盾。
其中，根据该第一存储器720中存储的界面编号和与该目标界面编号相邻的界面编号，确定待加载的界面编号对应的界面信息，具体为：根据该第一存储器720中存储的界面编号和与该目标界面编号相邻的界面编号，确定该第一存储器720中未存储的界面编号，该未存储的界面编号对应的界面信息为该第一存储器720中待加载的界面编号对应的界面信息。
需要说明的是，该处理器760可以调用该第一存储器720中存储的与该目标界面编号对应的界面元素、以及该界面元素显示于该界面编号对应的应用程序界面的位置信息，从而将该界面元素显示在与该目标界面编号对应的应用程序界面。其中，该界面元素可以是应用程序图标或widget桌面控件等。
本发明实施例中，该终端设备700还可以包括第二存储器721，该第二存储器721可以用于存储终端设备700的所有应用程序的界面信息。该处理器760加载该待加载的界面编号对应的界面信息到该第一存储器720中，具体为：该处理器760调用该第二存储器721中待加载的界面编号对应的界面信息，将该待加载的界面编 号对应的界面信息加载到该第一存储器720中。
可以理解的，该第二存储器721可以为该终端设备700的外存，该第一存储器720可以为该终端700的内存。该处理器760可以从该第二存储器721中加载预设数量的界面信息到该第一存储器720中。每个加载的界面信息在该第一存储器720中对应一个存储空间，每个存储空间可以相同。该第一存储器720可以为非易失性随机访问存储器(Non-Volatile Random Access Memory，NVRAM)、动态随机存取存储器(Dynamic Random Access Memory，DRAM)动态随机存储器、静态随机存取存储器(Static Random Access Memory，SRAM)静态随机存储器、闪存(Flash)等其中之一；该第二存储器721可以为硬盘、光盘、通用串行总线(Universal Serial Bus，USB)盘、软盘或磁带机等。
本发明实施例中，终端的所有界面信息可以存储在云服务器，该云服务器可以为第二存储器721。该处理器760加载该待加载的界面编号对应的界面信息到该第一存储器720中，具体为：该处理器760通过网络通道获取该云服务器中待加载的界面编号对应的界面信息，将该待加载的界面编号对应的界面信息加载到该第一存储器720中。
该输入单元730可用于接收输入的数字或字符信息，以及产生与终端700的用户设置以及功能控制有关的信号输入。具体地，本发明实施例中，该输入单元730可以包括触控面板731。触控面板731，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板731上或在触控面板731的操作)，并根据预先设定的程式驱动相应的连接装置。触控面板731可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给该处理器760，并能接收处理器760发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板731。除了触控面板731，输入单元730还可以包括其他输入设备732，其他输入设备732可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
该终端设备700还可以包括显示单元740，该显示单元740可用于显示由用户输入的信息或提供给用户的信息以及终端700的各种菜单界面。该显示单元740可 包括显示面板741，可以采用液晶显示器(Liquid Crystal Display，LCD)或有机发光二极管(Organic Light-Emitting Diode，OLED)等形式来配置显示面板741。
该显示单元740还可以显示上述待播放图像或待播放视频。
本发明实施例中，该触控面板731覆盖该显示面板741，形成触摸显示屏，当该触摸显示屏检测到在其上或附近的触摸操作后，传送给处理器760以确定触摸事件的类型，随后处理器760根据触摸事件的类型在触摸显示屏上提供相应的视觉输出。
本发明实施例中，该触摸显示屏包括应用程序界面显示区及常用控件显示区。该应用程序界面显示区及该常用控件显示区的排列方式并不限定，可以为上下排列、左右排列等可以区分两个显示区的排列方式。该应用程序界面显示区可以用于显示应用程序的界面。每一个界面可以包含至少一个应用程序的图标和/或widget桌面控件等界面元素。该应用程序界面显示区443也可以为不包含任何内容的空界面。该常用控件显示区用于显示使用率较高的控件，例如，设置按钮、界面编号、滚动条、电话本图标等应用程序图标等。
该处理器760是终端700的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在该第一存储器720内的软件程序和/或模块，以及调用存储在该第二存储器721内的数据，执行终端700的各种功能和处理数据，从而对终端700进行整体监控。该处理器760可包括一个或多个处理单元。
可以理解的，该处理器760初始化时，可以从该第二存储器721存储的界面信息中，加载预设数量的界面信息到该第一存储器720，并记录该预设数量的界面信息对应的界面编号，该处理器760读取该第一存储器720的任意一个或预定编号的界面信息，并根据该界面信息生成界面，控制该触摸显示屏的应用程序界面显示区显示该生成界面作为初始界面，并控制该常用控件显示区显示界面编号，提供用户选择界面，其中，该常用控件显示区显示的界面编号可以为该第一存储器720中已加载的界面信息对应的界面编号，也可以是该第二存储器721中存储的界面信息对应的界面编号。该预设数量不大于该第一存储器720所能存储该界面信息的最大数量。
进一步的，该处理器760可以控制该常用控件显示区显示的界面编号中的至少部分界面编号响应用户输入操作。
例如，该处理器760在该常用控件显示区显示的界面编号中，控制已加载的界面信息对应的界面编号可以响应用户的输入操作，未加载的界面信息对应的界面编号不响应用户的输入操作。
在本发明实施例中，该处理器760可以执行图11中方法400中的各步骤，这里，为了避免赘述，省略其详细说明。
图18是本发明实施例的VR装置800的示意性框图。如图18所示，该装置800包括：
处理单元810，用于执行上述方法100中的各步骤，以获取待播放图像，或，用于执行上述方法400中的各步骤，以获取待播放视频；
显示单元820，用于呈现该待播放图像或待播放视频。
该VR装置800可以对应(例如，可以包括、属于或本身即为)上述方法100或400中描述的处理节点，并且，该处理单元810用于执行上述方法100或400中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。并且，该显示单元820的动作可以与现有技术中VR显示装置呈现虚拟图像或虚拟视频时的动作相似，这里，为了避免赘述，省略其详细说明。
图19描述了本发明实施例提供的VR设备900的结构，该VR设备900包括：至少一个处理器901，至少一个网络接口904或者其他用户接口903，存储器905，显示器906，至少一个通信总线902。通信总线902用于实现这些组件之间的连接通信。
该显示器906可以为例如，触摸屏、LCD、CRT、全息成像设备或者投影设备等。
存储器905可以包括只读存储器和随机存取存储器，并向处理器901提供指令和数据。存储器905的一部分还可以包括非易失性随机存取存储器。
在一些实施方式中，存储器905存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集：
操作系统9051，包含各种系统程序，例如图1所示的框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务；
应用程序模块9052，包含各种应用程序，例如图1所示的桌面(launcher)、媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。
在本发明实施例中，通过调用存储器905存储的程序或指令，处理器901用于：执行上述方法100中的各步骤，以获取待播放图像，或，用于执行上述方法400中的各步骤，以获取待播放视频；该显示单元820，用于呈现该待播放图像或待播放视频。
该VR设备900可以对应(例如，可以包括、属于或本身即为)上述方法100或400中描述的处理节点，并且，该VR设备900中的处理器可以用于执行上述方法100或400中处理节点所执行的各动作或处理过程，这里，为了避免赘述，省略其详细说明。并且，该显示器906的动作可以与现有技术中VR显示器呈现虚拟图像或虚拟视频时的动作相似，这里，为了避免赘述，省略其详细说明。
应理解，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
应理解，在本发明实施例的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明实施例的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是 通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明实施例各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明实施例各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明实施例的具体实施方式，但本发明实施例的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明实施例揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明实施例的保护范围之内。

Claims (22)

  1. 一种图像处理方法，应用于图像处理装置，所述方法包括：
    获取背景图像，所述背景图像为球形全景图像或立方体全景图像；
    获取目标图像，所述目标图像为非全景图像；
    对所述目标图像和所述背景图像进行合成处理，以生成待播放图像，所述待播放图像为球形全景图像或立方体全景图像，且所述待播放图像包括第一区域和第二区域，所述第一区域包括根据所述背景图像获得的像素，所述第二区域包括根据所述目标图像获得的像素。
  2. 根据权利要求1所述的方法，其中，所述背景图像包括：背景区域和内容区域，以及
    所述对所述目标图像和所述背景图像进行合成处理，包括：
    确定所述背景图像的内容区域；
    根据所述背景图像的内容区域，对所述目标图像进行像素重构，以生成待拼接图像；
    对待拼接图像和所述背景图像进行合成处理，其中，所述第一区域包括所述背景区域中的像素，所述第二区域包括所述待拼接图像中的像素。
  3. 根据权利要求2所述的方法，其中，所述对待拼接图像和所述背景图像进行合成处理，包括：
    根据所述背景区域中的像素确定所述第一区域中的像素，并根据所述待拼接图像的像素确定所述第二区域中的像素；或
    利用所述待拼接图像的像素覆盖或替换所述背景图像的内容区域的部分或全部像素。
  4. 根据权利要求2或3所述的方法，其中，所述待拼接图像的大小与所述背景图像的内容区域的大小的差异在预设的第一范围内，所述待拼接图像的形状与所述背景图像的内容区域的形状的差异在预设的第二范围内。
  5. 根据权利要求2或3所述的方法，其中，所述根据所述背景图像的内容区域，对所述目标图像进行像素重构，包括：
    根据所述目标图像和以下至少一种参数，对所述目标图像进行像素重构：
    所述背景图像的内容区域在所述背景图像中的范围、所述背景图像的内容区域的形状、所述背景图像的内容区域的大小、所述背景图像的内容区域在所述背景图像中的位置、所述背景像的全景视角或所述背景图像的展开方式。
  6. 根据权利要求2或3所述的方法，其中，所述背景图像的形状为第一矩形，所述背景图像的内容区域位于所述第一矩形的中心位置，所述背景图像的内容区域的形状为第二矩形以第一方式变形后形成的形状，所述第一方式为将所述第二矩形的水平方向上的边变形为具有预设弧度的边，以及
    所述根据所述背景图像的内容区域，对所述目标图像进行像素重构，包括：
    根据所述第二矩形的长宽比例和所述预设弧度，对所述目标图像进行像素重构。
  7. 根据权利要求2或3所述的方法，其中，所述确定所述背景图像的内容区域，包括：
    获取第一提取模型，所述第一提取模型用于指示所述背景图像的内容区域在所述背景图像中的范围；
    根据所述第一提取模型，确定所述背景图像的内容区域。
  8. 根据权利要求7所述的方法，其中，所述获取第一提取模型，包括：
    获取预设的M个图像集合和M个提取模型之间的一一映射关系，其中，所述M个图像集合中的每个图像集合包括至少一个全景图像，每个全景图像包括内容区域，所述M个提取模型中的每个提取模型用于指示所述对应的图像集合中的全景图像中的内容区域的范围；
    确定所述背景图像所属的第一图像集合；
    根据所述映射关系，将所述第一图像集合对应的提取模型作为所述第一提取模型。
  9. 根据权利要求8所述的方法，其中，确定所述背景图像所属的第一图像集合包括：
    所述背景图像的来源、所述背景图像的名称或所述背景图像的格式确定所述背景图像所属的第一图像集合。
  10. 根据权利要求2或3所述的方法，其中，所述背景区域包括：第一边界区域，所述第一边界区域是所述背景区域中与所述内容区域相邻的区域，所述内容区域包括：第二边界区域，所述第二边界区域是所述内容区域中与所述背景区域相邻 的区域，所述第一边界区域中的任一像素的像素值大于或等于第一阈值，所述第二边界区域中的任一像素的像素值小于所述第一阈值，以及
    所述确定所述背景图像的内容区域，包括：
    根据所述第一阈值，确定所述背景图像的内容区域。
  11. 一种图像处理装置，其中，所述装置包括：处理器和存储器，所述处理器执行所述存储器中的机器可读指令，用于
    获取背景图像和目标图像，所述背景图像为球形全景图像或立方体全景图像，所述目标图像为非全景图像；
    对所述目标图像和所述背景图像进行合成处理，以生成待播放图像，所述待播放图像为球形全景图像或立方体全景图像，且所述待播放图像包括第一区域和第二区域，所述第一区域包括根据所述背景图像获得的像素，所述第二区域包括根据所述目标图像获得的像素。
  12. 根据权利要求11所述的装置，其中，所述背景图像包括：背景区域和内容区域，所述处理器进一步执行所述存储器中的机器可读指令，用于
    确定所述背景图像的内容区域，根据所述背景图像的内容区域，对所述目标图像进行像素重构，以生成待拼接图像，对待拼接图像和所述背景图像进行合成处理，其中，所述第一区域包括所述背景区域中的像素，所述第二区域包括所述待拼接图像中的像素。
  13. 根据权利要求12所述的装置，其中，所述处理器进一步执行所述存储器中的机器可读指令，用于根据所述目标图像和以下至少一种参数，对所述目标图像进行像素重构：
    所述背景图像的内容区域在所述背景图像中的范围、所述背景图像的内容区域的形状、所述背景图像的内容区域的大小、所述背景图像的内容区域在所述背景图像中的位置、所述背景像的全景视角或所述背景图像的展开方式。
  14. 根据权利要求12或13所述的装置，其中，所述处理器进一步执行所述存储器中的机器可读指令，用于获取第一提取模型，所述第一提取模型用于指示所述背景图像的内容区域在所述背景图像中的范围，根据所述第一提取模型，确定所述背景图像的内容区域；或
    所述背景区域包括第一边界区域，所述第一边界区域是所述背景区域中与所述 内容区域相邻的区域，所述内容区域包括第二边界区域，所述第二边界区域是所述内容区域中与所述背景区域相邻的区域，所述第一边界区域中的任一像素的像素值大于或等于第一阈值，所述第二边界区域中的任一像素的像素值小于所述第一阈值，所述处理器进一步执行所述存储器中的机器可读指令，用于根据所述第一阈值，确定所述背景图像的内容区域。
  15. 一种视频处理的方法，应用于视频处理装置，所述方法包括：
    获取背景视频，所述背景视频包括至少一帧图像，所述背景视频中的每一帧图像包括：球形全景图像或立方体全景图像；
    获取目标视频，所述目标视频包括至少一帧图像，所述目标视频中的每一帧图像包括：非全景图像；
    对所述目标视频和所述背景视频进行合成处理，以生成待播放视频，所述待播放视频中的每一帧图像包括：球形全景图像或立方体全景图像，所述待播放视频中的每一帧图像包括第一区域和第二区域，所述第一区域包括根据所述背景视频获得的像素，所述第二区域包括根据所述目标视频获得的像素。
  16. 根据权利要求15所述的方法，其中，所述对所述目标视频和所述背景视频进行合成处理包括：
    对目标图像和背景图像进行合成处理，以生成待播放图像，其中，所述目标图像是所述目标视频中的任意一帧图像，所述背景图像是所述背景视频中的任意一帧图像，且所述待播放图像的第一区域包括根据所述背景图像获得的像素，所述待播放图像的第二区域包括根据所述目标图像获得的像素；
    将生成的各待播放图像组合成所述待播放视频。
  17. 根据权利要求16所述的方法，其中，所述对目标图像和背景图像进行合成处理，包括：
    确定所述背景图像的内容区域；
    根据所述背景图像的内容区域，对所述目标图像进行像素重构，以生成待拼接图像；
    对待拼接图像和所述背景图像进行合成处理，其中，所述第一区域包括所述背景区域中的像素，所述第二区域包括所述待拼接图像中的像素。
  18. 一种视频处理装置，其中，所述装置包括：处理器和存储器，所述处理器 执行所述存储器中的机器可读指令，用于
    获取背景视频和目标视频，所述背景视频包括至少一帧图像，所述背景视频中的每一帧图像包括：球形全景图像或立方体全景图像，所述目标视频包括至少一帧图像，所述目标视频中的每一帧图像包括：非全景图像；
    对所述目标视频和所述背景视频进行合成处理，以生成待播放视频，其中，所述待播放视频中的每一帧图像包括：球形全景图像或立方体全景图像，所述待播放视频中的每一帧图像包括第一区域和第二区域，所述第一区域包括根据所述背景视频获得的像素，所述第二区域包括根据所述目标视频获得的像素。
  19. 根据权利要求18所述的装置，其中，所述处理器进一步执行所述存储器中的机器可读指令，用于对目标图像和背景图像进行合成处理，以生成待播放图像，其中，所述目标图像是所述目标视频中的任意一帧图像，所述背景图像是所述背景视频中的任意一帧图像，且所述待播放图像的第一区域包括根据所述背景图像获得的像素，所述待播放图像的第二区域包括根据所述目标图像获得的像素，将生成的各待播放图像组合成所述待播放视频。
  20. 根据权利要求19所述的装置，其中，所述处理器进一步执行所述存储器中的机器可读指令，用于确定所述背景图像的内容区域，根据所述背景图像的内容区域，对所述目标图像进行像素重构，以生成待拼接图像，对待拼接图像和所述背景图像进行合成处理，其中，所述第一区域包括所述背景区域中的像素，所述第二区域包括所述待拼接图像中的像素。
  21. 一种虚拟现实装置，其中，包括：
    处理单元，用于根据权利要求1至10中任一项所述的方法获取所述待播放图像，或用于根据权利要求15至17中任一项所述的方法获取所述待播放视频；
    显示单元，用于呈现所述待播放图像或所述待播放视频。
  22. 一种非易失性存储介质，用于存储机器可读指令，当所述机器可读指令被执行时，执行所述权利要求1至10或15至17中任一项所述的方法。
PCT/CN2018/081699 2017-04-11 2018-04-03 图像、视频处理方法和装置、虚拟现实装置和存储介质 WO2018188499A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710234566.4 2017-04-11
CN201710234566.4A CN107018336B (zh) 2017-04-11 2017-04-11 图像处理的方法和装置及视频处理的方法和装置

Publications (1)

Publication Number Publication Date
WO2018188499A1 true WO2018188499A1 (zh) 2018-10-18

Family

ID=59445346

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/081699 WO2018188499A1 (zh) 2017-04-11 2018-04-03 图像、视频处理方法和装置、虚拟现实装置和存储介质

Country Status (2)

Country Link
CN (1) CN107018336B (zh)
WO (1) WO2018188499A1 (zh)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107018336B (zh) * 2017-04-11 2018-11-09 腾讯科技（深圳）有限公司 图像处理的方法和装置及视频处理的方法和装置
CN107493503A (zh) * 2017-08-24 2017-12-19 深圳Tcl新技术有限公司 播放终端的虚拟现实视频呈像方法、系统及存储介质
CN107590859A (zh) * 2017-09-01 2018-01-16 广州励丰文化科技股份有限公司 一种混合现实画面处理方法及服务设备
CN107705253B (zh) * 2017-10-26 2021-05-04 中国科学院长春光学精密机械与物理研究所 一种生成视频激励源的方法及装置
CN107809659A (zh) * 2017-10-27 2018-03-16 北京雷石天地电子技术有限公司 一种多路视频融合播放的方法及装置
CN108364353A (zh) * 2017-12-27 2018-08-03 广东鸿威国际会展集团有限公司 引导观看者观看场景的三维直播流的系统和方法
CN108536286A (zh) * 2018-03-22 2018-09-14 上海皮格猫信息科技有限公司 一种融合真实物体的vr工作辅助系统、方法及vr设备
CN110321765A (zh) * 2018-03-30 2019-10-11 深圳市掌网科技股份有限公司 一种图像处理方法和系统
CN110349226B (zh) * 2018-04-01 2021-06-01 浙江大学 一种全景图像处理方法及装置
CN108965695A (zh) * 2018-06-27 2018-12-07 努比亚技术有限公司 一种拍摄方法、移动终端及计算机可读存储介质
CN108920707B (zh) * 2018-07-20 2022-03-15 百度在线网络技术（北京）有限公司 用于标注信息的方法及装置
CN110825989A (zh) * 2018-07-23 2020-02-21 北京京东尚科信息技术有限公司 图片展示方法、装置、电子设备及可读介质
CN109743584B (zh) * 2018-11-13 2021-04-06 百度在线网络技术（北京）有限公司 全景视频合成方法、服务器、终端设备及存储介质
CN111263191B (zh) * 2018-11-30 2023-06-27 中兴通讯股份有限公司 视频数据的处理方法、装置、相关设备及存储介质
CN109902577A (zh) * 2019-01-25 2019-06-18 华中科技大学 一种轻量级手势检测卷积神经网络模型的构建方法及应用
CN110047033A (zh) * 2019-03-18 2019-07-23 北京奇艺世纪科技有限公司 投影变形方法、设备、图像处理设备及介质
CN110045941A (zh) * 2019-04-24 2019-07-23 深圳市微光视界科技有限公司 一种全息投影显示的方法、装置、移动终端及存储介质
CN110087128A (zh) * 2019-04-30 2019-08-02 广州虎牙信息科技有限公司 直播互动方法、直播互动装置及直播设备
CN110766611A (zh) * 2019-10-31 2020-02-07 北京沃东天骏信息技术有限公司 图像处理方法、装置、存储介质及电子设备
CN112995491B (zh) * 2019-12-13 2022-09-16 阿里巴巴集团控股有限公司 视频生成方法、装置、电子设备及计算机存储介质
CN111710020B (zh) * 2020-06-18 2023-03-21 腾讯科技（深圳）有限公司 动画渲染方法和装置及存储介质
CN112351287B (zh) * 2020-09-30 2022-10-14 深圳市为汉科技有限公司 一种多媒体数据处理方法、装置、电子设备及计算机可读存储介质
CN114299407A (zh) * 2020-11-30 2022-04-08 海信视像科技股份有限公司 虚拟现实设备及vr场景图像识别方法
CN113206993A (zh) * 2021-04-13 2021-08-03 聚好看科技股份有限公司 一种调整显示屏幕的方法及显示设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105916022A (zh) * 2015-12-28 2016-08-31 乐视致新电子科技（天津）有限公司 一种基于虚拟现实技术的视频图像处理方法及装置
CN105915972A (zh) * 2015-11-16 2016-08-31 乐视致新电子科技（天津）有限公司 一种虚拟现实中4k视频优化方法和装置
US20160277772A1 (en) * 2014-09-30 2016-09-22 Telefonaktiebolaget L M Ericsson (Publ) Reduced bit rate immersive video
CN105979360A (zh) * 2015-12-04 2016-09-28 乐视致新电子科技（天津）有限公司 一种渲染图像的处理方法及装置
CN106303289A (zh) * 2015-06-05 2017-01-04 福建凯米网络科技有限公司 一种将真实对象与虚拟场景融合显示的方法、装置及系统
CN106385576A (zh) * 2016-09-07 2017-02-08 深圳超多维科技有限公司 立体虚拟现实直播方法、装置及电子设备
CN107018336A (zh) * 2017-04-11 2017-08-04 腾讯科技（深圳）有限公司 图像处理的方法和装置及视频处理的方法和装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105376547A (zh) * 2015-11-17 2016-03-02 广州市英途信息技术有限公司 一种基于3d虚拟合成技术的微课录制系统及方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160277772A1 (en) * 2014-09-30 2016-09-22 Telefonaktiebolaget L M Ericsson (Publ) Reduced bit rate immersive video
CN106303289A (zh) * 2015-06-05 2017-01-04 福建凯米网络科技有限公司 一种将真实对象与虚拟场景融合显示的方法、装置及系统
CN105915972A (zh) * 2015-11-16 2016-08-31 乐视致新电子科技（天津）有限公司 一种虚拟现实中4k视频优化方法和装置
CN105979360A (zh) * 2015-12-04 2016-09-28 乐视致新电子科技（天津）有限公司 一种渲染图像的处理方法及装置
CN105916022A (zh) * 2015-12-28 2016-08-31 乐视致新电子科技（天津）有限公司 一种基于虚拟现实技术的视频图像处理方法及装置
CN106385576A (zh) * 2016-09-07 2017-02-08 深圳超多维科技有限公司 立体虚拟现实直播方法、装置及电子设备
CN107018336A (zh) * 2017-04-11 2017-08-04 腾讯科技（深圳）有限公司 图像处理的方法和装置及视频处理的方法和装置

Also Published As

Publication number Publication date
CN107018336A (zh) 2017-08-04
CN107018336B (zh) 2018-11-09

Similar Documents

Publication Publication Date Title
WO2018188499A1 (zh) 图像、视频处理方法和装置、虚拟现实装置和存储介质
US11272165B2 (en) Image processing method and device
US9886102B2 (en) Three dimensional display system and use
US20100128112A1 (en) Immersive display system for interacting with three-dimensional content
JP2022537614A (ja) マルチ仮想キャラクターの制御方法、装置、およびコンピュータプログラム
JP7182920B2 (ja) 画像処理装置、画像処理方法およびプログラム
US11044398B2 (en) Panoramic light field capture, processing, and display
US11430192B2 (en) Placement and manipulation of objects in augmented reality environment
CN110554770A (zh) 静态遮挡物
US9754398B1 (en) Animation curve reduction for mobile application user interface objects
CN111467803B (zh) 游戏中的显示控制方法及装置、存储介质、电子设备
CN114327700A (zh) 一种虚拟现实设备及截屏图片播放方法
CN114514493A (zh) 增强设备
JP2020173529A (ja) 情報処理装置、情報処理方法、及びプログラム
CN110267079B (zh) 待播放视频中人脸的替换方法和装置
US20230368432A1 (en) Synthesized Camera Arrays for Rendering Novel Viewpoints
WO2019042028A1 (zh) 全视向的球体光场渲染方法
KR102176805B1 (ko) 뷰 방향이 표시되는 vr 컨텐츠 제공 시스템 및 방법
Wischgoll Display systems for visualization and simulation in virtual environments
CN116958344A (zh) 虚拟形象的动画生成方法、装置、计算机设备及存储介质
Narducci et al. Enabling consistent hand-based interaction in mixed reality by occlusions handling
CN113476835B (zh) 一种画面显示的方法及装置
JP7475625B2 (ja) ３次元空間における入力受付および入力表示方法、プログラム、および３次元空間における入力受付および入力表示装置
US20230405475A1 (en) Shooting method, apparatus, device and medium based on virtual reality space
JP2011198138A (ja) 漫画作成支援装置、漫画作成支援方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18784235

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18784235

Country of ref document: EP

Kind code of ref document: A1