WO2019114328A1 - 一种基于增强现实的视频处理方法及其装置 - Google Patents

一种基于增强现实的视频处理方法及其装置 Download PDF

Info

Publication number
WO2019114328A1
WO2019114328A1 PCT/CN2018/103602 CN2018103602W WO2019114328A1 WO 2019114328 A1 WO2019114328 A1 WO 2019114328A1 CN 2018103602 W CN2018103602 W CN 2018103602W WO 2019114328 A1 WO2019114328 A1 WO 2019114328A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
data
user
augmented reality
video stream
Prior art date
Application number
PCT/CN2018/103602
Other languages
English (en)
French (fr)
Inventor
汤锦鹏
Original Assignee
广州市动景计算机科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州市动景计算机科技有限公司 filed Critical 广州市动景计算机科技有限公司
Publication of WO2019114328A1 publication Critical patent/WO2019114328A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships

Definitions

  • the embodiments of the present application relate to the field of augmented reality technologies, and in particular, to a video processing method based on augmented reality and an apparatus thereof.
  • Augmented reality technology is a new technology that integrates real world information and virtual world information "seamlessly". It is an entity information (visual information, sound, taste, etc.) that is difficult to experience in a certain time and space of the real world. Touch, etc.), through computer and other science and technology, simulation and then superimposition, the virtual information is applied to the real world, perceived by human senses, thus achieving a sensory experience beyond reality.
  • the real environment and virtual objects are superimposed in real time on the same picture or space.
  • Augmented reality technology not only displays the information of the real world, but also displays the virtual information at the same time. The two kinds of information complement each other and superimpose.
  • users can use the helmet display to combine the real world with computer graphics to see the real world around it.
  • the first virtual scene is generated based on the stored sample rendering, and the first virtual scene is used to implement product promotion and user purchase.
  • video technology based on the fun of video content itself and the comprehensiveness of product display, product promotion and user purchase are becoming more and more popular by watching videos.
  • One of the technical problems solved by the embodiments of the present application is to provide a video processing method and device based on the augmented reality, which can utilize the augmented reality technology to more fully display the products to be promoted in the video.
  • the embodiment of the present application provides a video processing method based on augmented reality, including:
  • the processing, at least one image of the image in the first video stream, to obtain the at least one first object data in the image includes:
  • the performing the augmented reality processing based on the first object data and the obtained user data, and the rendering obtaining the first virtual scene in which the user is combined with the image includes:
  • the MTCNN algorithm is used to perform the augmented reality processing on the face data and the obtained user data, and the first virtual scene in which the user is combined with the image is obtained by rendering;
  • the non-face data and the obtained user data are subjected to augmented reality processing by using an SSD algorithm, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the method further includes:
  • the method further includes:
  • the first object includes at least one of a face, a garment, a shoe and hat, an accessory, a makeup effect, a hairstyle, a piece of furniture, a decoration, a scene, and a character.
  • the method further includes:
  • the method further includes:
  • an embodiment of the present application provides a video processing device based on augmented reality, including:
  • a data acquiring module configured to: process at least one frame image in the first video stream to obtain at least one first object data in the image;
  • a scene generating module configured to: perform augmented reality processing based on the first object data and the obtained user data, and obtain a first virtual scene in which the user is combined with the image.
  • the data obtaining module includes:
  • An identification processing unit configured to: perform image recognition processing on at least one frame of image data in the first video stream by using an image recognition algorithm
  • a first object processing unit configured to: obtain the at least one first object and the first object data that meet a user input instruction and/or a preset instruction.
  • the scenario generating module includes:
  • An object determining unit configured to determine whether the first object data is face data
  • a first algorithm unit configured to: perform the augmented reality processing on the face data and the obtained user data by using an MTCNN algorithm, and obtain the first combination of the user and the image by using an MTCNN algorithm.
  • a second algorithm unit configured to perform the augmented reality processing on the non-face data and the obtained user data by using an SSD algorithm, and use the SSD algorithm to combine the obtained user with the image.
  • the first virtual scene configured to perform the augmented reality processing on the non-face data and the obtained user data by using an SSD algorithm, and use the SSD algorithm to combine the obtained user with the image.
  • the apparatus further includes:
  • a video generating module configured to: generate an image according to the first virtual scene, and/or generate a second video stream according to the video stream and the first virtual scene.
  • the apparatus further includes:
  • a video sharing module configured to: share the image, and/or the second video stream.
  • the first object includes at least one of a face, a garment, a shoe and hat, an accessory, a makeup effect, a hairstyle, a piece of furniture, a decoration, a scene, and a character.
  • the apparatus further includes:
  • the information providing module is configured to: provide detailed information about the first object.
  • the apparatus further includes:
  • the purchase providing module is configured to: provide purchase information of the first object.
  • the embodiment of the present application processes at least one frame image in the first video stream to obtain at least one first object data in the image. Further, an augmented reality process is performed based on the first object data and the obtained user data, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the present application can perform augmented reality processing by processing the video stream and using the first object data generated by the video stream processing in combination with the user data to obtain a more realistic user-first virtual scene combined with the image. Therefore, the present application can utilize augmented reality technology to more fully display the products to be promoted in the video.
  • FIG. 1 is a hardware structural diagram of a computer device to which an embodiment of the present application is applied;
  • FIG. 2 is a flow chart of an embodiment of a video processing method based on augmented reality provided by the present application
  • step S1 of another embodiment of a video processing method based on augmented reality provided by the present application is a flowchart of step S1 of another embodiment of a video processing method based on augmented reality provided by the present application
  • FIG. 4 is a schematic diagram of a user selection interface in an augmented reality based video processing method provided by the present application
  • FIG. 5 is a schematic diagram of another user selection interface in an augmented reality based video processing method provided by the present application.
  • FIG. 6 is a schematic diagram of still another user selection interface in an augmented reality based video processing method provided by the present application.
  • FIG. 7 is a flowchart of step S2 of still another embodiment of an augmented reality based video processing method provided by the present application.
  • FIG. 8 is a flowchart of another embodiment of a video processing method based on augmented reality provided by the present application.
  • FIG. 9 is a flowchart of still another embodiment of a video processing method based on augmented reality provided by the present application.
  • FIG. 10 is a flow chart of still another embodiment of a video processing method based on augmented reality provided by the present application.
  • FIG. 11 is a flowchart of still another embodiment of a video processing method based on augmented reality provided by the present application.
  • FIG. 12 is a structural diagram of an embodiment of a video processing device based on augmented reality provided by the present application.
  • FIG. 13 is a structural diagram of a data acquisition module in another embodiment of an augmented reality-based video processing device provided by the present application.
  • FIG. 14 is a structural diagram of a scene generation module according to still another embodiment of an augmented reality-based video processing apparatus provided by the present application.
  • FIG. 15 is a structural diagram of another embodiment of a video processing device based on augmented reality provided by the present application.
  • 16 is a structural diagram of still another embodiment of a video processing device based on augmented reality provided by the present application.
  • FIG. 17 is a structural diagram of still another embodiment of a video processing device based on augmented reality provided by the present application.
  • FIG. 18 is a structural diagram of still another embodiment of a video processing device based on augmented reality provided by the present application.
  • 19 is a structural diagram of hardware devices applied to an augmented reality based video processing device provided by the present application.
  • the embodiment of the present application processes at least one frame image in the first video stream to obtain at least one first object data in the image. Further, an augmented reality process is performed based on the first object data and the obtained user data, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the present application can perform augmented reality processing by processing the video stream and using the first object data generated by the video stream processing in combination with the user data to obtain a more realistic user-first virtual scene combined with the image. Therefore, the present application can utilize augmented reality technology to more fully display the products to be promoted in the video.
  • the terms “a” or “an” are defined as one or more than one.
  • the term “plurality” is defined as two or more than two.
  • the term “other” is defined as at least one more or more.
  • the terms “include” and/or “have” are defined to include (ie, open language).
  • the term “coupled” is defined as connected, but is not necessarily a direct connection, and is not necessarily mechanically connected.
  • program or “computer program” or like terms is defined as a sequence of instructions designed to be executed on a computer system.
  • Program or “computer program” may include subprograms, functions, procedures, first object methods, first object implementations, executable applications, applets, servlets, source code, object code, shared libraries/dynamic loading Library and/or other sequences of instructions designed for execution on a computer system.
  • references to "one embodiment”, “an embodiment”, “an embodiment” or the like in this document means that the specific features, structures, or characteristics described in connection with the embodiments are included in at least one implementation of the present application. In the example. Therefore, the appearances of such phrases in various places throughout the specification are not necessarily all referring to the same embodiments. In addition, the particular features, structures, or characteristics may be combined, without limitation, in any suitable manner in one or more embodiments.
  • An embodiment of the present application provides a video processing method based on augmented reality, which can be applied to mobile terminals such as mobile phones and PADs, and can also be applied to terminals such as a PC end or an advertising machine.
  • the terminal generally includes a main control chip 11, a memory 12, an input/output device 13, and other hardware 14.
  • the main control chip 11 controls each functional module, and the memory 12 stores each application and data.
  • the method includes:
  • S1 Processing at least one frame image in the first video stream to obtain at least one first object data in the image.
  • the present application may process each frame of the image in the first video stream, or may only process an image that requires an operation of inserting an advertisement or performing a product display, that is, processing an image of a specific frame.
  • the position of the first object in the first video stream may be represented by a fixed or moving logo, or a pop-up page may be used to indicate the location of the first object, or the additional transparent layer may be used to indicate the first The location of an object.
  • the first object may include at least one of a face, a garment, a shoe and hat, an accessory, a makeup effect, a hairstyle, a piece of furniture, a decoration, a scene, and a character.
  • the present application can pre-process the video stream, that is, perform format conversion and/or reduction processing, and convert the video stream received or captured by the terminal into a unified image format that the image processing engine of the terminal can process.
  • the efficiency of image processing can be improved.
  • the face recognition processing is performed by using an existing face recognition algorithm to obtain face data.
  • the face recognition algorithm may use skin color recognition, template recognition or morphological recognition. Since it belongs to the existing face recognition algorithm, it will not be described again.
  • the existing recognition algorithm is used to perform the recognition processing of the chandelier to obtain chandelier data.
  • the processing method is the same as the face recognition algorithm, and only the chandelier data can be replaced with the face data.
  • the three-eye five-eye feature of the face is replaced with the lamp head, the lamp ear feature, and the like of the chandelier.
  • the application can set a general image recognition algorithm to perform recognition processing on the first object, and then change the specific recognition language or the identification parameter according to the first object feature that needs to be identified, and replace the first object that needs to be identified by the recognition algorithm.
  • the present application can also set a plurality of image recognition algorithms to respectively identify different first objects, and then perform adaptive adjustment according to the first object that needs to be identified.
  • the step S1 includes:
  • the image recognition algorithm is pre-stored, and the first object identified according to the need is different and adaptively selected.
  • the server may periodically push the updated image recognition algorithm to the terminal, and the user may also log in to the server to download the required image recognition. algorithm.
  • the present application has a good expansion capability, and can continuously update the image recognition algorithm according to the difference of the first object to be identified and the development of the recognition algorithm, so that it can be applied to the identification of different first objects, and the identification efficiency can be selected more. High recognition algorithm.
  • the identifying the first object is a fixed identification first object set in advance, or the user inputs an instruction to select.
  • the user selects a face first object that needs to be identified, and the present application automatically configures a face recognition algorithm to perform image recognition processing on at least one frame of image data in the first video stream.
  • the user may also select multiple first objects (such as faces, chandeliers, etc.) that need to be identified.
  • the present application automatically configures a face recognition algorithm and a chandelier recognition algorithm for at least the first video stream.
  • One frame of image data is subjected to image recognition processing.
  • the user of the present application can select the first object that needs to be identified, is flexible to use, and is convenient to operate.
  • the user input instruction may be selected from a plurality of first object options provided on the terminal interface, or the user may input the first object name in the input box.
  • the terminal may automatically log in to the server to download an available image recognition algorithm that can perform image recognition. Or, the user is notified that there is no local recognition algorithm for inputting the first object, and the user is required to replace the other input first object, as shown in FIG. 6.
  • the application may perform image recognition on the first object corresponding to the preset instruction according to a preset instruction.
  • the preset command is AM 7-9
  • the first object is the face
  • the PM 7-9 point is the first object.
  • the present application can customize various first objects according to user input instructions, and use flexible to improve the user experience.
  • the application can also preset the first object according to the preset instruction, without operation, and is convenient to use.
  • S2 Perform augmented reality processing based on the first object data and the obtained user data, and render a first virtual scene in which the user is combined with the image.
  • the user data may be user image data obtained by an imaging device of the terminal, image data input by a user, and other image data obtained through the Internet.
  • the user data is the face image data of the user obtained by the camera of the terminal, and the application performs augmented reality processing on the face first object data identified in step S1 and the face image data of the user. Rendering gets the first virtual scene.
  • the augmented reality processing effect of the face in the image of the user's face instead of the video stream can be obtained.
  • the user data is the room image data input by the user.
  • the application performs the augmented reality processing by the furniture first object data identified in step S1 and the room image data input by the user, and renders the first virtual scene.
  • the augmented reality processing effect of the furniture placed in the room in the image of the video stream is obtained.
  • the step S2 includes:
  • the first object data is face data
  • the face data and the obtained user data are subjected to augmented reality processing by using an MTCNN algorithm, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the first object data is non-face data
  • the non-face data and the obtained user data are subjected to augmented reality processing by using an SSD algorithm, and the first user is combined with the image. Virtual scene.
  • the present application performs augmented reality processing on the first object data and the user data by using different algorithms, thereby improving recognition efficiency and recognition effect of different first objects, and obtaining a more realistic first virtual scene.
  • the embodiment of the present application processes at least one frame image in the first video stream to obtain at least one first object data in the image. Further, an augmented reality process is performed based on the first object data and the obtained user data, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the present application can perform augmented reality processing by processing the video stream and using the first object data generated by the video stream processing in combination with the user data to obtain a more realistic user-first virtual scene combined with the image. Therefore, the present application can utilize augmented reality technology to more fully display the products to be promoted in the video.
  • the method of the present application includes the above steps S1-S2, and further includes the steps of:
  • the first virtual scene may be converted into an image.
  • the application may replace an object in the first virtual scene with a first object in at least one frame image in the first video stream, thereby generating an image.
  • the application may also take a picture for the first virtual scene to generate an image; or project the first virtual scene into a two-dimensional space and convert it into an image using a three-dimensional algorithm.
  • the second video stream may also be generated according to the first virtual scene.
  • the image generated by the virtual scene can be converted into a video by using an editing software such as a digital master.
  • the present application can also directly replace the first object in each frame of the video stream with the object in the virtual scene, thereby forming a second. Video stream.
  • the second video stream may be a short video, and the user may perform at least one operation: the user adds various effects and text, graphics, and the like through the video editing software; the user may also add the purchase information (for example, a purchase link) or a detailed description; the user can also intercept the second video stream or add other video elements.
  • the user may perform at least one operation: the user adds various effects and text, graphics, and the like through the video editing software; the user may also add the purchase information (for example, a purchase link) or a detailed description; the user can also intercept the second video stream or add other video elements.
  • the embodiment may generate an image and/or a second video according to the first virtual scene, so that the user can display the first virtual scene through the image and/or the second video so that the user obtains the first virtual scene more. Real perception.
  • the method of the present application includes the above steps S1-S3, and further includes the steps of:
  • the user may share an image formed according to the first virtual scene, and/or a second video stream by inputting an instruction, that is, sharing the image, and/or the second video stream to a microblog, a WeChat friend, a WeChat friend circle, and the like.
  • an instruction that is, sharing the image, and/or the second video stream to a microblog, a WeChat friend, a WeChat friend circle, and the like.
  • the sharing includes: directly loading the image, and/or the second video stream to a server, the server transmitting the image, and/or the second video stream to other clients; or Other users may choose to log in to the server to view the image, and/or the second video stream.
  • the sharing may directly share the image, and/or a second video stream; or the image, and/or a link to a second video stream, other users viewing the image by clicking on the link, and/or the
  • the second video stream, other users may also discuss the image at the same time, and/or the second video stream in the discussion area.
  • the discussion area may be superimposed to the image, and/or a pop-up window, a transparent additional layer in the second video stream, the discussion area may also be the image, and/or the second video stream Show other windows than the window.
  • the discussion area in the embodiment of the present application may adopt an augmented reality manner, and use the method in the step S1 to identify the image, and/or at least one second object in the second video stream, and the at least one The two object data and the input data of the user participating in the discussion are rendered into a second virtual scene using augmented reality technology.
  • the second object data may also be input data of a user participating in the discussion, or other data obtained in the discussion area.
  • the user participating in the discussion may further input voice data, and combine the voice data with the second virtual scene to form an effect that the discussion area user performs a real discussion in the virtual scene.
  • the voice data may be converted into text, and characters may be displayed in the second virtual scene, and the characters may be displayed in a manner of using a barrage in the second virtual scene.
  • the method of the present application includes the above steps S1-S2, and further includes the steps of:
  • S5. Provide detailed information of the first object.
  • the present application provides detailed information of the first object by using a pop-up window, a transparent additional layer, or the like, and the detailed information includes a description of a model, an attribute, a size, and the like of the first object, and may also be the first Instructions for the use of the object.
  • the method of the present application includes the above steps S1-S2, and further includes the steps of:
  • S6 Provide purchase information of the first object.
  • the present application provides a purchase link of the first object, or a purchase route of the first object, and the like.
  • the purchase information may further include at least one of the following: an evaluation of other users, and a first virtual scene generated by another user.
  • the present application further provides a video processing device based on augmented reality, which can be applied to mobile terminals such as mobile phones and PADs, and can also be applied to terminals such as PCs or advertisement machines.
  • the terminal generally includes a main control chip 11, a memory 12, an input/output device 13, and other hardware 14.
  • the main control chip 11 controls each functional module, and the memory 12 stores each application and data.
  • the apparatus includes:
  • the data obtaining module 121 is configured to: process at least one frame image in the first video stream to obtain at least one first object data in the image;
  • the scene generating module 122 is configured to: perform augmented reality processing based on the first object data and the obtained user data, and obtain a first virtual scene in which the user is combined with the image.
  • the present application may process each frame of the image in the first video stream, or may only process an image that requires an operation of inserting an advertisement or performing a product display, that is, processing an image of a specific frame.
  • the position of the first object in the first video stream may be represented by a fixed or moving logo, or a pop-up page may be used to indicate the location of the first object, or the additional transparent layer may be used to indicate the first The location of an object.
  • the first object includes at least one of a face, a garment, a shoe and hat, an accessory, a makeup effect, a hairstyle, a piece of furniture, a decoration, a scene, and a character.
  • the present application can pre-process the video stream, that is, perform format conversion and/or reduction processing, and convert the video stream received or captured by the terminal into a unified image format that the image processing engine of the terminal can process.
  • the efficiency of image processing can be improved.
  • the face recognition processing is performed by using an existing face recognition algorithm to obtain face data.
  • the face recognition algorithm may use skin color recognition, template recognition or morphological recognition. Since it belongs to the existing face recognition algorithm, it will not be described again.
  • the existing recognition algorithm is used to perform the recognition processing of the chandelier to obtain chandelier data.
  • the processing method is the same as the face recognition algorithm, and only the chandelier data can be replaced with the face data.
  • the three-eye five-eye feature of the face is replaced with the lamp head, the lamp ear feature, and the like of the chandelier.
  • the application can set a general image recognition algorithm to perform the identification process of the first object, and then modify the specific recognition language or the identification parameter according to the first object feature that needs to be identified, and replace the first object that needs to be identified by the recognition algorithm.
  • the present application can also set a plurality of image recognition algorithms to respectively identify and process different first objects, and then perform adaptive adjustment according to the first object that needs to be identified.
  • the data obtaining module 121 includes:
  • the identification processing unit 1211 is configured to perform image recognition processing on at least one frame of image data in the first video stream by using an image recognition algorithm.
  • the first object processing unit 1212 is configured to: obtain the at least one first object and the first object data that meet a user input instruction and/or a preset instruction.
  • the image recognition algorithm is pre-stored, and the first object identified according to the need is different and adaptively selected.
  • the server may periodically push the updated image recognition algorithm to the terminal, and the user may also log in to the server to download the required image recognition. algorithm.
  • the present application has a good expansion capability, and can continuously update the image recognition algorithm according to the difference of the first object to be identified and the development of the recognition algorithm, so that it can be applied to the identification of different first objects, and the identification efficiency can be selected more. High recognition algorithm.
  • the identifying the first object is a fixed identification first object set in advance, or the user inputs an instruction to select.
  • the user selects a face first object that needs to be identified, and the present application automatically configures a face recognition algorithm to perform image recognition processing on at least one frame of image data in the first video stream.
  • the user may also select multiple first objects (face first object, chandelier) that need to be identified.
  • the present application automatically configures a face recognition algorithm and a chandelier recognition algorithm for the first video stream. At least one frame of image data is subjected to image recognition processing.
  • the user of the present application can select the first object that needs to be identified, is flexible to use, and is convenient to operate.
  • the user input instruction may be selected from a plurality of first object options provided on the terminal interface, or the user may input the first object name in the input box.
  • the terminal may automatically log in to the server to download an available image recognition algorithm that can perform image recognition. Or, the user is notified that there is no local recognition algorithm for inputting the first object, and the user is required to replace the other input first object, as shown in FIG. 6.
  • the application may perform image recognition on the first object corresponding to the preset instruction according to a preset instruction.
  • the preset command is AM 7-9
  • the first object is the face
  • the PM 7-9 point is the first object.
  • the present application can customize various first objects according to user input instructions, and use flexible to improve the user experience.
  • the application can also preset the first object according to the preset instruction, without operation, and is convenient to use.
  • the user data may be user image data obtained by an imaging device of the terminal, image data input by a user, and other image data obtained through the Internet.
  • the user data is the face image data of the user obtained by the camera of the terminal, and the application performs the face first object data identified by the data acquisition module 121 and the face image data of the user.
  • Augmented reality processing rendering to obtain the first virtual scene.
  • the augmented reality processing effect of the face in the image of the user's face instead of the video stream can be obtained.
  • the user data is the room image data input by the user.
  • the application performs the augmented reality processing by the furniture first object data identified in step S1 and the room image data input by the user, and renders the first virtual scene.
  • the augmented reality processing effect of the furniture placed in the room in the image of the video stream is obtained.
  • the scenario generation module 122 includes:
  • the object determining unit 1221 is configured to determine whether the first object data is face data.
  • the first algorithm unit 1222 is configured to, if the first object data is the face data, perform the augmented reality processing on the face data and the obtained user data by using an MTCNN algorithm, and the rendering obtains the user and the image is combined.
  • the first virtual scene is configured to, if the first object data is the face data, perform the augmented reality processing on the face data and the obtained user data by using an MTCNN algorithm, and the rendering obtains the user and the image is combined.
  • the first virtual scene is configured to, if the first object data is the face data, perform the augmented reality processing on the face data and the obtained user data by using an MTCNN algorithm, and the rendering obtains the user and the image is combined.
  • the second algorithm unit 1223 is configured to: if the first object data is non-face data, perform an augmented reality process on the non-face data and the obtained user data by using an SSD algorithm, and obtain the user and the The first virtual scene in which the images are combined.
  • the present application performs augmented reality processing on the first object data and the user data by using different algorithms, thereby improving recognition efficiency and recognition effect of different first objects, and obtaining a more realistic first virtual scene.
  • the embodiment of the present application processes at least one frame image in the first video stream to obtain at least one first object data in the image. Further, an augmented reality process is performed based on the first object data and the obtained user data, and the first virtual scene in which the user is combined with the image is obtained by rendering.
  • the present application can perform augmented reality processing by processing the video stream and using the first object data generated by the video stream processing in combination with the user data to obtain a more realistic user-first virtual scene combined with the image. Therefore, the present application can utilize augmented reality technology to more fully display the products to be promoted in the video.
  • the device of the present application includes the data acquiring module 121 and the scenario generating module 122, and further includes:
  • the video generating module 123 is configured to: generate an image according to the first virtual scene, and/or generate a second video stream according to the video stream and the first virtual scene.
  • the first virtual scene may be converted into an image.
  • the application may replace an object in the first virtual scene with a first object in at least one frame image in the first video stream, thereby generating an image.
  • the application may also take a picture for the first virtual scene to generate an image; or project the first virtual scene into a two-dimensional space and convert it into an image using a three-dimensional algorithm.
  • the second video stream may also be generated according to the first virtual scene.
  • the image generated by the virtual scene can be converted into a video by using an editing software such as a digital master.
  • the present application can also directly replace the first object in each frame of the video stream with the object in the virtual scene, thereby forming a second. Video stream.
  • the second video stream may be a short video, and the user may perform at least one operation: the user adds various effects and text, graphics, and the like through the video editing software; the user may also add the purchase information (for example, a purchase link) or a detailed description; the user can also intercept the second video stream or add other video elements.
  • the user may perform at least one operation: the user adds various effects and text, graphics, and the like through the video editing software; the user may also add the purchase information (for example, a purchase link) or a detailed description; the user can also intercept the second video stream or add other video elements.
  • the embodiment may generate an image and/or a second video according to the first virtual scene, so that the user can display the first virtual scene through the image and/or the second video so that the user obtains the first virtual scene more. Real perception.
  • the device of the present application includes the data acquiring module 121, the scenario generating module 122, and the video generating module 123, and further includes:
  • the video sharing module 124 is configured to: share the image, and/or the second video stream.
  • the user may share an image formed according to the first virtual scene, and/or a second video stream by inputting an instruction, that is, sharing the image, and/or the second video stream to a microblog, a WeChat friend, a WeChat friend circle, and the like.
  • an instruction that is, sharing the image, and/or the second video stream to a microblog, a WeChat friend, a WeChat friend circle, and the like.
  • the sharing includes: directly loading the image, and/or the second video stream to a server, the server transmitting the image, and/or the second video stream to another client; or Other users may choose to log in to the server to view the image, and/or the second video stream.
  • the sharing may directly share the image, and/or a second video stream; or the image, and/or a link to a second video stream, other users viewing the image by clicking on the link, and/or the
  • the second video stream, other users may also discuss the image at the same time, and/or the second video stream in the discussion area.
  • the discussion area may be superimposed to the image, and/or a pop-up window, a transparent additional layer in the second video stream, the discussion area may also be the image, and/or the second video stream Show other windows than the window.
  • the discussion area in the embodiment of the present application may adopt an augmented reality manner, and use the method in the step S1 to identify the image, and/or at least one second object in the second video stream, and the at least one The two object data and the input data of the user participating in the discussion are rendered into a second virtual scene using augmented reality technology.
  • the second object data may also be input data of a user participating in the discussion, or other data obtained in the discussion area.
  • the user participating in the discussion may further input voice data, and combine the voice data with the second virtual scene to form an effect that the discussion area user performs a real discussion in the virtual scene.
  • the voice data may be converted into text, and characters may be displayed in the second virtual scene, and the characters may be displayed in a manner of using a barrage in the second virtual scene.
  • the device of the present application includes the data acquiring module 121 and the scenario generating module 122, and further includes:
  • the information providing module 125 is configured to: provide detailed information about the first object.
  • the present application provides detailed information of the first object by using a pop-up window, a transparent additional layer, or the like, and the detailed information includes a description of a model, an attribute, a size, and the like of the first object, and may also be the first Instructions for the use of the object.
  • the device of the present application includes the data acquiring module 121 and the scenario generating module 122, and further includes:
  • the purchase providing module 126 is configured to: provide purchase information of the first object.
  • the present application provides a purchase link of the first object, or a purchase route of the first object, and the like.
  • the purchase information may further include at least one of the following: an evaluation of other users, and a first virtual scene generated by another user.
  • FIG. 19 is a schematic structural diagram of hardware of an electronic device based on augmented reality video processing method according to an embodiment of the present disclosure. As shown in FIG. 19, the device includes:
  • processors 1910 and memory 1920 one processor 1910 is exemplified in FIG.
  • the electronic device that performs the augmented reality based video processing method may further include: an input device 1930 and an output device 1940.
  • the processor 1910, the memory 1920, the input device 1930, and the output device 1940 may be connected by a bus or other means, as exemplified by a bus connection in FIG.
  • the memory 1920 is a non-volatile computer readable storage medium, and can be used for storing a non-volatile software program, a non-volatile computer executable program, and a module, such as an augmented reality-based video processing method in the embodiment of the present application.
  • Corresponding program instructions/modules for example, data acquisition module 121, scene generation module 122 shown in FIG. 12.
  • the processor 1910 executes various functional applications of the server and data processing by executing non-volatile software programs, instructions, and modules stored in the memory 1920, that is, implementing the augmented reality-based video processing method of the above method embodiments.
  • the memory 1920 can include a storage program area and a storage data area, wherein the storage program area can store an operating system, an application required for at least one function; and the storage data area can store data created by use of the augmented reality-based video processing electronic device. Wait.
  • memory 1920 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.
  • memory 1920 can optionally include memory remotely disposed relative to processor 1910, which can be connected to a processing device of augmented reality based video processing over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Input device 1930 can receive input numeric or character information, as well as generate user settings for the augmented reality based video processing electronics and key signal inputs related to function control.
  • Output device 1940 can include a display device such as a display screen.
  • the one or more modules are stored in the memory 1920, and when executed by the one or more processors 1910, perform an augmented reality based video processing method in any of the above method embodiments.
  • the electronic device of the embodiment of the present application exists in various forms, including but not limited to:
  • Mobile communication devices These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication.
  • Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.
  • Ultra-mobile personal computer equipment This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has mobile Internet access.
  • Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.
  • Portable entertainment devices These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.
  • the server consists of a processor, a hard disk, a memory, a system bus, etc.
  • the server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.
  • embodiments of the present application can be provided as a method, apparatus (device), or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • embodiments of the present application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Abstract

一种基于增强现实的视频处理方法及其装置，所述方法包括：对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据（S1）；基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景（S2）。该方法能够利用增强现实技术，更加全面地对视频中的需推广产品进行展示。

Description

一种基于增强现实的视频处理方法及其装置 技术领域
本申请实施例涉及增强现实技术领域，具体涉及一种基于增强现实的视频处理方法及其装置。
背景技术
增强现实技术是一种将真实世界信息和虚拟世界信息“无缝”集成的新技术，是把原本在现实世界的一定时间空间范围内很难体验到的实体信息(视觉信息,声音,味道,触觉等),通过电脑等科学技术，模拟仿真后再叠加，将虚拟的信息应用到真实世界，被人类感官所感知，从而达到超越现实的感官体验。真实的环境和虚拟的物体实时地叠加到了同一个画面或空间同时存在。
增强现实技术，不仅展现了真实世界的信息，而且将虚拟的信息同时显示出来，两种信息相互补充、叠加。在视觉化的增强现实中，用户利用头盔显示器，把真实世界与电脑图形多重合成在一起，便可以看到真实的世界围绕着它。
传统的增强现实技术中，主要利用图像处理技术，对用户的姿态和图像进行捕获后，基于存储的样本渲染生成第一虚拟场景，利用所述第一虚拟场景实现产品推广以及用户购买。而随着视频技术的发展，基于视频内容本身的趣味性和产品展示的全面性，通过观看视频来进行产品推广与用户购买越来越普及。
因此，如何利用增强现实技术，优化视频中产品展示的全面性成为现有技术亟待解决的技术问题。
发明内容
本申请实施例解决的技术问题之一在于提供一种基于增强现实的视频处理方法及其装置，其能够利用增强现实技术，更加全面的对视频中的需推广 产品进行展示。
本申请实施例提供一种基于增强现实的视频处理方法，包括：
对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据；
基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
在本申请一具体实施例中，所述对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据包括：
利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理；
获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
在本申请一具体实施例中，所述基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景包括：
判断所述第一对象数据是否为人脸数据；
如所述第一对象数据为人脸数据，则采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景；
如所述第一对象数据为非人脸数据，则采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
在本申请一具体实施例中，所述方法还包括：
根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
在本申请一具体实施例中，所述方法还包括：
分享所述图像，和/或所述第二视频流。
在本申请一具体实施例中，所述第一对象包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
在本申请一具体实施例中，所述方法还包括：
提供所述第一对象的详细信息。
在本申请一具体实施例中，所述方法还包括：
提供所述第一对象的购买信息。
对应于上述方法，本申请一实施例提供一种基于增强现实的视频处理装置，包括：
数据获取模块，用于：对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据；
场景生成模块，用于：基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
在本申请一具体实施例中，所述数据获取模块包括：
识别处理单元，用于：利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理；
第一对象处理单元，用于：获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
在本申请一具体实施例中，所述场景生成模块包括：
对象判断单元，用于判断所述第一对象数据是否为人脸数据；
第一算法单元，用于如所述第一对象数据为人脸数据，则采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景；
第二算法单元，用于如所述第一对象数据为非人脸数据，则采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
在本申请一具体实施例中，所述装置还包括：
视频生成模块，用于：根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
在本申请一具体实施例中，所述装置还包括：
视频分享模块，用于：分享所述图像，和/或所述第二视频流。
在本申请一具体实施例中，所述第一对象包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
在本申请一具体实施例中，所述装置还包括：
信息提供模块，用于：提供所述第一对象的详细信息。
在本申请一具体实施例中，所述装置还包括：
购买提供模块，用于：提供所述第一对象的购买信息。
本申请实施例对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据。进而，基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。本申请可以通过对视频流的处理，并利用视频流处理所产生的第一对象数据结合用户数据进行增强现实处理，获得更加真实的用户与所述图像相结合的第一虚拟场景。因此，本申请能够利用增强现实技术，更加全面的对视频中的需推广产品进行展示。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1是本申请实施例所应用的计算机设备的硬件结构图；
图2是本申请提供的一种基于增强现实的视频处理方法一实施例流程图；
图3是本申请提供的一种基于增强现实的视频处理方法另一实施例步骤S1的流程图；
图4是本申请提供的一种基于增强现实的视频处理方法中一用户选择界面的示意图；
图5是本申请提供的一种基于增强现实的视频处理方法中另一用户选择界面的示意图；
图6是本申请提供的一种基于增强现实的视频处理方法中再一用户选择界面的示意图；
图7是本申请提供的一种基于增强现实的视频处理方法再一实施例步骤S2的流程图；
图8是本申请提供的一种基于增强现实的视频处理方法另一实施例流程图；
图9是本申请提供的一种基于增强现实的视频处理方法再一实施例流程图；
图10是本申请提供的一种基于增强现实的视频处理方法再一实施例流 程图；
图11是本申请提供的一种基于增强现实的视频处理方法再一实施例流程图；
图12是本申请提供的一种基于增强现实的视频处理装置一实施例结构图；
图13是本申请提供的一种基于增强现实的视频处理装置另一实施例中数据获取模块的结构图；
图14是本申请提供的一种基于增强现实的视频处理装置再一实施例中场景生成模块的结构图；
图15是本申请提供的一种基于增强现实的视频处理装置另一实施例结构图；
图16是本申请提供的一种基于增强现实的视频处理装置再一实施例结构图；
图17是本申请提供的一种基于增强现实的视频处理装置再一实施例结构图；
图18是本申请提供的一种基于增强现实的视频处理装置再一实施例结构图；
图19是本申请提供的一种基于增强现实的视频处理装置所应用硬件设备结构图。
具体实施方式
本申请实施例对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据。进而，基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。本申请可以通过对视频流的处理，并利用视频流处理所产生的第一对象数据结合用户数据进行增强现实处理，获得更加真实的用户与所述图像相结合的第一虚拟场景。因此，本申请能够利用增强现实技术，更加全面的对视频中的需推广产品进行展示。
尽管本申请能够具有许多不同形式的实施例，但在附图中显示并且将在本文详细描述的特定实施例，应该理解，这种实施例的公开应该被视为原理的示例，而非意图把本申请限制于显示和描述的特定实施例。在以下的描述 中，相同的标号用于描述附图的几个示图中的相同、相似或对应的部分。
如本文所使用，术语“一个”或“一种”被定义为一个(种)或超过一个(种)。如本文所使用，术语“多个”被定义为两个或超过两个。如本文所使用，术语“其他”被定义为至少再一个或更多个。如本文所使用，术语“包含”和/或“具有”被定义为包括(即，开放式语言)。如本文所使用，术语“耦合”被定义为连接，但未必是直接连接，并且未必是以机械方式连接。如本文所使用，术语“程序”或“计算机程序”或类似术语被定义为设计用于在计算机系统上执行的指令序列。“程序”或“计算机程序”可包括子程序、函数、过程、第一对象方法、第一对象实现、可执行应用、小应用程序、小服务程序、源代码、目标代码、共享库/动态加载库和/或设计用于在计算机系统上执行的其它指令序列。
在整个本文件中对“一个实施例”、“某些实施例”、“实施例”或类似术语的提及表示结合实施例描述的特定特征、结构或特性被包括在本申请的至少一个实施例中。因此，在整个本说明书的各种地方的这种词语的出现不必全部表示相同的实施例。另外，所述特定特征、结构或特性可非限制性地在一个或多个实施例中以任何合适的方式组合。
如本文所使用，术语“或者”应该被解释为是包括性的或者表示任何一种或任何组合。因此，“A、B或者C”表示“下面的任何一种：A；B；C；A和B；A和C；B和C；A，B和C”。仅当元件、功能、步骤或动作的组合以某种方式固有地相互排斥时，将会发生这种定义的例外。
为了使本领域的人员更好地理解本申请实施例中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员所获得的所有其他实施例，都应当属于本申请保护的范围。
下面结合本申请实施例附图进一步说明本申请实施例具体实现。
本申请一实施例提一种基于增强现实的视频处理方法，可应用于手机、PAD等移动终端，也可应用于PC端或者广告机等终端。
参见图1，所述终端通常包括：主控芯片11、存储器12、输入输出装置 13以及其他硬件14。所述主控芯片11控制各功能模块，存储器12存储各应用程序和数据。
参见图2，所述方法包括：
S1、对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据。
本申请可以对所述第一视频流中的每帧图像进行处理，也可以仅对需要插入广告或者进行商品展示等操作的图像进行处理，即针对特定帧的图像进行处理。
所述第一视频流中所述第一对象所在的位置可以采用固定或者移动的标志表示，也可以采用弹出页面表示所述第一对象所在的位置，或者采用附加透明层的方式表示所述第一对象所在的位置。
具体地，所述第一对象可以包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
本申请可以对视频流进行预处理，即进行格式转换和/或降阶处理，将所述终端接收或者捕获的视频流转换为所述终端的图像处理引擎能够处理的统一的图像格式。而对图像进行降阶处理，可以提高图像处理的效率。
如果所述第一对象为人脸，则利用现有的人脸识别算法进行人脸识别处理，获得人脸数据。所述人脸识别算法可以采用肤色识别、模板识别或者形态识别，由于其属于现有人脸识别算法，因此不再赘述。
如果所述第一对象为装饰，例如，吊灯，则利用现有的识别算法进行吊灯的识别处理，得到吊灯数据。处理方法和人脸识别算法相同，仅将吊灯数据替换人脸数据即可。例如，将人脸的三庭五眼特征替换为吊灯的灯头、灯穗特征等。
本申请可以设置一通用图像识别算法进行对第一对象的识别处理，然后根据需要进行识别的第一对象特点，更改特定识别语言或者识别参数，更换识别算法所需要进行识别的第一对象。
本申请也可以设置多种图像识别算法分别对不同的第一对象进行识别处理，然后根据需要进行识别的第一对象，进行自适应调整。
在本申请一具体实现中，参见图3，所述步骤S1包括：
S11、利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
具体地，所述图像识别算法为预先存储，根据需要进行识别的第一对象不同而自适应选择，服务器可以定期向终端推送更新的图像识别算法，用户也可以根据需要登陆服务器下载需要的图像识别算法。
因此，本申请具有良好的扩展能力，可以根据需要识别的第一对象不同，以及识别算法的发展，不断更新图像识别算法，令其可以应用于不同第一对象的识别，且可以选择识别效率更高的识别算法。
所述识别第一对象为预先设置的固定的识别第一对象，或者用户输入指令进行选择。
比如，参见图4，用户选择需要进行识别的为人脸第一对象，本申请自动配置人脸识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
比如，参见图5，用户也可以选择多个需要进行识别的第一对象(例如人脸、吊灯等)，本申请自动配置人脸识别算法和吊灯识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
因此，本申请用户可以对需要进行识别的第一对象进行选择，使用灵活，操作方便。
S12、获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
具体地，所述用户输入指令可以为在终端界面上提供的多个第一对象选项中进行点选，也可以为用户在输入框中输入第一对象名称。
如果用户在输入框中输入的第一对象的识别算法并未预先存储于所述终端，所述终端可以自动登陆服务器下载可以进行图像识别的可用图像识别算法。或者，通知用户本地并不存在输入第一对象的识别算法，请用户更换其它输入第一对象，参见图6所示。
本申请可以根据预设指令针对所述预设指令对应的第一对象进行图像识别。比如，预设指令为AM 7-9点第一对象为人脸，PM 7-9点第一对象为发型。
因此，本申请可以根据用户输入指令，自定义选择各种第一对象，使用灵活，改善用户体验。本申请也可以根据预设指令，预先设置第一对象，无需操作，使用方便。
S2、基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染 获得用户与所述图像相结合的第一虚拟场景。
具体地，所述用户数据可以为通过所述终端的摄像装置获得的用户图像数据，也可以为用户输入的图像数据，还可以为通过互联网获得的其它图像数据。
比如，用户数据为通过所述终端的摄像装置获得的用户的人脸图像数据，本申请将通过步骤S1识别得到的人脸第一对象数据与所述用户的人脸图像数据进行增强现实处理，渲染获得第一虚拟场景。即可获得用户人脸替代视频流的图像中人脸的增强现实处理效果。
再比如，用户数据为用户输入的房间图像数据，本申请将通过步骤S1识别得到的家具第一对象数据与所述用户输入的房间图像数据进行增强现实处理，渲染获得第一虚拟场景。即可获得视频流的图像中家具摆放在所述房间的增强现实处理效果。
在本申请另一具体实现中，参见图7，所述步骤S2包括：
S21、判断所述第一对象数据是否为人脸数据。
S22、如是，即所述第一对象数据为人脸数据，则采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
S23、如否，所述第一对象数据为非人脸数据，则采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
本申请根据第一对象数据的不同，采用不同的算法对第一对象数据与用户数据进行增强现实处理，从而提高不同第一对象的识别效率与识别效果，获得更加逼真的第一虚拟场景。
本申请实施例对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据。进而，基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。本申请可以通过对视频流的处理，并利用视频流处理所产生的第一对象数据结合用户数据进行增强现实处理，获得更加真实的用户与所述图像相结合的第一虚拟场景。因此，本申请能够利用增强现实技术，更加全面的对视频中的需推广产品进行展示。
在本申请另一具体实施例中，参见图8，本申请所述方法包括上述步骤S1-S2外，还包括步骤：
S3、根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
具体地，所述第一虚拟场景生成后，可以转换为图像。
本申请可以将第一虚拟场景中的对象替换所述第一视频流中的至少一帧图像中的第一对象，从而生成图像。
本申请也可以将针对第一虚拟场景进行拍摄，生成图像；或者，将所述第一虚拟场景投射至二维空间，利用三维-二维算法，将其转换为图像。
所述第一虚拟场景生成后，也可以根据所述第一虚拟场景生成第二视频流。
本实施例可以利用数码大师等编辑软件将虚拟场景生成的图像再转换为视频，本申请也可以直接将视频流中每帧图像中的第一对象替换为虚拟场景中的对象，从而形成第二视频流。
在本申请具体实现中，所述第二视频流可以为短视频，用户可以进行如下至少一操作：用户通过视频编缉软件增加各种效果以及文字、图形等信息；用户也可以添加购买信息(比如，购买链接)或者详细说明；用户还可以对所述第二视频流进行截取或者增加其他视频元素。
因此，本实施例可以根据第一虚拟场景生成图像和/或第二视频，从而令用户可以通过所述图像和/或第二视频展示所述第一虚拟场景以便用户对第一虚拟场景获得更加真实的观感。
在本申请再一具体实施例中，参见图9，本申请所述方法包括上述步骤S1-S3外，还包括步骤：
S4、分享所述图像，和/或所述第二视频流。
具体地，用户可以通过输入指令分享根据第一虚拟场景形成的图像，和/或第二视频流，即将所述图像，和/或第二视频流分享至微博、微信好友、微信朋友圈等，从而通过分享行为改善用户体验。
所述分享包括：直接将所述图像，和/或所述第二视频流加载至服务端，所述服务端将所述图像，和/或所述第二视频流发送至其他用户端；或者，其他用户可以选择登录所述服务端查看所述图像，和/或所述第二视频流。
所述分享可以直接分享所述图像，和/或第二视频流；或者所述图像，和/或第二视频流的链接，其他用户通过点击所述链接查看所述图像，和/或所述第二视频流，其他用户也可以同时针对所述图像，和/或所述第二视频流在讨论区中进行讨论。
所述讨论区可以为叠加至所述图像，和/或所述第二视频流中的弹窗、透明附加层，所述讨论区还可以为所述图像，和/或所述第二视频流显示窗口以外的其他窗口。
本申请实施例所述讨论区可以采用增强现实的方式，采用所述步骤S1中方法识别出所述图像，和/或所述第二视频流中至少一第二对象，将所述至少一第二对象数据与参与讨论的用户的输入数据利用增强现实技术渲染为第二虚拟场景。
此外，所述第二对象数据也可以采用参与讨论的用户的输入数据，或者根据所述讨论区中获得的其他数据。
所述参与讨论的用户还可以输入语音数据，将所述语音数据与所述第二虚拟场景结合，形成讨论区用户在虚拟场景中进行真实讨论的效果。
具体地，还可以将所述语音数据转换为文字，在所述第二虚拟场景中显示文字，所述文字可以在所述第二虚拟场景中采用弹幕的方式进行显示。
在本申请再一具体实施例中，参见图10，本申请所述方法包括上述步骤S1-S2外，还包括步骤：
S5、提供所述第一对象的详细信息。
具体地，本申请通过弹窗、透明附加层等方式提供所述第一对象的详细信息，所述详细信息包括所述第一对象的型号、属性、尺寸等说明，也可以为所述第一对象的使用说明。
在本申请再一具体实施例中，参见图11，本申请所述方法包括上述步骤S1-S2外，还包括步骤：
S6、提供所述第一对象的购买信息。
具体地，本申请提供所述第一对象的购买链接，或者所述第一对象的购买途径等。所述购买信息还可以包括下述至少其一：其他用户的评价，以及其他用户生成的第一虚拟场景。
对应上述方法，本申请还提供一种基于增强现实的视频处理装置，可应 用于手机、PAD等移动终端，也可应用于PC端或者广告机等终端。
参见图1，所述终端通常包括：主控芯片11、存储器12、输入输出装置13以及其他硬件14。所述主控芯片11控制各功能模块，存储器12存储各应用程序和数据。
参见图12，所述装置包括：
数据获取模块121，用于：对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据；
场景生成模块122，用于：基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
本申请可以对所述第一视频流中的每帧图像进行处理，也可以仅对需要插入广告或者进行商品展示等操作的图像进行处理，即针对特定帧的图像进行处理。
所述第一视频流中所述第一对象所在的位置可以采用固定或者移动的标志表示，也可以采用弹出页面表示所述第一对象所在的位置，或者采用附加透明层的方式表示所述第一对象所在的位置。
具体地，所述第一对象包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
本申请可以对视频流进行预处理，即进行格式转换和/或降阶处理，将所述终端接收或者捕获的视频流转换为所述终端的图像处理引擎能够处理的统一的图像格式。而对图像进行降阶处理，可以提高图像处理的效率。
如果所述第一对象为人脸，则利用现有的人脸识别算法进行人脸识别处理，获得人脸数据。所述人脸识别算法可以采用肤色识别、模板识别或者形态识别，由于其属于现有人脸识别算法，因此不再赘述。
如果所述第一对象为装饰，例如，吊灯，则利用现有的识别算法进行吊灯的识别处理，得到吊灯数据。处理方法和人脸识别算法相同，仅将吊灯数据替换人脸数据即可。例如，将人脸的三庭五眼特征替换为吊灯的灯头、灯穗特征等。
本申请可以设置一通用图像识别算法进行第一对象的识别处理，然后根据需要进行识别的第一对象特点，更改特定识别语言或者识别参数，更换识别算法所需要进行识别的第一对象。
本申请也可以设置多种图像识别算法分别对不同的第一对象进行识别处 理，然后根据需要进行识别的第一对象，进行自适应调整。
在本申请一具体实现中，参见图13，所述数据获取模块121包括：
识别处理单元1211，用于：利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
第一对象处理单元1212，用于：获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
具体地，所述图像识别算法为预先存储，根据需要进行识别的第一对象不同而自适应选择，服务器可以定期向终端推送更新的图像识别算法，用户也可以根据需要登陆服务器下载需要的图像识别算法。
因此，本申请具有良好的扩展能力，可以根据需要识别的第一对象不同，以及识别算法的发展，不断更新图像识别算法，令其可以应用于不同第一对象的识别，且可以选择识别效率更高的识别算法。
所述识别第一对象为预先设置的固定的识别第一对象，或者用户输入指令进行选择。
比如，参见图4，用户选择需要进行识别的为人脸第一对象，本申请自动配置人脸识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
比如，参见图5，用户也可以选择多个需要进行识别的第一对象(人脸第一对象、吊灯)，本申请自动配置人脸识别算法和吊灯识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理。
因此，本申请用户可以对需要进行识别的第一对象进行选择，使用灵活，操作方便。
具体地，所述用户输入指令可以为在终端界面上提供的多个第一对象选项中进行点选，也可以为用户在输入框中输入第一对象名称。
如果用户在输入框中输入的第一对象的识别算法并未预先存储于所述终端，所述终端可以自动登陆服务器下载可以进行图像识别的可用图像识别算法。或者，通知用户本地并不存在输入第一对象的识别算法，请用户更换其它输入第一对象，参见图6所示。
本申请可以根据预设指令针对所述预设指令对应的第一对象进行图像识别。比如，预设指令为AM 7-9点第一对象为人脸，PM 7-9点第一对象为发型。
因此，本申请可以根据用户输入指令，自定义选择各种第一对象，使用灵活，改善用户体验。本申请也可以根据预设指令，预先设置第一对象，无需操作，使用方便。
具体地，所述用户数据可以为通过所述终端的摄像装置获得的用户图像数据，也可以为用户输入的图像数据，还可以为通过互联网获得的其它图像数据。
比如，用户数据为通过所述终端的摄像装置获得的用户的人脸图像数据，本申请将通过所述数据获取模块121识别得到的人脸第一对象数据与所述用户的人脸图像数据进行增强现实处理，渲染获得第一虚拟场景。即可获得用户人脸替代视频流的图像中人脸的增强现实处理效果。
再比如，用户数据为用户输入的房间图像数据，本申请将通过步骤S1识别得到的家具第一对象数据与所述用户输入的房间图像数据进行增强现实处理，渲染获得第一虚拟场景。即可获得视频流的图像中家具摆放在所述房间的增强现实处理效果。
在本申请另一具体实现中，参见图14，所述场景生成模块122包括：
对象判断单元1221，用于判断所述第一对象数据是否为人脸数据。
第一算法单元1222，用于如是，即所述第一对象数据为人脸数据，则采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
第二算法单元1223，用于如否，所述第一对象数据为非人脸数据，则采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
本申请根据第一对象数据的不同，采用不同的的算法对第一对象数据与用户数据进行增强现实处理，从而提高不同第一对象的识别效率与识别效果，获得更加逼真的第一虚拟场景。
本申请实施例对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据。进而，基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。本申请可以通过对视频流的处理，并利用视频流处理所产生的第一对象数据结合用户数据进行增强现实处理，获得更加真实的用户与所述图像相结合的第一虚拟场景。因此，本申请能够利用增强现实技术，更加全面的对视频中的需 推广产品进行展示。
在本申请另一具体实施例中，参见图15，本申请所述装置包括上述数据获取模块121、场景生成模块122外，还包括：
视频生成模块123，用于：根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
具体地，所述第一虚拟场景生成后，可以转换为图像。
本申请可以将第一虚拟场景中的对象替换所述第一视频流中的至少一帧图像中的第一对象，从而生成图像。
本申请也可以将针对第一虚拟场景进行拍摄，生成图像；或者，将所述第一虚拟场景投射至二维空间，利用三维-二维算法，将其转换为图像。
所述第一虚拟场景生成后，也可以根据所述第一虚拟场景生成第二视频流。
本实施例可以利用数码大师等编辑软件将虚拟场景生成的图像再转换为视频，本申请也可以直接将视频流中每帧图像中的第一对象替换为虚拟场景中的对象，从而形成第二视频流。
在本申请具体实现中，所述第二视频流可以为短视频，用户可以进行如下至少一操作：用户通过视频编缉软件增加各种效果以及文字、图形等信息；用户也可以添加购买信息(比如，购买链接)或者详细说明；用户还可以对所述第二视频流进行截取或者增加其他视频元素。
因此，本实施例可以根据第一虚拟场景生成图像和/或第二视频，从而令用户可以通过所述图像和/或第二视频展示所述第一虚拟场景以便用户对第一虚拟场景获得更加真实的观感。
在本申请再一具体实施例中，参见图16，本申请所述装置包括上述数据获取模块121、场景生成模块122、视频生成模块123外，还包括：
视频分享模块124，用于：分享所述图像，和/或所述第二视频流。
具体地，用户可以通过输入指令分享根据第一虚拟场景形成的图像，和/或第二视频流，即将所述图像，和/或第二视频流分享至微博、微信好友、微信朋友圈等，从而通过分享行为改善用户体验。
所述分享包括：直接将所述图像，和/或所述第二视频流加载至服务端， 所述服务端将所述图像，和/或所述第二视频流发送至其他用户端；或者，其他用户可以选择登录所述服务端查看所述图像，和/或所述第二视频流。
所述分享可以直接分享所述图像，和/或第二视频流；或者所述图像，和/或第二视频流的链接，其他用户通过点击所述链接查看所述图像，和/或所述第二视频流，其他用户也可以同时针对所述图像，和/或所述第二视频流在讨论区中进行讨论。
所述讨论区可以为叠加至所述图像，和/或所述第二视频流中的弹窗、透明附加层，所述讨论区还可以为所述图像，和/或所述第二视频流显示窗口以外的其他窗口。
本申请实施例所述讨论区可以采用增强现实的方式，采用所述步骤S1中方法识别出所述图像，和/或所述第二视频流中至少一第二对象，将所述至少一第二对象数据与参与讨论的用户的输入数据利用增强现实技术渲染为第二虚拟场景。
此外，所述第二对象数据也可以采用参与讨论的用户的输入数据，或者根据所述讨论区中获得的其他数据。
所述参与讨论的用户还可以输入语音数据，将所述语音数据与所述第二虚拟场景结合，形成讨论区用户在虚拟场景中进行真实讨论的效果。
具体地，还可以将所述语音数据转换为文字，在所述第二虚拟场景中显示文字，所述文字可以在所述第二虚拟场景中采用弹幕的方式进行显示。
在本申请再一具体实施例中，参见图17，本申请所述装置包括上述数据获取模块121、场景生成模块122外，还包括：
信息提供模块125，用于：提供所述第一对象的详细信息。
具体地，本申请通过弹窗、透明附加层等方式提供所述第一对象的详细信息，所述详细信息包括所述第一对象的型号、属性、尺寸等说明，也可以为所述第一对象的使用说明。
在本申请再一具体实施例中，参见图18，本申请所述装置包括上述数据获取模块121、场景生成模块122外，还包括：
购买提供模块126，用于：提供所述第一对象的购买信息。
具体地，本申请提供所述第一对象的购买链接，或者所述第一对象的购买途径等。所述购买信息还可以包括下述至少其一：其他用户的评价，以及其他用户生成的第一虚拟场景。
图19是本申请实施例提供的一种基于增强现实的视频处理方法的电子设备的硬件结构示意图，如图19所示，该设备包括：
一个或多个处理器1910以及存储器1920，图19中以一个处理器1910为例。
执行基于增强现实的视频处理方法的电子设备还可以包括：输入装置1930和输出装置1940。
处理器1910、存储器1920、输入装置1930和输出装置1940可以通过总线或者其他方式连接，图19中以通过总线连接为例。
存储器1920作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本申请实施例中的基于增强现实的视频处理方法对应的程序指令/模块(例如，附图12所示的数据获取模块121、场景生成模块122)。处理器1910通过运行存储在存储器1920中的非易失性软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例基于增强现实的视频处理方法。
存储器1920可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储基于增强现实的视频处理电子设备的使用所创建的数据等。此外，存储器1920可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中，存储器1920可选包括相对于处理器1910远程设置的存储器，这些远程存储器可以通过网络连接至基于增强现实的视频处理的处理装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置1930可接收输入的数字或字符信息，以及产生基于增强现实的视频处理电子设备的用户设置以及功能控制有关的键信号输入。输出装置1940可包括显示屏等显示设备。
所述一个或者多个模块存储在所述存储器1920中，当被所述一个或者多个处理器1910执行时，执行上述任意方法实施例中的基于增强现实的视频处理方法。
上述产品可执行本申请实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本申请实施 例所提供的方法。
本申请实施例的电子设备以多种形式存在，包括但不限于:
(1)移动通信设备:这类设备的特点是具备移动通信功能，并且以提供话音、数据通信为主要目标。这类终端包括:智能手机(例如iPhone)、多媒体手机、功能性手机，以及低端手机等。
(2)超移动个人计算机设备:这类设备属于个人计算机的范畴，有计算和处理功能，一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等，例如iPad。
(3)便携式娱乐设备:这类设备可以显示和播放多媒体内容。该类设备包括:音频、视频播放器(例如iPod)，掌上游戏机，电子书，以及智能玩具和便携式车载导航设备。
(4)服务器:提供计算服务的设备，服务器的构成包括处理器、硬盘、内存、系统总线等，服务器和通用的计算机架构类似，但是由于需要提供高可靠的服务，因此在处理能力、稳定性、可靠性、安全性、可扩展性、可管理性等方面要求较高。
(5)其他具有数据交互功能的电子装置。
本领域的技术人员应明白，本申请的实施例可提供为方法、装置(设备)、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照实施例的方法、装置(设备)和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器 中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。显然，本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请实施例的这些修改和变型属于本申请实施例权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (19)

  1. 一种基于增强现实的视频处理方法，其特征在于，包括：
    对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据；
    基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
  2. 如权利要求1所述的方法，其特征在于，所述对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据包括：
    利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理；
    获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
  3. 如权利要求1或2所述的方法，其特征在于，所述基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景包括：
    判断所述第一对象数据是否为人脸数据；
    如所述第一对象数据为人脸数据，则采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景；
    如所述第一对象数据为非人脸数据，则采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
  4. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
  5. 如权利要求4所述的方法，其特征在于，所述方法还包括：
    分享所述图像，和/或所述第二视频流。
  6. 如权利要求1所述的方法，其特征在于，所述第一对象包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
  7. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    提供所述第一对象的详细信息。
  8. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    提供所述第一对象的购买信息。
  9. 一种基于增强现实的视频处理装置，其特征在于，包括：
    数据获取模块，用于：对第一视频流中的至少一帧图像进行处理，获得所述图像中的至少一第一对象数据；
    场景生成模块，用于：基于所述第一对象数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
  10. 如权利要求9所述的装置，其特征在于，所述数据获取模块包括：
    识别处理单元，用于：利用图像识别算法对所述第一视频流中的至少一帧图像数据进行图像识别处理；
    第一对象处理单元，用于：获得符合用户输入指令和/或预设指令的所述至少一第一对象以及所述第一对象数据。
  11. 如权利要求9或10所述的装置，其特征在于，所述场景生成模块包括：
    对象判断单元，用于判断所述第一对象数据是否为人脸数据；
    第一算法单元，用于在所述第一对象数据为人脸数据的情况下，采用MTCNN算法将所述人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景；
    第二算法单元，用于所述第一对象数据为非人脸数据的情况下，采用SSD算法将所述非人脸数据与获得的用户数据进行增强现实处理，渲染获得用户与所述图像相结合的第一虚拟场景。
  12. 如权利要求9所述的装置，其特征在于，所述装置还包括：
    视频生成模块，用于：根据所述第一虚拟场景生成图像，和/或根据所述视频流以及所述第一虚拟场景生成第二视频流。
  13. 如权利要求12所述的装置，其特征在于，所述装置还包括：
    视频分享模块，用于：分享所述图像，和/或所述第二视频流。
  14. 如权利要求9所述的装置，其特征在于，所述第一对象包括：人脸、服装、鞋帽、配饰、化妆效果、发型、家具、装饰、场景、人物中至少一种。
  15. 如权利要求9所述的装置，其特征在于，所述装置还包括：
    信息提供模块，用于：提供所述第一对象的详细信息。
  16. 如权利要求9所述的装置，其特征在于，所述装置还包括：
    购买提供模块，用于：提供所述第一对象的购买信息。
  17. 一种终端设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如权利要求1-8中任一项方法对应的操作。
  18. 一种计算设备，包括：
    处理器；以及
    存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求1-8中任何一项所述的方法。
  19. 一种非暂时性机器可读存储介质，其上存储有可执行代码，当所述可执行代码被电子设备的处理器执行时，使所述处理器执行如权利要求1至8中任一项所述的方法。
PCT/CN2018/103602 2017-12-11 2018-08-31 一种基于增强现实的视频处理方法及其装置 WO2019114328A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711309645.3A CN108109209A (zh) 2017-12-11 2017-12-11 一种基于增强现实的视频处理方法及其装置
CN201711309645.3 2017-12-11

Publications (1)

Publication Number Publication Date
WO2019114328A1 true WO2019114328A1 (zh) 2019-06-20

Family

ID=62209582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103602 WO2019114328A1 (zh) 2017-12-11 2018-08-31 一种基于增强现实的视频处理方法及其装置

Country Status (2)

Country Link
CN (1) CN108109209A (zh)
WO (1) WO2019114328A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111010599A (zh) * 2019-12-18 2020-04-14 浙江大华技术股份有限公司 一种处理多场景视频流的方法、装置及计算机设备
CN112862657A (zh) * 2019-11-28 2021-05-28 阿里巴巴集团控股有限公司 图像处理方法、装置、电子设备及计算机存储介质
CN111240482B (zh) * 2020-01-10 2023-06-30 北京字节跳动网络技术有限公司 一种特效展示方法及装置

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108109209A (zh) * 2017-12-11 2018-06-01 广州市动景计算机科技有限公司 一种基于增强现实的视频处理方法及其装置
CN109743584B (zh) * 2018-11-13 2021-04-06 百度在线网络技术（北京）有限公司 全景视频合成方法、服务器、终端设备及存储介质
CN110636365B (zh) * 2019-09-30 2022-01-25 北京金山安全软件有限公司 视频字符添加方法、装置、电子设备及存储介质
CN111243101B (zh) * 2019-12-31 2023-04-18 浙江省邮电工程建设有限公司 基于人工智能增加用户ar环境沉浸度方法、系统及装置
CN113784148A (zh) * 2020-06-10 2021-12-10 阿里巴巴集团控股有限公司 数据处理方法、系统、相关设备和存储介质
CN111915744A (zh) * 2020-08-31 2020-11-10 深圳传音控股股份有限公司 增强现实图像的交互方法、终端和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107221346A (zh) * 2017-05-25 2017-09-29 亮风台（上海）信息科技有限公司 一种用于确定ar视频的识别图片的方法与设备
CN107343211A (zh) * 2016-08-19 2017-11-10 北京市商汤科技开发有限公司 视频图像处理方法、装置和终端设备
US20170337949A1 (en) * 2016-03-01 2017-11-23 Meograph, Inc. Mobile device video personalization
CN107391060A (zh) * 2017-04-21 2017-11-24 阿里巴巴集团控股有限公司 图像显示方法、装置、系统及设备、可读介质
CN108109209A (zh) * 2017-12-11 2018-06-01 广州市动景计算机科技有限公司 一种基于增强现实的视频处理方法及其装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104240277B (zh) * 2013-06-24 2019-07-19 腾讯科技（深圳）有限公司 基于人脸检测的增强现实交互方法和系统
CN104834897A (zh) * 2015-04-09 2015-08-12 东南大学 一种基于移动平台的增强现实的系统及方法
CN105872588A (zh) * 2015-12-09 2016-08-17 乐视网信息技术（北京）股份有限公司 视频中加载广告的方法及装置
CN106604147A (zh) * 2016-12-08 2017-04-26 天脉聚源(北京)传媒科技有限公司 一种视频处理方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170337949A1 (en) * 2016-03-01 2017-11-23 Meograph, Inc. Mobile device video personalization
CN107343211A (zh) * 2016-08-19 2017-11-10 北京市商汤科技开发有限公司 视频图像处理方法、装置和终端设备
CN107391060A (zh) * 2017-04-21 2017-11-24 阿里巴巴集团控股有限公司 图像显示方法、装置、系统及设备、可读介质
CN107221346A (zh) * 2017-05-25 2017-09-29 亮风台（上海）信息科技有限公司 一种用于确定ar视频的识别图片的方法与设备
CN108109209A (zh) * 2017-12-11 2018-06-01 广州市动景计算机科技有限公司 一种基于增强现实的视频处理方法及其装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112862657A (zh) * 2019-11-28 2021-05-28 阿里巴巴集团控股有限公司 图像处理方法、装置、电子设备及计算机存储介质
CN111010599A (zh) * 2019-12-18 2020-04-14 浙江大华技术股份有限公司 一种处理多场景视频流的方法、装置及计算机设备
CN111240482B (zh) * 2020-01-10 2023-06-30 北京字节跳动网络技术有限公司 一种特效展示方法及装置

Also Published As

Publication number Publication date
CN108109209A (zh) 2018-06-01

Similar Documents

Publication Publication Date Title
WO2019114328A1 (zh) 一种基于增强现实的视频处理方法及其装置
US11012740B2 (en) Method, device, and storage medium for displaying a dynamic special effect
US10147239B2 (en) Content creation tool
US11798201B2 (en) Mirroring device with whole-body outfits
US20150185825A1 (en) Assigning a virtual user interface to a physical object
CN113287118A (zh) 用于面部再现的系统和方法
US11842433B2 (en) Generating personalized videos with customized text messages
EP3912136A1 (en) Systems and methods for generating personalized videos with customized text messages
US20170213394A1 (en) Environmentally mapped virtualization mechanism
WO2022146890A1 (en) Detection and obfuscation of display screens in augmented reality content
JP2019537397A (ja) 映像のための効果共有方法およびシステム
US11430158B2 (en) Intelligent real-time multiple-user augmented reality content management and data analytics system
US20130050190A1 (en) Dressing simulation system and method
CN109636917B (zh) 三维模型的生成方法、装置、硬件装置
US11107129B1 (en) Dynamic media content for in-store screen experiences
US11095956B2 (en) Method and system for delivering an interactive video
US20180160133A1 (en) Realtime recording of gestures and/or voice to modify animations
KR102658960B1 (ko) 얼굴 재연을 위한 시스템 및 방법
US20240096033A1 (en) Technology for creating, replicating and/or controlling avatars in extended reality
WO2022246373A1 (en) Hardware encoder for stereo stitching
EP4342174A1 (en) Hardware encoder for stereo stitching
KR20240050463A (ko) 얼굴 재연을 위한 시스템 및 방법
WO2023211738A1 (en) Augmented reality experiences with dual cameras
TW201814433A (zh) 虛擬實境環境中選定物件之管理方法及系統，及其相關電腦程式產品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18889724

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18889724

Country of ref document: EP

Kind code of ref document: A1