WO2019024853A1 - 一种图像处理方法、装置及存储介质 - Google Patents

一种图像处理方法、装置及存储介质 Download PDF

Info

Publication number
WO2019024853A1
WO2019024853A1 PCT/CN2018/097860 CN2018097860W WO2019024853A1 WO 2019024853 A1 WO2019024853 A1 WO 2019024853A1 CN 2018097860 W CN2018097860 W CN 2018097860W WO 2019024853 A1 WO2019024853 A1 WO 2019024853A1
Authority
WO
WIPO (PCT)
Prior art keywords
real object
social network
image data
feature
real
Prior art date
Application number
PCT/CN2018/097860
Other languages
English (en)
French (fr)
Inventor
林经纬
朱莹
廖戈语
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2020505230A priority Critical patent/JP7098120B2/ja
Priority to KR1020207004027A priority patent/KR102292537B1/ko
Publication of WO2019024853A1 publication Critical patent/WO2019024853A1/zh
Priority to US16/780,891 priority patent/US11182615B2/en
Priority to US17/478,860 priority patent/US20220004765A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • G06F16/784—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content the detected or recognised objects being people
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9536—Search customisation based on social or collaborative filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/01—Social networking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/30—Scenes; Scene-specific elements in albums, collections or shared content, e.g. social network photos or video
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/52—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services

Definitions

  • the present application relates to the field of image technologies, and in particular, to an image processing method, apparatus, and storage medium.
  • Displaying the image of an object in various scenarios of a social network is a basic function of the client, and the presently displayed method is single, taking the object as a user as an example, and the related technology usually uses a virtual character image or a self-portrait avatar to display the image of the user. And play a role in the identification of social networks; however, this approach is currently difficult to adapt to social networks to personalize the needs of users, has become a constraint on the diversity of social networks.
  • the embodiments of the present invention provide an image processing method, device, and storage medium, which can solve the above technical problems and effectively expand the presentation manner of objects in a social network.
  • an image processing method including:
  • the virtual object in the augmented reality model is rendered according to a position of the real object in the rendered image to form the real object and the virtual object that are displayed together.
  • an image processing apparatus including:
  • An identification module configured to identify a feature of a real object in the environment from the obtained image data
  • a querying module configured to query a social network with the feature of the real object, and determine that the real object has an attribute of the social network
  • a model module configured to obtain an augmented reality model in the social network that is adapted to the real object
  • a rendering module configured to render according to the obtained image data, and render the virtual object in the augmented reality model according to a position of the real object in the rendered image to form a common display The real object and the virtual object.
  • the embodiment of the present application provides a storage medium, where an executable program is stored, and when the executable program is executed by a processor, the image processing method provided by the embodiment of the present application is implemented.
  • an image processing apparatus including:
  • a memory for storing an executable program
  • the image processing method provided by the embodiment of the present application is implemented when the processor is configured to execute the executable program stored in the memory.
  • the real object belonging to the social network can be quickly identified for the image data in any scene of the social network, and the corresponding scene is merged.
  • the augmented reality model of the real object in the social network forms a real object and a virtual object that are displayed together, and provides a way of expanding the object in the social network to achieve the effect of combining virtual reality;
  • the augmented reality model for different real objects in the social network has diversified features, so that when applied to the rendering of image data, the differentiated display effect of different objects is realized.
  • 1-1 is a schematic structural diagram of an optional hardware of an image processing apparatus according to an embodiment of the present disclosure
  • 1-2 is a schematic structural diagram of an optional function of an image processing apparatus according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of an optional system implemented as an AR device according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of another optional structure of an image processing apparatus according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic flowchart of an optional implementation process of an image processing method according to an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of another optional implementation process of an image processing method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of facial feature points provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an effect of jointly displaying a real object and a virtual object according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an effect of jointly displaying a real object and a virtual object according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an effect of jointly displaying a real object and a virtual object according to an embodiment of the present application.
  • FIG. 10-1 and FIG. 10-2 are schematic diagrams showing effects of a cartoon character dressing and a custom network virtual character according to an embodiment of the present application;
  • FIG. 11 is a schematic diagram of still another optional implementation process of an image processing method according to an embodiment of the present disclosure.
  • AR Augmented Reality
  • Augmented reality technology can seamlessly integrate real-world information with information in the virtual world, and use information technology (visual, sound, taste, etc.) that cannot be experienced in the real world to be superimposed by scientific technology simulation, and then apply virtual information to The real world is perceived by human senses to achieve a sensory experience that combines virtual reality.
  • information technology visual, sound, taste, etc.
  • calculating the position and posture of a real object in a real image ie, only a photo or video of a real object in the real world
  • an image including a virtual object such as an image or a video
  • a three-dimensional (3D, Three-Dimensional) model, etc. to add an image including a virtual object to a real image in a three-dimensional space.
  • the virtual item based on the face positioning is added to realize the effect of the face dressing; for example, according to the two-dimensional code of the scanned product, near the displayed two-dimensional code Display product information and/or store and address where the item can be purchased, and so on.
  • Augmented reality can also realize real-time interaction according to the scene.
  • the fighting action in the game is controlled by the glove or hand stick of the AR system; or, in the chess game of AR, You can control the pieces through the gloves of the AR system, and so on.
  • Client refers to the client installed in the device, or the third-party client in the device, used to support various applications based on social networks, to achieve a variety of social functions, such as video call function or send pictures Features, etc.
  • HMD head-mounted displays
  • Social network a network that supports multiple users to communicate with each other through a client (such as QQ, enterprise IM) on a server deployed on a network (such as a wide area network or a local area network).
  • a client such as QQ, enterprise IM
  • server deployed on a network (such as a wide area network or a local area network).
  • Image data is the representation of the intensity and spectrum (color) of each point of light on the image of the real object in the environment. According to the intensity of the light and the spectrum information, the image information of the real world is converted into data information. Image data for easy digitization and analysis.
  • the augmented reality model is a digital scene for augmented reality outlined by the image processing device through digital graphic technology, such as personalized AR dressing in a social network, which may be a hat, glasses, and background image.
  • the image data includes people and objects in real life, including natural scenery such as rivers and mountains, and human landscapes such as urban landscapes and architectural landscapes or other types of objects.
  • virtual object when the client renders the image data, it needs to render the virtual object that does not exist in the environment where the image data is collected, and realize the fusion of the real object and the virtual object, thereby realizing the improvement of the display effect or the enhancement of the information amount; for example, when the real object When it is a character, the virtual object may be various props and virtual backgrounds for dressing up the character image, or may be a personal business card.
  • Rendering the visual image of the real object and the virtual object that the rendering engine outputs to the screen using the rendering engine.
  • some appropriate rendering is performed on the image or video including the real object, such as adding some virtuality in the image or video of the user to the current social scene.
  • Objects to create special effects are possible.
  • FIG. 1-1 is an optional hardware structure diagram of an image processing apparatus according to an embodiment of the present application.
  • various devices such as desktop computers and notebooks, running the client may be implemented.
  • Computer and smartphone The image processing apparatus 100 shown in FIG. 1-1 includes at least one processor 101, a memory 102, a display component 103, at least one communication interface 104, and a camera 105.
  • the various components in image processing device 100 are coupled together by a bus system 106. It will be appreciated that the bus system 106 is used to implement connection communication between these components.
  • the bus system 106 includes, in addition to the configuration data bus, a power bus, a control bus, and a status signal bus. However, for clarity of description, various buses are labeled as bus system 106 in FIG.
  • the display component 103 can include an image processing device display, a mobile phone display, a tablet display, etc. for display.
  • the communication interface 104 may include an antenna system, Bluetooth, Wireless Fidelity, Near Field Communication (NFC) modules, and/or data lines, and the like.
  • NFC Near Field Communication
  • the camera 105 can be a standard camera, a telephoto camera, a wide-angle lens, a zoom camera, a digital light field camera, and a digital camera.
  • memory 102 can be either volatile memory or non-volatile memory, and can include both volatile and nonvolatile memory.
  • the memory 102 in the embodiment of the present application is used to store various types of configuration data to support the operation of the image processing apparatus 100.
  • Examples of such configuration data include a program for operating on the image processing apparatus 100, such as the client 1021, and an operating system 1022 and a database 1023, wherein the program implementing the method of the embodiment of the present application may be included in the client 1021.
  • Processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the image processing method may be completed by an integrated logic circuit of hardware in the processor 101 or an instruction in a form of software.
  • the processor 101 described above may be a general purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like.
  • DSP digital signal processor
  • the processor 101 can implement or execute the methods, steps, and logic blocks provided in the embodiments of the present application.
  • a general purpose processor can be a microprocessor or any conventional processor or the like.
  • the steps of the method provided by the embodiment of the present application may be directly implemented as a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a storage medium, and the storage medium is located in the memory 102.
  • the processor 101 reads the information in the memory 102 and completes the image processing method provided by the embodiment of the present application.
  • FIG. 1-1 The function structure of the image processing apparatus shown in FIG. 1-1 is described.
  • the software implementation is taken as an example.
  • FIG. 1-2 FIG. 1-2 is a running client according to an embodiment of the present application.
  • An optional functional structure diagram of the image processing apparatus in which the local client and the peer client are opposite concepts are described in conjunction with the respective functional modules shown in FIG. 1-2.
  • FIG. 1-1 the figure can be understood.
  • 1-2 shows the implementation of the functional modules on the hardware.
  • the identification module 210 is configured to identify features of the real object in the environment from the obtained image data.
  • the identification module 210 receives image data of the peer client collection environment and transmitted in the social network, and identifies features of the real object located in the peer client environment from the received image data; And/or, the collection environment forms image data, and identifies features of the real object located in the local client environment from the collected image data.
  • the identification module 210 is specifically configured to: when communicating with a peer client in the social network, collect the local client environment to form image data for transmission to the peer client, and collect the image data from the collected client.
  • the image data identifies the feature of the real object in the local client environment; or, when responding to the local client's collection operation, the local client environment is collected to form image data, and the local client is identified from the collected image data.
  • the characteristics of real objects in the environment are specifically configured to: when communicating with a peer client in the social network, collect the local client environment to form image data for transmission to the peer client, and collect the image data from the collected client.
  • the image data identifies the feature of the real object in the local client environment; or, when responding to the local client's collection operation, the local client environment is collected to form image data, and the local client is identified from the collected image data.
  • the characteristics of real objects in the environment are specifically configured to: when communicating with a peer client in the social network, collect the local client environment to form image data for transmission to the peer
  • the identifying module 210 is specifically configured to: before obtaining an augmented reality model adapted to the real object in the social network, determine that the feature of the identified real object meets a condition that the social network can recognize, including: At least one of the following: when the image feature points are identified, the number of identified image feature points exceeds the feature point data amount threshold; when the biometric is identified, the integrity of the identified biometrics exceeds the integrity threshold.
  • the query module 220 is configured to query the social network with the characteristics of the real object to determine whether the real object has attributes belonging to the social network.
  • the attributes of the social network involved in the embodiments of the present application are for functions carried by the social network, such as media functions (such as content aggregation), social, e-commerce, and payment, etc., and the members involved in the process of implementing these functions are of type/ Functional induction, for example, includes:
  • the payment object attribute indicates that the member is an account that receives the payment
  • the shared object attribute also known as the shared item attribute, indicates that the member is a shared item in the social network, such as food, goods, and the like;
  • the shared media information attribute indicates that the member is a shared media information in the social network, such as video, audio, and mobile games, and various products that do not have actual forms.
  • the query module 220 is specifically configured to: query a feature database of the social network by using a feature of the real object; and determine that the real object belongs to the social network when the real object matches the feature of the registered user of the social network. Registered user, the real object has the registered user attribute of the social network; when the real object matches the feature of the shared object of the social network, the real object is determined to be the shared object of the social network, and the real object has the social network at this time The object properties are shared.
  • the model module 230 is configured to obtain an augmented reality model adapted to the real object in the model library of the social network.
  • the model module 230 is specifically configured to: when the real object is a registered user in the social network, obtain a virtual object preset by the registered user in the social network, where the virtual object includes at least one of the following: a virtual item, a virtual background, and a filter; when the real object is a shared object in the social network, obtaining a virtual object in the social network for the shared object, the virtual object includes at least one of the following: a social network for the shared object Article; an advertisement for a shared object in a social network.
  • the model module 230 is specifically configured to: invoke the identification service of the server, and identify the feature of the real object from the obtained image data; or, open the image recognition thread, and open the image recognition The obtained image data is identified in the thread to obtain the characteristics of the real object.
  • the rendering module 240 is configured to perform rendering according to the obtained image data, and render the virtual object in the augmented reality model according to the position of the real object in the rendered image to form a real object and a virtual object that are jointly displayed. .
  • the rendering module 240 is specifically configured to: detect a pose change of the real object in the image data; and render the output in the augmented reality model and the pose in the position of the real object in the output image.
  • the adaptively adapted virtual objects form superimposed real objects and virtual objects.
  • the query module 220 is specifically configured to query an augmented reality model adapted to a real object in a local cache or database; when not queried, the social network query is adapted to the real object.
  • Augmented reality model adapted to a real object in a local cache or database
  • the structure of the AR device is implemented as an AR device when the image processing device provided by the embodiment of the present application is implemented as an AR lens.
  • FIG. 2 is an embodiment of the image processing device provided by the embodiment of the present application.
  • FIG. 3 is another embodiment of an image processing apparatus provided by an embodiment of the present application.
  • FIGS. 2 and 3 Although the structure of the image processing apparatus is shown in FIGS. 2 and 3, respectively, it can be understood that the structures shown in FIGS. 2 and 3 can be used in combination, and image data from the acquisition environment can be realized to render the output image data and virtual.
  • the composite display effect of the object will be described with respect to the components involved in FIGS. 2 and 3.
  • the camera is configured to acquire image data of an environment including a real object, including an image or a video, and send the acquired image or video to an image synthesizer to perform a synthesizing operation with the virtual object of the augmented reality model.
  • a scene generator configured to acquire position information of the head in the image data according to the position information of the real object in the image data, extract a virtual object corresponding to the position information in the augmented reality model, and extract the virtual object The object is sent to the image synthesizer.
  • the scene generator is further configured to generate a virtual object according to the location information, and send the virtual object to the display, where the virtual object is used to superimpose the real object on the image synthesizer.
  • the image synthesizer is configured to synthesize the acquired image or video of the real object and the virtual object, and render the composite image or the composite video, and the rendering result is periodically refreshed to the display.
  • the display is configured to display the composite image or the composite video sent by the image synthesizer to form a common display effect of the real object and the virtual object of the augmented reality model.
  • FIG. 4 is an embodiment of the present application.
  • An optional implementation flow diagram of the provided image processing method the image processing device obtains the image data formed by the environment including the real object, and the virtual image of the augmented reality model, and the following steps are involved:
  • Step 501 Obtain image data including a real object.
  • Obtaining the image data of the real object is the first step to realize the augmented reality. Only the image in the real world is input into the image processing device, and the generated virtual image extracted from the augmented reality model by the image processing device is synthesized and output to the above display. On the component, the user can see the final enhanced scene image.
  • the image data of the real object can be collected by the above-mentioned camera.
  • the digital light field camera can acquire complete light field information when shooting a real object such as a person or a natural scene, so that the user is in the process of using the image processing apparatus. It can realize where the human eye wants to see and where to autofocus; moreover, the acquired light is the set of light collected in the real light field. When synthesized with the virtual image, it can be seen from the glasses that it cannot be distinguished. Of course, it is also possible to receive image data collected and transmitted by other image processing devices.
  • the image processing apparatus collects image data in a real environment through a camera. Since a real object exists in the real environment, the collected image data includes a real object. In another possible manner, the image data including the real object is acquired by another image processing device and then transmitted to the image processing device of the embodiment, and the image processing device receives the image data.
  • Step 502 Detect location information of a real object.
  • the virtual objects must be merged into the exact position in the real world. Therefore, the position of real objects in the image data is detected in real time, even in the direction of real object motion. Tracking, in order to help the system decide which virtual object in the augmented reality model is displayed and where the virtual object is displayed, and reconstruct the coordinate system according to the observer's field of view.
  • a video detection method for recognizing a predefined mark, an object or a reference point in a video image according to a pattern recognition technique, and then calculating a coordinate transformation matrix according to the offset and the rotation angle thereof.
  • the coordinate conversion matrix is used to represent the position information of the real object; or the angle of the user's head rotation is measured by the gyroscope to determine the position information of the real object to determine how to convert the coordinates and content of the virtual object in the field of view.
  • the image processing apparatus may acquire a plurality of image data including the real object, and track the trajectory of the real object motion according to the position and posture change between the image data. To determine the location information of the real object in each image data.
  • the position and posture change of each image data can be detected by a gyroscope, or a tracking algorithm can be used to track two adjacent image data.
  • Step 503 Obtain a virtual object from the augmented reality model.
  • the display In order to obtain the immersion of the AR device, the display must be displayed with a realistic image and simulated and displayed in an augmented reality scene. Therefore, the image processing apparatus obtains a virtual object from the augmented reality model.
  • the coordinate transfer matrix from the predefined mark to the mark in the current augmented reality scene is reconstructed, and the image processing device draws the virtual object in the augmented reality model according to the coordinate transfer matrix, and Rendering.
  • Step 504 Combine the real object and the virtual object into a video or directly display according to the location information.
  • the image synthesizer of the image processing device first calculates an affine transformation of the virtual object coordinates to the camera plane according to the position information of the camera and the positioning mark of the real object, and then draws the virtual object on the viewing plane according to the affine transformation matrix, thereby The virtual object is merged with the video or photo of the real object and displayed on the display to form an effect of the real object and the virtual object being displayed together.
  • a virtual object is synthesized with a video or image of a real object, and displayed on a call interface of the client, such as a video of a caller's video or image.
  • a call interface of the client such as a video of a caller's video or image.
  • Superimposing virtual objects such as hats and glasses in real time greatly enhances the fun of video conversations; or, in the scene of scanning real objects under the client line using social networks, the user is displayed together on the image of the real object.
  • Personal business cards in social networks enable seamless access to offline social and online social networking.
  • FIG. 5 is a schematic flowchart of another optional implementation of the image processing method provided by the embodiment of the present application. The steps are explained.
  • Step 601 The local client obtains image data.
  • the image data may be obtained by the user himself or herself by calling the camera, that is, collecting the environment to form image data in the process of the local client; or using the local client.
  • the peer client sends, that is, receives the peer client collection environment in the social network and transmits in the process of the local client.
  • Image data identifying features of real objects in the environment from the received image data.
  • Step 602 The local client identifies the feature of the real object from the obtained image data.
  • the real objects may be natural scenes, human scenes, and living objects (including humans) in nature.
  • the real object has a variety of feature types, such as image features, including: feature points of the face, contour features of the object, texture features, etc.; and biometric features, including voiceprint features, iris features, fingerprint features, and the like.
  • image features including: feature points of the face, contour features of the object, texture features, etc.
  • biometric features including voiceprint features, iris features, fingerprint features, and the like.
  • the local client captures one or more facial images including the face of the user by calling a camera of the host device of the host device, and performs face detection on the captured facial image.
  • the identification of the feature points for example, from the dimension recognition of the shape features, the detection of different facial organs by the external contour features, and the recognition of facial feature points of different parts of the facial organs.
  • a multi-frame facial image may also be acquired, and the multi-frame captured facial image is respectively recognized to obtain a plurality of facial feature points in each facial image.
  • the position, for example, the facial feature point includes any one or more of an eye feature point, a nose feature point, a lip feature point, a eyebrow feature point, and a face edge feature point.
  • the multi-frame facial image may be continuously captured.
  • the facial image may be a continuous multi-frame facial image in the captured video within a specified time period, for example, 1 second or 0.5 second;
  • the face image can also be a multi-frame face image that is discretely distributed on the time axis in the captured video.
  • each facial feature point obtained by digital marker recognition for example, 1 to 20 shown in FIG. 6 represents the facial expression.
  • Edge feature points, 21 to 28 and 29 to 36 correspond to the user's left eye feature point and right eye feature point
  • 37 to 44 and 88 represent the user's left eye feature point, of which 88 is the left eye pupil
  • 89 represents the right eye feature point of the user, wherein 89 is a right eye pupil feature point
  • 53-65 represents a user's nose feature point
  • 66-87 represents a user's lip feature point.
  • the feature recognition of the real object is described by taking the facial feature recognition as an example, wherein the facial feature recognition technology is generally divided according to the different criteria adopted, according to the different features.
  • the local feature based method may utilize the local geometric features of the face, such as the relative position and relative distance of some facial organs (eyes, nose, mouth, etc.) to describe the face. Its characteristic components usually include Euclidean distance, curvature and angle between feature points, which can achieve an efficient description of the salient features of the face.
  • the integral feature method is used to locate the facial feature points, and the multi-dimensional facial feature vector is identified by the Euclidean distance between the feature points as the feature component for classification.
  • the characteristic components mainly include: the vertical distance between the eyebrow and the center of the eye: a plurality of description data of the curvature of the eyebrow; the width of the nose and the vertical position of the nose; the position of the nostril and the width of the face, etc., through the identification of the facial feature information, in the identification process Get 100% correct recognition rate.
  • the local feature based method may also be an empirical description of the general characteristics of the facial features.
  • facial images have some obvious basic features.
  • facial regions usually include facial features such as eyes, nose and mouth, and their brightness is generally lower than that of the surrounding area; the eyes are roughly symmetrical, and the nose and mouth are distributed on the axis of symmetry.
  • the local feature-based method is not limited to the type based on the local feature method in the embodiment of the present application, in addition to the above-described integral projection method and the prior rule method.
  • the method based on the whole is to make the face image as a whole, and perform some transformation processing to identify the feature.
  • the method considers the overall attribute of the face, and also preserves the topological relationship between the face parts and the components. Information of its own.
  • the subspace analysis method can be used to find a linear or nonlinear spatial transformation according to a certain target, and compress the original high dimensional data into a low dimensional subspace, so that the distribution of data in this subspace is more compact and the calculation is reduced. The complexity.
  • a set of rectangular mesh nodes may be placed on the facial image, and the features of each node are described by multi-scale wavelet features at the node, and the connection relationship between the nodes is represented by geometric distance, thereby forming a two-dimensional structure.
  • the face representation of the topology map In the face recognition process, recognition is based on the similarity between nodes and connections in the two images.
  • the ensemble-based method is not limited to the type based on the overall method, in addition to the subspace analysis method and the elasticity map matching method described above, and the neural network-based method.
  • the feature recognition of the image data may be divided into the following two manners according to different execution subjects of the recognition feature:
  • Manner 1 The local client invokes the identification service of the server, and sends image data to the server's identification service.
  • the server identifies the feature of the real object from the obtained image data, and returns to the local client.
  • the first method is especially applicable to the case where the computing resources of the local client are limited, and the computing resources consumed by the local client for feature recognition and the delay caused thereby can be effectively reduced.
  • the computing resources of the local client are limited, and the computing resources consumed by the local client for feature recognition and the delay caused thereby can be effectively reduced.
  • the identification service of the server may be invoked, and the identity of the real object is identified from the obtained image data by the identification service of the server, and returned to the local client.
  • Manner 2 The image recognition thread is opened in the process of the local client, and the obtained image data is identified in the opened image recognition thread to obtain the feature of the real object.
  • the image recognition thread can be turned on in the process of the host device of the client.
  • the feature recognition operation is completed by the client device itself, because during the recognition process, the user may still listen to music, or open a game, or a video process, in order to not occupy other application resources, the process may be in the client.
  • the number of threads that can be opened can be determined according to the computational complexity of the recognition (such as the frame rate of the video, the resolution of the photo, etc.). If the computational complexity is low, only a relatively small number of threads can be opened, if the complexity of the calculation is recognized. Higher, you can open multiple threads.
  • the obtained image data is identified in the opened image recognition thread to obtain the feature of the real object, thereby ensuring that the feature information of the recognized image data is normally performed, and also avoids interrupting processes or threads of other applications.
  • the feature of the identified real object satisfies a condition capable of recognizing the real object, and the condition includes at least one of the following: when the image feature point is recognized, the identified The number of image feature points exceeds the feature point data amount threshold; when the biometric is identified, the integrity of the identified biometric exceeds the integrity threshold; if the condition is met, the subsequent steps are performed, otherwise return to step 601 until the satisfaction is satisfied Characteristics of the condition.
  • any of the following may occur: 1) in a dark environment; 2) the real object is in motion; 3) the camera is in motion; 4) the real object
  • the feature portion is occluded, for example, when the user himself is photographing the face, most of the face is blocked.
  • the acquired feature information is insufficient to complete the subsequent operation. Therefore, before the social network is queried by the feature of the real object, the quantity or integrity of the corresponding feature information is judged, and the A full feature query results in the consumption of computing resources by the social network.
  • the eyes, nose, and mouth are no less than 100 feature points.
  • the light is too dark, or the user and the user The camera is in a state of relative motion, or most of the user's face is occluded. Therefore, after the captured image is removed from the same feature point and the invalid feature point, the eyes, nose, and mouth are all below 100 feature points. If the collection fails, you need to re-collect it; otherwise, you can perform the next step.
  • Step 603 The local client queries the social network according to the characteristics of the real object to determine whether the real object belongs to the social network. If yes, step 604 is performed; otherwise, the process returns to step 601.
  • the feature database running in the server in the social network searches whether the preset feature information matching the feature of the real object is stored, and if so, determines the real The object belongs to the social network; if not, it does not belong to the social network.
  • the user selects the “QQ-AR” function option to take an image of himself or another user.
  • the QQ client collects the user's face in the photo.
  • Feature information according to the feature information, whether the feature information of the user exists in the social network. If the user uploads the image of the user in advance, the preset feature information of the user's face is pre-stored in the social network, thereby The preset feature information of the user may be found to determine that the user belongs to the social network; if the user does not upload his own image, it is determined that the user does not belong to the social network.
  • the corresponding features are used to query the social network through the feature database, such as The feature database of the registered user, the texture feature of the shared object, the graphic code, and the like are used to query the feature database of the social network; then, according to the query result, the following two scenarios can be divided.
  • Scenario 1 The type of the queried object is a registered user of the social network.
  • the feature database of the social network is queried with the feature of the real object; when the real object matches the feature of the registered user of the social network, the real object is determined to be a registered user belonging to the social network.
  • the local client when the user of the local client captures an image of the user or other user, the local client obtains image data about the person, and queries the feature database in the network according to the feature in the image data, and pre-stores the feature data.
  • the image of the user in the image data it may be determined that the user is a registered user belonging to the social network, and the ID of the registered user on the social network is obtained.
  • the image feature of the registered user is stored in the feature database in advance, and the image feature of the unregistered user is not stored. Therefore, whether the user is a registered user of the social network can be determined according to whether the feature in the image data of the user is stored in the feature database.
  • Scenario 2 The type of the queried object is the shared object in the social network.
  • the feature database of the social network is queried with the feature of the real object; when the real object matches the feature of the shared object of the social network, the real object is determined to be the shared object of the social network.
  • the client acquires image data about the real object, obtains feature information about the real object, such as a product QR code or a silhouette of the scene, and then according to the obtained feature.
  • the real object may be determined as the shared object of the social network, and the ID of the shared object in the social network is obtained, and the social Related content in the network about shared content is supported based on ID queries.
  • a common application is: when the user sees the product shared by a certain user on the social network, when not knowing where to buy it, then only need to scan the QR code or After the scanning is completed, the store can be jointly displayed on the image processing device screen or the HMD in an AR manner, and the address information of the store, wherein the address information can be an actual address or a website address, such as an e-commerce. The network address is thus purchased.
  • Step 604 The local client obtains an augmented reality model adapted from the real object from the social network.
  • the virtual object in the augmented reality model preset by the registered user in the social network is obtained, and the virtual object may be used to implement a dressing effect, for example, including At least one of the following: virtual props, virtual backgrounds, and filters.
  • the above filter may be an internal filter, a built-in filter and an external filter; of course, the virtual object can also achieve the effect of information display, such as displaying a user's business card in a social network and sharing information index.
  • the server of the social network by identifying and matching the facial features of the user, an image matching the facial features of the user in the social network is found, and the ID in the corresponding social network is obtained through the matched image. According to the ID, the associated augmented reality model is found as an adapted augmented reality model.
  • the augmented reality model of the registered user of the social network may be a personal business card that is randomly assigned to the network to display at least the registered user, and may also be Personalized settings based on the user.
  • the virtual object when the real object is a shared object in the social network, obtaining a virtual object in the social network for the shared object, the virtual object includes at least one of: a social network for the shared object Article; an advertisement for a shared object in a social network.
  • the user can align the item or the scene through "QQ-AR", and then the screen will An animation of the item or scene being scanned appears, and after the animation ends, it indicates that the product or the scene is scanned successfully, and then, according to the package, shape, barcode or QR code of the product, the article or advertisement associated with the item is found. Or purchase the store's store and address, etc.; or, based on information such as the characteristics, shape, and location of the scene, find an article or advertisement associated with it.
  • a solution for buffering the augmented reality model in the cache of the local client is provided, for example, for the user of the local client, the social network calculates the potential friend, the interested user or the product. And pre-push the corresponding augmented reality model to the local client for caching to speed up the rendering speed of the virtual object and avoid delay.
  • social network such as query augmented reality model prioritization, involving the following two different query results:
  • Method 1 Store in the cache or database of the host device
  • an augmented reality model adapted from a real object from a social network before obtaining an augmented reality model adapted from a real object from a social network, first, in the client's cache or the database of the host device, the ID of the real object in the social network is queried with the real object.
  • the augmented reality model is configured such that, for the case where the local client has stored the corresponding augmented reality model, it is not necessary to request the social network every time, and the rendering speed of the virtual object in the real model can be enhanced to minimize the delay.
  • the user obtains the facial feature parameter of the user after aligning himself with the captured image or a video.
  • the client queries the cache according to the feature parameter to cache the previously used augmented reality model. For example, a personalized AR dress is set, and if so, the augmented reality model is obtained from the cache, thus improving the efficiency of acquiring the augmented reality model.
  • Method 2 Store on a social network server
  • the augmented reality model of the real object is not queried in the cache and the database of the host device of the local client, the augmented reality model storing the real object is queried to the server of the social network by the ID of the real object.
  • Step 605 The local client performs rendering according to the obtained image data.
  • Step 606 The local client renders the virtual object in the augmented reality model according to the position of the real object in the rendered image, and forms a real object and a virtual object that are commonly displayed.
  • Method 1 Devices such as smartphones and computers
  • the client installed in the device such as the smart phone and the computer acquires the augmented reality model
  • the augmented reality model is synthesized with the real object carried in the image data transmitted during the instant communication process, The way the synthesized video or image is displayed on the smartphone screen or computer screen.
  • VR glasses are based on the display mode of the transmissive HMD of video synthesis technology.
  • the real-world video or image is acquired by the camera, and then the generated or acquired virtual object is synthesized with the real-world video or image, and corresponding rendering is performed. And then display it on the display through the HMD.
  • the user performs video chat with other users of the social network through the local client, and receives image data of the opposite client (bearing images of other users), and the local client performs face data on the image data.
  • Feature recognition 71 identifying that the user is a registered user of the social network, querying the augmented reality model predetermined by the user in the social network as AR dressing-diving glasses, in the process of rendering, according to the relative position of the AR glasses dressing and the user's human eye,
  • the diving glasses 72 are rendered in front of the eyes of the human eye.
  • the local client performs video collection on the environment where the host device is located, including collecting image data of the face in the environment, performing face feature recognition 81, and identifying the user of the local client as a social network.
  • the registered user querying the social network to obtain a predetermined augmented reality model for the AR, including the background 83 corresponding to the water wave and the diving glasses 82; the relative position according to the diving glasses 82, the virtual background 83 and the user's human eye, and the background 83 and the user
  • the hierarchical relationship is to place the virtual background 83 on the bottom layer of the user, avoiding the background 83 occluding the user.
  • the user uses the scanning function of the local client to call the camera of the host device to scan the face of the newly recognized friend, that is, collect image data for the face in the environment, and perform face feature recognition 91 to identify
  • the newly-recognized friend is a registered user of the social network, and the predetermined augmented reality model is queried for the AR dressing.
  • the AR dressing in the interface of the local client displaying the face, the rabbit face 92 and the mouth opening action are performed according to the face position rendering. After the personalized dressing of 93, after the synthesis, the user appeared in the head of the rabbit's ears and the mouth opened.
  • the local client detects a pose change of the real object in the image data, where the pose change may be the user and The relative position between the client devices changes, or the angle changes.
  • the change in the angle may be a change in the side view angle, the top view angle, or the look up angle between the user and the client.
  • the virtual object adapted to the pose change in the augmented reality model is rendered, and the superposed real object and the virtual object are formed to ensure the seamless fusion of the real object and the virtual object.
  • the local client detects that the user's location moves according to the scanned image data, and the local client uses the AR software development kit (SDK) of the device such as HDM or mobile phone to track and match the rendered real object. That is, as the real object highlights the movement or the distance between the local client and the real object and the angle of the object, the corresponding widget and background of the augmented reality model will also perform corresponding rendering changes, thereby forming a better augmented reality effect.
  • SDK AR software development kit
  • FIG. 11 is a schematic flowchart of another optional implementation of the image processing method according to the embodiment of the present application. Identify the server and social dressing server, including the following steps:
  • Step 801 The client performs an acquisition operation.
  • the client can acquire an image containing a face and perform an operation of acquiring features on the image to acquire feature points included in the image.
  • the user to be scanned here is referred to as user C.
  • Step 802 The client determines whether there are enough feature points, if yes, step 803 is performed, otherwise step 802 is continued.
  • the number of the collected feature points is obtained, and it is determined whether the number of the feature points exceeds the threshold value of the feature point data amount. If it is exceeded, the number of the feature points is sufficient, the scanning face is successful, and if not, the feature point is indicated. Insufficient quantity, need to continue to collect.
  • Step 803 The client detects whether there is a cache of the AR dressing in the local area. If yes, step 804 is performed; if not, step 805 is performed.
  • Step 804 The client displays an AR picture or video.
  • the AR picture or video is: a composite picture of the AR dressing and the image taken by the user, and the AR dresses a composite video of the video taken by the user.
  • the client After obtaining the AR dressing, the client combines the AR dress with the image or video captured by the user to obtain an AR picture or video, and implements an effect of adding a dress to the user in the AR picture or video.
  • Step 805 The client uploads a photo to the face recognition server.
  • face recognition is required at the face recognition server to perform a matching operation with the image stored in the face recognition server based on the recognized result.
  • Step 806 The face recognition server identifies that the matching is successful.
  • scenario 1 user C is the user who uses the client, and has not set the AR dress
  • scenario 2 the user C is another.
  • Step 807 The face recognition server acquires a social network account.
  • the social network account number may be a QQ number, or may be a micro signal, or other IM account number.
  • the social dressing server stores the personalized dressing corresponding to each social network account. After the face recognition server recognizes the registered user, the social network account of the registered user is obtained, and the social network account is obtained for the social dressing server to pull through the social network account. Personalized dress up.
  • Step 808 The face recognition server sends a request to the social dressing server to pull the personalized dress model.
  • the request carries the acquired social network account.
  • Step 809 The social dressing server pulls the personalized dressing model.
  • the face recognition server will obtain a personalized dressing model from the social dressing server, and then recommend a corresponding personalized dressing model to the client through the social dressing server.
  • the face server will obtain the AR dressing set by the user C from the social dressing server, and then recommend the corresponding personalized dressing model to the client through the social dressing server. If the user does not set the AR dressup, then the operation ends.
  • Step 810 Send the personalized dressing model to the client.
  • Step 811 The client loads the model according to the local ARSDK.
  • the client uses the AR SDK of the device such as HDM or mobile phone to track and match the displayed content and graphics, so that the personalized dressing follows the user's motion and renders changes, thereby forming a better augmented reality. Effect.
  • Scene 1 Online Social - Instant Video Chat, AR Dress Up
  • the user of the local client uses instant communication (including QQ, WeChat, etc.) to perform video chat with the peer user (such as friends and relatives), the user calls the camera in the local client to capture the video or image in real time, thereby obtaining the real object.
  • Video or image parameters in order to highlight the personalized, active chat atmosphere, before the video or image is taken (also in the process of capturing video or images) to add corresponding virtual objects, such as personalized dress and virtual background.
  • the user of the peer client when the user uses the camera to take a video or photo, it can also be similar to the local user, dress the captured video or picture, or directly transfer the captured video or photo to the local user, by the local user. Perform the above dressing operation.
  • the AR dressing described above can be replaced with other information of the user in the social network, such as a personal business card, including an account number, a graphic code, and the like in the social network.
  • Scenario 2 Realizing AR Dress Up in the Process of Online Social-Video Transmission
  • both users may send a funny, good-looking video or photo that they think they have taken to the other party, for example, a local user ( Or the opposite user) took a photo of the meal, the instant messaging client will identify the features of the real object in the photo, in order to match the corresponding dress according to the identified features, and then add a matching dress on the photo Send to the peer user (or local user).
  • instant messaging including QQ, WeChat, etc.
  • Scenario 3 Offline Social - Client scans other users
  • the local client such as the user of the mobile QQ, clicks the “sweep” option on the client, then selects “QQ-AR”, aligns the face of the user to be scanned, and then a real scan is displayed on the screen.
  • the animation of the object When the animation ends, it indicates that the scan is successful, that is, the feature of the real object in the environment is identified from the collected image data, and the corresponding ID is extracted from the social network based on the feature query, and the AR preset by the user is pulled according to the queried ID.
  • the client side instantly forms the effect of dressing on the scanned person's face.
  • Scene 4 Offline Social - Client scans the user himself
  • the local client such as the user of the mobile QQ, uses the camera to align the user's own face through the “QQ-AR”, and then an animation of the face being scanned appears on the screen, and then the animation is finished, indicating that the face is scanned. success.
  • At least one sexual dressing can be selected at the bottom of the screen. After the user selects the personalized dress that he likes, the personalized dress will be applied to the screen.
  • personalized dressing can be virtual items, virtual backgrounds and filters, etc.
  • the virtual items can be hats, glasses or other facial pendants.
  • Augmented reality models for different real objects in social networks have diversified features, such as AR-style dressings, social business cards, etc., as needed, so that when applied to image data rendering, different objects are differentiated. display effect.
  • the client identifies the feature from the image data locally or by calling the server's identification service according to the situation, which is beneficial to reduce the delay and realize the synchronous display of the real object and the virtual object.
  • the virtual object can be displayed in time on the client, and the real object and the virtual object are not displayed due to the network reason. The problem is not synchronized.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Library & Information Science (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Strategic Management (AREA)
  • Data Mining & Analysis (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Computer Graphics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Finance (AREA)
  • Tourism & Hospitality (AREA)
  • Primary Health Care (AREA)
  • Human Resources & Organizations (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Computational Linguistics (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种图像处理方法、装置及存储介质；方法包括：获得图像数据，并从所获得的图像数据识别出真实对象的特征；以真实对象的特征查询社交网络，确定真实对象具有社交网络的属性；从社交网络获得与真实对象适配的增强现实模型；根据所获得的图像数据进行渲染，以及，根据真实对象在所渲染形成的图像中的位置，对增强现实模型中的虚拟对象进行渲染，形成共同显示的真实对象与所述虚拟对象。本申请能够快速识别出归属于社交网络的真实对象，在相应的场景中融合社交网络中适配真实对象的增强现实效果。

Description

一种图像处理方法、装置及存储介质
本申请要求于2017年8月4日提交中国国家知识产权局、申请号为201710661746.0、发明名称为“一种图像处理方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像技术领域，尤其涉及一种图像处理方法、装置及存储介质。
背景技术
随着互联网特别是移动互联网的快速发展，出现了各式各样的基于社交网络的客户端，为用户进行网上社交提供了极大的方便，用户可以使用各种智能终端设备安装社交网络的客户端，随时随地与社交网络中的用户进行线下或线上的互动。
在社交网络的各种场景中展示对象的形象是客户端的基本功能，而目前展示的方式单一，以对象是用户为例，相关技术通常是采用虚拟人物形象或者自拍头像的方式来展示用户的形象，并起到在社交网络的辨识性；但是，这种方式目前难以适配社交网络个性化展现用户的需求，已经成为社交网络多元化特性的掣肘。
由于社交网络的应用场景日益多元化，如果提供针对多个应用场景分别设置展示方式的方案，一方面由于需要用户就不同应用场景进行设置而效率较低，不具有实施的现实意义，另一方面会导致社交网络后台的处理复杂化，导致很大的开销。
对于在社交网络的各种应用场景中拓展对象的展现方式以适配社交网络的多元化的需求，目前尚无有效解决方案。
发明内容
本申请实施例提供一种图像处理方法、装置及存储介质，可以解决上述技 术问题，实现有效拓展社交网络中对象的展现方式。
为达到上述目的，本申请实施例的技术方案是这样实现的：
一方面，本申请实施例提供一种图像处理方法，包括：
从所获得的图像数据识别出环境中真实对象的特征；
以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性；
获得所述社交网络中与所述真实对象适配的增强现实模型；
根据所获得的图像数据进行渲染，以及，
根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，形成共同显示的所述真实对象与所述虚拟对象。
另一方面，本申请实施例提供一种图像处理装置，包括：
识别模块，用于从所获得的图像数据识别出环境中真实对象的特征；
查询模块，用于以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性；
模型模块，用于获得所述社交网络中与所述真实对象适配的增强现实模型；
渲染模块，用于根据所获得的图像数据进行渲染，以及，根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，形成共同显示的所述真实对象与所述虚拟对象。
另一方面，本申请实施例提供一种存储介质，存储有可执行程序，所述可执行程序被处理器执行时，实现本申请实施例提供的图像处理方法。
另一方面，本申请实施例提供一种图像处理装置，包括：
存储器，用于存储可执行程序；
处理器，用于执行所述存储器中存储的可执行程序时，实现本申请实施例提供的图像处理方法。
本申请实施例至少具有以下有益效果：
1)基于从图像数据中识别真实对象的特征并查询社交网络的方式，能够对于社交网络的任意场景中的图像数据，都可以快速识别出归属于社交网络的真实对象，在相应的场景中融合社交网络中适配真实对象的增强现实模型，形成共同显示的真实对象与虚拟对象，提供了在社交网络中拓展对象的展现方式，达到虚拟现实结合的效果；
2)社交网络中针对不同的真实对象的增强现实模型具有多元化的特点，从而应用到图像数据的渲染时，实现了不同对象的差异化的显示效果。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1-1为本申请实施例提供的图像处理装置的一个可选的硬件结构示意图；
图1-2为本申请实施例提供的图像处理装置的一个可选的功能结构示意图；
图2为本申请实施例提供的基于图像处理装置实施为AR设备的一个可选的系统结构示意图；
图3为本申请实施例提供的图像处理装置实施为AR设备的另一个可选的结构示意图；
图4为本申请实施例提供的图像处理方法的一个可选的实现流程示意图；
图5为本申请实施例提供的图像处理方法的另一个可选的实现流程示意图；
图6为本申请实施例提供的脸部特征点的示意图；
图7为本申请实施例提供的将真实对象与虚拟对象进行共同显示的效果示意图；
图8为本申请实施例提供的将真实对象与虚拟对象进行共同显示的效果示意图；
图9为本申请实施例提供的将真实对象与虚拟对象进行共同显示的效果示意图；
图10-1和图10-2为本申请实施例提供的卡通人物装扮和自定义网络虚拟角色的效果示意图；
图11为本申请实施例提供的图像处理方法的又一个可选的实现流程示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
对本申请进行进一步详细说明之前，对本申请实施例中涉及的名词和术语进行说明，本申请实施例中涉及的名词和术语适用于如下的解释。
1)增强现实(AR，Augmented Reality)，将虚拟的信息应用到真实世界的技术，将真实的对象和虚拟的对象实时地叠加到了同一个画面或空间而同时存在。
增强现实技术能够将真实世界的信息与虚拟世界的信息无缝集成，将在真实世界无法体验到的信息(视觉、声音、味道等)采用科学技术模拟仿真后再叠加，将虚拟的信息应用到真实世界，被人类感官所感知，从而达到虚拟现实相结合的感官体验。
例如，计算真实影像(即仅包括现实世界中真实对象的照片或视频)中真实对象的位置和姿态，在真实影像中根据真实对象的位置和姿态，应用包括虚拟对象的影像，如图像、视频或者三维(3D，Three-Dimensional)模型等技术，在三维空间的真实影像中增添包括虚拟对象的影像。如根据真实对象的照片或视频中脸部的位置和姿态，添加基于脸部定位的虚拟道具，实现脸部装扮的效果；又例如，根据扫描商品的二维码，在显示的二维码附近显示商品信息和/或可以购买到该商品的商店及地址，等等。
增强现实还可以实现根据场景实现实时交互的特性，例如，在实现AR类游戏过程中，通过AR系统配套的手套或手棒控制游戏中的打斗动作；或者，在进行AR的棋类比赛中，可以通过AR系统配套的手套控制棋子，等等。
2)客户端，本文中是指在设备中安装的客户端，或设备中第三方的客户端，用于支持基于社交网络的各种应用，实现多种社交功能，如视频通话功能或者发送图片功能等。
3)设备，支持运行客户端的电子设备，如智能手机、平板电脑、基于图像处理装置显示器和头盔式显示器(HMD，Head-mounted display)的设备等，其中，HMD可以是基于光学原理的穿透式HMD或基于视频合成技术的穿透式HMD。文本中设备也称为客户端的宿主设备。
4)社交网络，基于网络(如广域网或局域网)部署的服务器上实现支持多个用户通过客户端(如QQ、企业IM)相互通信的网络。
5)图像数据，是对环境中真实对象的图像上每一点光的强弱和频谱(颜色)的表示，根据光的强弱和频谱信息，将真实世界的图像信息转换成数据信息，即为图像数据，以便于数字化处理和分析。
6)增强现实模型，是图像处理装置通过数字图形技术勾勒出的用于增强现实的数字化场景，例如社交网络中的个性化AR装扮，可以是帽子、眼镜和背景图像等装扮。
7)真实对象，图像数据中包括现实生活中的人和物，其中物包括河流、山川等自然景物、以及城市景观、建筑景观等人文景物或者其他类型的物体等。
8)虚拟对象，客户端渲染图像数据时需要渲染在采集图像数据的环境中不存在的虚拟对象，实现真实对象与虚拟对象的融合，实现显示效果的提升或信息量的增强；例如当真实对象为人物时，虚拟对象可以是用于装扮人物形象的各种道具和虚拟背景，也可以是个人名片。
9)渲染，客户端中使用渲染引擎输出到屏幕的真实对象和虚拟对象的可视影像。例如，在使用社交客户端进行社交的过程中，为了增加社交的活跃气氛，对包括真实对象的图像或视频中进行一些适当的渲染，如在用户的图像或视频增加一些符合当前社交场景的虚拟对象以形成特效。
现在将参考附图描述实现本申请实施例的图像处理装置。图像处理装置可以以各种形式来实施，下面对本申请实施例的图像处理装置的硬件结构做说明。
参见图1-1，图1-1为本申请实施例提供的图像处理装置的一个可选的硬件结构示意图，实际应用中可以实施为前述的运行客户端的各种设备，如台式机电脑、笔记本电脑和智能手机。图1-1所示的图像处理装置100包括：至少一个处理器101、存储器102、显示组件103、至少一个通信接口104和摄像头105。图像处理装置100中的各个组件通过总线系统106耦合在一起。可以理解，总线系统106用于实现这些组件之间的连接通信。总线系统106除包括配置数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图1中将各种总线都标为总线系统106。
其中，显示组件103可以包括图像处理装置显示器、手机显示屏和平板电 脑显示屏等，用于显示。
通信接口104可以包括天线系统、蓝牙(Bluetooth)、无线局域网(WiFi，Wireless Fidelity)、近场通信(NFC，Near Field Communication)模块和/或数据线等。
摄像头105可以是定标准摄像头、长焦摄像头、广角镜头、变焦摄像头、数字光场摄像头和数码摄像头等。
可以理解，存储器102可以是易失性存储器或非易失性存储器，也可包括易失性和非易失性存储器两者。
本申请实施例中的存储器102用于存储各种类型的配置数据以支持图像处理装置100的操作。这些配置数据的示例包括：用于在图像处理装置100上操作的程序如客户端1021，还包括操作系统1022和数据库1023，其中，实现本申请实施例方法的程序可以包含在客户端1021。
本申请实施例揭示的方法可以应用于处理器101中，或者由处理器101实现。处理器101可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，图像处理方法的各步骤可以通过处理器101中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器101可以是通用处理器、数字信号处理器(DSP，Digital Signal Processor)，或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。处理器101可以实现或者执行本申请实施例中提供的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所提供的方法的步骤，可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于存储介质中，该存储介质位于存储器102，处理器101读取存储器102中的信息，结合其硬件完成本申请实施例提供的图像处理方法。
再对图1-1示出的图像处理装置的功能结构进行说明，以软件实施为例，参见图1-2，图1-2为本申请实施例提供的运行本端客户端(可以理解，本端客户端与对端客户端是相对的概念)的图像处理装置的一个可选的功能结构示意图，结合图1-2示出的各个功能模块进行说明，根据图1-1，可以理解图1-2示出的功能模块在硬件上的实现。
识别模块210，用于从所获得的图像数据识别出环境中真实对象的特征。
在本申请可选实施例中，识别模块210接收社交网络中的对端客户端采集 环境并传输的图像数据，从所接收的图像数据中识别位于对端客户端环境的真实对象的特征；和/或，采集环境形成图像数据，从所采集形成的图像数据中识别位于本端客户端环境的真实对象的特征。
在本申请可选实施例中，识别模块210，具体用于当与社交网络中的对端客户端通信时，采集本端客户端环境形成用于传输至对端客户端的图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征；或者，当响应本端客户端的采集操作时，采集本端客户端环境形成图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征。
在本申请可选实施例中，识别模块210，具体用于当获得社交网络中与真实对象适配的增强现实模型之前，判断所识别出真实对象的特征满足社交网络能够识别的条件，条件包括以下至少之一：当识别出图像特征点时，所识别出的图像特征点的数量超出特征点数据量阈值；当识别出生物特征时，所识别出的生物特征的完整程度超出完整程度阈值。
查询模块220，用于以真实对象的特征查询社交网络，确定真实对象是否具有属于社交网络的属性。
本申请实施例所涉及社交网络的属性，是针对社交网络所承载的功能，如媒体功能(例如内容聚合)、社交、电商和支付等，对实现这些功能的过程所涉及的成员在类型/功能上进行的归纳，例如，包括：
注册用户属性，表示成员是社交网络的注册用户；
支付对象属性，表示成员是接收支付的账户；
被分享对象属性，也称为被分享物品属性，表示成员是社交网络中被分享的物品，如美食、商品等各种物品；
被分享媒体信息属性，表示成员是社交网络中被分享的媒体信息，如视频、音频和手机游戏等各种不具有实际形态的产品。
在本申请可选实施例中，查询模块220，具体用于：以真实对象的特征查询社交网络的特征数据库；当真实对象与社交网络的注册用户的特征匹配时，确定真实对象为属于社交网络的注册用户，此时真实对象具有社交网络的注册用户属性；当真实对象与社交网络的被分享对象的特征匹配时，确定真实对象为社交网络的被分享对象，此时真实对象具有社交网络的被分享对象属性。
模型模块230，用于获得社交网络的模型库中与真实对象适配的增强现实模型。
在本申请可选实施例中，模型模块230，具体用于：当真实对象为社交网络中的注册用户时，获得注册用户在社交网络中预设的虚拟对象，虚拟对象包括以下至少之一：虚拟道具、虚拟背景和滤镜；当真实对象为社交网络中的被分享对象时，获得社交网络中针对被分享对象的虚拟对象，虚拟对象包括以下至少之一：社交网络中针对被分享对象的文章；社交网络中针对被分享对象的广告。
在本申请可选实施例中，模型模块230，具体用于：调用服务器的识别服务，从所获得的图像数据中识别出真实对象的特征；或者，开启图像识别线程，在所开启的图像识别线程中识别所获得的图像数据得到真实对象的特征。
渲染模块240，用于根据所获得的图像数据进行渲染，以及，根据真实对象在所渲染形成的图像中的位置，对增强现实模型中的虚拟对象进行渲染，形成共同显示的真实对象与虚拟对象。
在本申请可选实施例中，渲染模块240，具体用于：检测真实对象在图像数据中的位姿变化；在真实对象在所输出的图像中的位置，渲染输出增强现实模型中与位姿变化适配的虚拟对象，形成叠加的真实对象与虚拟对象。
在本申请可选实施例中，查询模块220，具体用于在本地的缓存或数据库中查询与真实对象适配的增强现实模型；当未查询到时，从社交网络查询得到与真实对象适配的增强现实模型。
将对本申请实施例提供的图像处理装置实施为AR眼镜时实现AR功能的结构进行示例性说明，参见图2和图3，图2为本申请实施例提供的图像处理装置实施为AR设备的一个可选的结构示意图，用于实现环境的图像数据的采集、图像数据与用于实现增强现实模型的虚拟对象的合成；图3为本申请实施例提供的图像处理装置实施为AR设备的另一个可选的结构示意图，用于实现图像数据与虚拟对象的合成输出。
虽然图像处理装置的结构是分别在图2和图3示出的，但是可以理解，图2和图3示出的结构可以结合使用，与实现从采集环境的图像数据到渲染输出图像数据与虚拟对象的合成显示效果，就图2和图3涉及的组件进行说明。
摄像头，用于获取包括真实对象的环境的图像数据，包括图像或视频，将获取到的图像或视频发送至图像合成器，以与增强现实模型的虚拟对象进行合成操作。
场景产生器，用于根据图像数据中真实对象的位置信息，例如，通过头部跟踪器获取头部在图像数据中的位置信息，提取增强现实模型中对应位置信息的虚拟对象，并将该虚拟对象发送至图像合成器。
场景产生器，还用于根据位置信息生成虚拟对象，并将该虚拟对象发送至显示器，虚拟对象用以在图像合成器上与真实对象进行叠加。
图像合成器，用于将获取到的真实对象的图像或视频，以及虚拟对象进行合成，将合成图像或合成视频进行渲染，渲染结果定时刷新到显示器显示。
显示器，用于将图像合成器发送的合成图像或合成视频进行显示，形成真实对象和增强现实模型的虚拟对象共同显示的效果。
现在根据参考附图描述实现本申请实施例的图像处理方法的实现过程，以图像处理装置根据图1至图3实施为智能手机或AR眼镜为例，参见图4，图4为本申请实施例提供的图像处理方法的一个可选的实现流程示意图，对图像处理装置获得包括真实对象的环境形成的图像数据、以及增强现实模型的虚拟图像进行说明，涉及以下步骤：
步骤501：获得包括真实对象的图像数据。
获取真实对象的图像数据是实现增强现实的首要步骤，只有将真实世界中的图像输入到图像处理装置中，与图像处理装置从增强现实模型提取的产生的虚拟图像合成，并输出到上述的显示组件上，用户才能看到最终的增强场景图像。
这里，真实对象的图像数据可以通过上述的摄像头采集，例如数字光场摄像头在拍摄真实对象，如人或自然景物时，可以获取到完整的光场信息，使得用户在使用图像处理装置的过程中能实现人眼想看哪里、哪里就能自动对焦；而且，获取的光线是真实光场中采集的光线集，当与虚拟图像合成后，从眼镜中看出去无法辨真假。当然，也可以是接收到其他图像处理装置采集并发送的图像数据。
也即是，在一种可能实现方式中，图像处理装置通过摄像头采集真实环境中的图像数据，由于该真实环境中存在真实对象，因此采集到的图像数据中包括真实对象。在另一种可能方式中，由其他图像处理装置采集包括真实对象的图像数据后发送给本实施例的图像处理装置，该图像处理装置接收该图像数据。
步骤502：检测真实对象的位置信息。
实现虚拟对象和真实对象如人和物的完美结合，须将虚拟对象合并到现实世界中的准确的位置，因此，实时地检测图像数据中真实对象的位置，甚至是对真实对象运动的方向进行跟踪，以便用来帮助系统决定显示何种增强现实模型中的哪个虚拟对象以及该虚拟对象显示的位置，并按照观察者的视场重建坐标系。测定真实对象的位置信息的方法可以有多种，例如，视频检测方法，根据模式识别技术识别视频图像中预先定义好的标记、物体或基准点，然后根据其偏移和转动角度计算坐标转换矩阵，以坐标转换矩阵来表示真实对象的位置信息；或者，通过陀螺仪测定使用者的头部转动的角度，确定真实对象的位置信息，以判定如何转换视场中虚拟对象的坐标和内容。
也即是，图像处理装置获取到图像数据后，采用任一种上述测定位置信息的方法，检测图像数据中真实对象的位置信息。并且，在真实对象在真实环境中进行移动的过程中，图像处理装置可以获取到多个包括真实对象的图像数据，并根据图像数据之间的位置和姿态变化，对真实对象运动的轨迹进行跟踪，从而确定每个图像数据中真实对象的位置信息。其中，每个图像数据的位置和姿态变化可以通过陀螺仪检测得到，或者采用跟踪算法，对相邻两个图像数据进行跟踪得到。
步骤503：从增强现实模型中获得虚拟对象。
为了获得AR设备的沉浸感，须让显示器显示具有真实感的图像，并在增强现实场景中进行模拟和显示。因此，图像处理装置从增强现实模型中获得虚拟对象。
当真实对象在增强现实场景中的定位标记识别后，重建从预定义标记到当前增强现实场景中标记的坐标转移矩阵，图像处理装置根据这个坐标转移矩阵，绘制增强现实模型中的虚拟对象，并进行渲染。
步骤504：根据位置信息，将真实对象和虚拟对象合并视频或直接显示。
图像处理装置的图像合成器首先根据摄像头的位置信息和真实对象的定位标记，来计算虚拟对象坐标到摄像头视平面的仿射变换，然后按照仿射变换矩阵在视平面上绘制虚拟对象，从而将虚拟对象与真实对象的视频或照片合并后一起显示在显示器上，形成真实对象与虚拟对象共同显示的效果。
举例来说，在使用社交网络的客户端进行视频通话的场景中，将虚拟对象与真实对象的视频或图像进行合成，显示在客户端的通话界面上，如在通话者 的视频或图像的脸部实时叠加帽子、眼镜等虚拟对象，在很大程度上提高了视频对话或的趣味性；或者，在使用社交网络的客户端线下扫描真实对象的场景中，在真实对象的影像上共同显示用户在社交网络中的个人名片，实现了线下社交与线上社交的无缝接入。
基于图4说明的图像处理装置对包括真实对象的图像数据与增强现实模型的虚拟对象融合的处理过程，下面，以用户使用社交网络的客户端(由于涉及到与对端客户端的通信，因此这里也称为本端客户端)在社交网络的应用场景中进行图像处理的过程进行说明，图5为本申请实施例提供的图像处理方法的另一个可选的实现流程示意图，将根据图5示出的步骤进行说明。
步骤601：本端客户端获得图像数据。
在本申请可选实施例中，图像数据的获取方式可以是用户本人通过本端客调用摄像头拍摄获得，即在本端客户端的进程中采集环境形成图像数据；也可以是在使用本端客户端与社交网络中其他用户的对端客户端(数量不限)进行视频通信的过程中，由对端客户端发送，即在本端客户端的进程中接收社交网络中对端客户端采集环境并传输的图像数据，从所接收的图像数据中识别环境中真实对象的特征。
步骤602，本端客户端从所获得的图像数据识别出真实对象的特征。
这里，真实对象可以是自然景物、人文景物和自然界中具有生命的物体(包括人类)。其中，真实对象的特征类型有多种，如图像特征，包括：脸部的特征点、物体的轮廓特征、纹理特征等；又如生物特征，包括声纹特征、虹膜特征、指纹特征等。在本申请实施例将主要以人为例进行阐述，例如从图像数据中识别人的脸部特征信息，对于其他类型特征的实施可以根据下文的说明而理解。
在本申请可选实施例中，本端客户端通过调用自身的宿主设备的摄像头拍摄，获取包括用户的脸部的一张或多张的脸部图像，对拍摄获取得到的脸部图像进行脸部特征点的识别，例如从形状特征的维度识别来说，通过外部轮廓特征检测不同脸部器官，识别脸部器官的不同部位的脸部特征点。
在本申请另一可选实施例中，还可以获取到多帧的脸部图像，分别对多帧拍摄到的脸部图像进行识别，得到每张脸部图像中的多个脸部特征点的位置，例如，脸部特征点包括眼睛特征点、鼻子特征点、嘴唇特征点、眉部特征点以 及脸部边缘特征点中的任一种或多种。
多帧的脸部图像可以是连续拍摄到的，例如脸部图像可以为指定时长内，拍摄到的视频中的连续的多帧的脸部图像，指定时长例如为1秒或0.5秒；当然，脸部图像也可以拍摄到的视频中在时间轴上离散分布的多帧的脸部图像。
示例性的，可以得到如图6所示的脸部特征点的识别结果，为了下文方便说明，采用数字标记识别得到的各个脸部特征点，例如图6中所示的1～20表示脸部边缘特征点，21～28以及29～36对应表示用户的左眉部特征点和右眉部特征点，37～44以及88表示用户的左眼特征点，其中88为左眼瞳孔，45～51以及89表示用户的右眼特征点，其中89为右眼瞳孔特征点，53～65表示用户的鼻子特征点、66～87表示用户的嘴唇特征点。需要指出的是，以上仅为示例，在可选实施例中可以在以上脸部特征点中仅识别部分或更多的特征点，或采用其他方式标记各个特征点，均属于本申请实施例的范畴。
在本申请实施例中，以脸部特征识别为例对上述的真实对象的特征识别进行阐述，其中，脸部特征识别技术按照其采用的准则的不同，通常根据所识别的特征的不同分为两类：
(1)基于局部特征的方法
在本申请可选实施例中，基于局部特征的方法可以利用脸部的局部几何特征，如一些脸部器官(眼、鼻、嘴等)的相对位置和相对距离来描述脸部。其特征分量通常包括特征点间的欧氏距离、曲率和角度等，可以实现对脸部显著特征的一个高效描述。
例如，使用积分投影法定位脸部特征点，以特征点间欧氏距离作为特征分量识别出多维的脸部特征向量用于分类。特征分量主要包括：眉毛与眼睛中心的垂直距离：眉毛弧度的多个描述数据；鼻宽及鼻的垂直位置；鼻孔位置以及脸宽等，通过上述脸部特征信息的识别，在识别过程中可以取得100％正确识别率。
在本申请可选实施例中，基于局部特征的方法还可以是关于脸部特征一般特点的经验描述。
例如，脸部图像有一些明显的基本特征，如脸部区域通常包括双眼、鼻和嘴等脸部特征，其亮度一般低于周边区域；双眼大致对称，鼻、嘴分布在对称轴上等。
基于局部特征的方法除了上述的积分投影法和先验规则方法，还有基于几 何形状信息方法等，在本申请实施例中，对基于局部特征方法的类型不做限制。
(2)基于整体的方法
这里，基于整体的方法则是把脸部图像作为一个整体，对其进行某种变换处理识别特征，该方法考虑了脸部的整体属性，也保留了脸部部件之间的拓扑关系和各部件本身的信息。
由于脸部图像的维数通常非常高，且脸部图像在高维空间中的分布很不紧凑，因而不利于分类，并且在计算上的复杂度也非常大。可采用子空间分析的方法，根据一定的目标来寻找一个线性或非线性的空间变换，把原始高维数据压缩到一个低维子空间，使数据在此子空间内的分布更加紧凑，降低计算的复杂度。
此外，也可在脸部图像上放置一组矩形网格节点，每个节点的特征用该节点处的多尺度小波特征描述，各节点之间的连接关系用几何距离表示，从而构成基于二维拓扑图的脸部表述。在脸部识别过程中，根据两幅图像中各节点和连接之间的相似性进行识别。
基于整体的方法除了上述的子空间分析法和弹性图匹配法，还有基于神经网络的方法等，在本申请实施例中，对基于整体方法的类型不做限制。
在本申请可选实施例中，对于图像数据的特征识别，根据识别特征的执行主体不同，可以分以下两种方式：
方式一：本端客户端调用服务器的识别服务，向服务器的识别服务发送图像数据，由服务器从所获得的图像数据中识别出真实对象的特征，并返回本端客户端。
方式一尤其适用于本端客户端计算资源有限的情况，能够有效降低本端客户端进行特征识别而消耗的计算资源以及导致的延迟。例如，对于视频中的特征识别，由于视频中的对象一般来说是处于运动状态的，运动中的对象的特征点识别，相应的操作复杂，占用客户端的宿主设备的开销也大，此时，可以调用服务器的识别服务，由服务器的识别服务从所获得的图像数据中识别出真实对象的特征，返回给本端客户端。
方式二：在本端客户端的进程中开启图像识别线程，在所开启的图像识别线程中识别所获得的图像数据得到真实对象的特征。
对于简单图像数据的特征识别，为了更快速的识别到特征点，可以在客户端的宿主设备的进程中开启图像识别线程。
例如，通过客户端的宿主设备本身完成这项特征识别操作，由于在识别过程中，用户可能还在听音乐、或开启了游戏、或视频进程，为了不占用其它应用的资源，可以在客户端的进程中开启图像识别的线程。其中，线程的开启数量，可以根据识别的计算复杂程度(如视频的帧率、照片的分辨率等)决定，若计算复杂程度较低，可以只开启相对少的线程，如果识别计算的复杂程度较高，可以开启多个线程。开启完成后，在所开启的图像识别线程中识别所获得的图像数据得到真实对象的特征，从而保证了识别图像数据的特征信息正常进行，同时也避免中断其它应用的进程或线程。
在本申请可选实施例中，对于所识别的特征，可以判断所识别出真实对象的特征满足能够识别真实对象的条件，条件包括以下至少之一：当识别出图像特征点时，所识别出的图像特征点的数量超出特征点数据量阈值；当识别出生物特征时，所识别出的生物特征的完整程度超出完整程度阈值；如果满足条件则执行后续步骤，否则返回步骤601，直至获得满足条件的特征。
由于在用户采集关于真实对象的图像数据时，可能会出现以下任一种情况：1)处于光线较暗的环境；2)真实对象处于运动状态；3)摄像头处于运动状态；4)真实对象的特征部分被遮挡，例如用户本人在拍摄脸部的时候，脸部的大部分被遮挡。当出现上述任一种情况时，造成所获取的特征信息不足以完成后续操作，因此，在以真实对象的特征查询社交网络之前，对相应的特征信息的数量或完整性进行判断，能够避免不完整特征查询导致社交网络的计算资源消耗的情况。
举例来说，假设脸部识别需要眼、鼻和嘴等脸部特征，且眼、鼻和嘴均不低于100个特征点，用户在拍摄图像时，由于拍摄的光线过暗、或用户与摄像头处于相对运动的状态、或用户的大部分脸部被遮挡，因此，拍摄出来的图像，除去相同的特征点和无效特征点之后，眼、鼻和嘴均低于100个特征点，那么，此次采集失败，需要重新进行采集；否则，可以执行下一步的操作。
步骤603：本端客户端根据真实对象的特征查询社交网络，确定真实对象是否属于社交网络，如果属于，执行步骤604；否则，返回步骤601。
本申请可选实施例中，根据真实对象的特征，在社交网络中的服务器运行的特征数据库，查找是否存储有与该真实对象的特征相匹配的预设特征信息，若有，则确定该真实对象属于该社交网络；若否，则不属于社交网络。
例如，以本端客户端为QQ客户端为例，用户选择“QQ-AR”功能选项， 拍摄自己或其它用户的图像，拍摄完成后，QQ客户端便采集到照片中的关于用户脸部的特征信息，根据该特征信息，在社交网络中查找是否存在该用户的特征信息，若用户预先上传了自己的图像时，那么，社交网络中已经预先存储用户的脸部的预设特征信息，从而可以查找到该用户的预设特征信息，则确定该用户属于该社交网络；若用户未上传自己的图像时，则确定该用户不属于该社交网络。
对于社交网络中的各种真实对象而言，不仅可以包括社交网络的注册用户，还可包括社交网络中的被分享对象如商品等各种物品，通过特征数据库记录对应的特征查询社交网络，如以注册用户的人脸特征，被分享对象的纹理特征、图形码等查询社交网络的特征数据库；那么，根据查询结果可以分为以下两个场景。
场景一：查询到的对象的类型为社交网络的注册用户
在本申请可选实施例中，以真实对象的特征查询社交网络的特征数据库；当真实对象与社交网络的注册用户的特征匹配时，确定真实对象为属于社交网络的注册用户。
例如，本端客户端的用户拍摄本人或其他用户的图像时，本端客户端获取到的是关于人的图像数据，根据图像数据中的特征查询网络中的特征数据库，当特征数据中预先存储了关于图像数据中的用户的图像，则可以确定该用户为属于社交网络的注册用户，并获得注册用户在社交网络的ID。
其中，特征数据库中预先存储了注册用户的图像特征，而不存储未注册用户的图像特征，因此根据特征数据库中是否存储用户的图像数据中的特征，可以判断用户是否为社交网络的注册用户。
场景二：查询到的对象的类型为社交网络中的被分享对象
在本申请可选实施例中，以真实对象的特征查询社交网络的特征数据库；当真实对象与社交网络的被分享对象的特征匹配时，确定真实对象为社交网络的被分享对象。
例如，用户拍摄真实对象如拍摄商品或景物，客户端获取到的是关于真实对象的图像数据，获得关于真实对象的特征信息，如商品二维码或景物轮廓等特征，然后根据所获得的特征查询网络中的特征数据库，当特征数据中预先存储了与真实对象匹配的被分享对象的图像，则可以确定真实对象为社交网络的被分享对象，获得被分享对象在社交网络中的ID，社交网络中关于被分享的相 关内容支持基于ID查询得到。
通过查询的对象类型为被分享对象时，一个常见的应用是：当用户看到社交网络中某个用户分享的商品，当不知道从哪里可以购买到，那么，只需要扫描其二维码或条形码，扫描完成后，可以在图像处理装置屏幕或者HMD上以AR的方式共同显示可以购买的商店，以及商店的地址信息，其中，地址信息可以是实际的地址，也可以是网址，如电商的网络地址，从而进行购买。
步骤604：本端客户端从社交网络获得与真实对象适配的增强现实模型。
在本申请可选实施例中，当真实对象为社交网络中的注册用户时，获得注册用户在社交网络中预设的增强现实模型中的虚拟对象，虚拟对象可以用于实现装扮效果，例如包括以下至少之一：虚拟道具、虚拟背景和滤镜。这里，上述滤镜可以是内阙滤镜、内置滤镜和外挂滤镜；当然，虚拟对象也可以实现信息展示的效果，如用于展示用户在社交网络中的名片和分享的信息索引等。
例如，在社交网络的服务器中，通过对用户的脸部特征的识别与匹配，查找到社交网络中与该用户脸部特征相匹配的图像，通过匹配到的图像获取对应的社交网络中的ID，根据ID查找到关联的增强现实模型作为适配的增强现实模型，可以理解，社交网络的注册用户的增强现实模型可以是涉及网络随机分配的以至少用于显示注册用户的个人名片，还可以根据用户实现个性化的设置。
在本申请可选实施例中，当真实对象为社交网络中的被分享对象时，获得社交网络中针对被分享对象的虚拟对象，虚拟对象包括以下至少之一：社交网络中针对被分享对象的文章；社交网络中针对被分享对象的广告。
例如，当用户发现某个商品很喜欢，或者某个景点很漂亮，但不知道关于该商品或景物的相关信息时，用户可以通过“QQ-AR”，对准商品或景物，然后屏幕上会出现一个正在扫描的商品或景物的动画，然后动画结束后，表示扫描商品或景物成功，然后，根据商品的包装、外形、条形码或二维码等信息，查找到与之关联的文章或广告，或者购买该商品的商店与地址等；或者，根据景物的特征、外形及地理位置等信息，查找到与之关联的文章或广告。
本申请可选实施例中，对于增强现实模型的获取，提供在本端客户端的缓存中缓存增强现实模型的方案，例如对于本端客户端的用户，社交网络计算潜在好友，感兴趣的用户或商品，并将对应的增强现实模型预先推送到本端客户端进行缓存以加速虚拟对象的渲染速度，避免延迟。
那么，根据本地缓存、社交网络这样的查询增强现实模型的优先级排序，涉及以下两种不同的查询结果：
方式一：存储于宿主设备的缓存或数据库中
在本申请可选实施例中，从社交网络获得与真实对象适配的增强现实模型之前，首先在客户端的缓存或宿主设备的数据库中，以真实对象在社交网络中的ID查询与真实对象适配的增强现实模型，这样，对于本端客户端已经存储对应的增强现实模型的情况，无需每次都向社交网络请求，可以增强现实模型中虚拟对象的渲染速度，最大程度减小延迟。
例如，用户通过“QQ-AR”，对准自己拍摄图像或一段视频后，获取到用户本人的脸部特征参数，客户端根据该特征参数在缓存中查询是否缓存了之前使用过的增强现实模型，例如设置过个性化AR装扮，若有，则从缓存中获取该增强现实模型，这样，提高了获取增强现实模型的效率。
方式二：存储于社交网络服务器
在本端客户端的宿主设备的缓存和数据库中均未查询到真实对象的增强现实模型时，以真实对象的ID向社交网络的服务器中查询存储真实对象的增强现实模型。
步骤605：本端客户端根据所获得的图像数据进行渲染。
步骤606：本端客户端根据真实对象在所渲染形成的图像中的位置，对增强现实模型中的虚拟对象进行渲染，形成共同显示的真实对象与虚拟对象。
这里，根据显示的方式不同，可以分为以下两种情况：
方式一：智能手机和电脑等设备
在社交网络的即时通信的场景中，智能手机和电脑等设备中安装的客户端获取到增强现实模型时，将增强现实模型与即时通信过程中传输的图像数据中承载的真实对象进行合成，以合成后的视频或图像的方式在智能手机屏幕或电脑屏幕上显示。
方式二：设置HMD的VR眼镜
VR眼镜基于视频合成技术的穿透式HMD的显示方式，通过摄像头获取真实世界的视频或图像，然后将产生或获取到的虚拟对象与该真实世界的视频或图像进行合成，并进行相应的渲染，然后通过HMD在显示器进行显示。
在渲染的时候，需要考虑增强现实模型的虚拟对象与真实对象之间的位置关系，下面举例说明。
1)如图7所示，用户通过本端客户端与社交网络的其他用户进行视频聊天，接收对端客户端的图像数据(承载有其他用户的影像)，本端客户端对图像数据进行人脸特征识别71，识别到用户为社交网络的注册用户，查询社交网络中该用户预定的增强现实模型为AR装扮-潜水眼镜，在渲染的过程中，根据AR眼镜装扮与用户人眼的相对位置，将潜水眼镜72渲染在人眼的“眼前”。
2)如图8所示，本端客户端对宿主设备所在的环境进行视频采集，包括对环境中的人脸采集图像数据，进行人脸特征识别81，识别到本端客户端的用户为社交网络的注册用户，查询社交网络得到预定的增强现实模型为AR装扮，包括对应水波的背景83和潜水眼镜82；根据潜水眼镜82、虚拟的背景83与用户人眼的相对位置，以及背景83与用户的层次关系，即将虚拟的背景83置于用户的底层，避免背景83将用户遮挡。
3)如图9所示，用户使用本端客户端的扫描功能，调用宿主设备的摄像头扫描新认识朋友的脸部，即对环境中的人脸采集图像数据，进行人脸特征识别91，识别到新认识的朋友为社交网络的注册用户，查询到预定的增强现实模型为AR装扮，根据AR装扮，在本端客户端显示人脸的界面中，根据人脸位置渲染加兔子耳朵92和张嘴动作93的个性化装扮，合成后，用户出现头部长出兔子耳朵和嘴部张开的画面。
在本申请另一可选实施例中，对于图像数据是视频数据或一系列的照片的情况，本端客户端检测真实对象在图像数据中的位姿变化，这里，位姿变化可以是用户与客户端设备之间的相对位置发生变化、或角度发生变化，其中，角度发生变化可以是用户与客户端之间的侧视角度、俯视角度或仰视角度发生变化。根据真实对象在所输出的图像中的位置，渲染输出增强现实模型中与位姿变化适配的虚拟对象，形成叠加的真实对象与虚拟对象，保证真实对象与虚拟对象的无缝融合的效果。
本端客户端根据扫描的图像数据，检测用户的位置发生移动，本端客户端会利用HDM或手机等设备的AR软件开发工具包(SDK，Software Development Kit)，对渲染的真实对象进行追踪匹配，即随着真实对象凸显的移动或者本端客户端与真实对象距离和物体角度的变化，增强现实模型对应的挂件和背景也会进行对应的渲染变化，从而形成更好地增强现实的效果。
目前，许多IM客户端中都支持给自己网络虚拟角色(Avatar)进行设置， 在聊天中表现出来，如图10-1和图10-2所示，允许用户选择自己喜欢的3D Avatar形象并在视频聊天中应用，头部移动或张嘴的时候，用户选择的Avatar也会跟随作出相应的动作；此外，还有一种是卡通形象装扮，例如用户选择一个卡通形象代表虚拟世界中的自己，用户可以为卡通形象换套装、换脸型等等。而在新技术层出不穷的今天，以上的场景已经无法很好地满足用户诉求了。一方面是卡通人物与自己并不具备相关性，对于年轻用户来说，与其给自己添加一个卡通人物形象，不如让好友直接看到自己扮演某个卡通人物的效果，才能更好地彰显个性化。
本申请实施例提供的图像处理方法的方案可以用以解决上述问题，参见图11，图11为本申请实施例提供的图像处理方法的又一个可选的实现流程示意图，社交网络中设置人脸识别服务器和社交装扮服务器，包括以下步骤：
步骤801：客户端进行采集操作。
客户端可以获取包含人脸的图像，对图像进行采集特征的操作，从而获取图像中包含的特征点。
例如，可以通过手机QQ客户端中调用扫描功能，例如点击“+号”选择“扫一扫”，再选择“QQ-AR”，对准脸部(用户自己或他人)，然后进行扫描。为了描述方便，这里将被扫描的用户称为用户丙。
步骤802：客户端判断是否有足够特征点，如果有则执行步骤803，否则继续执行步骤802。
进行采集操作之后，获取采集到的特征点的数量，判断特征点的数量是否超过特征点数据量阈值，如果超过，表示特征点的数量足够，扫描人脸成功，如果没有超过，表示特征点的数量不足，需要继续采集。
判断是否有足够特征点，可以通过观察屏幕上出现的一个正在扫描的正脸的动画，一般来说，动画结束后表示扫描人脸成功；若不成功，静止扫描1秒后，然后继续执行步骤802。
步骤803：客户端检测本地是否有AR装扮的缓存，若有，执行步骤804；若无，执行步骤805。
对于首次使用AR装扮的用户，客户端本地无相应的AR装扮缓存；此外，对于拍摄的是他人的人脸信息时，一般来说，本地无相应的AR装扮缓存；当然，实际应用中，QQ客户端可以接收后台服务器推送的其他用户(如潜在的好友、可能感兴趣的用户，等等)的AR装扮，那么，一旦用户在线下社交的 过程中认识了潜在的用户并使用QQ客户端扫描，将即时获得AR装扮，无需到后台服务器查询，最大程度减小延迟。
步骤804：客户端显示AR图片或视频。
这里，AR图片或视频为：AR装扮与用户所拍摄的图像的合成图片，AR装扮与用户所拍摄的视频的合成视频。客户端获取到AR装扮之后，将AR装扮与用户所拍摄的图像或视频进行合成，得到AR图片或视频，在AR图片或视频中实现了为用户添加装扮的效果。
步骤805：客户端上传照片到人脸识别服务器。
当在本地库未查找到相应的AR装扮时，需要在人脸识别服务器进行人脸识别，以根据识别后的结果与存储在人脸识别服务器中的图像进行匹配操作。
步骤806：人脸识别服务器识别匹配成功。
若该人脸识别服务器中有匹配的图像，则表示匹配成功，说明该人脸识别服务器中存储了用户丙的图像，用户丙为社交网络中的注册用户。
进入步骤806，有两种情景：情景一：用户丙为使用客户端的用户本人，其未设置过AR装扮；情景二：用户丙为他人。
步骤807：人脸识别服务器获取社交网络账号。
这里，社交网络账号可以是QQ号，也可以是微信号，或其它IM账号。
社交装扮服务器会存储每个社交网络账号对应的个性化装扮，人脸识别服务器识别出注册用户后，获取注册用户的社交网络账号，获取社交网络账号用于社交装扮服务器通过社交网络账号来拉取个性化装扮。
步骤808：人脸识别服务器向社交装扮服务器发送拉取个性化装扮模型的请求。该请求携带获取的社交网络账号。
步骤809：社交装扮服务器拉取个性化装扮模型。
若为情景一：用户丙为使用客户端的用户本人，其未设置过AR装扮。
这里，人脸识别服务器将会从社交装扮服务器获取个性化装扮模型，然后通过社交装扮服务器向客户端推荐相应的个性化装扮模型。
若为情景二：用户丙为他人。
这里，用户丙若是设置了AR装扮，那么，人脸服务器将从社交装扮服务器中获取用户丙设置的AR装扮，然后通过社交装扮服务器向客户端推荐相应的个性化装扮模型。用户丙若是未设置AR装扮，那么，此操作结束。
步骤810：将个性化装扮模型发送至客户端。
步骤811：客户端根据本地ARSDK加载模型。
这里，当用户移动的时候，客户端会利用HDM或手机等设备的AR SDK，对显示的内容和图形进行追踪匹配，使个性化装扮跟随用户的运动而渲染变化，从而形成更好地增强现实的效果。
不难理解，通过上述实施，以本端客户端为QQ客户端为例，可以应用到如下几个典型的场景。
场景一：线上社交-即时视频聊天中，实现AR装扮
本端客户端的用户使用即时通信(包括QQ、微信等)与对端的用户(如亲朋好友)进行视频聊天时，用户在本端客户端中调用摄像头实时拍摄视频或图像，从而获得针对真实对象的视频或图像的参数，为了凸显个性化，活跃聊天气氛，在拍摄视频或图像之前(也可以是在拍摄视频或图像的过程中)为真实对象添加相应的虚拟对象，如个性化的装扮和虚拟背景。对于对端客户端的用户，当该用户使用摄像头实施拍摄视频或照片时，也可以类似本地用户，将拍摄的视频或图片经过装扮，或者直接将拍摄的视频或照片传输到本地用户，由本地用户执行上述的装扮操作。
当然，上述的AR装扮可以替换为用户在社交网络中的其他信息，如个人名片，包括在社交网络的账号、图形码等。
场景二：线上社交-视频传输的过程中实现AR装扮
本地用户使用即时通信(包括QQ、微信等)与对端的用户(如亲朋好友)进行聊天时，用户双方可能会将自己认为拍摄的搞笑、好看的视频或照片发送给对方，例如，本地用户(或对端用户)拍摄了一张吃饭的照片，即时通信客户端将会识别该照片中的真实对象的特征，以便根据识别出来的特征匹配相应的装扮，然后在该照片上添加匹配的装扮后发送给对端用户(或本地用户)。
场景三：线下社交-客户端扫描其他用户
本端客户端如手机QQ的用户，点击客户端上的“扫一扫”选项，再选择“QQ-AR”，对准欲扫描的用户人脸，然后屏幕上会出现一个正在扫描的关于真实对象的动画。当动画结束后，表示扫描成功，即从所采集的图像数据中识别环境中真实对象的特征，从社交网络基于特征查询对应的ID，根据查询到的ID拉取用户预先设置的AR装扮到本端客户端，在扫描的人脸上即时形成装扮的效果。
场景四：线下社交-客户端扫描用户本人
本端客户端如手机QQ的用户，通过“QQ-AR”，使用摄像头对准用户自己的脸部，然后屏幕上会出现一个正在扫描的脸部的动画，然后动画结束后，表示扫描人脸成功。屏幕下方会出现至少一个性化装扮可以选择，用户选择了喜欢的个性化装扮之后，个性化装扮将会应用在屏幕上。其中，个性化装扮可以是虚拟道具、虚拟背景和滤镜等，虚拟道具可以是帽子、眼镜或其它脸部挂件等。当用户点击确定上传个性化装扮后，用户的人脸照片和个性化AR装扮会分别上传到服务器上，并与该用户的QQ号绑定。
综上所述，本申请实施例实现以下有益效果：
1)基于从图像数据中识别特征并查询社交网络的方式，能够对于社交网络的任意场景中的图像数据，都可以快速识别出归属于社交网络的真实对象，在相应的场景中融合社交网络中适配真实对象的增强现实效果。
2)社交网络中针对不同的真实对象的增强现实模型具有多元化的特点，例如根据需要可以AR形式的装扮、社交名片等，从而应用到图像数据的渲染时，实现了不同对象的差异化的显示效果。
3)通过将虚拟装扮与对方人物相结合，为用户提供了一个新的泛社交的话题切入点，有利于线下社交向线上社交的无缝接入。
4)客户端根据情况在本地或者调用服务器的识别服务从图像数据识别特征，有利于减小延迟，实现真实对象和虚拟对象同步显示。
5)根据优先在客户端的宿主设备查询增强现实模型的方式，对于客户端本地预存了增强现实模型的情况，能够实现虚拟对象在客户端的及时显示，避免了网络原因导致真实对象与虚拟对象显示不不同步的问题。
6)根据从图像数据识别特征的完整程度决定是否向社交网络请求增强现实模型，避免社交网络后台的无效计算，有效节约社交网络的计算资源。
以上所述，仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。凡在本申请的精神和范围之内所作的任何修改、等同替换和改进等，均包含在本申请的保护范围之内。

Claims (23)

  1. 一种图像处理方法，其特征在于，应用于图像处理装置中，包括：
    从所获得的图像数据识别出环境中真实对象的特征；
    以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性；
    获得所述社交网络中与所述真实对象适配的增强现实模型；
    根据所获得的图像数据进行渲染，以及，
    根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，形成共同显示的所述真实对象与所述虚拟对象。
  2. 如权利要求1所述的方法，其特征在于，所述从所获得的图像数据识别出环境中真实对象的特征，包括：
    接收所述社交网络中的对端客户端采集环境并传输的图像数据，从所接收的图像数据中识别位于对端客户端环境的真实对象的特征；
    和/或，
    采集环境形成图像数据，从所采集形成的图像数据中识别位于本端客户端环境的真实对象的特征。
  3. 如权利要求2所述的方法，其特征在于，所述从所采集形成的图像数据中识别位于本端客户端环境的真实对象的特征，包括：
    当与所述社交网络中的对端客户端通信时，采集本端客户端环境形成用于传输至所述对端客户端的图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征；
    或者，
    当响应所述本端客户端的采集操作时，采集本端客户端环境形成图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征。
  4. 如权利要求1所述的方法，其特征在于，还包括：
    当获得所述社交网络中与所述真实对象适配的增强现实模型之前，
    判断所识别出真实对象的特征满足社交网络能够识别的条件，所述条件包括以下至少之一：
    当识别出图像特征点时，所识别出的图像特征点的数量超出特征点数据量阈值；
    当识别出生物特征时，所识别出的生物特征的完整程度超出完整程度阈值。
  5. 如权利要求1所述的方法，其特征在于，所述以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性，包括：
    以所述真实对象的特征查询所述社交网络的特征数据库；
    当所述真实对象与所述社交网络的注册用户的特征匹配时，确定所述真实对象具有所述社交网络的注册用户属性；
    当所述真实对象与所述社交网络的被分享对象的特征匹配时，确定所述真实对象具有所述社交网络的被分享对象属性。
  6. 如权利要求1所述的方法，其特征在于，所述获得所述社交网络中与所述真实对象适配的增强现实模型，包括：
    当所述真实对象为所述社交网络中的注册用户时，获得所述注册用户在所述社交网络中预设的虚拟对象，所述虚拟对象包括以下至少之一：虚拟道具、虚拟背景和滤镜；
    当所述真实对象具有所述社交网络的被分享对象的属性时，获得所述社交网络中针对所述被分享对象的虚拟对象，所述虚拟对象包括以下至少之一：
    所述社交网络中针对所述被分享对象的文章；
    所述社交网络中针对所述被分享对象的广告。
  7. 如权利要求1所述的方法，其特征在于，所述从所获得的图像数据识别出环境中真实对象的特征，包括：
    调用服务器的识别服务，从所获得的图像数据中识别出环境中真实对象的特征；
    或者，开启图像识别线程，在所开启的图像识别线程中识别所获得的图像数据，得到环境中真实对象的特征。
  8. 如权利要求1所述的方法，其特征在于，所述根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，包括：
    检测所述真实对象在所述图像数据中的位姿变化；
    在所述真实对象在所输出的图像中的位置，渲染输出所述增强现实模型中与所述位姿变化适配的虚拟对象。
  9. 如权利要求1所述的方法，其特征在于，所述从所述社交网络获得与所述真实对象适配的增强现实模型，包括：
    在本端客户端查询与所述真实对象适配的增强现实模型；
    当未查询到时，从所述社交网络查询得到与所述真实对象适配的增强现实模型。
  10. 一种图像处理装置，其特征在于，包括：
    识别模块，用于从所获得的图像数据识别出环境中真实对象的特征；
    查询模块，用于以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性；
    模型模块，用于获得所述社交网络中与所述真实对象适配的增强现实模型；
    渲染模块，用于根据所获得的图像数据进行渲染，以及，根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，形成共同显示的所述真实对象与所述虚拟对象。
  11. 如权利要求10所述的装置，其特征在于，
    所述识别模块，具体用于：
    接收所述社交网络中的对端客户端采集环境并传输的图像数据，从所接收的图像数据中识别位于对端客户端环境的真实对象的特征；
    和/或，
    采集环境形成图像数据，从所采集形成的图像数据中识别位于本端客户端环境的真实对象的特征。
  12. 如权利要求11所述的装置，其特征在于，
    所述识别模块，具体用于：
    当与所述社交网络中的对端客户端通信时，采集本端客户端环境形成用于传输至所述对端客户端的图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征；
    或者，
    当响应所述本端客户端的采集操作时，采集本端客户端环境形成图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征。
  13. 如权利要求10所述的装置，其特征在于，
    所述识别模块，还用于当获得所述社交网络中与所述真实对象适配的增强现实模型之前，判断所识别出真实对象的特征满足社交网络能够识别的条件，所述条件包括以下至少之一：
    当识别出图像特征点时，所识别出的图像特征点的数量超出特征点数据量阈值；
    当识别出生物特征时，所识别出的生物特征的完整程度超出完整程度阈值。
  14. 一种图像处理装置，其特征在于，包括：
    存储器，用于存储可执行程序；
    处理器，用于执行所述存储器中存储的可执行程序时，实现如下操作：
    从所获得的图像数据识别出环境中真实对象的特征；
    以所述真实对象的特征查询社交网络，确定所述真实对象具有所述社交网络的属性；
    获得所述社交网络中与所述真实对象适配的增强现实模型；
    根据所获得的图像数据进行渲染，以及，
    根据所述真实对象在所渲染形成的图像中的位置，对所述增强现实模型中的虚拟对象进行渲染，形成共同显示的所述真实对象与所述虚拟对象。
  15. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    接收所述社交网络中的对端客户端采集环境并传输的图像数据，从所接收的图像数据中识别位于对端客户端环境的真实对象的特征；
    和/或，
    采集环境形成图像数据，从所采集形成的图像数据中识别位于本端客户端环境的真实对象的特征。
  16. 根据权利要求15所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    当与所述社交网络中的对端客户端通信时，采集本端客户端环境形成用于传输至所述对端客户端的图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征；
    或者，
    当响应所述本端客户端的采集操作时，采集本端客户端环境形成图像数据，从所采集的图像数据中识别本端客户端环境中真实对象的特征。
  17. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    当获得所述社交网络中与所述真实对象适配的增强现实模型之前，
    判断所识别出真实对象的特征满足社交网络能够识别的条件，所述条件包括以下至少之一：
    当识别出图像特征点时，所识别出的图像特征点的数量超出特征点数据量阈值；
    当识别出生物特征时，所识别出的生物特征的完整程度超出完整程度阈值。
  18. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    以所述真实对象的特征查询所述社交网络的特征数据库；
    当所述真实对象与所述社交网络的注册用户的特征匹配时，确定所述真实对象具有所述社交网络的注册用户属性；
    当所述真实对象与所述社交网络的被分享对象的特征匹配时，确定所述真实对象具有所述社交网络的被分享对象属性。
  19. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    当所述真实对象为所述社交网络中的注册用户时，获得所述注册用户在所述社交网络中预设的虚拟对象，所述虚拟对象包括以下至少之一：虚拟道具、虚拟背景和滤镜；
    当所述真实对象具有所述社交网络的被分享对象的属性时，获得所述社交网络中针对所述被分享对象的虚拟对象，所述虚拟对象包括以下至少之一：
    所述社交网络中针对所述被分享对象的文章；
    所述社交网络中针对所述被分享对象的广告。
  20. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    调用服务器的识别服务，从所获得的图像数据中识别出环境中真实对象的特征；
    或者，开启图像识别线程，在所开启的图像识别线程中识别所获得的图像数据，得到环境中真实对象的特征。
  21. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行所述可执行程序时，实现如下操作：
    检测所述真实对象在所述图像数据中的位姿变化；
    在所述真实对象在所输出的图像中的位置，渲染输出所述增强现实模型中与所述位姿变化适配的虚拟对象。
  22. 根据权利要求14所述的装置，其特征在于，所述处理器，还用于执行 所述可执行程序时，实现如下操作：
    在本端客户端查询与所述真实对象适配的增强现实模型；
    当未查询到时，从所述社交网络查询得到与所述真实对象适配的增强现实模型。
  23. 一种存储介质，其特征在于，存储有可执行程序，所述可执行程序被处理器执行时，实现如权利要求1至9任一项所述的图像处理方法。
PCT/CN2018/097860 2017-08-04 2018-08-01 一种图像处理方法、装置及存储介质 WO2019024853A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2020505230A JP7098120B2 (ja) 2017-08-04 2018-08-01 画像処理方法、装置及記憶媒体
KR1020207004027A KR102292537B1 (ko) 2017-08-04 2018-08-01 이미지 처리 방법 및 장치, 및 저장 매체
US16/780,891 US11182615B2 (en) 2017-08-04 2020-02-03 Method and apparatus, and storage medium for image data processing on real object and virtual object
US17/478,860 US20220004765A1 (en) 2017-08-04 2021-09-17 Image processing method and apparatus, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710661746.0 2017-08-04
CN201710661746.0A CN108305317B (zh) 2017-08-04 2017-08-04 一种图像处理方法、装置及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/780,891 Continuation US11182615B2 (en) 2017-08-04 2020-02-03 Method and apparatus, and storage medium for image data processing on real object and virtual object

Publications (1)

Publication Number Publication Date
WO2019024853A1 true WO2019024853A1 (zh) 2019-02-07

Family

ID=62872576

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/097860 WO2019024853A1 (zh) 2017-08-04 2018-08-01 一种图像处理方法、装置及存储介质

Country Status (6)

Country Link
US (2) US11182615B2 (zh)
JP (1) JP7098120B2 (zh)
KR (1) KR102292537B1 (zh)
CN (1) CN108305317B (zh)
TW (1) TWI708152B (zh)
WO (1) WO2019024853A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112672185A (zh) * 2020-12-18 2021-04-16 脸萌有限公司 基于增强现实的显示方法、装置、设备及存储介质
WO2021079829A1 (ja) * 2019-10-25 2021-04-29 Ｎｅｃソリューションイノベータ株式会社 表示装置、イベント支援システム、表示方法、及びイベント支援システムの生産方法

Families Citing this family (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI439960B (zh) 2010-04-07 2014-06-01 Apple Inc 虛擬使用者編輯環境
KR101988319B1 (ko) * 2013-09-09 2019-06-12 엘지전자 주식회사 이동 단말기 및 이의 제어 방법
US9912860B2 (en) 2016-06-12 2018-03-06 Apple Inc. User interface for camera effects
DK180859B1 (en) 2017-06-04 2022-05-23 Apple Inc USER INTERFACE CAMERA EFFECTS
CN108305317B (zh) 2017-08-04 2020-03-17 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质
DK180078B1 (en) 2018-05-07 2020-03-31 Apple Inc. USER INTERFACE FOR AVATAR CREATION
US11722764B2 (en) 2018-05-07 2023-08-08 Apple Inc. Creative camera
US10375313B1 (en) * 2018-05-07 2019-08-06 Apple Inc. Creative camera
CN109165571B (zh) 2018-08-03 2020-04-24 北京字节跳动网络技术有限公司 用于插入图像的方法和装置
CN109040824B (zh) * 2018-08-28 2020-07-28 百度在线网络技术（北京）有限公司 视频处理方法、装置、电子设备和可读存储介质
US10573057B1 (en) * 2018-09-05 2020-02-25 Citrix Systems, Inc. Two-part context-based rendering solution for high-fidelity augmented reality in virtualized environment
DK201870623A1 (en) 2018-09-11 2020-04-15 Apple Inc. USER INTERFACES FOR SIMULATED DEPTH EFFECTS
WO2020056689A1 (zh) * 2018-09-20 2020-03-26 太平洋未来科技（深圳）有限公司 一种ar成像方法、装置及电子设备
US10674072B1 (en) 2019-05-06 2020-06-02 Apple Inc. User interfaces for capturing and managing visual media
US11770601B2 (en) 2019-05-06 2023-09-26 Apple Inc. User interfaces for capturing and managing visual media
US11087430B2 (en) * 2018-09-28 2021-08-10 Apple Inc. Customizable render pipelines using render graphs
US11128792B2 (en) 2018-09-28 2021-09-21 Apple Inc. Capturing and displaying images with multiple focal planes
US11321857B2 (en) 2018-09-28 2022-05-03 Apple Inc. Displaying and editing images with depth information
CN109933788B (zh) * 2019-02-14 2023-05-23 北京百度网讯科技有限公司 类型确定方法、装置、设备和介质
US11706521B2 (en) 2019-05-06 2023-07-18 Apple Inc. User interfaces for capturing and managing visual media
CN112102498A (zh) * 2019-06-18 2020-12-18 明日基金知识产权控股有限公司 用于将应用虚拟地附接到动态对象并实现与动态对象的交互的系统和方法
JP7221156B2 (ja) 2019-06-28 2023-02-13 富士フイルム株式会社 画像処理システム、画像処理方法及びプログラム
CN110868635B (zh) * 2019-12-04 2021-01-12 深圳追一科技有限公司 视频处理方法、装置、电子设备及存储介质
US11921998B2 (en) 2020-05-11 2024-03-05 Apple Inc. Editing features of an avatar
DK202070624A1 (en) 2020-05-11 2022-01-04 Apple Inc User interfaces related to time
CN111652107B (zh) * 2020-05-28 2024-05-21 北京市商汤科技开发有限公司 对象计数方法及装置、电子设备和存储介质
US11054973B1 (en) 2020-06-01 2021-07-06 Apple Inc. User interfaces for managing media
CN111899350A (zh) * 2020-07-31 2020-11-06 北京市商汤科技开发有限公司 增强现实ar图像的呈现方法及装置、电子设备、存储介质
US11212449B1 (en) 2020-09-25 2021-12-28 Apple Inc. User interfaces for media capture and management
CN112348889B (zh) * 2020-10-23 2024-06-07 浙江商汤科技开发有限公司 视觉定位方法及相关装置、设备
US11449155B2 (en) * 2021-01-11 2022-09-20 Htc Corporation Control method of immersive system
KR102472115B1 (ko) * 2021-02-04 2022-11-29 (주)스마트큐브 다자간 온라인 업무 협업을 위한 증강현실 기반의 화상회의를 제공하기 위한 장치 및 이를 위한 방법
JP7427786B2 (ja) * 2021-02-09 2024-02-05 北京字跳▲網▼絡技▲術▼有限公司 拡張現実に基づく表示方法、機器、記憶媒体及びプログラム製品
US11539876B2 (en) 2021-04-30 2022-12-27 Apple Inc. User interfaces for altering visual media
US11778339B2 (en) 2021-04-30 2023-10-03 Apple Inc. User interfaces for altering visual media
TWI768913B (zh) 2021-05-20 2022-06-21 國立中正大學 眼睛中心定位方法及其定位系統
US11776190B2 (en) 2021-06-04 2023-10-03 Apple Inc. Techniques for managing an avatar on a lock screen
CN113479105A (zh) * 2021-07-20 2021-10-08 钟求明 一种基于自动驾驶车辆的智能充电方法及智能充电站
TWI792693B (zh) * 2021-11-18 2023-02-11 瑞昱半導體股份有限公司 用於進行人物重辨識的方法與裝置
CN116645525B (zh) * 2023-07-27 2023-10-27 深圳市豆悦网络科技有限公司 一种游戏图像识别方法及处理系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102332095A (zh) * 2011-10-28 2012-01-25 中国科学院计算技术研究所 一种人脸运动跟踪方法和系统以及一种增强现实方法
CN106295504A (zh) * 2016-07-26 2017-01-04 车广为 人脸识别基础上的增强显示方法
GB2544885A (en) * 2015-10-30 2017-05-31 2Mee Ltd Communication system and method
CN107004290A (zh) * 2015-01-06 2017-08-01 索尼公司 效果生成装置、效果生成方法以及程序
CN108305317A (zh) * 2017-08-04 2018-07-20 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质

Family Cites Families (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4367424B2 (ja) * 2006-02-21 2009-11-18 沖電気工業株式会社 個人識別装置，個人識別方法
US10872322B2 (en) * 2008-03-21 2020-12-22 Dressbot, Inc. System and method for collaborative shopping, business and entertainment
JP5237394B2 (ja) * 2009-01-30 2013-07-17 富士通フロンテック株式会社 認証装置、撮像装置、認証方法および認証プログラム
US20100208033A1 (en) * 2009-02-13 2010-08-19 Microsoft Corporation Personal Media Landscapes in Mixed Reality
US20110052012A1 (en) * 2009-03-31 2011-03-03 Myspace Inc. Security and Monetization Through Facial Recognition in Social Networking Websites
WO2012139270A1 (en) * 2011-04-11 2012-10-18 Intel Corporation Face recognition control and social networking
KR101189043B1 (ko) * 2011-04-27 2012-10-08 강준규 영상통화 서비스 및 그 제공방법, 이를 위한 영상통화서비스 제공서버 및 제공단말기
US8332424B2 (en) * 2011-05-13 2012-12-11 Google Inc. Method and apparatus for enabling virtual tags
CN102916986A (zh) * 2011-08-01 2013-02-06 环达电脑(上海)有限公司 人脸辨识的搜寻系统及其方法
EP2759127A4 (en) 2011-09-23 2014-10-15 Tangome Inc REINFORCEMENT OF A VIDEO CONFERENCE
KR20130063876A (ko) * 2011-12-07 2013-06-17 (주)엘에이치에스지 클라우드 컴퓨팅을 이용한 증강 현실 시스템 및 생성 방법
US9258626B2 (en) * 2012-01-20 2016-02-09 Geun Sik Jo Annotating an object in a video with virtual information on a mobile terminal
CN103368816A (zh) * 2012-03-29 2013-10-23 深圳市腾讯计算机系统有限公司 基于虚拟人物形象的即时通讯方法及系统
WO2014031899A1 (en) * 2012-08-22 2014-02-27 Goldrun Corporation Augmented reality virtual content platform apparatuses, methods and systems
US10027727B1 (en) * 2012-11-21 2018-07-17 Ozog Media, LLC Facial recognition device, apparatus, and method
JP5991536B2 (ja) * 2013-02-01 2016-09-14 パナソニックＩｐマネジメント株式会社 メイクアップ支援装置、メイクアップ支援方法、およびメイクアップ支援プログラム
CN103544724A (zh) * 2013-05-27 2014-01-29 华夏动漫集团有限公司 一种利用增强现实与卡片识别技术在移动智能终端实现虚拟动漫角色的系统及方法
WO2015027196A1 (en) * 2013-08-22 2015-02-26 Bespoke, Inc. Method and system to create custom products
JP2015095031A (ja) * 2013-11-11 2015-05-18 アルパイン株式会社 情報処理装置およびコメント投稿方法
CN103679204A (zh) * 2013-12-23 2014-03-26 上海安琪艾可网络科技有限公司 基于智能移动设备平台的图像识别与创作应用系统及方法
US10901765B2 (en) * 2014-01-22 2021-01-26 Nike, Inc. Systems and methods of socially-driven product offerings
KR20150126289A (ko) * 2014-05-02 2015-11-11 한국전자통신연구원 증강현실 기반 소셜 네트워크 서비스 정보를 제공하는 내비게이션 장치와 메타데이터 처리장치 및 그 방법
US20160042557A1 (en) * 2014-08-08 2016-02-11 Asustek Computer Inc. Method of applying virtual makeup, virtual makeup electronic system, and electronic device having virtual makeup electronic system
US9798143B2 (en) * 2014-08-11 2017-10-24 Seiko Epson Corporation Head mounted display, information system, control method for head mounted display, and computer program
DE102014226625A1 (de) * 2014-10-31 2016-05-04 Cortado Ag Verfahren zur Übertragung von Druckdaten, Server und mobiles Endgerät
CN104504423B (zh) * 2014-12-26 2018-02-13 山东泰宝防伪制品有限公司 Ar增强现实立体防伪系统及其方法
US9754355B2 (en) * 2015-01-09 2017-09-05 Snap Inc. Object recognition based photo filters
EP3272078B1 (en) * 2015-03-18 2022-01-19 Snap Inc. Geo-fence authorization provisioning
CN104901873A (zh) * 2015-06-29 2015-09-09 曾劲柏 一种基于场景和动作的网络社交系统
JP6589573B2 (ja) * 2015-11-06 2019-10-16 ブラザー工業株式会社 画像出力装置
US11237629B2 (en) * 2016-02-06 2022-02-01 Maximilian Ralph Peter von und zu Liechtenstein Social networking technique for augmented reality
CN106200917B (zh) * 2016-06-28 2019-08-30 Oppo广东移动通信有限公司 一种增强现实的内容显示方法、装置及移动终端
CN106200918B (zh) * 2016-06-28 2019-10-01 Oppo广东移动通信有限公司 一种基于ar的信息显示方法、装置和移动终端
US20180047200A1 (en) * 2016-08-11 2018-02-15 Jibjab Media Inc. Combining user images and computer-generated illustrations to produce personalized animated digital avatars
CN109952610B (zh) * 2016-11-07 2021-01-08 斯纳普公司 图像修改器的选择性识别和排序
JP6753276B2 (ja) * 2016-11-11 2020-09-09 ソニー株式会社 情報処理装置、および情報処理方法、並びにプログラム
US11635872B2 (en) * 2016-11-22 2023-04-25 Snap Inc. Smart carousel of image modifiers
CN106780757B (zh) * 2016-12-02 2020-05-12 西北大学 一种增强现实的方法
US10636175B2 (en) * 2016-12-22 2020-04-28 Facebook, Inc. Dynamic mask application
CN106846237A (zh) * 2017-02-28 2017-06-13 山西辰涵影视文化传媒有限公司 一种基于Unity3D的增强实现方法
GB2560031B (en) * 2017-02-28 2020-05-27 PQ Solutions Ltd Binding data to a person's identity
CN107820591A (zh) * 2017-06-12 2018-03-20 美的集团股份有限公司 控制方法、控制器、智能镜子和计算机可读存储介质
US9980100B1 (en) * 2017-08-31 2018-05-22 Snap Inc. Device location based on machine learning classifications
KR101968723B1 (ko) * 2017-10-18 2019-04-12 네이버 주식회사 카메라 이펙트를 제공하는 방법 및 시스템
US11488359B2 (en) * 2019-08-28 2022-11-01 Snap Inc. Providing 3D data for messages in a messaging system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102332095A (zh) * 2011-10-28 2012-01-25 中国科学院计算技术研究所 一种人脸运动跟踪方法和系统以及一种增强现实方法
CN107004290A (zh) * 2015-01-06 2017-08-01 索尼公司 效果生成装置、效果生成方法以及程序
GB2544885A (en) * 2015-10-30 2017-05-31 2Mee Ltd Communication system and method
CN106295504A (zh) * 2016-07-26 2017-01-04 车广为 人脸识别基础上的增强显示方法
CN108305317A (zh) * 2017-08-04 2018-07-20 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021079829A1 (ja) * 2019-10-25 2021-04-29 Ｎｅｃソリューションイノベータ株式会社 表示装置、イベント支援システム、表示方法、及びイベント支援システムの生産方法
CN112672185A (zh) * 2020-12-18 2021-04-16 脸萌有限公司 基于增强现实的显示方法、装置、设备及存储介质
CN112672185B (zh) * 2020-12-18 2023-07-07 脸萌有限公司 基于增强现实的显示方法、装置、设备及存储介质

Also Published As

Publication number Publication date
KR20200020960A (ko) 2020-02-26
US11182615B2 (en) 2021-11-23
TWI708152B (zh) 2020-10-21
CN108305317B (zh) 2020-03-17
JP2020529084A (ja) 2020-10-01
KR102292537B1 (ko) 2021-08-20
US20220004765A1 (en) 2022-01-06
CN108305317A (zh) 2018-07-20
TW201911082A (zh) 2019-03-16
JP7098120B2 (ja) 2022-07-11
US20200285851A1 (en) 2020-09-10

Similar Documents

Publication Publication Date Title
TWI708152B (zh) 圖像處理方法、裝置及儲存介質
US10789453B2 (en) Face reenactment
US11670059B2 (en) Controlling interactive fashion based on body gestures
US11734866B2 (en) Controlling interactive fashion based on voice
US11816926B2 (en) Interactive augmented reality content including facial synthesis
WO2023070021A1 (en) Mirror-based augmented reality experience
US11900506B2 (en) Controlling interactive fashion based on facial expressions
EP4315256A1 (en) Facial synthesis in augmented reality content for third party applications
US20240096040A1 (en) Real-time upper-body garment exchange
CN114779948B (zh) 基于面部识别的动画人物即时交互控制方法、装置及设备
WO2023121897A1 (en) Real-time garment exchange
WO2023121896A1 (en) Real-time motion and appearance transfer
US20240161242A1 (en) Real-time try-on using body landmarks
US20230316665A1 (en) Surface normals for pixel-aligned object
WO2023192426A1 (en) Surface normals for pixel-aligned object
EP4396781A1 (en) Controlling interactive fashion based on body gestures
CN117041670A (zh) 图像处理方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18841835

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020505230

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20207004027

Country of ref document: KR

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 18841835

Country of ref document: EP

Kind code of ref document: A1