CN110503724A

CN110503724A - A kind of AR expression resource construction management system and method based on human face characteristic point

Info

Publication number: CN110503724A
Application number: CN201910757570.8A
Authority: CN
Inventors: 华晨; 张云龙
Original assignee: Beijing Cat Eye Vision Technology Co Ltd
Current assignee: Beijing Cat Eye Vision Technology Co Ltd
Priority date: 2019-08-19
Filing date: 2019-08-19
Publication date: 2019-11-26

Abstract

The AR expression resource construction management method based on human face characteristic point that the present invention provides a kind of, includes the following steps: S11, AR expression project management；S12, AR expression special effective function management；S13, AR expression material function management；S14, AR expression parameter setting management；S15, AR expression layout check management；S16, AR expression generation management.Mobile terminal H5 face paster can be completed by the editor to fine arts resource in the present invention, substantially increase the independence and entertainment of the application of AR expression, the type of special efficacy and some parameters can give user oneself editor, embody very strong adaptability and creativeness, enhance user experience.

Description

A kind of AR expression resource construction management system and method based on human face characteristic point

Technical field

The present invention relates to AR (Augmented Reality, augmented reality) and expression scene process fields, specifically, The present invention relates to a kind of AR expression resource configuration system and resource allocation method based on human face characteristic point.

Background technique

Configuration management with virtual reality and augmented reality in the continuous application of every profession and trade, for AR expression resource Ratio shared by the repetitive operation that this technical threshold is lower, time-consuming is gradually increased, and increasingly causes people's Pay attention to.

Currently, portion's majority mobile phone photograph big absolutely or be related to the application of Image Acquisition, both provide paster, head portrait decoration, The functions such as decoration simulation can allow user to decorate in taking pictures using some display effects paster outstanding.Camera from Clap APP in, so that it may provide a variety of pasters for user select, such as to photographer plus cat ear, plus red lip, In addition various face colors, plus various ornaments, put on a hat.This is improved take pictures in interest, in traditional figure As increasing new selection on the basis of filtration treatment U.S. face.But the processing of this paster, in entire industry, there are no shapes At unified standard, developer is also rested on for the use of resource in most of application and developments and imports and configure by hand rank Section, all paster production and configuration work will be by developers and supplier's employee's completion, such as the one of the exploitation of certain company Money social activity chat software just supports AR expression, but it is only capable of providing a limited number of pasters and text type selection, user Independence it is relatively low.A chat software of certain company exploitation for another example, can also support AR expression to show, but it exists The problem of be also that the expression packet paster packet type that can provide is limited, be difficult to meet the individual requirement of user.It is such soft at present The operating mode of part, usually big internet platform company exploitation main program, such as input method software, social software etc., In It is wherein embedded in AR expression paster program, is reserved with AR expression program access entrance, expression developer (including third party developer) A series of Miscellaneous Documents (including obj ect file, configuration file, Parameter File etc.) are made according to the format of platform corporate policy, Packing is uploaded to platform company, is conducted into main program by platform company and is supplied to user's selection use.But with personalization Demand gradually increases, and it is low that this development scheme encounters working efficiency, and tool is high using threshold, and difficulty of test is big, independence Difference, normalization lack the problems such as supervision.

In addition, present AR expression, is typically only capable to the plane expression of addition 2D, is equivalent to outside face and has tiled one Expression, expression itself and paster be not just lively, and with the rotation of head part, paster and expression meeting serious distortion cannot be embodied The characteristics of AR expression true reappearance.

In addition, the position of paster be also cannot from main regulation, be usually at present according to certain AI algorithm, allow paster from The dynamic a certain partial movement for following face, such as sunglasses will follow always eyes mobile, cover eyes always.There is also one for this A problem is exactly that user can not be participated in completely in the adjustment of paster position, and all work is all complete according to algorithm by APP oneself At, even if because the position of certain reason pasters is not highly desirable, but user can not also be adjusted it.

Summary of the invention

The present invention provides a kind of AR expression resource construction management method based on human face characteristic point, includes the following steps:

S11, AR expression project management: then creation AR expression project creates item objects and project in the memory unit After importing the initial parameter of resource in item data structure, item data structure and resource initial parameter are believed for data structure Breath is saved in storage unit；

S12, AR expression special effective function management: the initial animation special effect parameters for generating animation effect object are obtained, according to institute It states initial animation special effect parameters and establishes animation effect data structure, and according to the animation effect data structure and initial parameter Value generates initial animation special efficacy object, and animation effect data structure and initial parameter value are saved in storage unit, realizes The storage and display of initial animation special efficacy object；

S13, AR expression material function management: the static material parameter of primary for generating static material object is obtained, according to institute It states static material parameter and establishes static material data structure, and is raw according to the static material data structure and initial parameter value At initial static material object, static material data structure and initial parameter value are saved in storage unit, realized initial The storage and display of static material object；

S14, AR expression parameter setting management: according to the animation effect data knot in input instruction setting steps S12 The parameter value of static material in the static material data structure in structure in the parameter value of animation effect and/or step S13, will The parameter value and/or static material data structure and the parameter value of setting of animation effect data structure and setting are saved in and deposit In storage unit, and final animation effect object and/or static element are generated according to the parameter value of the data structure and setting Material object；

S15, AR expression layout check management: final animation effect object and/or final static element in obtaining step S14 The data structure and parameter value of material object, and be arranged final animation effect object and/or final static material object relative to The layout attributes parameter of default face grid base map, layout attributes parameter is saved in storage unit, is realized under a bureau with cloth Property final animation effect object and/or final static material object storage and display；

S16, AR expression generation management: according to the final animation effect object with layout attributes and/or final quiet State material object generates AR expression and executes code, runs the AR expression and executes code, generates AR expression.

Further, layout checks that management can carry out under two kinds of display modes of 2D and 3D in step S15, and supports 2D With 3D pattern switching, 2D material can show and edit in 3 d mode, but 3D material is divided into two kinds in 2 d mode, Yi Zhongwei It shows and edits according to 2D display mode, another kind is that 3D material cannot show and edit in 2 d mode.

Further, layout checks that management when 2D pattern switching is to 3D mode, is understood in 3D mode in step S15 One special efficacy object of lower creation, the x and y-axis of special efficacy object are updated using x, the y location under 2D mode, and z-axis first defaults one Then special efficacy is rendered the position fixed in x, y and z axes by a initial value, when editing in 3 d mode to 2D special efficacy object Afterwards, then after switching back into 2D mode, it only updates the x of special efficacy object and y-axis position, z-axis position is not processed；When again by 2D mode After being switched to 3D mode, x and y-axis position are updated, z-axis position passes through this mistake using last edited position Journey, come complete the switching between 2D and 3D mode with it is synchronous.

Further, step S11 to S16 is manipulated by human-computer interaction interface and is realized, human-computer interaction interface includes: system master Forms area, QML menu area, tool forms area and the part of operating space four, it is left that tool forms area can rest in system Main form Right two sides, operating space can be located at centre, can also be based on each area of habit reasonable Arrangement human-computer interaction interface of operator Rational position；

Main form area can be arranged according to system selects suitable component to realize at the end PC or mobile terminal, when system is arranged At the end PC, the realization of ApplicationWindow component is can be used in Main form area, creates window abundant with fast and easy；

QML menu area is realized using the MenuBar mode in QML or the ListView mode used；

It is realized using Column Layout in tool forms area；

Intermediary operation area is mainly used for special display effect integral layout and realizes the layout editor of special efficacy object, uses Rectangle component is realized, be can be used as container and is placed all QML components.

User edits 2D and 3D both of which by intermediary operation area, and both of which is all using Rectangle Component is realized, the difference is that 3D mode is that nesting Scene3D component is realized in Rectangle.It intermediary operation area can basis The difference of special efficacy type and checked and edited using corresponding mode automatically, 3D special efficacy object is looked into using 3D mode automatically It sees, 2D special efficacy object default is checked using 2D mode, can be switched to 3D mode by switching push button and be checked；But 3D material exists It is checked under 2D mode and is divided into two kinds, for one kind to show and editing according to 2D display mode, another kind is that 3D material cannot be in 2D mould It shows and edits under formula.

Further, resource construction management method realizes each management process by Qt Quick in AR scene, is providing Under the 2D mode of source configuring management method, 2D model selection Rectangle can according to need and be designed to respectively as basic module Kind different component and a variety of different Style, while Rectangle is also used as container assemblies, can be embedded in inside it any Other kinds of component；Under the 3D mode of resources configuration management method, it can be realized while realizing the rendering of 3D model Superposition of the 2D pel on 3D model is loaded 3D model using Qt 3D component and carries out 3D rendering, also interior in Qt 3D component A variety of solution to model parsers are set, convenient for importing the model of multiple format.

Further, it is deposited in order to compatible 2D and 3D resource management, face coordinate architecture simultaneously using three-dimensional structure Storage, i.e., stored in a manner of x, y, z axis, and the public face the most universal collected by big data is as tracking sample, by mouth The central point of straight line line between center and posterior neck only tracks the coordinate system as tracking origin, i.e. coordinate origin Origin, and the tracking sample that is used as of public face the most universal collected based on big data constructs the reference of face key position Coordinate, and by all animation effect objects that will be arranged and static material object all correspondences in face coordinate system with coordinate It is origin anchoring, so that the object described in when showing after AR expression generation can follow the movement of face and move, with people The relative position of face will not change.

The AR expression resource construction management system based on human face characteristic point that the present invention also provides a kind of has and realizes AR expression Multiple functional modules of configuration management function, comprising: project management module 31, layout management module 32, parameter management module 33, Special efficacy management module 34, material management module 35, outgoing management module 36, storage unit, human-computer interaction interface, in which:

Then project management module 31 creates item data structure, In for creating AR expression project in the memory unit After importing the initial parameter of resource in item data structure, by item data structure and resource initial parameter information preservation to storing In unit；

Special effective function module 32, for obtaining the initial animation special effect parameters for generating animation effect object, according to described first Initiating draws special effect parameters and establishes animation effect data structure, and raw according to the animation effect data structure and initial parameter value At initial animation special efficacy object, animation effect data structure and initial parameter value are saved in storage unit, realized initial The storage and display of animation effect object；

Material functional module 33, for obtaining the static material parameter of primary for generating static material object, according to described quiet State material parameter establishes static material data structure, and is generated just according to the static material data structure and initial parameter value Begin static material object, and static material data structure and initial parameter value are saved in storage unit, realize initial static The storage and display of material object；

Parameter configuration function module 34, for according to the animation effect in input instruction modification special effective function module 32 Static element in the static material data structure in data structure in the parameter value of animation effect and/or material functional module 33 The parameter value of material, by animation effect data structure and modified parameter value and/or static material data structure and modification Parameter value afterwards is saved in storage unit, and generates final animation according to the data structure and modified parameter value Special efficacy object and/or static material object；

It is laid out look facility module 35, final animation effect object in the configuration feature module 34 that gets parms and/or most The data structure and parameter value of static state material object eventually, and final animation effect object and/or final static material pair are set As the layout attributes parameter relative to default face grid base map, layout attributes parameter is saved in storage unit, realizes band There are the storage and display of the final animation effect object and/or final static material object of layout attributes；

Expression generation management module 36: according to the final animation effect object with layout attributes and/or final quiet State material object generates AR expression and executes code, runs the AR expression and executes code, generates AR expression.

Storage unit, by project management module 31, special effective function module 32, material database functional module 33, parameter configuration Functional module 34, layout look facility module 35 treated object data structure and parameter value are stored so as to subsequent tune With.

It further, further include human-computer interaction interface, human-computer interaction interface includes: system Main form area, QML menu area, Tool forms area and the part of operating space four, tool forms area can rest at left and right sides of system Main form, and operating space can be with It, can also be based on the rational position in each area of habit reasonable Arrangement human-computer interaction interface of operator positioned at centre；

It is realized using Column Layout in tool forms area；

Further, which generates AR expression by method described in any one of claim 1 to 5.

The AR expression resource construction system based on human face characteristic point that the present invention also provides a kind of, which includes that right such as is wanted Ask any one of 6 to 8 described in any item AR expression resource construction management systems and application client end system, claim 6 to 8 The AR expression resource construction management system and application client end system is as the AR expression resource structure based on human face characteristic point The subsystem of system is built, synergistic effect realizes AR expression true reappearance；

The AR expression resource construction management system is as the AR expression resource construction system based on human face characteristic point One of subsystem is set to the end PC；

The application client end system, as one of them of the AR expression resource construction system based on human face characteristic point Subsystem is set to mobile terminal, and powerful calculating ability, material obtaining ability based on the end PC set resource construction management system It sets at the end PC, the matched downloading by completing resource packet of application client end system, decompression and operation execute code and show AR expression.

The beneficial effects of the present invention are:

Mobile terminal H5 face paster, the correlations such as face special efficacy binding can be completed by the editor to fine arts resource in the present invention The function of application greatly reduces the development difficulty of application, and non-technical personnel quickly save by upper hand, raising whole work efficiency The shortcomings that in the past for expression it is necessary to develop a whole set of multiple mating corresponding document, the function of writing associated documents Be incorporated into expression edition program, user can participate directly in the editor of expression, the type of expression be no longer it is fixed, But can be selected oneself to make by user oneself, substantially increase the independence and entertainment of the application of AR expression, the kind of special efficacy Class and some parameters can give user oneself editor, embody very strong adaptability and creativeness, not have to again be AI algorithm It is not smart enough and have no way out, enhance user experience.

Detailed description of the invention

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this The some embodiments recorded in application, for those of ordinary skill in the art, in the premise of not making the creative labor property Under, it is also possible to obtain other drawings based on these drawings.

Fig. 1 is the resources configuration management operation flow object structure diagram of one embodiment；

Fig. 2 is the resources configuration management system process flow diagram of one embodiment；

Fig. 3 is the functional structure chart of the resources configuration management system and method for one embodiment；

Fig. 4 is the layout management flow chart in the resources configuration management system and method for one embodiment；

Fig. 5 is the face coordinate system schematic diagram of the resources configuration management system and method for one embodiment；

Fig. 6 is the UI interfacing relationship schematic diagram of the resources configuration management system and method for one embodiment.

Specific embodiment

In order to make those skilled in the art better understand the technical solutions in the application, below in conjunction with the application reality The attached drawing in example is applied, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described implementation Example is merely a part but not all of the embodiments of the present application.Based on the embodiment in the application, this field is common The application protection all should belong in technical staff's every other embodiment obtained without creative efforts Range.

The embodiment of the present invention provides a kind of AR expression resource construction management method based on human face characteristic point, working service Process is as shown in Figure 1, there are four objects altogether: i.e. project, resource, editor, parameter, wherein taking project as leading, resource, volume It collects and parameter all depends in AR expression item objects, such as the project of one specific AR expression of building, pass through foundation Item objects add resource, the parameter of editor and setting in AR expression item objects, to reach the function of AR expression building management Energy.

Operation flow is the insertion of module shape in AR expression resource construction management method based on human face characteristic point, can be according to reality Oneself suitable system of border demand assembling, system process flow diagram of the invention import money as shown in Fig. 2, after creation project Source, by generating AR expression after the completion of editor's adjustment, parameter setting, then packing export, which synchronizes, checks AR expression.

Present invention preferably employs the multi-layer framework of C/S model, which can pass through the end PC It realizes, can also be realized by mobile terminals such as mobile phone, PAD, it can also be by the end PC and mobile terminal cooperative achievement, in view of the end PC Powerful calculating ability, material obtaining ability, therefore preferably realized by the end PC, the functional structure of the method for the present invention such as Fig. 3 institute Show.

In conjunction with Fig. 2, Fig. 3, the AR expression resource construction management method of the invention based on human face characteristic point, specifically include as Lower step:

Such as the one existing AR table worn glasses of the AR expression task items being branded as or selection of creation one newly Feelings task items, item objects are face, create the data structure of face, and the various initial of face is imported in data structure Parameter, so as to materials such as subsequent addition cap, glasses, and the animations such as be branded as, wear glasses.

Wherein, animation effect object includes three-dimensional or two-dimensional ornament, picture, artistic font etc., animation effect data Structure can load the animation effect parameter for generating animation effect object, animation effect data structure and animation effect parameter Codetermine the dynamic operation of animation effect object, for example the flashing of lightning, the flowing of flowing water, pyrotechnics are bursted forth, it is initial special Effect parameter value is preset parameter value, and preset dynamic operation can allow user to see the operating status of animation effect object, and User can obtain the operating status of oneself preference, such as adjust direction of flashing, dodge by modifying to initial parameter value Gleaming degree, flicker frequency.

The step preferably realizes that ListView component can show data with tabular form by ListView component, It cascade can also be carried out to data shows with tree-like, new special efficacy object can be added by adding function by special efficacy object, can also To be renamed, be deleted to special efficacy object, etc. operation.

Special effective function management and material function management are mutually juxtaposed, and can be run parallel, and material is biased to static state, such as cap Son, mask, clothes, sunglasses etc., animation effect are biased to dynamic, such as lightning, flowing water, pyrotechnics etc., can also add and be branded as, wear Mask plucks the animation effects such as sunglasses.

The parameter value of animation effect determines the essential attribute of special efficacy object, tracking attribute and triggering attribute, such as by repairing The track of flashing for changing parameter value setting lightning, duration of flashing, speed of flashing, brightness of flashing, opportunity of flashing etc., setting flowing water flow Speed, direction, the size of wave etc., splendid degree of pyrotechnics etc. is set

Its workflow is as shown in figure 4, layout attributes parameter not only determines animation effect object and/or final static material The layout attributes of object, including display state layout attributes in different modes, such as the display under 2D mode and 3D mode State layout attributes show whole cloth of the special efficacy type relative to face grid base map in intermediate region after selecting object Office in integral layout after selected object, can be dragged or be pulled to the object and change its size, this can be completed pair The adjustment of special efficacy position, for example the mask for covering eyes is pulled down, while mouse roller or scaling choosing can also be used Function is selected, integral layout is zoomed in and out,.

It is laid out the display mode that look facility module supports 2D and 3D both of which, presets a face grid as background With reference to, and support 2D and 3D pattern switching, 2D element can show and edit in 3 d mode, but 3D element divides in 2 d mode It is two kinds, for one kind to show and editing according to 2D display mode, another kind is that 3D element cannot show and edit in 2 d mode. In general, element is exactly that we want pair of the 2D photo loaded on head portrait, 3D article and some other paster class As, such as the cat ear customized.

In 3 d mode, when 2D pattern switching is to 3D mode, special efficacy object, the x of special efficacy can be created in 3 d mode Be updated with y-axis using x, the y location under 2D mode, z-axis first defaults an initial value, then special efficacy rendered in x, y and The fixed position of z-axis, after being edited in 3 d mode to 2D special efficacy, then after switching back into 2D mode, only update special efficacy x and Y-axis position, z-axis position are not processed；After again by 2D pattern switching to 3D mode, x and y-axis position are updated, z-axis Position is using last edited position, by this process, come complete the switching between 2D and 3D mode with it is synchronous.

All data in project are packaged first in the step, resource packet is generated, can be the lattice such as zip or rar Formula generates downloading two dimensional code then according to the IP address of local computer and port, and customer end A pp can pass through scanning two dimension Code downloading resource packet, and run execution code building expression and check demonstrating effect.In addition, this resource packet includes that some column can be with Add new expression.

The present invention realizes concerned process steps in AR scene by Qt Quick, opens in resource construction management process of the invention After beginning, can be operated by human-computer interaction interface, be set in advance good person's face in human-computer interaction interface, start special efficacy and/ Or material import operation, special efficacy and paster etc. be added, have related display on the face in the people, but be all it is static, by right These special efficacys and paster are operated, be sized with the parameters such as position, direction of rotation, color, function bar to special efficacy into Row operation, including the parameters such as position and size also include special efficacy trigger condition, and the parameters such as duration, the end PC and mobile terminal are equal It is applicable.

Human-computer interaction interface includes: system Main form area, QML menu area, tool forms area and the part of operating space four, work Tool forms area can rest at left and right sides of system Main form, and operating space can be located at centre, can also be based on operator's It is accustomed to the rational position in each area of reasonable Arrangement human-computer interaction interface.

Main form area can be realized according to the demand of developer using suitable component, when resources configuration management process When the end PC is run, the realization of ApplicationWindow component is can be used in Main form, and ApplicationWindow provides letter Single mode creates program window, because it has the attributes such as attribute MenuBar, ToolBar, Tabview, can be convenient quick wound Build a window abundant.

QML menu area, it is general to be realized there are two types of mode, one is directly being realized using the MenuBar mode in QML, separately One is the ListView modes used to realize, the preferred MenuBar mode of system of the invention is realized.

Tool forms area realizes that CoulumnLayout is commonly used layout types using CoulumnLayout, I Be referred to as Column Layout, CoulumnLayout provides various definitions mode to realize the control of layout.

Intermediary operation area is mainly used for special display effect integral layout and realizes the layout editor of special efficacy object, uses Rectangle component realizes that Rectangle is visual object most basic in QML component, and is made in QtQuick With most visualization components, it is used as container, almost all of QML component can be placed.

The integral layout in intermediary operation area supports that 2D and 3D both of which, both of which are all using Rectangle component It realizes, the difference is that 3D mode is that nesting Scene3D component is realized in Rectangle.It intermediary operation area can be according to special efficacy The difference of type and checked and edited using corresponding mode automatically, as 3D special efficacy object is checked using 3D mode automatically； 2D special efficacy object default is checked using 2D mode, can be switched to 3D mode by switching push button and be checked；But 3D material is in 2D mode Under be divided into two kinds, one kind show and edit according to 2D display mode, it is another for 3D material cannot show in 2 d mode with Editor.

In QtQuick, the manipulation event of QML is realized by invisible elements Mo useArea, can pass through mouse Mark, keyboard, voice etc. issue manipulation instruction and constitute manipulation event, and common event has click, double-click, long-pressing, shortcut key, voice Deng.Commonly it is provided with scope setting, key setting etc..The usage of MouseArea element is the item mono- visual with one Synergistic effect is played, the manipulation event response of item is handled.

The present invention realizes concerned process steps by Qt Quick in AR scene, and Qt Quick is a kind of advanced user interfaces skill Art can easily be created for the dynamic touch formula interface and light weight level application that mobile and embedded device uses using it, Three kinds of completely new technologies together constitute Qt Quick user interface creation kit: improved Qt Creator IDE, A kind of newly-increased language (QML) simple and convenient and easy to study and a new module that entitled QtDeclarative in the library Qt is added, these make It obtains Qt and easily facilitates developer and designer's use.

QML is a kind of statement formula language.It provides a group interface and is used to describe narration visualization component and they it Between interaction.It is the readable language of a height, and is designed to be connected with each other component in such a way that one dynamic.Together It makes component very easy be re-used and create the user interface customized when one.Using QtQuick module, designer and Kai Hair personnel can be very easy to establish the user interface with smooth animation using QML, and these interfaces are connected to rear end Above library.

In the present invention under the 2D mode of resources configuration management method, QML 2D mode is based primarily upon Qt Quick group substantially Part exploitation, in the present invention, mainly used Rectangle, Button, Text, Menu, ListView, Image, SplitView, Column, Row, Label etc., wherein Rectangle is most basic, most flexible component, be can according to need It is designed to a variety of different components and a variety of different Style, while Rectangle is also container assemblies, can put inside it The component of any other type is set, therefore is most commonly used component.

In the present invention, most of component is all placed on used in Rectangle, between multiple Rectangle mutually Nesting combination constitutes in the present invention complicated and orderly interface.

In the present invention under the 3D mode of resources configuration management method, QML3D mode is based primarily upon Scene 3D component and opens Hair, the bottom layer realization of Scene 3D is Scene Graph, is that Qt 5 proposes a new rendering bottom, when substituting Qt4 The Graphics View of phase.Scene Graph mainly utilizes the rendering advantage of OpenGL (ES) 2, in 2D and 3D to flow very much Smooth speed is rendered, and meets growing interfacial effect demand, while Scene Graph has reserved various connect Mouthful, meet everybody and defines the needs of display and rendering effect.

Common requirement using Scene Graph is exactly to realize the superposition of the rendering and 2D pel of 3D model on it (Overlay)。

Qt 3D is QML based on load 3D model in the Scene Graph upper component encapsulated and QML and carries out 3D Render most commonly used component.

The built-in obj solution to model parser of Qt 3D, can easily import the model of obj format.

In order to compatible 2D and 3D resource management simultaneously, face coordinate architecture is using three-dimensional structure storage, i.e., with x, The mode of y, z axis stores, and for the convenience of calculating and simple, when being tracked to face, passes through what big data was collected Public face the most universal only tracks an origin, i.e. face coordinate origin, is then based on this origin as tracking sample Coordinate by the way that the coordinates of other points of face and head is calculated, such as midpoint between upper lower lip, in upper lip The point of point, nose, the point of two centres can be set as tracking origin, and tracking origin position of the invention will be as shown in figure 5, will The central point of straight line line between mouth center and posterior neck is received as tracking origin, i.e. coordinate origin, and based on big data The tracking sample that the public face the most universal of collection is used as constructs the reference coordinate of face key position,

It is as shown in table 1:

Position	X-coordinate	Y-coordinate	Z coordinate
				Nose	0	3.3	13.5
Left eye	3.3	6.6	10.4
				Right eye	-3.3	6.6	10.4
Forehead	0	11	11
				Left ear	8.2	4.4	0
Auris dextra	-8.2	4.4	0

It can be former based on tracking using human face characteristic point algorithm using the collected face realtime graphic of mobile phone camera Point quickly calculates the coordinate of face coordinate origin, and according to this coordinate, upward 1 unit is exactly nose, upward 2 units It is exactly the ordinate of eyes, a unit, is exactly the abscissa of eyes to the left and right, if it is ear, also to add the seat of Z-direction Mark forms space multistory sense.This is also the reason of present invention can import 3D model material, can be determined in space coordinates The spatial position of the body and animation effect object of people and static material object, and then real-time display has on mobile phone screen Three-dimensional sense and the material paster special efficacy etc. that human motion and movement can be followed.Corresponding, by space coordinates These points can support adjustment of the resource construction management system to 3D material position, size and state in turn, realize user certainly The purpose of main modulation material parameter.

And all objects by that will be arranged all are corresponded to and are anchored in face coordinate system with coordinate origin, so that in AR All objects can follow the movement of face and move when showing after expression generation, will not change with the relative position of face.

The production of AR face paster expression can be completed by the editor to resources such as the fine arts or art by the present invention, and face is special The function of the related applications such as effect binding.The development difficulty of application is greatly reduced, non-technical personnel quickly goes up hand, improves whole work Make efficiency.The shortcomings that eliminating in the past for an expression it is necessary to develop a whole set of multiple mating corresponding document, writing correlation The function of file is incorporated into expression edition program, and user can participate directly in the editor of expression, the type of expression No longer it is fixed, but can be selected oneself to make by user oneself, the type of special efficacy and some parameters can gives use Family oneself editor, embodies very strong adaptability and creativeness, without for AI algorithm is not smart enough and has no way out, mentioning significantly again The high independence and entertainment of AR expression application, enhances user experience.

Powerful calculating ability, material obtaining ability based on the end PC further exist the setting of resource construction management system It is preferably adopted so that the resources configuration management method based on human face characteristic point is applied to the end PC, preferably multi-layer framework at the end PC It is realized with language technologies such as QtCuik insertion C++.

And resource construction management system is arranged at the end PC, the matched completion corresponding function of application client end system, The function of realization is relatively easy, it is only necessary to complete the downloading of resource packet, decompression and and to the calling of SDK.

Application client end system can handle library with incoming image, download resource, downloading from server by two-dimensional barcode information Resolving resource APMB package after the completion, can viewing effect after parsing.

Image procossing library is preferably the library ZXing, for the 1D/2D bar code image for the multiple format of open source realized with Java Library is handled, the port for relating to other language is contained, may be implemented to complete bar code using the built-in camera of mobile phone Scanning and decoding.

Resource APMB package packaging structure include res catalogue, scripts catalogue, common_config.json file, Default.lua file, face_metadata.json file, targets.json file etc., wherein res catalogue stores ar Resource file used in resource folder, scripts catalogue store script file.

In the present embodiment, when operating to the resource construction management system that the end PC is arranged in, the resource at the end PC is set Building management system can mutually carry out instant messaging with the application client end system that mobile terminal is arranged in, and pass through resource construction pipe Manage system configuration AR expression can application client end system with real-time display in mobile terminal, such as operator passes through the end PC resource Building management system is added to a special efficacy cartoon material (such as cap) on virtual face, the end PC resource construction management system Picture in system human-computer interaction interface be it is static, only cap, this material is added to head part to display operation person, and at this time Mobile terminal application client end system and the end PC resource construction management system carry out instant messaging, will be real-time on the screen of mobile terminal Show head and this cap of user, and this cap can follow our head mobile using AI algorithm.Certainly, this It is a kind of editable state, we, can be to the position of material inside the human-computer interaction interface of the end PC resource construction management system It sets and is adjusted, scale, for example move end subscriber and think that cap is somewhat big or somewhat small, it can be in the end PC resource construction management system System is carried out the operation of cap zoomed in or out, at this point, corresponding variation can be generated with the cap in account on mobile phone screen, This has just given the sizable independence of user, and not only material can be with unrestricted choice, and size position color of material etc. can be into Row adjustment.

The positioning of this resources configuration management method is to realize the building management of resource, mainly operated with UI based on, the interface UI is closed Be schematic diagram as shown in fig. 6, therefore most of function is handled by way of human-computer interaction manual intervention, into It is to be realized automatically by program, while application client end system is needed through manual scanning two dimension when row reading and writing data The modes such as code carry out the downloading of resource packet.

Based on the same inventive concept, another embodiment of the present invention provides a kind of AR expression resource structure based on human face characteristic point Build management system, it is preferred to use the multi-layer framework of C/S model, the system can be set in the end PC, also can be set mobile phone, In the terminals such as PAD, powerful calculating ability, material obtaining ability in view of the end PC, therefore the system is all preferably mounted on PC End, the structure of the system is as shown in figure 3, include the multiple functional modules for realizing multiple functions: project management module 31, special efficacy function Energy management module 32, material function management module 33, parameter setting function management module 34 are laid out look facility management module 35, expression generation management module 36, storage unit, human-computer interaction interface, in which:

Such as project management module 31 creates the new AR expression task items being branded as or selection one is existing The AR expression task items worn glasses, item objects are face, create the data structure of face, and import in data structure The various initial parameters of face, so as to materials such as subsequent addition cap, glasses, and the animations such as be branded as, wear glasses.

Special effective function management module 32, for obtaining the initial animation special effect parameters for generating animation effect object, according to institute It states initial animation special effect parameters and establishes animation effect data structure, and according to the animation effect data structure and initial parameter Value generates initial animation special efficacy object, and animation effect data structure and initial parameter value are saved in storage unit, realizes The storage and display of initial animation special efficacy object；

Material function management module 33, for obtaining the static material parameter of primary for generating static material object, according to institute It states static material parameter and establishes static material data structure, and is raw according to the static material data structure and initial parameter value At initial static material object, static material data structure and initial parameter value are saved in storage unit, realized initial The storage and display of static material object；

Parameter setting function management module 34, for according to the animation in input instruction modification special effective function module 32 It is quiet in the static material data structure in special effects data structure in the parameter value of animation effect and/or material functional module 33 The parameter value of state material, by animation effect data structure and modified parameter value and/or static material data structure and Modified parameter value is saved in storage unit, and is generated finally according to the data structure and modified parameter value Animation effect object and/or static material object；

Be laid out look facility management module 35, final animation effect object in the configuration feature module 34 that gets parms and/ Or the data structure and parameter value of final static material object, and final animation effect object and/or final static element are set Layout attributes parameter of the material object relative to default face grid base map, layout attributes parameter is saved in storage unit, real The now storage and display of the final animation effect object with layout attributes and/or final static material object；

The present invention realizes concerned process steps in AR scene by Qt Quick, transports in resources configuration management system of the invention It after after row, just will appear human-computer interaction interface, good person's face be set in advance in human-computer interaction interface, start material import operation, Special efficacy and paster are added, have related display on the face in the people, but be all it is static, by these special efficacys and paster progress Operation is sized with parameters such as position, direction of rotation, colors, operates in function bar to special efficacy, including position and big The parameters such as small also include special efficacy trigger condition, and the parameters such as duration, the end PC and mobile terminal are applicable.

Based on the same inventive concept, another embodiment of the present invention provides a kind of resource construction system based on human face characteristic point System, the system include resource construction management system, application client end system, resource construction management system and application client end system As the subsystem of the resource construction system based on human face characteristic point, synergistic effect realizes AR expression true reappearance.

Resource construction management system is resource construction management system described in previous embodiment, as based on face characteristic One of subsystem of the resource construction system of point, is set to the end PC, preferably multi-layer framework, it is preferred to use QtQuik is embedded in C+ + wait language technologies to realize.

Application client end system, as one of subsystem of the resource construction system based on human face characteristic point, setting In mobile terminal, resource construction management system is arranged at the end PC for powerful calculating ability, material obtaining ability based on the end PC, The matched completion corresponding function of application client end system, the function of realization are relatively easy, it is only necessary to complete resource packet downloading, Decompression and and to the calling of SDK, operation executes code and shows AR expression.

In the embodiment of the present invention, when operating to the resources configuration management system that the end PC is arranged in, the end PC is set Resources configuration management system and the application client end system that mobile terminal is arranged in can mutually carry out instant messaging, be matched by resource Set management system configuration AR expression can application client end system with real-time display in mobile terminal, such as operator passes through the end PC Resources configuration management system is added to a special efficacy cartoon material (such as cap) on virtual face, the end PC resource distribution pipe Picture in reason system human-computer interaction interface be it is static, only cap, this material is added to head part to display operation person, and Mobile terminal application client end system and the end PC resources configuration management system carry out instant messaging at this time, will on the screen of mobile terminal Real-time display goes out head and this cap of user, and this cap can follow our head mobile using AI algorithm.When So, this is a kind of editable state, we, can be to material inside the human-computer interaction interface of the end PC resources configuration management system Position be adjusted, scale, for example move end subscriber and think that cap is somewhat big or somewhat small, can be in the end PC resource distribution pipe Reason system is carried out the operation of cap zoomed in or out, at this point, can be generated accordingly on mobile phone screen with the cap in account Variation, this has just given the sizable independence of user, and not only material can all may be used with unrestricted choice, size position color of material etc. To be adjusted.

This system positioning is to realize the configuration management of resource, mainly operated with UI based on, therefore most of function is all logical It crosses what the mode of human-computer interaction manual intervention was handled, when carrying out reading and writing data, is realized automatically by program, Application client end system needs the downloading of manual scanning two dimensional code progress resource packet simultaneously.

The present invention is general resource distribution plateform system, can support a variety of industrial applications, for example, industry, education, amusement, Game etc..In the selection of platform, cross-platform is supported, such as: Windows, Linux and MACOS.By comprehensive assessment, choosing It selects QT5.8 and realizes cross-platform requirement, and can support the application of PC and mobile platform, to meet the subsequent extension demand of system, Industry development is adapted to, can be improved the degree of participation and enjoyment level of user, improves developer's whole work efficiency.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution The range of scheme should all cover within the scope of the claims and the description of the invention.

Claims

1. a kind of AR expression resource construction management method based on human face characteristic point, includes the following steps:

S11, AR expression project management: then creation AR expression project creates item objects and project data in the memory unit After importing the initial parameter of resource in item data structure, item data structure and resource initial parameter information are protected for structure It is stored in storage unit；

S12, AR expression special effective function management: obtaining the initial animation special effect parameters for generating animation effect object, according to described first Initiating draws special effect parameters and establishes animation effect data structure, and raw according to the animation effect data structure and initial parameter value At initial animation special efficacy object, animation effect data structure and initial parameter value are saved in storage unit, realized initial The storage and display of animation effect object；

S13, AR expression material function management: the static material parameter of primary for generating static material object is obtained, according to described quiet State material parameter establishes static material data structure, and is generated just according to the static material data structure and initial parameter value Begin static material object, and static material data structure and initial parameter value are saved in storage unit, realize initial static The storage and display of material object；

S14, AR expression parameter setting management: according in the animation effect data structure in input instruction setting steps S12 The parameter value of static material in the parameter value of animation effect and/or the static material data structure in step S13, by animation It is single that the parameter value and/or static material data structure and the parameter value of setting of special effects data structure and setting are saved in storage In member, and final animation effect object and/or static material pair are generated according to the parameter value of the data structure and setting As；

S15, AR expression layout check management: final animation effect object and/or final static material pair in obtaining step S14 The data structure and parameter value of elephant, and final animation effect object and/or final static material object are set relative to default The layout attributes parameter of face grid base map, layout attributes parameter is saved in storage unit, is realized with layout attributes The storage and display of final animation effect object and/or final static material object；

S16, AR expression generation management: according to the final animation effect object with layout attributes and/or final static element Material object generates AR expression and executes code, runs the AR expression and executes code, generates AR expression.

2. the method according to claim 1, wherein layout checks that management can be in 2D and 3D two in step S15 It being carried out under the display mode of kind mode, and supports 2D and 3D pattern switching, 2D element can show and edit in 3 d mode, but 3D element is divided into two kinds in 2 d mode, one kind show and edit according to 2D display mode, it is another cannot be for 3D element It shows and edits under 2D mode.

3. according to the method described in claim 2, it is characterized in that, layout checks management in 2D pattern switching in step S15 When to 3D mode, special efficacy object can be created in 3 d mode, and the x and y-axis of special efficacy object use x, the y location under 2D mode It is updated, z-axis first defaults an initial value, then special efficacy is rendered to the position fixed in x, y and z axes, when in 3 d mode After editing to 2D special efficacy object, then after switching back into 2D mode, x and the y-axis position of special efficacy object are only updated, z-axis position is not It processes；After again by 2D pattern switching to 3D mode, x and y-axis position are updated, z-axis position is compiled using last Position after volume, by this process, come complete the switching between 2D and 3D mode with it is synchronous.

4. method according to any one of claims 1 to 3, which is characterized in that this method is by Qt Quick in AR scene Middle to realize each management process, under the 2D mode of resources configuration management method, 2D model selection Rectangle is as basic group Part can according to need and be designed to a variety of different components and a variety of different Style, while Rectangle is also used as container Component can be embedded in the component of any other type inside it；Under the 3D mode of resources configuration management method, 3D mould is being realized It can be realized superposition of the 2D pel on 3D model while the rendering of type, 3D model and progress loaded using Qt 3D component 3D is rendered, also built-in a variety of solution to model parsers in Qt 3D component, convenient for importing the model of multiple format.

5. the method according to claim 1, wherein in order to be compatible with 2D and 3D resource management, face simultaneously Coordinate architecture is stored using three-dimensional structure, i.e., is stored in a manner of x, y, z axis, is collected by big data the most universal big Many faces are as tracking sample, using the central point of the straight line line between mouth center and posterior neck as tracking origin, i.e. coordinate system Origin only tracks the coordinate origin, and the tracking sample being used as based on the public face the most universal that big data is collected The reference coordinate of face key position is constructed, and all animation effect objects by that will be arranged and static material object are all It is corresponding to be anchored in face coordinate system with coordinate origin, so that object described in when showing after AR expression generation can be with It moves with the movement of face, will not change with the relative position of face.

6. a kind of AR expression resource construction management system based on human face characteristic point has and realizes AR expression configuration management function Multiple functional modules, comprising: project management module 31, layout management module 32, parameter management module 33, special efficacy management module 34, material management module 35, outgoing management module 36, storage unit, in which:

Then project management module 31 creates item data structure, in project for creating AR expression project in the memory unit After importing the initial parameter of resource in data structure, by item data structure and resource initial parameter information preservation to storage unit In；

Special effective function management module 32, for obtaining the initial animation special effect parameters for generating animation effect object, according to described first Initiating draws special effect parameters and establishes animation effect data structure, and raw according to the animation effect data structure and initial parameter value At initial animation special efficacy object, animation effect data structure and initial parameter value are saved in storage unit, realized initial The storage and display of animation effect object；

Material function management module 33, for obtaining the static material parameter of primary for generating static material object, according to described quiet State material parameter establishes static material data structure, and is generated just according to the static material data structure and initial parameter value Begin static material object, and static material data structure and initial parameter value are saved in storage unit, realize initial static The storage and display of material object；

Parameter setting function management module 34, for according to the animation effect in input instruction modification special effective function module 32 Static element in the static material data structure in data structure in the parameter value of animation effect and/or material functional module 33 The parameter value of material, by animation effect data structure and modified parameter value and/or static material data structure and modification Parameter value afterwards is saved in storage unit, and generates final animation according to the data structure and modified parameter value Special efficacy object and/or static material object；

It is laid out look facility management module 35, final animation effect object in the configuration feature module 34 that gets parms and/or most The data structure and parameter value of static state material object eventually, and final animation effect object and/or final static material pair are set As the layout attributes parameter relative to default face grid base map, layout attributes parameter is saved in storage unit, realizes band There are the storage and display of the final animation effect object and/or final static material object of layout attributes；

Expression generation management module 36: according to the final animation effect object with layout attributes and/or final static element Material object generates AR expression and executes code, runs the AR expression and executes code, generates AR expression.

Storage unit, by project management module 31, special effective function module 32, material database functional module 33, parameter configuration function Module 34, layout look facility module 35 treated object data structure and parameter value are stored for subsequent calls.

7. system according to claim 6, which is characterized in that the system further includes human-computer interaction interface, human-computer interaction circle Face includes: system Main form area, and QML menu area, tool forms area and the part of operating space four, tool forms area can rest in At left and right sides of system Main form, operating space can be located at centre, can also be based on the man-machine friendship of habit reasonable Arrangement of operator The rational position in the mutual each area in interface；

Main form area can be arranged according to system selects suitable component to realize in the end PC or mobile terminal, when system setting When the end PC, the realization of ApplicationWindow component is can be used in Main form area, creates window abundant with fast and easy；

It is realized using Column Layout in tool forms area；

Intermediary operation area is mainly used for special display effect integral layout and realizes the layout editor of special efficacy object, using Rectangle Component is realized, be can be used as container and is placed all QML components.

User edits 2D and 3D both of which by intermediary operation area, and both of which is all using Rectangle component It realizes, the difference is that 3D mode is that nesting Scene3D component is realized in Rectangle.It intermediary operation area can be according to special efficacy The difference of type and checked and edited using corresponding mode automatically, 3D special efficacy object is checked using 3D mode automatically, 2D Special efficacy object default is checked using 2D mode, can be switched to 3D mode by switching push button and be checked；But 3D material is in 2 d mode It checks and is divided into two kinds, for one kind to show and editing according to 2D display mode, another kind is that 3D material cannot be shown in 2 d mode And editor.

8. system according to claim 6 or 7, which is characterized in that the system passes through described in any one of claim 1 to 5 Method generate AR expression.

9. a kind of AR expression resource construction system based on human face characteristic point, which includes such as any one of claim 6 to 8 institute The AR expression resource construction management system and application client end system stated, the described in any item AR expression resources of claim 6 to 8 Construct the subsystem of management system and application client end system as the AR expression resource construction system based on human face characteristic point, association Same-action realizes AR expression true reappearance；

The AR expression resource construction management system as the AR expression resource construction system based on human face characteristic point wherein One subsystem, is set to the end PC；

The application client end system, one of subsystem as the AR expression resource construction system based on human face characteristic point System, is set to mobile terminal, powerful calculating ability, material obtaining ability based on the end PC, and the setting of resource construction management system is existed When the end PC, the matched downloading by completing resource packet of application client end system, decompression and operation execute code and show AR table Feelings.

10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor The step of any one of claim 1-5 the method is realized when execution.