WO2019105093A1 - Data encapsulating method and apparatus, data processing device and apparatus, and electronic device - Google Patents

Abstract

A data encapsulating method and apparatus, a data processing device and apparatus, an electronic device and a non-temporary machine readable storage medium. The data encapsulating method comprises: providing an algorithm registration interface for registering an AR algorithm and an AR service interface for conducting AR interaction with a webpage on the basis of a webpage view component (S102); providing a channel for communication between an algorithm interface of the AR algorithm successfully registered on the basis of the webpage view component and the AR service interface (S104); and encapsulating and generating an AR service callable file according to the webpage view component, the algorithm registration interface of the AR algorithm, the AR service interface, and the established channel (S106). By means of the method, the AR distribution and deployment costs are greatly reduced.

Description

Data encapsulation, processing method, device and electronic device Technical field

The embodiments of the present invention relate to the field of computer technologies, and in particular, to a data encapsulation and processing method, apparatus, and electronic device.

Background technique

AR (Augmented Reality) is a new technology that integrates real world information and virtual world information "seamlessly". It uses information that is difficult to experience in a certain time and space of the real world (eg, Visual information, sound information, etc.), superimposed on the real information after the simulation, the real environment and the virtual object are superimposed in the same picture or space at the same time.

With the development of mobile communication technology and the improvement of processing capabilities of intelligent terminals, augmented reality technology applied to smart terminals has begun to receive attention. However, in the prior art, an augmented reality technology is implemented by means of an application (application) in which an embedded AR algorithm is installed. In this manner, a new AR service needs to be upgraded, replaced or added with an AR algorithm. App version upgrades are also required, and even the same AR service cannot be run simultaneously on different apps. It can be seen that this method makes the distribution and deployment cost of the AR high, which is not conducive to the popularization and development of the AR.

Summary of the invention

In view of this, the embodiments of the present invention provide a data encapsulation and processing method, apparatus, and electronic device to solve the problem of high cost of existing AR distribution and deployment.

According to a first aspect of the embodiments of the present invention, a data encapsulation method is provided, comprising: setting, according to a web page view component, an algorithm registration interface for registering an AR algorithm and an AR service interface for performing AR interaction with a webpage; a channel for communicating between the algorithm interface of the AR algorithm and the AR service interface of the web page view component; and the AR service may be encapsulated according to the web page view component, the algorithm registration interface, the AR service interface, and the channel. Call the file.

According to a second aspect of the embodiments of the present invention, a data processing method is provided, including: receiving an AR service request, where an AR service request carries an AR service parameter; and an AR service parameter in an AR service request is sent to an AR service. The callable file, wherein the AR service callable file is generated according to the algorithm registration interface of the AR algorithm, the webpage view component, the AR service interface, and the communication channel between the AR algorithm interface and the AR service interface; The AR service parameters are processed, and the processing result is returned; and the AR service is displayed according to the processing result.

According to a third aspect of the embodiments of the present invention, a data encapsulating apparatus is provided, including: a first setting module, configured to set an algorithm registration interface for registering an AR algorithm and perform AR interaction with a webpage based on a webpage view component; The AR service interface, the second setting module, configured to set a channel for communicating between the algorithm interface of the AR algorithm and the AR service interface based on the webpage view component; the encapsulation module is configured according to the webpage view component, The algorithm registers an interface, an AR service interface, and the channel, and encapsulates an AR service callable file.

According to a fourth aspect of the present invention, a data processing apparatus is provided, including: a receiving module, configured to receive an AR service request, where an AR service request carries an AR service parameter, and a sending module is configured to use the AR service The AR service parameter in the request is sent to the AR service callable file, wherein the AR service callable file is encapsulated according to the algorithm registration interface of the AR algorithm, the web view component, the AR service interface, and the communication channel encapsulation between the AR algorithm interface and the AR service interface. And a processing module, configured to process the AR service parameter by using an AR service callable file, and return a processing result; and a display module, configured to perform AR service display according to the processing result.

According to a fifth aspect of the embodiments of the present invention, there is provided an electronic device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete each other through the communication bus The memory is for storing at least one executable instruction that causes the processor to perform operations corresponding to the data encapsulation method of the first aspect.

According to a sixth aspect of the embodiments of the present invention, there is provided an electronic device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete each other through the communication bus The memory is for storing at least one executable instruction that causes the processor to perform an operation corresponding to the data processing method of the second aspect.

According to a seventh aspect of the embodiments of the present invention, there is provided a non-transitory machine readable storage medium having stored thereon executable code for causing the executable code to be executed by a processor of an electronic device The method performs the method as described above.

According to the solution of the embodiment of the present invention, an algorithm registration interface of an AR algorithm, a webpage view component for implementing a webpage function, an AR service interface for implementing an AR service, and a communication channel between an AR algorithm interface and an AR service interface are encapsulated and generated into an AR. Business callable files. The AR algorithm itself is independent of the AR service callable file, and only needs to register with the AR service callable file, and informs the AR service that the corresponding algorithm information of the file can be called, that is, the AR service callable file can be called. The AR service callable file does not need to understand the modification of the AR algorithm, or the specific implementation of adding or deleting, and only needs to obtain the corresponding algorithm information through the registration operation. In addition, the AR service end only needs to interact with the AR service callable file to implement the corresponding AR function. The AR service end does not need to embed the AR algorithm, and does not need to understand the specific implementation details of the AR algorithm. Therefore, when the new AR service is launched, the new AR service can be used only by making small changes to the front end such as the HTML page; the host application itself where the AR service can call the file itself may not have the AR algorithm, and The registered AR algorithm can implement the AR service of the host App itself; when replacing or adding the AR algorithm that does not belong to the host App itself, there is no need to change the host App itself; and, the same AR service can be easily implemented on different apps. Run at the same time. It can be seen that the distribution and deployment cost of the AR is greatly reduced by the solution of the embodiment of the present invention.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a few embodiments described in the embodiments of the present invention, and other drawings can be obtained according to the drawings for those skilled in the art.

1 is a flow chart showing the steps of a data encapsulation method according to Embodiment 1 of the present invention;

2 is a flow chart showing the steps of a data encapsulation method according to Embodiment 2 of the present invention;

3 is a schematic structural diagram of an AR service callable file according to an embodiment of the present invention;

4 is a schematic diagram of an AR data interaction according to an embodiment of the present invention;

5 is a flow chart showing the steps of a data processing method according to Embodiment 3 of the present invention;

6 is a flow chart showing the steps of a data processing method according to Embodiment 4 of the present invention;

7 is a structural block diagram of a data encapsulating apparatus according to Embodiment 5 of the present invention;

FIG. 8 is a structural block diagram of a data processing apparatus according to Embodiment 6 of the present invention; FIG.

9 is a schematic structural diagram of an electronic device according to Embodiment 7 of the present invention;

FIG. 10 is a schematic structural diagram of an electronic device according to Embodiment 8 of the present invention.

Detailed ways

For a better understanding of the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments are only a part of the embodiments of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art should be within the scope of protection of the embodiments of the present invention based on the embodiments in the embodiments of the present invention.

The specific implementation of the embodiments of the present invention is further described below in conjunction with the accompanying drawings.

Embodiment 1

Referring to FIG. 1, a flow chart of steps of a data encapsulation method according to a first embodiment of the present invention is shown.

The data encapsulation method of this embodiment includes the following steps:

Step S102: Based on the webpage view component, set an algorithm registration interface for registering the AR algorithm and an AR service interface for performing AR interaction with the webpage.

The AR algorithm is an underlying algorithm for implementing AR functions, such as detecting a face in an image and returning information of a face key point, detecting a target object set in the image and returning the position information of the target object, and detecting the target in the image. Whether the object has a certain attribute, and so on. The AR algorithm may be implemented in an appropriate language, optionally in an interpreted language such as the Java language or the C# language, or in a compiled language such as C or C++. Compared with scripting languages such as JS language to implement AR algorithm, compiling language or interpreted language is more efficient and faster, which can effectively meet the needs of AR business for timeliness. However, it is not limited thereto, and in actual use, the scripting language is also applicable to the AR algorithm of the embodiment of the present invention.

Algorithm registration is a way to inform the algorithm caller of relevant information of the algorithm (such as algorithm identification (name or ID, etc.), class name, etc.). Algorithm registration can be implemented by function registration or function declaration.

With the registration method, the AR algorithm can implement flexible modification within the algorithm without the caller knowing the specific details of the modification; when a new AR algorithm occurs, the new AR algorithm can be realized by informing the algorithm caller related algorithm information through registration. Call; when you need to delete the registered AR algorithm, you can no longer register or call. The AR algorithm can return corresponding information through the AR algorithm interface, for example, returning corresponding information to the HTML5 page through the AR service interface, so that the HTML5 page performs further AR processing based on the information.

In the embodiment of the present invention, the algorithm registration interface is configured to receive the registration of the AR algorithm, and the AR algorithm notifies the algorithm caller of the algorithm information through the algorithm registration interface. The specific implementation manner of the algorithm registration interface may be appropriately set by a person skilled in the art according to actual requirements, which is not limited by the embodiment of the present invention.

The AR service is set at the front end (as implemented by JavaScript (JS) code in the front-end HTML5 page), and functions implemented include, but are not limited to, camera data rendering or other image data rendering, AR special effects rendering, and the like. The AR service does not have the AR algorithm capability. The AR service interface needs to obtain the result returned by the AR algorithm. Based on the result, the AR service is finally implemented. The AR service interface is an interface for performing AR interaction with a webpage, and can be implemented by a scripting language. Alternatively, the AR service interface can be implemented in the form of a JS API. The AR service interface interacts with the AR algorithm interface through the webpage view component to deliver AR service parameters.

The web view component is a component that implements browser functions, such as WEBVIEW, which can be implemented as a web-based rendering engine based on the browser kernel, which can display the corresponding interface and webpage by loading HTML data.

In this embodiment, the algorithm registration interface and the AR service interface are set based on the webpage view component, that is, the backend AR algorithm and the front end, such as the HTML5 webpage, can implement algorithm registration and front-end AR through the corresponding interface set in the webpage view component. Data interaction.

Step S104: Set a channel for communicating between the algorithm interface of the AR algorithm based on the web page view component and the AR service interface.

When the AR service is specifically used, the front-end webpage transfers the service parameters of the AR service request to the webpage view component through the AR service interface, and the webpage view component not only performs the corresponding function according to the corresponding function module of the internal service parameter, but also is the AR. The service interface and the AR algorithm interface establish a communication channel for performing transmission, interaction, processing, and the like of the AR data and the web page data. The communication channel between the AR service interface and the AR algorithm interface may be represented by parameter transfer and/or interface call between each other.

Step S106: Encapsulate and generate an AR service callable file according to the webpage view component, the algorithm registration interface of the AR algorithm, the AR service interface, and the established channel.

Through the AR service callable file, the interaction between the front-end AR service and the back-end AR algorithm can be realized, and finally the AR function is implemented.

The AR service callable file may take any suitable file format, including but not limited to a component form, such as an AAR component or a COM component. Through the callable file, the related App or browser can directly call the AR algorithm for AR service processing, without implementing the AR algorithm itself, without knowing the implementation details of the AR algorithm, thereby making the AR function more flexible and compatible, and easier. Distribution and deployment.

It should be noted that, in practical applications, the algorithm is not limited to the algorithm registration interface of the AR algorithm, the web page view component, the AR service interface, and the established channel. Those skilled in the art may also use other suitable interfaces or functional components according to actual needs. Or the function module is encapsulated into an AR service callable file.

In this embodiment, the algorithm registration interface of the AR algorithm, the webpage view component that implements the webpage function, the AR service interface that implements the AR service, and the communication channel between the AR algorithm interface and the AR service interface are encapsulated and generated by the AR service. file. The AR algorithm itself is independent of the AR service callable file, and only needs to register with the AR service callable file, and informs the AR service that the corresponding algorithm information of the file can be called, that is, the AR service callable file can be called. The AR service callable file does not need to understand the modification of the AR algorithm, or the specific implementation of adding or deleting, and only needs to obtain the corresponding algorithm information through the registration operation. In addition, the AR service end only needs to interact with the AR service callable file to implement the corresponding AR function. The AR service end does not need to embed the AR algorithm, and does not need to understand the specific implementation details of the AR algorithm. Therefore, when the new AR service is launched, the new AR service can be used only by making small changes to the front end such as the HTML page; the host application itself where the AR service can call the file itself may not have the AR algorithm, and The registered AR algorithm can implement the AR service of the host App itself; when replacing or adding the AR algorithm that does not belong to the host App itself, there is no need to change the host App itself; and, the same AR service can be easily implemented on different apps. Run at the same time. It can be seen that the distribution and deployment cost of the AR is greatly reduced by the solution of the embodiment of the present invention.

The data encapsulation method of this embodiment may be implemented by any suitable device or device having data processing functions, including but not limited to various terminals and servers.

Embodiment 2

Referring to FIG. 2, a flow chart of steps of a data encapsulation method according to a second embodiment of the present invention is shown.

The data encapsulation method of this embodiment includes the following steps:

Step S202: Set an AR service interface for performing AR interaction with the webpage based on the webpage view component.

The web view component is a component that implements a browser function, and can be implemented as a webpage rendering engine based on a browser kernel, and the corresponding interface and webpage can be displayed by loading HTML data. In practical applications, those skilled in the art can adopt any suitable web page view component according to actual conditions. In this embodiment, the WEBVIEW web page view component is taken as an example.

WEBVIEW is a control based on WEBKIT engine that displays WEB pages. In addition to the general VIEW properties and settings, WEBVIEW can also handle URL requests, page loads, renderings, and page interactions. WEBVIEW allows you to display and render web pages, use HTML files (on the web or in local assets) for layout, interact with JavaScript, and more. The browser app can implement the browser function through WEBVIEW, while the non-browser app can embed the browser function into its own application through WEBVIEW.

WEBVIEW also provides methods to extend its own functionality. For example, the addJavascriptInterface() method provided by WEBVIEW is used to set the extended JS (JavaScript) interface to WEBVIEW, and forward the calling methods of these extended JS interfaces to the interface registration. Native function. Based on this, the embodiment of the present invention sets an AR service interface for performing AR interaction with a webpage, so as to implement various AR functions required for the front-end webpage. For example, the AR service interface can receive the AR service parameters sent by the front end HTML5 page. On the one hand, the AR service parameters are forwarded to the algorithm interface of the matched AR algorithm through WEBVIEW; on the other hand, the WEBVIEW invokes the corresponding according to the service parameters. An image device interface or an image acquisition interface to obtain an image. In addition, after processing the image according to the AR service parameters, the AR algorithm returns the processing result to the AR service interface, and the AR service interface hands it to the HTML5 page for processing. In one possible approach, the AR service interface can be implemented in the JavaScript language.

In addition, based on the extended function of WEBVIEW, a query interface for acquiring algorithm information of the AR algorithm may be set, and the query interface also interacts with the front-end webpage, and the front-end webpage can learn the registered various AR algorithms through the query interface. The algorithm information is used to provide a reference for determining the AR algorithm corresponding to the AR service for subsequent selection.

Step S204: Set an algorithm registration interface for registering the AR algorithm based on the web page view component.

The algorithm registration interface of the AR algorithm is used to implement the registration of the AR algorithm, and the algorithm information of the AR algorithm caller AR algorithm is notified by registration for subsequent calls. The AR algorithm may be an algorithm set in the App, or may be an independently existing algorithm such as an AR algorithm that is included in the operating system. The AR algorithm can also be embedded in the AR service callable file, that is, directly encapsulated into the AR service callable file. In one possible approach, the AR algorithm can be implemented as an underlying algorithm component.

The AR algorithm may include one or more (two or more). In one feasible solution, the AR algorithm may be configured to use a unified interface externally, and multiple AR algorithms are registered through the same interface class to simplify the interface. Settings to improve algorithm compatibility.

Step S206: Set an image device interface for controlling the image capturing device based on the web page view component, and/or set an image obtaining interface for acquiring image data.

The image device interface is used to control the image capture device, such as controlling camera opening, shooting, closing, data transfer, and the like. In a feasible way, the image device interface can also be set based on the extended function of WEBVIEW. However, the present invention is not limited thereto. In practical applications, those skilled in the art may also package the image collection and processing into independent functional modules according to actual needs, and provide corresponding interfaces for calling. Through the image device interface, images can be acquired in real time to realize AR processing of real-time images. In a specific implementation, the image device interface may control the image collection device by using an image device control algorithm, where the image device control algorithm may be an algorithm encapsulated in an AR service callable file, or may be an AR service callable file but may be called. Algorithm.

The image acquisition interface is used to acquire image data, such as a still image (such as a locally stored or network downloaded image), or a video frame image in a recorded video. The image acquisition interface can also be based on the extended function settings of WEBVIEW. However, the present invention is not limited thereto. In practical applications, those skilled in the art may also package them into independent functional modules according to actual needs, and provide corresponding interfaces for calling. Through the image acquisition interface, AR processing of non-real time images can be realized. In a specific implementation, the image acquisition interface may acquire image data by using an image acquisition algorithm. The image acquisition algorithm may be an algorithm encapsulated in an AR service callable file, or may be an algorithm that can be called outside the AR service callable file.

Both the image device interface and the image acquisition interface accept the call of WEBVIEW, and according to the call, the image is acquired or the image is acquired. On the one hand, the collected or acquired image is sent to the AR algorithm interface for processing, and on the other hand, it is returned to the AR service. Interface for the AR business interface to be rendered by the front-end HTML5 page.

It should be noted that, for convenience of description, the embodiment describes the setting of the corresponding interface according to the sequence of steps S202-S204, but those skilled in the art should understand that in actual applications, the execution of the above steps may be in no particular order. , can also be executed in parallel.

Step S208: Set a channel for communicating between the algorithm interface of the AR algorithm based on the web page view component and the AR service interface.

Data transmission and information exchange are implemented between the AR algorithm interface and the AR service interface through a communication channel. In a feasible manner, the AR service parameter received by the webpage view component through the AR service interface may be set to be transmitted to the communication channel of the successfully registered AR algorithm interface. The AR service parameters include, but are not limited to, the service type of the AR service, the operation parameters of the AR service, and the like.

When at least one of the image device interface and the image acquisition interface is also provided, it is also necessary to establish a communication channel between the image device interface and/or the image acquisition interface and the AR service interface, and the image device interface and/or image acquisition. The communication channel between the interface and the AR algorithm interface. That is, setting a communication channel between the image device interface and the AR service interface and the algorithm interface of the AR algorithm via the web page view component; and/or setting the image acquisition interface via the web page view component and the AR service interface respectively The communication channel between the algorithm interfaces of the AR algorithm is configured to send the image data to be processed to the AR algorithm interface for processing by the AR algorithm, and return the image data to be processed to the AR service interface for the AR service interface to be handed over to the front page. Render rendering.

Step S210: According to at least one interface of the image device interface and the image acquisition interface, and a webpage view component, an algorithm registration interface of the AR algorithm, an AR service interface, a communication channel between the AR algorithm interface and the AR service interface, and a query interface Encapsulation generates AR business components.

In this embodiment, the AR service callable file is implemented as an AR service callable component, such as a aar component. However, it will be apparent to those skilled in the art that other forms of callable files are equally applicable.

In a feasible manner, at least one of the image device interface and the image acquisition interface, and the algorithm registration interface, the web view component, the AR service interface, the AR algorithm interface, and the AR service interface of the AR algorithm may be communicated. The channel and the query interface are encapsulated to generate an AR service component.

In another possible manner, the webpage view component may not be encapsulated into the AR service component. In this case, a call interface for calling the webpage view component may be set, and at least one interface of the image device interface and the image acquisition interface may be set. And, the algorithm registration interface of the AR algorithm, the calling interface of the webpage view component, the AR service interface, the communication channel between the AR algorithm interface and the AR service interface, and the query interface are encapsulated to generate an AR service component.

An example of an AR service callable file implemented based on the above process is shown in FIG. 3. In FIG. 3, the AR service callable file is implemented as a component in the form of a WebAR framework, that is, the WebAR framework is integrally packaged into components. (referred to as the WebAR component), let the host App (such as the UC browser application, Taobao application, Alipay application in Figure 3) reference. For Android system, packaged into aar; for iOS system, packaged into Framework.

The host App uses the corresponding AR algorithm registration interface in the WebAR component (such as the Android Java interface and the iOS OC interface in FIG. 3) and the corresponding algorithm class in the AR algorithm component, and the AR algorithm (such as the A algorithm in FIG. 3) A algorithm 2, B algorithm) is registered in the WebAR component. The AR algorithm includes an AR algorithm that the host App itself has and/or an AR algorithm that is independent of the host App.

In this example, the front end is provided with an AR service 1, an AR service 2, and an AR service 3 through an HTML5 page, and the AR service implements interaction with the WebAR component through a JS interface (AR service interface). The JS interface (AR service interface) is implemented by the AR environment management class ARSession. ARSession is a shared object that provides AR functions by managing device cameras and motion processing. It coordinates AR's numerous processes, including data acquisition from device motion sensors, control of built-in cameras, and camera image processing. ARSession achieves AR effects by combining all the results to establish a connection between the real space and the virtual coordinate system space.

In this example, the front-end HTML5 page initializes the AR environment through the JS interface (AR service interface) implemented via ARSession, opens the camera and selects the Detector interface required for the AR service. After obtaining the image data captured by the camera through the JS interface (AR service interface), the HTML5 page converts the original format of the image data into a format suitable for drawing, and renders through WebGL. The Detector interface can call the corresponding capability of the AR algorithm. The WebAR component is responsible for transmitting the call message received through the JS interface and the image data captured by the camera to the AR algorithm, and then returning the processing result of the AR algorithm to the HTML5 page.

In this example, each AR algorithm adopts a separate component form, and all AR algorithms implement a unified interface type IDetector interface. The IDetector interface includes a unified process interface, a detection interface, and a callback interface, such as a start() interface and a pause( ) interface, stop() interface, resume() interface, init() interface, track() interface, onResult() interface, etc. These interfaces have corresponding JS (JavaScript) mapping interfaces, ie Detector interfaces, on the front end, so the front end The HTML5 page looks like the AR algorithm is called directly.

The host App needs to register the required AR algorithm with the WebAR component during initialization. The front-end HTML5 page can query the currently available AR algorithm through the JS interface (query interface) and select the AR algorithm required for activation. The WebAR component creates an AR algorithm instance through the selection of the JS interface (AR service interface), but the interaction process is completed by the IDetector. Therefore, whether the specific AR algorithm is implemented by the A algorithm 1 or the B algorithm, or even the function of the A algorithm 1 is different, but the implementation details are different. The A algorithm 2 is the same for the WebAR component, so the random replacement of the AR algorithm can be implemented.

The WebAR component delivers camera data to the currently activated AR algorithm instance, waits for the AR algorithm to complete the detection and returns the returned result to the JS interface (AR service interface), and then the JS interface (AR service interface) is processed by the HTML5 page. .

In this implementation, the control and management of the image acquisition device is implemented by the camera management component shown in FIG. 3, which acquires and transmits the camera data through the corresponding interface. For the Android system, the camera data can be obtained through the system interface android.hardware.Camera() or android.hardware.Camera2(); for the iOS system, the camera data can be obtained through the system interface UIImagePickerController(). The acquired camera data is passed to the front-end HTML5 page through the JS interface (AR service interface). The HTML5 page acquires the image data currently captured by the camera in real time by calling the JS interface (AR service interface) of the WebAR component. At the same time, the WebGL technology is used to convert the input YUV color-encoded streaming media data into RGB format and draw it onto the Canvas canvas to realize the rendering of the camera image.

The WebAR component extends the JS interface for the AR service through the extension function of WebView, thereby implementing the AR extension interface of the WebAR component to the front end HTML5 page. The WebAR component establishes a communication channel for the JS interface and the AR algorithm interface, so that the call message received through the JS interface can reach the AR algorithm component, and the algorithm interface of the AR algorithm component can notify the HTML5 page of the processing result through the JS interface. For the Android system, WebView can be embedded as a component in the WebAR framework; for the iOS system, the WebView needs to be referenced by the host App. For example, the host App registers the UI WebView or WK Webview into the WebAR component. FIG. 3 shows an example of a WebAR component of the Android system in which a WebView is embedded as a component in a WebAR component.

Based on the WebAR component shown in Figure 3, a process of AR data interaction is shown in Figure 4. As shown in Figure 4, when the current HTML5 page initiates an AR service, the AR service controls the camera management component to start the camera through the JS interface (AR service interface and image device interface) implemented by ARSession, and queries the registered AR from the Detector registration list. The algorithm selects the AR algorithm for determining the AR service, obtains the algorithm information of the AR algorithm, for example, the information of the IDetector interface; the JS interface (the image device interface) transmits the camera data collected by the camera to the determined IDetector interface, and is handed over to the IDetector interface. After the AR algorithm corresponding to the interface is processed, the processing result is sent to the front end HTML5 page through the JS interface (AR service interface), and further processed by the front end HTML5 page, for example, rendering rendering, or virtual object superposition based on the processing result.

In this example, the data is exchanged with the WebView in the WebAR component, the data transmission of the camera data is transmitted to the JS interface (AR service interface), and the JS interface (AR service interface and query interface) of the WebAR component is interfaced with the AR algorithm. Map binding. The JS interface (AR service interface and query interface) provides a unified AR related capability interface to the front end, so the front end AR service logic can be replaced at any time; the algorithm registration interface of the AR algorithm is used to bridge various AR algorithms, so the AR algorithm can be implemented at any time. Replacement; the entire WebAR framework serves as a component for the host App to reference, so after the different host App accesses, it can provide the same JS interface (AR service interface and query interface) and AR algorithm capabilities to the front end.

Through the WebAR component, the AR service of the front-end HTML5 page, the underlying AR algorithm and the host App are only interacted with the WebAR component respectively, and the three do not depend on each other. Therefore, the ARAR service, the AR algorithm, and the host App are all freely pluggable and independent of each other without being affected by each other.

For example, if you need to replace the front-end AR service, you only need to re-write the HTML5 page, and the host App and AR algorithms remain unchanged, so that the AR service can be quickly launched. Users only need to access the new HTML5 page, and there is no need to update the App version. Therefore, the problem that the new AR service on the line needs to be upgraded by the App is solved; if the AR algorithm needs to be added or the existing AR algorithm is replaced, only the new algorithm needs to implement the IDetector interface and register in the WebAR component to make The host app reference, the HTML5 page can directly use the new AR algorithm, and the HTML5 page and the host App do not need to be changed, thereby solving the problem that the replacement, the new AR algorithm needs to be upgraded by the App; if there is an AR service needs more Simultaneously running on the App, you only need to access the WebAR component on multiple target host App, you can achieve the effect of one AR service running on multiple App, no difference with the normal HTML5 webpage, thus solving the same An issue where an AR service cannot run simultaneously on different apps.

It can be seen that, through this example, a pluggable WebAR component is implemented, so that the front-end AR service, the specific AR algorithm, and the host application can be flexibly replaced, so that the AR service change and the AR algorithm update do not need to wait for the App upgrade, and the same Front-end AR services can run on multiple different host applications.

In addition, in this example, WEBVIEW is used as the WebAR component as an example, but is not limited thereto. Those skilled in the art may also use Weex or RN (React Native) instead of WebView to implement the solution according to the description of the embodiments of the present invention. Weex or RN can parse the JS language like WEBVIEW, and only need to follow the Weex or RN framework when implementing it.

In this embodiment, the algorithm registration interface of the AR algorithm, the webpage view component that implements the webpage function, the AR service interface that implements the AR service, and the communication channel between the AR algorithm interface and the AR service interface are encapsulated and generated by the AR service. file. The AR algorithm itself is independent of the AR service callable file, and only needs to register with the AR service callable file, and informs the AR service that the corresponding algorithm information of the file can be called, that is, the AR service callable file can be called. The AR service callable file does not need to understand the modification of the AR algorithm, or the specific implementation of adding or deleting, and only needs to obtain the corresponding algorithm information through the registration operation. In addition, the AR service end only needs to interact with the AR service callable file to implement the corresponding AR function. The AR service end does not need to embed the AR algorithm, and does not need to understand the specific implementation details of the AR algorithm. Therefore, when the new AR service is launched, the new AR service can be used only by making small changes to the front end such as the HTML page; the host application itself where the AR service can call the file itself may not have the AR algorithm, and The registered AR algorithm can implement the AR service of the host App itself; when replacing or adding the AR algorithm that does not belong to the host App itself, there is no need to change the host App itself; and, the same AR service can be easily implemented on different apps. Run at the same time. It can be seen that the distribution and deployment cost of the AR is greatly reduced by the solution of the embodiment of the present invention.

The data encapsulation method of this embodiment may be implemented by any suitable device or device having data processing functions, including but not limited to various terminals and servers.

Embodiment 3

Referring to FIG. 5, a flow chart of steps of a data processing method according to a third embodiment of the present invention is shown.

This embodiment is described as an example of a host in which an AR service callable file is installed. The AR service callable file uses the AR service callable file described in Embodiment 1 or 2 to implement data processing of the AR service, and the AR. The contents related to the business callable file can be referred to the foregoing embodiment.

The data processing method of this embodiment includes the following steps:

Step S302: Receive an AR service request.

The AR service request carries the AR service parameters, such as the service type of the AR service, the operation parameters of the AR service, and the like. The AR service request can be initiated by the front end HTML5 page, and the WebAR component is called for processing.

Step S304: Send the AR service parameter in the AR service request to the AR service callable file.

The AR service callable file is generated according to at least the algorithm registration interface of the AR algorithm, the webpage view component, the AR service interface, and the communication channel encapsulation between the AR algorithm interface and the AR service interface, as described in Embodiment 1 or 2. For example, the WebAR component in the second embodiment.

When the user triggers the AR service through a webpage such as an HTML5 page, the host application or the browser where the AR service can invoke the file generates a corresponding AR service request, and the AR service callable file is called to process the AR service request, and the AR service request is processed. The AR service parameters are sent to the AR service interface in the AR service callable file.

Step S306: The AR service parameters are processed by the AR service callable file, and the processing result is returned.

After the AR service callable file is revoked, the AR algorithm is selected according to the AR service parameters, and the image acquisition device, such as the camera, is controlled to collect images, and the AR service parameters and image data are sent to the AR algorithm interface, and the AR algorithm interface is used to invoke the AR. The algorithm processes the image data according to the AR service parameters, and returns the processing result through the AR service interface.

Step S308: Perform AR service presentation according to the processing result.

After receiving the processing result, the front-end HTML5 page performs corresponding AR service processing and display based on the processing result, for example, mapping the face in the current image.

With this embodiment, an AR-enabled file can be used to register an interface, a web page view component that implements a web page function, an AR service interface that implements an AR service, and an AR service callable file of a communication channel between an AR algorithm interface and an AR service interface. To achieve front-end AR services. The front end does not need to implement the AR algorithm, and does not need to understand the implementation details of the AR algorithm, which greatly reduces the distribution and deployment cost of the AR.

The data processing method of this embodiment may be implemented by any suitable device or device having data processing functions, including but not limited to various terminals and servers.

Embodiment 4

Referring to Figure 6, there is shown a flow chart of the steps of a data processing method in accordance with a fourth embodiment of the present invention.

This embodiment is described as an example of a host in which an AR service callable file is installed. The AR service callable file uses the AR service callable file described in Embodiment 1 or 2 to implement data processing of the AR service, and the AR. The contents related to the business callable file can be referred to the foregoing embodiment.

The data processing method of this embodiment includes the following steps:

Step S402: Receive a registration request of the AR algorithm, and register the AR algorithm by using an algorithm registration interface of the AR service callable file.

Through the AR algorithm registration, the AR service can call the file to learn the algorithm information of the AR algorithm, and the registration of the AR algorithm can adopt the interface registration method.

The AR service callable file may be an AR service callable component, such as the WebAR component in the second embodiment.

Step S404: Receive an AR algorithm query request, and return the algorithm information of the AR algorithm through the query interface of the AR service callable file.

After the AR algorithm registers with the AR service callable file, the AR service can call the file to store the algorithm information of the successfully registered AR algorithm, and the current HTML5 page can query the AR algorithm through the query interface encapsulated in the AR service callable file, and send the AR algorithm. When the AR algorithm queries the request, the AR service callable file can receive and return the algorithm information of the AR algorithm to the front end HTML5 page according to the AR algorithm query request, and the subsequent HTML5 page can pass the AR service interface and the AR according to the actual AR service requirement. The algorithm interface calls the AR algorithm.

Step S406: Receive an AR service request.

The AR service request carries an AR service parameter, such as an AR service type, an AR operation parameter, and the like.

Step S408: Send the AR service parameter in the AR service request to the AR service callable file.

As described above, in the embodiment, the AR service callable file adopts the AR service callable file in the first embodiment or the second embodiment.

The host application or browser of the AR service callable file sends the AR service parameters to the AR service interface in the AR service callable file.

Step S410: The AR service parameters are processed by the AR service callable file, and the processing result is returned.

In a feasible manner, the AR service parameter can be received by the AR service interface of the AR service callable file, and the matched AR algorithm is determined according to the AR service parameter, and the AR service parameter is sent to the AR of the matched AR algorithm by using the webpage view component. The algorithm interface processes and receives the AR service data returned by the AR algorithm interface through the AR service interface.

When the AR service callable file encapsulates an image device interface for controlling the image capture device, and/or encapsulates an image acquisition interface for acquiring image data, it may also pass after determining the matched AR algorithm according to the AR service parameter. The AR service may invoke an image device interface of the file, invoke an image acquisition device for image collection, and/or acquire a still image or a video frame image through an image acquisition interface of the AR service callable file. In this case, the processing of the AR service parameter by the web page view component to the AR algorithm interface of the matched AR algorithm includes: sending the image acquired by the image acquisition device or the image acquisition interface through the web page view component, and sending the AR service parameter. The AR algorithm interface is processed.

Step S412: Perform AR service presentation according to the processing result.

For example, after receiving the AR service callable file, the face key position data in the face image returned by the AR service interface is subjected to texture processing on the face in the image, and the like.

With this embodiment, an AR-enabled file can be used to register an interface, a web page view component that implements a web page function, an AR service interface that implements an AR service, and an AR service callable file of a communication channel between an AR algorithm interface and an AR service interface. To achieve front-end AR services. The front end does not need to implement the AR algorithm, and does not need to understand the implementation details of the AR algorithm, which greatly reduces the distribution and deployment cost of the AR.

The data processing method of this embodiment may be implemented by any suitable device or device having data processing functions, including but not limited to various terminals and servers.

Embodiment 5

Referring to FIG. 7, a block diagram of a structure of a data encapsulating apparatus according to Embodiment 5 of the present invention is shown.

The data encapsulating device of this embodiment includes: a first setting module 502, configured to set an algorithm registration interface for registering an AR algorithm and an AR service interface for performing AR interaction with a webpage based on a webpage view component; and a second setting module 504 And a channel for performing communication between the algorithm interface of the AR algorithm based on the web page view component and the AR service interface; the encapsulation module 506 is configured to register the interface, the webpage view component, and the AR according to the algorithm of the AR algorithm. The service interface and the channel encapsulate the AR service callable file.

Optionally, the data encapsulating device of this embodiment further includes: a third setting module 508, configured to set an image device interface for controlling the image capturing device based on the web page view component, and/or configured to acquire an image An image acquisition interface of the data; the encapsulation module 506 is configured to encapsulate at least one of the image device interface and the image acquisition interface, and the algorithm registration interface, the webpage view component, the AR service interface, and the channel, and the encapsulation Generate an AR business callable file.

Optionally, the encapsulating module 506 is configured to encapsulate, generate, and generate, at least one of the image device interface and the image obtaining interface, and the algorithm registration interface, the webpage view component, the AR service interface, and the channel. The AR service may invoke a file; or, set a call interface for invoking a web page view component, at least one interface of the image device interface and the image acquisition interface, and the algorithm registration interface, a webpage view component call The interface, the AR service interface, and the channel are encapsulated to generate an AR service callable file.

Optionally, the second setting module 504 is configured to set, by the communication channel, the AR service parameter received by the webpage view component through the AR service interface to the AR algorithm interface that is successfully registered.

Optionally, the second setting module 504 is further configured to set, by using the webpage view component, communication between the image device interface for controlling the image collection device and the algorithm interface of the AR service interface and the AR algorithm, respectively. a channel; and/or a communication channel between the image acquisition interface for acquiring image data via the web page view component and the algorithm interface of the AR service interface and the AR algorithm, respectively.

Optionally, the data encapsulating device of the embodiment further includes: a fourth setting module 510, configured to set a query interface for acquiring algorithm information of the AR algorithm, and encapsulate the query interface into the AR service callable file.

Optionally, the AR service callable file is an AR service callable component.

Optionally, the AR algorithm includes multiple, and multiple AR algorithms are registered by the same interface class.

The data encapsulating device of this embodiment is used to implement the corresponding data encapsulation method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.

Embodiment 6

Referring to Fig. 8, there is shown a block diagram showing the structure of a data processing apparatus according to a sixth embodiment of the present invention.

The data processing apparatus of this embodiment includes: a receiving module 602, configured to receive an AR service request, where the AR service request carries an AR service parameter, and a sending module 604, configured to send the AR service parameter in the AR service request to the AR a business callable file, wherein the AR service callable file is generated according to an algorithm registration interface of the AR algorithm, a web page view component, an AR service interface, and a communication channel encapsulation between the AR algorithm interface and the AR service interface; and the processing module 606 is configured to pass The AR service can process the AR service parameters and return the processing result. The display module 608 is configured to perform the AR service display according to the processing result.

Optionally, the processing module 606 is configured to receive an AR service parameter by using an AR service interface of the AR service callable file, and determine a matched AR algorithm according to the AR service parameter; send the AR service parameter to the matching by using the webpage view component. The AR algorithm interface of the AR algorithm processes, and receives the AR service data returned by the AR algorithm interface through the AR service interface.

Optionally, the AR service callable file is further encapsulated with an image device interface for controlling the image collection device, and/or an image acquisition interface for acquiring image data is encapsulated; the processing module 606 is further configured to use the AR service parameter according to the AR service parameter. After determining the matched AR algorithm, the image collection device is invoked to perform image acquisition through the image device interface of the AR service callable file; and/or, the static image or video frame is obtained through the image acquisition interface of the AR service callable file. And the image obtained by the image acquisition device or the image acquisition interface, and the AR service parameter are sent to the AR algorithm interface for processing by the webpage view component.

Optionally, the data processing apparatus of this embodiment further includes: a registration module 610, configured to: before the receiving module 602 receives the AR service request, receive a registration request of the AR algorithm, and register an interface to the AR algorithm by using an algorithm of the AR service callable file. Sign up.

Optionally, the AR service callable file is further encapsulated with a query interface for acquiring the algorithm information of the AR algorithm. The data processing apparatus of the embodiment further includes: a query module 612, configured to receive the AR service request before receiving the AR service request The AR algorithm queries the request, and returns the algorithm information of the AR algorithm through the query interface of the AR service callable file.

Optionally, the AR service callable file is an AR service callable component.

The data processing apparatus of this embodiment is used to implement the corresponding data processing method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.

Example 7

Referring to FIG. 9, a schematic structural diagram of an electronic device according to Embodiment 7 of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in FIG. 9, the electronic device can include a processor 702, a communications interface 704, a memory 706, and a communications bus 708.

among them:

Processor 702, communication interface 704, and memory 706 complete communication with one another via communication bus 708.

The communication interface 704 is configured to communicate with other terminal devices or servers.

The processor 702 is configured to execute the program 710, and specifically, perform the related steps in the foregoing data encapsulation method embodiment.

In particular, program 710 can include program code, the program code including computer operating instructions.

The processor 702 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the terminal device may be the same type of processor, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 706 is configured to store the program 710. Memory 706 may include high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The program 710 may be specifically configured to cause the processor 702 to perform an operation of: setting an algorithm registration interface for registering an AR algorithm and an AR service interface for performing AR interaction with a webpage based on a webpage view component; setting based on the webpage view component And a channel for communicating between the algorithm interface of the AR algorithm and the AR service interface; and registering the interface, the webpage view component, the AR service interface, and the channel according to the algorithm, and encapsulating the AR service callable file.

In an optional implementation, the program 710 is further configured to enable the processor 702 to set an image device interface for controlling the image collection device based on the web page view component; and/or to set an image for acquiring image data. Obtaining an interface; the program 710 is further configured to: when the processor 702 generates an AR service callable file according to the algorithm registration interface, the webpage view component, the AR service interface, and the channel, according to the image device interface and the At least one interface in the image acquisition interface, and the algorithm registration interface, the web page view component, the AR service interface, and the channel, and the package generates an AR service callable file.

In an optional implementation, the program 710 is further configured to cause the processor 702 to perform at least one interface according to the image device interface and the image acquisition interface, and the algorithm registration interface, a webpage view component, And the at least one interface of the image device interface and the image acquisition interface, and the algorithm registration interface, the webpage view component, and the AR service interface, when the AR service interface and the channel are configured to generate an AR service callable file Encapsulating with the channel to generate an AR service callable file; or setting a call interface for calling a web page view component, at least one interface of the image device interface and the image acquisition interface, and the algorithm The registration interface, the calling interface of the webpage view component, the AR service interface, and the channel are encapsulated to generate an AR service callable file.

In an optional implementation, the program 710 is further configured to enable the processor 702 to set a channel for communication between the algorithm interface of the AR algorithm based on the web page view component and the AR service interface. The AR service parameters received by the web page view component through the AR service interface are delivered to the communication channel of the AR algorithm interface that is successfully registered.

In an optional implementation, the program 710 is further configured to enable the processor 702 to set an image device interface for controlling the image collection device via the web page view component to respectively associate with the AR service interface and the AR algorithm. a communication channel between the algorithm interfaces; and/or a communication between the image acquisition interface for acquiring image data via the web page view component and the algorithm interface of the AR service interface and the AR algorithm, respectively aisle.

In an optional implementation, the program 710 is further configured to cause the processor 702 to set a query interface for acquiring algorithm information of the AR algorithm, and encapsulate the query interface into the AR service callable file.

In an optional implementation manner, the AR service callable file is an AR service callable component.

In an optional implementation manner, the AR algorithm includes multiple, and multiple AR algorithms are registered through the same interface class.

For a specific implementation of the steps in the program 710, reference may be made to the corresponding steps in the foregoing embodiment of the data encapsulation method and the corresponding description in the unit, and details are not described herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device and the module described above may be referred to the corresponding process description in the foregoing method embodiment, and details are not described herein again.

With the electronic device of the embodiment, the algorithm registration interface of the AR algorithm, the webpage view component that implements the webpage function, the AR service interface that implements the AR service, and the communication channel between the AR algorithm interface and the AR service interface are encapsulated and generated into an AR. Business callable files. The AR algorithm itself is independent of the AR service callable file, and only needs to register with the AR service callable file, and informs the AR service that the corresponding algorithm information of the file can be called, that is, the AR service callable file can be called. The AR service callable file does not need to understand the modification of the AR algorithm, or the specific implementation of adding or deleting, and only needs to obtain the corresponding algorithm information through the registration operation. In addition, the AR service end only needs to interact with the AR service callable file to implement the corresponding AR function. The AR service end does not need to embed the AR algorithm, and does not need to understand the specific implementation details of the AR algorithm. Therefore, when the new AR service is launched, the new AR service can be used only by making small changes to the front end such as the HTML page; the host application itself where the AR service can call the file itself may not have the AR algorithm, and The registered AR algorithm can implement the AR service of the host App itself; when replacing or adding the AR algorithm that does not belong to the host App itself, there is no need to change the host App itself; and, the same AR service can be easily implemented on different apps. Run at the same time. It can be seen that the distribution and deployment cost of the AR is greatly reduced by the solution of the embodiment of the present invention.

Example eight

Referring to FIG. 10, a schematic structural diagram of an electronic device according to Embodiment 7 of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in FIG. 10, the electronic device can include a processor 802, a communications interface 804, a memory 806, and a communications bus 808.

among them:

Processor 802, communication interface 804, and memory 806 complete communication with one another via communication bus 808.

The communication interface 804 is configured to communicate with other terminal devices or servers.

The processor 802 is configured to execute the program 810, and specifically may perform the related steps in the foregoing data processing method embodiment.

In particular, program 810 can include program code, the program code including computer operating instructions.

The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the terminal device may be the same type of processor, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 806 is configured to store the program 810. Memory 806 may include high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The program 810 is specifically configured to enable the processor 802 to perform the following operations: receiving an AR service request, where the AR service request carries an AR service parameter, and the AR service parameter in the AR service request is sent to the AR service callable file, where The AR service callable file is generated according to the algorithm registration interface of the AR algorithm, the webpage view component, the AR service interface, and the communication channel between the AR algorithm interface and the AR service interface; the AR service callable file is used to process the AR service parameters, and Returning the processing result; performing AR service display according to the processing result.

In an optional implementation manner, the program 810 is further configured to: when the processor 802 processes the AR service parameter by using an AR service callable file, and returns a processing result, the AR service of the file can be invoked through the AR service. The interface receives the AR service parameter, and determines the matched AR algorithm according to the AR service parameter. The AR service parameter is sent to the AR algorithm interface of the matched AR algorithm for processing by the webpage view component, and is received by the AR service interface. The AR service data returned by the AR algorithm interface.

In an optional implementation, the AR service callable file is further encapsulated with an image device interface for controlling the image capture device, and/or encapsulated with an image acquisition interface for acquiring image data; the program 810 is further configured to enable After determining the matched AR algorithm according to the AR service parameter, the processor 802 invokes the image collection device to perform image collection through the image device interface of the AR service callable file; and/or, through the AR service, the image of the file can be called. Obtaining an interface to obtain a still image or a video frame image; the program 810 is further configured to cause the processor 802 to pass the webpage when the AR service parameter is sent by the webpage view component to the AR algorithm interface of the matched AR algorithm The view component sends the image acquired by the image acquisition device or acquired by the image acquisition interface, and the AR service parameter to the AR algorithm interface for processing.

In an optional implementation manner, the program 810 is further configured to enable the processor 802 to receive a registration request of the AR algorithm before receiving the AR service request, and register the AR algorithm by using an algorithm registration interface of the AR service callable file.

In an optional implementation manner, the AR service callable file is further encapsulated with a query interface for acquiring algorithm information of the AR algorithm; the program 810 is further configured to enable the processor 802 to receive the AR algorithm query before receiving the AR service request. The request returns the algorithm information of the AR algorithm through the query interface of the AR service callable file.

In an optional implementation manner, the AR service callable file is an AR service callable component.

For a specific implementation of the steps in the program 810, reference may be made to the corresponding steps in the foregoing data processing method embodiment and the corresponding description in the unit, and details are not described herein. A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device and the module described above may be referred to the corresponding process description in the foregoing method embodiment, and details are not described herein again.

With the electronic device of the embodiment, an algorithm encapsulation interface encapsulating the AR algorithm, a web page view component implementing the webpage function, an AR service interface implementing the AR service, and an AR service of the communication channel between the AR algorithm interface and the AR service interface are used. The file can be called to implement the AR service of the front end. The front end does not need to implement the AR algorithm, and does not need to understand the implementation details of the AR algorithm, which greatly reduces the AR distribution and deployment costs.

It should be noted that the various components/steps described in the embodiments of the present invention may be split into more components/steps according to the needs of the implementation, or two or more components/steps or partial operations of the components/steps may be combined into one. New components/steps to achieve the objectives of embodiments of the present invention.

The above method according to an embodiment of the present invention may be implemented in hardware, firmware, or implemented as software or computer code that may be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or implemented by The network downloads computer code originally stored in a remote recording medium or non-transitory machine readable medium and stored in a local recording medium so that the methods described herein can be stored using a general purpose computer, a dedicated processor or programmable Such software processing on a recording medium of dedicated hardware such as an ASIC or an FPGA. It will be understood that a computer, processor, microprocessor controller or programmable hardware includes storage components (eg, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is The methods described herein are implemented when the processor or hardware is accessed and executed. Moreover, when a general purpose computer accesses code for implementing the methods shown herein, the execution of the code transforms the general purpose computer into a special purpose computer for performing the methods illustrated herein.

Those of ordinary skill in the art will appreciate that the elements and method steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the embodiments of the invention.

The above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention, and those skilled in the art can also make various kinds without departing from the spirit and scope of the embodiments of the present invention. Variations and modifications, therefore, all equivalent technical solutions are also within the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims (31)

1. A data encapsulation method, including:

   An algorithm registration interface for registering an augmented reality AR algorithm and an AR service interface for performing AR interaction with a webpage are set based on a webpage view component;

   Setting a channel for communicating between the algorithm interface of the AR algorithm successfully registered based on the web page view component and the AR service interface;

   Encapsulating and generating an AR service callable file according to the webpage view component, the algorithm registration interface, the AR service interface, and the channel.
2. The method of claim 1 wherein

   The method further includes: setting an image device interface for controlling the image capture device based on the web page view component; and/or setting an image acquisition interface for acquiring image data;

   And generating, according to the webpage view component, the algorithm registration interface, the AR service interface, and the channel, an AR service callable file, including:

   And encapsulating an AR service callable file according to at least one of the image device interface and the image acquisition interface, and the webpage view component, the algorithm registration interface, the AR service interface, and the channel.
3. The method of claim 2, wherein at least one of the image device interface and the image acquisition interface, and the web page view component, the algorithm registration interface, the AR service interface, and the The channel, the package generates an AR service callable file, including:

   Encapsulating at least one of the image device interface and the image acquisition interface, and the web page view component, the algorithm registration interface, the AR service interface, and the channel to generate an AR service callable file ;or,

   Setting a call interface for invoking the webpage view component, at least one interface of the image device interface and the image acquisition interface, and a call interface of the webpage view component, the algorithm registration interface, the The AR service interface and the channel are encapsulated to generate an AR service callable file.
4. The method according to any one of claims 1-3, wherein the setting is based on a channel between the algorithm interface of the successfully registered AR algorithm and the AR service interface based on the web page view component, including:

   The AR service parameter received by the webpage view component through the AR service interface is set to be transmitted to the communication channel of the successfully registered AR algorithm interface.
5. The method of claim 4 wherein the method further comprises:

   Providing, by the web page view component, a communication channel between the image device interface for controlling the image capture device and the algorithm interface of the AR service interface and the AR algorithm; and/or

   And a communication channel between the image acquisition interface for acquiring image data via the web page view component and the algorithm interface of the AR service interface and the AR algorithm, respectively.
6. The method of any of claims 1-3, wherein the method further comprises:

   And a query interface for acquiring algorithm information of the AR algorithm, and encapsulating the query interface into the AR service callable file.
7. The method of any of claims 1-3, wherein the AR service callable file is an AR service callable component.
8. The method according to any one of claims 1 to 3, wherein the AR algorithm comprises a plurality of, and the plurality of the AR algorithms are registered by the same interface class.
9. A data processing method comprising:

   Receiving an augmented reality AR service request, where the AR service request carries an AR service parameter;

   Sending the AR service parameter in the AR service request to the AR service callable file, where the AR service callable file is registered according to the algorithm of the AR algorithm, the webpage view component, the AR service interface, the AR algorithm interface, and the AR service. Communication channel encapsulation between interfaces is generated;

   Processing, by the AR service callable file, the AR service parameter, and returning a processing result;

   The AR service is displayed according to the processing result.
10. The method according to claim 9, wherein the processing of the AR service parameters by the AR service callable file and returning the processing result comprises:

    Receiving, by the AR service interface of the AR service callable file, the AR service parameter, and determining a matched AR algorithm according to the AR service parameter;

    The AR service parameter is sent to the AR algorithm interface of the matched AR algorithm for processing by the webpage view component, and the AR service data returned by the AR algorithm interface is received by the AR service interface.
11. The method according to claim 10, wherein the AR service callable file is further encapsulated with an image device interface for controlling the image capture device, and/or encapsulated with an image acquisition interface for acquiring image data;

    After the determining the matched AR algorithm according to the AR service parameter, the method further includes:

    And acquiring, by the image device interface of the AR service callable file, an image collection device to perform image collection; and/or acquiring a still image or a video frame image by using an image acquisition interface of the AR service callable file;

    The processing, by the webpage view component, sending the AR service parameter to the AR algorithm interface of the matched AR algorithm, including: acquiring, by the webpage view component, an image acquired by the image acquisition device or acquired by the image acquisition interface, and The AR service parameters are sent to the AR algorithm interface for processing.
12. The method of any of claims 9-1, wherein before the receiving the augmented reality AR service request, the method further comprises:

    Receiving a registration request of the AR algorithm, registering the AR algorithm by using an algorithm registration interface of the AR service callable file.
13. The method according to any one of claims 9-11, wherein the AR service callable file is further encapsulated with a query interface for acquiring algorithm information of the AR algorithm;

    Before the receiving the augmented reality AR service request, the method further includes:

    The AR algorithm query request is received, and the algorithm information of the AR algorithm is returned through the query interface of the AR service callable file.
14. The method of any of claims 9-11, wherein the AR service callable file is an AR service callable component.
15. A data encapsulating device comprising:

    a first setting module, configured to: according to a webpage view component, an algorithm registration interface for registering an augmented reality AR algorithm and an AR service interface for performing AR interaction with a webpage;

    a second setting module, configured to set a channel for performing communication between the algorithm interface of the AR algorithm based on the webpage view component and the AR service interface;

    And an encapsulating module, configured to generate an AR service callable file according to the webpage view component, the algorithm registration interface, the AR service interface, and the channel.
16. The device according to claim 15, wherein

    The device further includes: a third setting module, configured to set an image device interface for controlling the image capturing device based on the webpage view component, and/or an image acquiring interface for acquiring image data;

    The encapsulating module is configured to: according to at least one interface of the image device interface and the image obtaining interface, and the webpage view component, the algorithm registration interface, the AR service interface, and the channel, and the encapsulation Generate an AR business callable file.
17. The apparatus according to claim 16, wherein the encapsulating module is configured to interface at least one of the image device interface and the image acquisition interface, and the webpage view component, the algorithm registration interface, Encapsulating the AR service interface and the channel to generate an AR service callable file; or setting a call interface for calling the webpage view component, and at least one of the image device interface and the image acquisition interface The interface, and the calling interface of the webpage view component, the algorithm registration interface, the AR service interface, and the channel are encapsulated to generate an AR service callable file.
18. The device according to any one of claims 15-17, wherein the second setting module is configured to set a communication of the AR service parameter received by the webpage view component through the AR service interface to the AR algorithm interface with successful registration. aisle.
19. The device according to any one of claims 15-17, wherein the second setting module is further configured to set an image device interface for controlling an image capturing device via the webpage view component and the a communication channel between the AR service interface and the algorithm interface of the AR algorithm; and/or an image acquisition interface for acquiring image data via the web page view component, respectively, and the AR service interface and the AR The communication channel between the algorithm's algorithm interfaces.
20. The device of any of claims 15-17, wherein the device further comprises:

    And a fourth setting module, configured to set a query interface for acquiring algorithm information of the AR algorithm, and encapsulate the query interface into the AR service callable file.
21. The apparatus of any of claims 15-17, wherein the AR service callable file is an AR service callable component.
22. The apparatus according to any one of claims 15-17, wherein the AR algorithm comprises a plurality of, and the plurality of the AR algorithms are registered by the same interface class.
23. A data processing device comprising:

    a receiving module, configured to receive an augmented reality AR service request, where the AR service request carries an AR service parameter;

    a sending module, configured to send an AR service parameter in the AR service request to an AR service callable file, where the AR service callable file is registered according to an algorithm of an AR algorithm, a webpage view component, an AR service interface, and an AR The communication channel encapsulation between the algorithm interface and the AR service interface is generated;

    a processing module, configured to process the AR service parameter by using the AR service callable file, and return a processing result;

    And a display module, configured to perform AR service display according to the processing result.
24. The apparatus according to claim 23, wherein the processing module is configured to receive the AR service parameter by using an AR service interface of the AR service callable file, and determine a matched AR algorithm according to the AR service parameter; The AR service parameter is sent to the AR algorithm interface of the matched AR algorithm for processing by the webpage view component, and the AR service data returned by the AR algorithm interface is received by the AR service interface.
25. The apparatus according to claim 24, wherein said AR service callable file is further encapsulated with an image device interface for controlling an image capture device, and/or encapsulated with an image acquisition interface for acquiring image data;

    The processing module is further configured to: after the determining the matched AR algorithm according to the AR service parameter, invoke an image collection device to perform image collection by using an image device interface of the AR service callable file; and/or Obtaining a static image or a video frame image by the image acquisition interface of the AR service callable file; and sending, by the webpage view component, the image acquired by the image acquisition device or the image acquisition interface, and the AR service parameter to the The AR algorithm interface is processed.
26. The device of any of claims 23-25, wherein the device further comprises:

    And a registration module, configured to receive a registration request of the AR algorithm before the receiving module receives the augmented reality AR service request, and register the AR algorithm by using an algorithm registration interface of the AR service callable file.
27. The apparatus according to any one of claims 23-25, wherein the AR service callable file is further encapsulated with a query interface for acquiring algorithm information of the AR algorithm;

    The device further includes: a querying module, configured to: before the receiving module receives the augmented reality AR service request, receive an AR algorithm query request, and return the algorithm information of the AR algorithm by using the query interface of the AR service callable file.
28. The apparatus of any of claims 23-25, wherein the AR service callable file is an AR service callable component.
29. An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus;

    The memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the data encapsulation method of any of claims 1-8.
30. An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus;

    The memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the data processing method of any of claims 9-14.
31. A non-transitory machine readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to perform as claimed in claims 1-8, 9-14 The method of any of the preceding claims.

Images