CN112631152A

CN112631152A - AR-based application program starting method, device, equipment and storage medium

Info

Publication number: CN112631152A
Application number: CN202011319468.9A
Authority: CN
Inventors: 张墨; 杨晨炜; 王莉媛
Original assignee: Hangzhou Yixian Advanced Technology Co ltd
Current assignee: Hangzhou Yixian Advanced Technology Co ltd
Priority date: 2020-11-23
Filing date: 2020-11-23
Publication date: 2021-04-09
Anticipated expiration: 2040-11-23
Also published as: CN112631152B

Abstract

The invention relates to an AR-based application program starting method, an AR-based application program starting device, electronic equipment and a storage medium, wherein the method comprises the following steps: the AR terminal device acquires a controller list and continuously detects whether controllers in the controller list exist in a preset sensing region, wherein the controllers comprise objects or/and pictures of the objects; in the event a controller is detected, one or more applications bound to the controller are launched. Through the application, the AR terminal equipment is started to detect each controller in the control list in the whole process during operation, and the AR application program bound with the AR terminal equipment can be quickly started once the controller is detected, so that the efficiency of starting the application program is improved, and the problem of low efficiency in starting the target APP based on the AR terminal equipment is solved.

Description

AR-based application program starting method, device, equipment and storage medium

Technical Field

The invention relates to the technical field of augmented reality, in particular to an AR-based application program starting method, device, equipment and storage medium.

Background

Augmented Reality (Augmented Reality) technology is a technology that skillfully fuses virtual information with the real world. Common AR terminal devices (AR glasses, AR projectors) each include a series of environment detection sensors such as a depth camera and a color camera, and recognize gestures, 3D objects, planes, images, and the like, and register virtual information (characters, images, three-dimensional models, video sounds, and the like) at a target position in the real world. In the related art, based on the AR terminal device, a user often needs to search a desired target APP from a plurality of different applications (APPs, abbreviated as APPs) through a gesture or a remote control handle to register the target APP to a target location, but when APP data is excessive, browsing and searching through the gesture or the remote control handle becomes cumbersome and time-consuming, resulting in low efficiency of starting the target APP.

Aiming at the problem that starting a target APP based on AR terminal equipment in the related technology has low efficiency, an effective solution is not provided at present.

Disclosure of Invention

The embodiment of the application provides an AR-based application program starting method, an AR-based application program starting device, AR-based application program starting equipment and a storage medium, and aims to at least solve the problem that starting of a target APP based on AR terminal equipment in the related art is low in efficiency.

In a first aspect, an embodiment of the present application provides an AR-based application starting method, where the method includes:

the AR terminal device acquires a controller list and continuously detects whether controllers in the controller list exist in a preset sensing region, wherein the controllers comprise objects or/and pictures of the objects;

in the event that the controller is detected, one or more applications bound with the controller are launched.

In some embodiments, before the obtaining the list of controllers, the method further comprises:

installing a plurality of application programs, copying controller detection data from an installation package of each application program into the AR terminal equipment, wherein the controller detection data comprises an initial binding relationship between the application programs and an initial controller;

and generating a first controller list according to the initial controllers having the initial binding relation with the application programs.

In some embodiments, after generating the first controller list according to the initial controllers having the initial binding relationship with each of the application programs, the method further includes:

receiving a first controller configuration modification instruction, wherein the first controller configuration modification instruction comprises a first binding relationship between a first application program and a first controller;

and updating the initial controller which has the initial binding relationship with the first application program in the first controller list to the first controller according to the first controller configuration modification instruction to obtain a second controller list.

receiving a second controller configuration modification instruction, wherein the second controller configuration modification instruction comprises a binding relationship between a second application program and a second controller;

and adding the second controller into the first controller list to obtain a third controller list under the condition that no initial controller having an initial binding relationship with the second application program exists in the first controller list.

In some embodiments, before the continuously detecting whether there are controllers in the controller list in the preset sensing region in the case that each controller in the controller list is preset to be prohibited from detecting, the method further includes:

and receiving a first controller starting instruction, and starting continuous detection on the first controller according to the first controller starting instruction.

In some embodiments, after the turning on of the continuous detection of the first controller according to the first controller turn-on instruction, the method further comprises:

and detecting whether the running of the CPU on the AR terminal equipment exceeds a preset load threshold value, if so, forbidding to start continuous detection on the second controller under the condition of receiving a second controller starting instruction.

In a second aspect, an embodiment of the present application provides an AR-based application starting apparatus, where the apparatus includes a detection module and a starting module;

the detection module is used for acquiring a controller list and continuously detecting whether controllers in the controller list exist in a preset sensing region, wherein the controllers comprise real objects or/and pictures of the real objects;

the starting module is used for starting one or more application programs bound with the controller under the condition that the controller is detected.

In some embodiments, the apparatus further includes a list generation module, where the list generation module is configured to install a plurality of applications, and copy controller detection data from an installation package of each of the applications to the AR terminal device, where the controller detection data includes an initial binding relationship between an application and an initial controller; and adding the initial controllers having the initial binding relation with the application programs into the controller list to obtain a first controller list.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor, when executing the computer program, implements the AR-based application starting method according to the first aspect.

In a fourth aspect, embodiments of the present application provide a computer storage medium, on which a computer program is stored, which when executed by a processor, implements the AR-based application launching method as described in the first aspect above.

Compared with the related art, the AR-based application starting method provided by the embodiment of the application obtains the controller list through the AR terminal device, and continuously detects whether the controller in the controller list exists in the preset sensing region, wherein the controller includes an object or/and a picture of the object; under the condition that the controller is detected, one or more application programs bound with the controller are started, the AR terminal equipment is enabled to start detection on each controller in the control list in the whole running process, and the AR application programs bound with the controller can be quickly started once the controller is detected, so that the efficiency of starting the application programs is improved, and the problem of low efficiency in starting a target APP based on the AR terminal equipment in the related technology is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of an application environment of an AR-based application launching method according to an embodiment of the present application;

FIG. 2 is a flow chart of an AR based application launching method according to an embodiment of the application;

FIG. 3 is a flowchart illustrating an AR-based application launching method according to a preferred embodiment of the present application;

FIG. 4 is a flow chart diagram illustration of a controller list update method according to a preferred embodiment of the present application;

FIG. 5 is a schematic diagram of an AR-based application launching device according to an embodiment of the present application;

fig. 6 is an internal structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.

It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.

Fig. 1 is a schematic application environment diagram of an AR-based application starting method according to an embodiment of the present application, and the AR-based application starting method provided in the present application may be applied to the application environment shown in fig. 1. The AR terminal device 10 includes an environment detection sensor 11 and a projection device 12. The AR terminal device 10 is not limited to one of AR glasses and AR projectors. The environment detection sensor 11 may be one or more of a depth camera, a color camera, and an infrared camera. Projection device 12 may be a structured light projection device. The environment detection sensor 11 acquires a controller list and continuously detects whether or not there is a controller in the controller list within a preset sensing region. The controller comprises the object or/and a picture of the object. The sensing area includes a projection plane of the projection device 12 at a target location such as a desktop, wall, or ground, and an area between the projection plane and the environment detection sensor. In the event a controller is detected, projection device 12 launches one or more applications bound to the controller and registers virtual information for the one or more applications with the target location. When detecting the object and/or the image of the object as the controller, the AR terminal device 10 can quickly start the AR application program bound with the AR terminal device, and does not need to browse and search, so that the starting efficiency of the application program can be improved, and the problem of low efficiency in starting the target APP based on the AR terminal device in the related art is solved.

The embodiment provides an AR-based application program starting method. Fig. 2 is a flowchart of an AR-based application starting method according to an embodiment of the present application, where the flowchart includes the following steps, as shown in fig. 2:

s210, the AR terminal device obtains the controller list and continuously detects whether the controllers in the controller list exist in a preset sensing area. The controller list includes a plurality of controllers, which may be objects or/and pictures of objects. The object can be a 3D object with high user frequency such as a cooker and tableware, namely a 3D object used by a user, and the picture of the object can be obtained by photographing the 3D object and can also be a schematic picture representing the 3D object. After the AR terminal device is started and the controller list is obtained, algorithm data for identifying each controller in the controller list is operated in the whole operation process. The sensing area includes a projection plane of a target position such as a desktop, a wall surface, or a ground surface, and an area between the projection plane and the AR terminal device.

And S220, starting one or more application programs bound with the controller under the condition that the controller is detected. When the AR terminal device detects that a certain controller in the controller list enters the sensing area, the application program bound with the controller can be started successfully. For example, the pan is a controller in the controller list, the application program bound to the pan is a cooking APP or a plurality of different cooking APPs, and when the AR terminal device detects the pan in the sensing area, the AR terminal device quickly starts the cooking APP or the plurality of different cooking APPs.

Through steps S210 to S220, a user does not need to spend more time browsing and searching for restarting through gestures or a remote control handle, the AR terminal device starts detection on each controller in the control list in the whole process of operation, and the AR application program bound with the controller can be quickly started once the controller is detected, so that the efficiency of starting the application program is improved, and the problem of low efficiency in starting a target APP based on the AR terminal device in the related technology is solved.

In some embodiments, before the AR terminal device obtains the controller list, when a plurality of applications are installed, controller detection data (i.e., APP internal algorithm data) is copied from an installation package of each application into the AR terminal device. The controller detection data includes an initial binding relationship between the application and the initial controller. And generating a first controller list according to the initial controllers having the initial binding relation with the application programs. By copying the APP internal algorithm data into the AR terminal device, the application level algorithm can run in the AR terminal device in a system level mode, under the condition that an initial controller is not bound in advance to a certain application program in the controller detection data, the controller bound with the application program is flexibly set according to the use frequency of the user to the object, the applicability of starting the application program by the AR terminal device according to various objects is improved, and user experience can be improved.

In some embodiments, after the first controller list is generated, when the AR terminal device receives the first controller configuration modification instruction, according to the first binding relationship between the first application program and the first controller in the first controller configuration modification instruction, the initial controller in the first controller list, which has the initial binding relationship with the first application program, is updated to the first controller, so as to obtain the second controller list. The first controller is a 3D real object with high user use frequency. If the AR terminal device does not receive a controller configuration modification instruction that the user needs to modify the controller configuration, the first controller list is not updated. And updating the controller list according to the configuration modification instruction of the controller, so that the personalized requirements of the user can be met, and the user experience is improved.

In some embodiments, after the first controller list is generated, when the AR terminal device receives the second controller configuration modification instruction, and in a case that there is no initial controller having an initial binding relationship with the second application program in the first controller list, the AR terminal device adds the second controller to the first controller list according to the binding relationship between the second application program and the second controller in the second controller configuration modification instruction, to obtain the third controller list. When the second application program is not bound with the controller, the controller bound with the second application program is flexibly set according to the use frequency of the object by the user, so that the efficiency of starting the second application program can be improved, the personalized requirements of the user can be met, and the user experience is improved.

In some embodiments, in a case where each controller in the controller list is preset to be prohibited from being detected, the AR terminal device receives a first controller-on instruction before continuously detecting whether there is a controller in the controller list within a preset sensing region, and starts continuous detection of the first controller according to the first controller-on instruction. The controller starting instruction comprises an instruction for starting detection of a first controller in the controller list, the first controller is started and detected in the whole running process of the AR terminal device according to the instruction, other controllers preset to be forbidden to detect are not detected, and the condition that the CPU is overloaded due to excessive algorithm detection can be avoided. Preferably, when the AR terminal device receives the start instruction of the other controller, the AR terminal device may start continuous detection on the other corresponding controller according to the start instruction of the other controller, so as to meet the personalized requirement of the user and improve the user experience.

In some embodiments, after the continuous detection of the first controller is started according to the first controller start instruction, it is detected whether the operation of the CPU on the AR terminal device exceeds a preset load threshold, and if so, the continuous detection of the second controller is prohibited from being started under the condition that the second controller start instruction is received, so that the condition that the CPU is overloaded due to excessive algorithm detection can be avoided. It should be noted that when the CPU operation exceeds the preset load threshold, the continuous detection of the other controllers is not started again when the other controller start instruction is received.

The embodiments of the present application are described and illustrated below by means of preferred embodiments. Fig. 3 is a flowchart illustrating an AR-based application launching method according to a preferred embodiment of the present application. As shown in fig. 3, after the AR terminal device is powered on, the controller list is acquired, then the controller background service is started, the detection of the controllers in the controller list is started in the whole process, and whether the controllers are detected is determined. And under the condition that the controller is detected, starting the APP bound with the controller, and under the condition that the controller is not detected, continuing to start the detection of the controller in the controller list in the whole course.

Preferably, fig. 4 is a flowchart illustration of a controller list update method according to a preferred embodiment of the present application. As shown in fig. 4, the AR terminal device installs an application program and automatically copies algorithm data (i.e., controller detection data) to the AR terminal device, and determines whether a user modifies the controller configuration according to the controller configuration modification instruction, if so, updates the controller list, and if not, does not update the controller list.

The embodiment of the application provides an AR-based application program starting device. Fig. 5 is a schematic structural diagram of an AR-based application launching device according to an embodiment of the present application, and as shown in fig. 5, the device includes a detection module 510 and a launching module 520: the detection module 510 is configured to acquire a controller list, and continuously detect whether there are controllers in the controller list in a preset sensing region, where the controllers include a real object or/and a picture of the real object; the launch module 520 is configured to launch one or more applications bound to the controller if the controller is detected.

Preferably, the AR-based application starting apparatus further includes a list generating module, where the list generating module is configured to install a plurality of applications, and copy controller detection data from an installation package of each application to the AR terminal device, where the controller detection data includes an initial binding relationship between the application and the initial controller; and adding the initial controllers having the initial binding relation with the application programs into the controller list to obtain a first controller list.

For specific limitations of the AR-based application launching device, reference may be made to the above limitations of the AR-based application launching method, which are not described herein again. The respective modules in the AR-based application launching device described above may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

An embodiment of the present application further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the method embodiments described above.

Optionally, the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.

It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.

In addition, in combination with the AR-based application starting method in the foregoing embodiments, the embodiments of the present application may provide a storage medium to implement. The storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements any of the AR-based application launching methods in the above embodiments.

In an embodiment, fig. 6 is a schematic internal structure diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 6, there is provided an electronic device, which may be a server, and its internal structure diagram may be as shown in fig. 6. The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the electronic device is used for storing data. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement an AR-based application launching method.

Those skilled in the art will appreciate that the configuration shown in fig. 6 is a block diagram of only a portion of the configuration associated with the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or may combine certain components, or have a different arrangement of components.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It should be understood by those skilled in the art that various features of the above-described embodiments can be combined in any combination, and for the sake of brevity, all possible combinations of features in the above-described embodiments are not described in detail, but rather, all combinations of features which are not inconsistent with each other should be construed as being within the scope of the present disclosure.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. An AR-based application launching method, the method comprising:

2. The method of claim 1, wherein prior to obtaining the list of controllers, the method further comprises:

3. The method of claim 2, wherein after generating the first list of controllers from the initial controllers having the initial binding relationship with each of the applications, the method further comprises:

4. The method of claim 2, wherein after generating the first list of controllers from the initial controllers having the initial binding relationship with each of the applications, the method further comprises:

5. The method of claim 1, wherein in the event that each controller in the list of controllers is preset to inhibit detection, the method further comprises, prior to continuously detecting whether there is a controller in the list of controllers within a preset sensing region:

6. The method of claim 5, wherein after initiating the continuous detection of the first controller in accordance with the first controller activation command, the method further comprises:

7. An AR-based application launching device, characterized in that the device comprises a detection module and a launching module;

8. The apparatus of claim 7, further comprising a list generation module, configured to install a plurality of applications, and copy controller detection data from an installation package of each of the applications into the AR terminal device, wherein the controller detection data includes an initial binding relationship between an application and an initial controller; and adding the initial controllers having the initial binding relation with the application programs into the controller list to obtain a first controller list.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor, when executing the computer program, implements the AR-based application launching method as recited in any of claims 1 to 6.

10. A computer storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the AR-based application launching method as claimed in any one of claims 1 to 6.