WO2022041155A1 - Application program identification device and electronic device - Google Patents

Abstract

Disclosed in the present application are an application program identification device and an electronic device, comprising: a main processor, used for receiving a first command triggered by a user, the first command being used for instructing a first image and a first application program to be associated; a co-processor, used for performing identification on the first image to obtain first image features, and for sending to the main processor first information used for indicating the first image features; the main processor is also used for associating the first image features and the first application program when determining that a database does not comprise the first information, and storing the first information and the first application program in the database; the database comprises information used for indicating at least one image feature and the application program associated with each image feature. Thus, the electronic device can identify an application program to be activated on the basis of image features, simplifying the user operation procedure and increasing processing efficiency.

Description

An application identification device and electronic device technical field

The present application relates to the technical field of electronic devices, and in particular, to an application identification device and an electronic device.

Background technique

With the development of smart electronic device technology, the number and functions of applications installed on smart electronic devices, such as mobile terminals, are increasing. Smart mobile terminals have been integrated into all aspects of daily life and can meet various needs of users. For example, users can use shared bicycles through mobile terminals, make payments through mobile terminals, and take public transportation through mobile terminals.

However, with the development of mobile terminal technology and application development technology, users have also put forward higher requirements for the convenience of using applications in mobile terminals. At present, if a user wants to use an application in a mobile terminal, he generally needs to find the icon of the application he wants to use on the main interface or application interface of the mobile terminal and click on it. When there are a large number of applications installed in the mobile terminal , the process of finding the application's icon may take a long time. In order to simplify the user operation, virtual keys can also be set, and the user can quickly start the application by touching the virtual keys, but the display interface of the mobile terminal may be messy; or, the corresponding touch track can also be set for the application in advance, as shown in the figure As shown in 1, the memory in the mobile terminal stores the preset correspondence between the touch track and the application program. After the user inputs the touch track, the touch track input by the user and the touch track in the above-mentioned corresponding relationship are sequentially matched. If successful, the corresponding application program is activated, but the user needs to remember the touch track corresponding to each application program, which increases the user's memory burden. In addition, in the above two methods, fingerprint recognition can also be added, that is, when the user uses different fingers to perform the same operation, they can correspond to different applications; however, in some scenarios, the user's fingers may be inconvenient. When the mobile terminal operates, for example, when the weather is cold and the user wears gloves, the user needs to take off the gloves first, or when the user's fingers are too wet, too dry or stick to objects, the fingerprint may not be recognized. Although the prior art proposes a solution for entering an application program through an image recognition technology, the types of images that can be recognized are limited, and the types of entering or opening an application program are limited, which reduces user experience.

SUMMARY OF THE INVENTION

The present application provides an identification device and electronic device for an application program, which supports users to customize an application program that can be activated based on an image recognition technology, simplifies the user operation process, and improves the user experience.

In a first aspect, an embodiment of the present application provides an application identification device, including: a main processor configured to receive a first instruction triggered by a user, where the first instruction is used to instruct a first image to be associated with a first associated with an application; a coprocessor, configured to identify the first image to obtain a first image feature, and send first information to the main processor for indicating the first image feature; the The main processor is further configured to associate the first image feature with the first application when it is determined that the first information is not included in the database, and save the first information and all the first information in the database the first application; wherein the database includes information for indicating at least one image feature and an application associated with each image feature. The above solutions support user customization to establish the relationship between image features and application programs. In subsequent image recognition, if an image feature associated with an application program is identified, the recognition device can automatically start the application program associated with the image feature, which not only simplifies the user's operation process, but also expands the activation process based on image recognition technology. type of application.

In a possible implementation manner, the first information includes a first image feature and/or an identification of the first image feature.

In a possible implementation manner, a signal processing device is further included for processing the first data from the image sensor to obtain the first image.

In a possible implementation manner, the first image is an image specified by a user in at least one preset image.

In a possible implementation manner, the signal processing device is further configured to process the second data from the image sensor to obtain a second image; the coprocessor is further configured to process the second image The image is identified to obtain a second image feature, and when it is determined that the database includes second information for indicating the second image feature, the second information is sent to the main processor. The main processor is further configured to determine the application program associated with the second image feature in the database as the target application program to be activated.

In a possible implementation manner, the coprocessor is further configured to, when the database does not include the second information used to indicate the characteristics of the second image, perform the processing on the second image or the first image Semantic analysis is performed on the two image features to obtain a semantic label; wherein, the semantic label includes text information used to characterize the second image or the second image feature; the main processor is also used for the slave The coprocessor receives the semantic tag, indicates to the user at least one application program associated with the semantic tag, and receives a second instruction triggered by the user, the second instruction is used to instruct the activation of the at least one application program. the second application, and determining that the second application is the target application to be activated.

In a possible implementation manner, after receiving the identifier of the target application from the coprocessor, the main processor is further configured to: determine whether the electronic device where the device is deployed has the target application installed target application; when the target application is installed in the electronic device, activate the target application; or, when the target application is not installed in the electronic device but a third application is installed , activate the target application program through the target application program interface built in the third application program; or, when the target application program is not installed in the terminal, prompt the user to download the target application program.

In a possible implementation manner, the signal processing device is specifically configured to process the second data from the image sensor periodically or in real time to obtain the second image.

In a possible implementation, the signal processing device includes at least one of an image signal processor ISP or a sensor processor.

In a possible implementation manner, the coprocessor is specifically configured to recognize the first image or the second image by using a small sample image recognition algorithm.

In a possible implementation manner, the coprocessor is at least one of a neural network processor, a digital signal processor, an image processing unit or a hardware accelerator.

In a second aspect, an embodiment of the present application provides a device for identifying an application, including: a main processing module configured to receive a user-triggered first instruction, where the first instruction is used to instruct a first image to be associated with a first application association; a co-processing module for identifying the first image to obtain a first image feature, and sending first information for indicating the first image feature to the main processing module; the main processing module , and is also used to associate the first image feature with the first application when it is determined that the first information is not included in the database, and save the first information and the first application in the database An application program; wherein the database includes information indicating at least one image feature and an application program associated with each image feature. The above solutions support user customization to establish the relationship between image features and application programs. In subsequent image recognition, if an image feature associated with an application program is identified, the recognition device can automatically start the application program associated with the image feature, which not only simplifies the user's operation process, but also expands the activation process based on image recognition technology. type of application.

In a third aspect, embodiments of the present application provide an electronic device, where the electronic device includes an image sensor and the device according to the first aspect and any possible implementation manner of the first aspect. Among them, the image sensor is used to collect data periodically or in real time.

In a fourth aspect, an embodiment of the present application further provides a computer storage medium, where a software program is stored in the storage medium, and when at least a first part of the software program is read and executed by a coprocessor, the first aspect or the above-mentioned first aspect can be implemented. The function performed by the coprocessor in any one of the designs of the one aspect; or, at least the second part of the software program, when read and executed by the main processor, can implement the first aspect or any one of the above-mentioned first aspects The function performed by the main processor in the implementation.

In a fifth aspect, the embodiments of the present application further provide a computer program product, when at least the first part runs on the coprocessor, the coprocessor executes the design of the first aspect or any one of the above-mentioned first aspects. The function performed by the coprocessor; or, when at least the second part of which runs on the main processor, causing the main processor to perform as performed by the main processor in the first aspect or any implementation manner of the above-mentioned first aspect function.

Description of drawings

Fig. 1 is a kind of schematic flow chart of activating application program;

FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a device for an application program provided by an embodiment of the present application;

FIG. 4 is one of schematic diagrams of an interactive interface provided by an embodiment of the present application;

FIG. 5 is the second schematic diagram of the interactive interface provided by the embodiment of the present application;

FIG. 6a is a third schematic diagram of an interactive interface provided by an embodiment of the present application;

FIG. 6b is the fourth schematic diagram of the interactive interface provided by the embodiment of the application;

FIG. 7a is a fifth schematic diagram of an interactive interface provided by an embodiment of the present application;

FIG. 7b is the sixth schematic diagram of the interactive interface provided by the embodiment of the application;

FIG. 8 is a schematic flowchart of a registration image provided by an embodiment of the present application;

9 is a schematic flowchart of an AI image recognition technology provided by an embodiment of the present application;

10 is a schematic flowchart of a small sample image recognition technology provided by an embodiment of the present application;

FIG. 11a is one of the captured images provided by the embodiment of the present application;

FIG. 11b is a schematic diagram of an application program activated corresponding to FIG. 11a provided by an embodiment of the present application;

Fig. 12a is the second captured image provided by this embodiment of the application;

FIG. 12b is a schematic diagram of an application program activated corresponding to FIG. 12a provided by an embodiment of the present application;

FIG. 12c is a schematic diagram of an application program page activated corresponding to FIG. 12a according to an embodiment of the present application;

Fig. 13a is the third acquisition image provided by the embodiment of the present application;

FIG. 13b is a schematic diagram of an application program related to FIG. 13a provided by an embodiment of the present application;

Fig. 14a is the fourth acquisition image provided by this embodiment of the application;

FIG. 14b is a schematic diagram of an application page activated corresponding to FIG. 14a provided by an embodiment of the present application;

Fig. 15a is the fourth acquisition image provided by this embodiment of the application;

FIG. 15b is a schematic diagram of an application page activated corresponding to FIG. 15a provided by an embodiment of the present application;

FIG. 16 is a schematic structural diagram of another application device provided by an embodiment of the present application.

detailed description

The embodiments of the present application will be described in further detail below with reference to the accompanying drawings. Various applications installed in electronic devices can provide users with convenience in different aspects and meet different needs of users in different scenarios. For example, bicycle-sharing applications allow users to conveniently use public bicycles; payment applications allow users to make payments through electronic devices without having to carry cash or bank cards; public transport inquiry applications, It is convenient for users to check bus routes and operating conditions.

Since the process of opening an application program in the prior art is relatively complicated, the traditional method of activating an application program still has some limitations. In order to simplify user operations, improve user experience, and improve processing efficiency, embodiments of the present application provide an application identification device and an electronic device. The identification device can be a circuit board, a circuit system, a chip or a chipset in an electronic device, and can run necessary software and be used to perform operations related to application activation, the software includes but is not limited to driver software, operating system software and application software. The electronic device is a movable electronic device, such as a mobile terminal such as a mobile phone and a tablet, and may optionally include or not include a communication capability, and the communication capability may be mobile communication or short-range communication. When the electronic device has no communication capability, it is equivalent to a movable local device. The electronic device needs to have a shooting capability, such as a built-in camera, that is, a sensor lens used for shooting pictures. The electronic device includes, but is not limited to, a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a laptop computer or a wearable device.

Taking the electronic device as a mobile phone as an example, FIG. 2 shows a possible partial structure of the mobile phone 200 . As shown, cell phone 200 includes: radio frequency (RF) circuitry 210, memory 220, other input devices 230, touch display screen 240, sensors 250, audio circuitry 260, I/O subsystem 270, processor 280, and Power 290 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 2 does not constitute a limitation on the mobile phone, and may include more or less components than the one shown in the figure, or combine some components, or disassemble some components, or Different component arrangements. Below in conjunction with FIG. 2, each constituent component of the mobile phone 200 will be specifically introduced:

The RF circuit 210 can be used for receiving and sending signals during sending and receiving of information or during a call. In particular, after receiving the downlink information of the base station, it is processed by the processor 280; in addition, the designed uplink data is sent to the base station. Typically, RF circuits include, but are not limited to, antennas, at least one amplifier, transceivers, couplers, low noise amplifiers (LNAs), duplexers, and the like. In addition, the RF circuit 210 may also communicate with the network and other devices via wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (code division multiple access) division multiple access, CDMA), wideband code division multiple access (WCDMA), long term evolution (long term evolution, LTE), email, short message service (short messaging service, SMS), etc.

The memory 220 may be used to store software programs and modules, and the processor 280 executes various functional applications and data processing of the mobile phone 200 by running the software programs and modules stored in the memory 220 . The memory 220 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function, and the like; The use of the cell phone 200 creates data (such as audio data, phone book, etc.) and the like. Additionally, memory 220 may include power-down volatile memory or non-power-down volatile memory. The memory 220 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of static storage device that can store information and instructions The dynamic storage device can also be an electrically erasable programmable read-only memory (EEPROM) or can be used to carry or store the desired program code in the form of instructions or data structures and can be stored by a computer. any other medium taken, but not limited to this.

Other input devices 230 may be used to receive input numerical or character information, and to generate key signal input related to user settings and function control of the cell phone 200 . Specifically, other input devices 230 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), operation sticks, and the like. In some embodiments of the present invention, other input devices may also include an image sensor (camera) for capturing images. The other input device 230 is connected to the other input device controller 271 of the I/O subsystem 270 , and performs signal interaction with the processor 280 under the control of the other input device controller 271 .

The touch display screen 240 may be used to display information input by or provided to the user and various menus of the mobile phone 200, and may also accept user input. The specific touch display screen 240 may include a display panel 241 and a touch panel 242 . The display panel 241 may be configured in the form of a liquid crystal display (liquid crystal display, LCD), an organic light emitting diode (organic light emitting diode, OLED) or the like. The touch panel 242, also known as a touch screen, touch sensitive screen, etc., can collect the user's contact or non-contact operations on or near it (such as the user using a finger, stylus, etc., any suitable object or accessory on the touch panel 242) Or the operation near the touch panel 242 may also include a somatosensory operation; the operation includes single-point control operation, multi-point control operation and other operation types), and the corresponding connection device is driven according to a preset program. Optionally, the touch panel 242 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into a processor capable of The processed information is then sent to the processor 280, and the commands sent by the processor 280 can be received and executed. In addition, the touch panel 242 can be realized by various types such as resistive, capacitive, infrared, and surface acoustic wave, and any technology developed in the future can also be used to realize the touch panel 242 . Further, the touch panel 242 can cover the display panel 241, and the user can display the content on the display panel 241 according to the content displayed on the display panel 241 (the display content includes, but not limited to, a soft keyboard, a virtual mouse, virtual keys, icons, etc.) on the display panel 241 The operation is performed on or near the covered touch panel 242. After the touch panel 242 detects the operation on or near it, the operation is transmitted to the processor 280 through the I/O subsystem 270 to determine the user input, and then the processor 280 according to the user Inputs provide corresponding visual outputs on display panel 241 through I/O subsystem 280 . Although in FIG. 2, the touch panel 242 and the display panel 241 are used as two independent components to realize the input and input functions of the mobile phone 200, in some embodiments, the touch panel 242 and the display panel 241 may be integrated And realize the input and output functions of the mobile phone 200 .

The cell phone 200 may also include at least one sensor 250, such as an acceleration sensor 251, a light sensor, and other sensors. Specifically, the acceleration sensor 251 can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometers) (posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the touch display screen 240 according to the brightness of the ambient light, The proximity sensor can turn off the screen and/or the backlight when the phone 200 is moved to the ear. As for other sensors, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., that can be configured in the mobile phone 200, the details are not repeated here.

The audio circuit 260 , the speaker 261 , and the microphone 262 can provide an audio interface between the user and the mobile phone 200 . The audio circuit 260 can transmit the received audio data converted signal to the speaker 261, and the speaker 261 converts it into a sound signal for output; on the other hand, the microphone 262 converts the collected sound signal into a signal, which is received by the audio circuit 260. Converted to audio data, the audio data is output to the RF circuit 210 for transmission to, for example, another mobile phone, or to the memory 220 for further processing.

The I/O subsystem 270 is used to control input and output external devices, which may include other input device controllers 271 , sensor controllers 272 , and display controllers 273 . Optionally, one or more other input device controllers 271 receive signals from and/or send signals to other input devices 230, which may include physical buttons (press buttons, rocker buttons, etc.), Dials, slide switches, cameras, and more. It should be noted that the other input device controller 271 may be connected to any one or more of the above-mentioned devices. The display controller 273 in the I/O subsystem 270 receives signals from and/or sends signals to the touch display screen 240 . After the touch display screen 240 detects the user input, the display controller 273 converts the detected user input into interaction with the user interface objects displayed on the touch display screen 240, that is, to realize human-computer interaction. Sensor controller 270 may receive signals from and/or send signals to one or more sensors 250 .

The processor 280 is used as the control center of the mobile phone 200, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the memory 220, and calling the software programs stored in the memory 220. Data, perform various functions of the mobile phone 200 and process data, so as to monitor the mobile phone as a whole. Optionally, the processor 280 may include one or more processing units, or one or more processors; preferably, the processor 280 may integrate an application processor and a modem processor, wherein the application processor mainly processes operations System, user interface and application programs, etc., the modem processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 280. The processor 280 may further include the processing functional units mentioned in the subsequent embodiments.

The mobile phone 200 also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 280 through a power management system, so as to manage charging, discharging, and power consumption functions through the power management system. Although not shown, the mobile phone 200 may also include other modules such as a Bluetooth module, which will not be repeated here.

A schematic structural diagram of an application identification device 300 provided by an embodiment of the present application may be as shown in FIG. 3 , including a main processor 301 and a coprocessor 302 .

The main processor 301 is configured to receive a first instruction triggered by a user, where the first instruction is used to instruct to associate the first image with the first application program.

In an optional implementation manner, the above-mentioned device 300 further includes a signal processing device 303; the signal processing device 303 is configured to process the first data from the image sensor to obtain the first image. Optionally, a user may trigger an image sensor in the electronic device to collect the first data. In specific implementation, before receiving the first instruction triggered by the user, the main processor 301 is further configured to receive a shooting instruction triggered by the user, and the shooting instruction is used to instruct to shoot the first image and control the image sensor in the electronic device to capture The first data is transmitted to the signal processing device 303, and the signal processing device 303 processes the first data to obtain a first image. In another optional implementation manner, the first image is an image designated (selected) by the user in at least one preset image. Specifically, the first image may be image data pre-stored by the user in the electronic device where the device 300 is located, for example, an image selected in a gallery on the electronic device. Optionally, the aforementioned first image may be one or more images, for example, the user can trigger shooting of multiple images from different angles of the same object, or select one or more images from the gallery on the electronic device, for this purpose. Not limited.

The coprocessor 302 is configured to identify the first image to obtain a first image feature, and send first information indicating the first image feature to the main processor 301 . Wherein, the first information includes the first image feature and/or the identifier of the first image feature. The first image features include the target object in the first image, and related parameters such as color, size, and shape of the target object. Optionally, when processing the first image, the coprocessor 302 may use an artificial intelligence AI image recognition algorithm to identify the first image to obtain corresponding first image features; or may also use a small sample (few- shot learning) image recognition algorithm to recognize the first image.

The main processor 301 is further configured to associate the first image feature with the first application when the database does not include the first information, and store the first information and all the first information in the database. the first application; wherein the database includes information for indicating at least one image feature and an application associated with each image feature. Associating the first image feature with the first application also means associating the first image with the first application.

Exemplarily, if the information used to indicate any image feature in the at least one image feature includes the identifier and image feature of the any image feature, it is assumed that the database has the identifier "21" and the image feature "Yellow Shared". Bicycle” and the app “ofo Shared Bicycle” associated with “21” and “Yellow Shared Bicycle”. If the first information received by the main processor 301 from the co-processor 302 includes the identifier "11", the image feature "black guitar", and the first application selected by the user is determined based on the first instruction triggered by the user as the "Tuner APP" ", you can associate "11" and "Black Guitar" with the "Tuner APP" and save them in the database.

Optionally, the content included in the database is divided into a preset part and a user-defined part. That is, there is some information used to indicate image features in the database and the application program associated with the information is pre-configured in the database, and there are also some information used to indicate image features and the application program associated with the information is user-defined. Save in the database, for example, according to the above process, save the first information and the first application in the database. To distinguish, the aforementioned application program preset in the database and associated with the relevant image features may be referred to as a default application program, and the preset part and the user-defined part may be stored in two tables in the database.

In the embodiment of the present application, user customization is supported, an association relationship between image features and application programs is established, and the types of application programs that can be activated based on the image recognition technology are expanded. In subsequent image recognition, if an image feature associated with an application program is identified, the recognition device can automatically start the application program associated with the image feature, which simplifies the user operation process, improves user experience, and improves processing efficiency.

During specific implementation, an interactive interface for the user to trigger the first instruction may be set on the electronic device. Exemplarily, referring to the interactive interfaces on the mobile phone shown in FIGS. 4 to 7 b , the user performs a related operation on the mobile phone to trigger the first instruction, so as to associate the first image with the first application program.

As shown in FIG. 4 , the function "registration image" can be deployed in the settings in the mobile phone; it is also possible to deploy the function "registration image" based on independent software in the mobile phone. After the user taps "register image", it jumps to an interface as shown in FIG. 5 that prompts the user to enter an image. If the user selects "take a picture", the user will jump to the camera interface as shown in Figure 6a for the user to take one or more images; if the user selects "Open Gallery", it will jump to the gallery interface as shown in Figure 6b , for the user to select images stored in the gallery. Specifically, Fig. 6b shows that the user selects 5 images. After the user completes inputting the image, the coprocessor in the mobile phone recognizes the image feature of the image entered by the user. If the image feature has been associated with an application, it is determined that the image feature has been registered, and the user is prompted that the image feature has been registered. Registration, as shown in Figure 7a, pops up on the mobile phone "the object in this image has been registered to the XX application, please select a new image or delete the association between the image and the XX application"; or, if the image features If the application is not associated, as shown in Figure 7b, the user is prompted to register the image, and the application list pops up on the mobile phone. If the user selects application 2, the image features of the image entered by the user and the application 2 are saved. association relationship.

Corresponding to the interactive interfaces shown in FIG. 4 to FIG. 7 b , referring to FIG. 8 , an embodiment of the present application provides a schematic flowchart of registering an image, including the following steps. S1, in response to a user operation, turn on a pre-deployed registered image function in the electronic device. For example, as shown in FIG. 4, the user taps "register image". S2, prompting the user to input an image, and acquiring an image triggered by the user to shoot or a stored image selected by the user. S3, determine whether the image feature of the acquired image is registered, that is, determine whether the image feature of the acquired image has been associated with an application program. If yes, go to S4; if not, go to S5.

S4, prompt the user that the image feature has been registered, re-enter the image or delete the association relationship between the image feature and its associated application program. S5, prompting that the user can register. S6, associate the image feature of the acquired image with the application program designated by the user to complete the registration.

Optionally, the application program specified by the user may be determined by referring to any one of the following two manners. Manner 1: After S2 is performed and before S3 is performed, a list of application programs is displayed to the user for the user to select the application program desired to be associated with the input image, so that the user can select the desired application program in advance. Manner 2: After S5 is performed and before S6 is performed, a list of application programs is displayed to the user for the user to select an application program desired to be associated with the input image.

Optionally, when the user triggers the shooting or the selected image does not contain salient features, for example, the image does not contain the target object, or most of the first image is the background of the wall, the ground, the ceiling, etc. and the features of the target object contained are relatively high. If less, the user is prompted to shoot or select an image with significant features, and the image with significant features is used as the aforementioned first image. The images captured or selected by the user are required to have salient features to avoid subsequent triggering of false identification.

Further, the signal processing device 303 is further configured to process the second data from the image sensor to obtain a second image. The coprocessor 302 is further configured to identify the second image to obtain a second image feature, and when it is determined that the database includes second information for indicating the second image feature, send the second information to the main processor. The main processor 301 is further configured to determine the application program associated with the second image feature in the database as the target application program to be activated.

Optionally, when processing the second image, the coprocessor 302 may use an artificial intelligence (AI) image recognition algorithm to identify the second image to obtain corresponding second image features; or a small sample (few- shot learning) image recognition algorithm to recognize the second image.

Among them, the process of AI image recognition technology can be shown in Figure 9, including the training process and the recognition process. In the training process, first obtain a large number of images to be trained, that is, obtain training samples, then perform preprocessing and feature extraction on the sample images to obtain an image training model, and use a deep learning engine, such as the engine software of the coprocessor 302, The processed image training model is processed to obtain a training classifier. For example, if the image features to be trained are yellow shared bicycles, first obtain images of a large number of yellow shared bicycles, and the obtained images may include images of yellow shared bicycles taken from different angles and in different scenes, and then separately for each yellow shared bicycle. Each image is preprocessed, such as difference, denoising, or sharpening, etc., and feature extraction is performed on the preprocessed image, that is, the characteristics of the yellow shared bicycle in the image are extracted. The extracted features about the yellow shared bicycle are trained to obtain a training model of the yellow shared bicycle, and the classifier of the yellow shared bicycle is trained according to the obtained training model. The training model and the classifier can exist in the form of software models, and are stored in the memory of the electronic device for subsequent calls by the coprocessor 302 . In the identification process, firstly acquire the image to be identified (such as the aforementioned second image), preprocess the image to be identified, and perform feature extraction on the preprocessed image to be identified, wherein the preprocessing includes removing the image to be identified Images that do not contain significant features in the image, for example, if the background of the image, such as the ceiling, wall, and ground, accounts for a large proportion, and the features about the target object account for a small proportion or even none, the image will be eliminated to reduce the subsequent recognition by the coprocessor 302. The amount of calculation of image features further improves processing efficiency. Then, the extracted features are subjected to dimensionality reduction processing, and then matched with the training model and the training classifier obtained during the training process to obtain the recognition result. The above training process can be completed offline by the server, and then the model and classifier obtained by training are sent to the electronic device for storage in the memory of the electronic device. The coprocessor 302 in the electronic device performs Identify, and obtain the second image feature.

The small sample image recognition technology can be shown in Figure 10, including an offline training process, an online training process and a recognition process. In the offline training process, a large number of images to be trained are obtained, that is, training samples are obtained, and then the sample images are preprocessed and feature extracted to obtain an image training model with meta-knowledge, and a deep learning engine such as the coprocessor 302 is used. The engine software processes the processed image training model to obtain a training classifier. Then in the online training process, a small number of images with user personalization are acquired, the personalized images are preprocessed and features are extracted to the personalized image training model, and a deep learning engine such as the engine software of the coprocessor 302 is used to perform The aforementioned personalized image training model is processed to obtain a classifier. In the identification process, firstly acquire the image to be identified (such as the aforementioned second image), preprocess the image to be identified, and perform feature extraction on the preprocessed image to be identified, wherein the preprocessing includes removing the image to be identified Images that do not contain significant features in the image, for example, if the background of the image, such as the ceiling, wall, and ground, accounts for a large proportion, and the features about the target object account for a small proportion or even none, the image will be eliminated to reduce the subsequent recognition by the coprocessor 302. The amount of calculation of image features further improves processing efficiency. Then, it is matched with the image recognition model and classifier obtained in the online training process to obtain the recognition result. If the matching is successful, the coprocessor recognizes and obtains the image feature corresponding to the image, for example, the coprocessor recognizes the second image and obtains the second image feature.

The following refers to the relationship between the user-defined saved image features and application programs shown in Table 1, and as shown in Table 2, the default application programs corresponding to some image features stored in the database, that is, not user-defined applications program, which existed in the database before the user selected it. An example is given to illustrate the solution of the above-mentioned optional implementation manner.

Table 1

logo	image features	application
11	black guitar	Tuner APP
12	silver drum kit	Metronome APP
13	air conditioner	Remote control APP
14	TV set	Remote control APP
15	Car Steering Wheel	Navigation app
16	yoga mat	Keep
17	Textbooks/Textbooks	learning app
18	orange shared bicycle	Mobike
...	...	...

Table 2

logo	image features	default application
twenty one	yellow bike sharing	ofo bike sharing
twenty two	blue bike sharing	little blue bike
twenty three	subway gate	Easy access
twenty four	Receipt QR code	Alipay (or WeChat)
25	code scanning gun	Alipay (or WeChat)
26	bus stop sign	the car is coming
...	...	...

For example, the user-defined "Mobike" application has an association with the orange shared bicycle. If the second image obtained by the co-processor 302 is shown in FIG. 11a, the co-processor 302 identifies that there is an orange shared bicycle in the second image, that is, determines that the second image corresponding to the second image has an orange shared bicycle, According to Table 1, it is determined that the associated application is "Mobike", the coprocessor 302 can notify the main processor 301 of the identifier "18", and the main processor 301 determines that the application corresponding to the identifier "18" is " Mobike”, then further identifies “Mobike” as the target application to be activated. If the application "Mobike" has been installed in the electronic device where the device 300 is located, the main processor 301 activates the "Mobike" application, and the content displayed by the electronic device to the user is shown in Figure 11b. If the application "Mobike" is not installed in the electronic device, the main processor 301 may prompt the user whether to download the application "Mobike" through the display screen of the electronic device.

For another example, in a scenario where a code scanning gun appears, the user may need to make a payment, so the "Alipay" application that can pay is set as the default application corresponding to the code scanning gun. If the image acquired by the co-processor 302 is shown in FIG. 12a, the co-processor 302 identifies that the image feature corresponding to the acquired second image is a scan code gun, and determines the application associated with the code scan gun that the user does not define according to Table 1 program, and according to Table 2, it is determined that the default application program corresponding to it is the "Alipay" application program, the coprocessor 302 can notify the main processor 301 of the identification "25", and the main processor 301 determines the application corresponding to the identification "25" If the program is "Alipay", then "Alipay" is further determined as the target application to be activated. If the application "Alipay" has been installed in the electronic device where the device 300 is located, the main processor 301 activates the "Alipay" application, and the content displayed by the electronic device to the user is shown in Figure 12b. If the application program "Alipay" is not installed in the electronic device, the main processor may prompt the user whether to download the application program "Alipay" through the display screen of the electronic device.

It should be understood that the above-mentioned Tables 1 and 2 are only examples, and the user-defined association relationship between the image feature and the application program may include the content shown in Table 1, but is not limited thereto. The preset image features in the database and the corresponding default application programs may include the contents shown in Table 2, but are not limited thereto. In addition, the column of "Identification" is also optional. If there is this item, the communication between the coprocessor 302 and the main processor 301 can be simplified, that is, the coprocessor 302 notifies the main processor of the determined identification of the application program. If this item does not exist, the coprocessor 302 can also notify the main processor of the application to be activated with the identification information corresponding to the image in other ways, such as sending the name of the application or image feature information to the main processor 301 etc.

In addition, the same image features in the database represented in Table 2 can also correspond to multiple default applications. For example, if the image feature identified by the co-processor 302 is "two-dimensional code for payment" or "scanner", the user may need to make payment, and both the "Alipay" application and the "WeChat" application can realize the payment, At this time, in an optional implementation manner, the two applications may be used as default applications corresponding to the same image feature corresponding candidates, and different priorities may be set for the two applications. If the priority of "Alipay" is higher, the main processor 301 will give priority to activate the "Alipay" application if the "Alipay" application is already installed in the electronic device, and if the "Alipay" application is not installed, then Activate the "WeChat" application. The above priority setting can be set when the corresponding relationship between the image feature and the default application is determined, or can be set by the user after the electronic device obtains the corresponding relationship between the image feature and the default application. In another optional embodiment, the main processor 301 may prompt the user to select one of "Alipay" and "WeChat" through the display screen of the electronic device, record the user's selection, and determine the priority based on the user's selection. For example, if the user's preference is to select "Alipay" multiple times, the "Alipay" application will be activated first, and if the "Alipay" application is not installed, the "WeChat" application will be activated.

Further, in an optional implementation manner, the coprocessor 302 is further configured to, when the database does not include the second information for indicating the characteristics of the second image, Or perform semantic analysis on the second image feature to obtain a semantic label; wherein, the semantic label includes text information used to characterize the second image or the second image feature; the main processor 301 further uses After receiving the semantic tag from the coprocessor, determine at least one application program associated with the semantic tag; and then indicate to the user at least one application program associated with the semantic tag, and receive the first application triggered by the user. Two instructions, where the second instruction is used to instruct to activate a second application in the at least one application, and determine that the second application is the target application to be activated.

Specifically, the semantic label may be the type of the target object in the second image. For example, FIG. 13a shows that the acquired second image contains a piano, and the semantic label obtained by semantic analysis may be a musical instrument. The main processor 301 can search for at least one application program related to the musical instrument according to the program description information such as the function introduction, development instructions, software classification, etc. of the application program in the electronic device. Instrument-related applications may include a recorder with a recording function; users may need to find tracks, and instrument-related applications may include music players with a function of finding playing tracks; users may need to learn to play the piano online, related to musical instruments 's applications may include related learning software that can learn to play an instrument. A candidate list corresponding to at least one application program is displayed to the user on the display screen of the electronic device, for example, for the user to select the application program to be activated. Exemplarily, as shown in FIG. 13b, the application software related to the “musical instrument” displayed to the user includes a recorder, a music player, a learning software, etc., and the user selects the application to be activated as a music player. Optionally, the user's selection may also be recorded, and the user's selection is used to determine the priority of the semantic tag-related applications. Further, at least one application program in the candidate list may be displayed according to the determined priority order.

Further, after determining the target application, the main processor 301 is further configured to determine whether the target application is installed in the electronic device where the device 300 is located. If it is determined that the target application is installed in the electronic device, the target application is activated. Specifically, if the target application is already installed in the electronic device, when the main processor 301 determines the target application, the target application may not be running, and the main processor 301 can run the target application in the foreground; or , when the main processor 301 determines the target application, the target application may also be in the background running state, then the main processor 301 can switch the target application from the background running to the foreground running, that is, through the display of the electronic device The screen displays the user interface of the target application, so that the user can operate the target application. If the target application program is not installed in the electronic device, the main processor 301 may prompt whether to download the target application program through the display screen of the electronic device. Further, if the target application program is not installed in the electronic device, but a third application program is installed in the electronic device, and the third application program has a built-in interface of the target application program, the main processor 301 can also pass the third application program through the third application program. The target application's interface built into the application activates the target application. In this device, the coprocessor acquires the image from the signal processing device, then recognizes the image and determines the corresponding target application, and the main processor activates the target application or prompts the user to download the target application. It can be seen that the device can accurately determine the application that the user may want to use according to the image from the image sensor, and activate the application, and the user does not need to click the application icon in the process, which simplifies the user operation. In addition, the feature extraction of the image and the determination of the application program to be activated are realized by the coprocessor, without the need for the main processor to perform related operations, the processing efficiency is high, and the power consumption is saved.

For example, installation-free applications, also known as small programs, are gradually favored by users due to the advantages of no installation, click-to-use, and release of storage space after closing. Specifically, application A has a built-in interface of application B. The user can click the icon of application B in application A, and the electronic device will load the package of application B and run it. Small can be downloaded quickly and without the installation process, users can quickly use application B to achieve the effect of click-to-use. Therefore, if the main processor 301 determines to activate application B, even if application B is not installed in the electronic device, but the interface of application B is built in application A, the main processor can activate application B through application A , to avoid the burden of adding storage to electronic devices. Optionally, if both the target application program to be activated and the third application program with a built-in target application program interface are installed in the electronic device, priorities can also be set for different activation methods. For example, considering the limitation on the package size of the applet, the function of the applet may not be as good as that of a normally installed application. Therefore, a high priority can be set for directly activating the target application, and a third application can be used to activate the target application directly. Activating the target application sets low priority.

In a possible implementation manner, the above-mentioned device 300 may further include a transceiver 304 for receiving part or all of the content in the database from the server. As mentioned above, the model training process can be completed by the server, and then the model obtained by training is sent to the electronic device, and the electronic device can receive the aforementioned at least one image feature sent by the server through the transceiver 304 in the device 300. Default application for image features. Specifically, the transceiver 304 may be a wired transceiver, that is, the content in the database is received through wired transmission, or the transceiver 304 may also be a wireless transceiver, that is, the content in the database is received through wireless transmission, such as in the mobile phone 200. the RF circuit.

In another possible implementation, the training process can also be completed by the provider of each application, and the model obtained by training is carried in the installation package of the application. If the electronic device downloads and installs the application, Then the coprocessor 302 can obtain the image feature corresponding to the application program. Therefore, if the coprocessor 302 recognizes the image feature existing in the database from the image, the electronic device is installed with the application program corresponding to the image feature. .

In order to further improve the accuracy of the activated application, the correspondence between the image features and the application stored in the database may also include location information. In a specific embodiment, the image features stored in the database and the application program corresponding to each image feature may be as shown in Table 3.

table 3

logo	image features	location information	application
twenty one	yellow bike sharing	-	ofo bike sharing
twenty two	blue bike sharing	-	little blue bike
twenty three	subway gate	Beijing	Easy access
twenty three	subway gate	Shanghai	Metro
twenty four	Receipt QR code		Alipay (or WeChat)
25	code scanning gun		Alipay (or WeChat)
26	bus stop sign	Shanghai	Shanghai bus query
26	bus stop sign	Beijing	Beijing real-time bus
…	…	…	…

For example, swiping the QR code is also used to take the subway. The "Easy Pass" app is currently used in Beijing, while the "Metro Metropolis" app is currently used in Shanghai. After swiping the QR code to enter the station, and the electronic device opens the "Metro Metropolis" application, it cannot provide services for the user.

For another example, at present, many bus applications can provide users with information on the running route of the bus and the current location of the bus, so as to facilitate the user to take the bus and estimate the waiting time for the bus. However, each bus application has some limitations. For example, the "Carriage Came" application can basically query the bus routes of each city, but cannot query the location information of the buses in each city. In Beijing , if you want to query the location information of buses of a certain bus line, you can use the "Beijing Real-time Bus" application, but this application cannot query the bus information of other cities. Therefore, increasing the matching of electronic device location information helps to improve the accuracy of activating the application. The location information may be implemented by a positioning device in the electronic device, such as a global positioning system (global positioning system, GPS) or a Beidou system.

In order to further improve the user experience, after the main processor 301 activates the determined target application, the page required by the user may be further opened. In a specific embodiment, after the association between the user-defined image feature and the application program is registered, the activation page of the application program can also be customized by the user. In addition, the image features stored in the database and the application program corresponding to each image feature may also include activation page information, as shown in Table 4 below.

Table 4

logo	Image category	application	page
twenty one	yellow bike sharing	ofo bike sharing	Scan the code to unlock the page
twenty two	blue bike sharing	little blue bike	Scan the code to unlock the page
twenty three	subway gate	Easy access	Scan the page
twenty four	Receipt QR code	Alipay (or WeChat)	Scan the page
25	code scanning gun	Alipay (or WeChat)	QR code payment page
26	bus stop sign	the car is coming	route page
…	…	…	…

The following is an example of the case where the image feature of the acquired image has not been customized by the user and the associated application program, but its corresponding default application program can be found in the database as an example. For example, since the scene of "scanning code gun" or "payment with QR code" is usually a scene where users need to pay, the application corresponding to "scanning code gun" or "payment with QR code" can be used for payment. "Alipay", that is, when the co-processor 302 recognizes that the image features of the acquired image (such as the aforementioned second image) are "scan code gun" or "payment QR code", the main processor 301 activates the "Alipay" application program . If activating the "Alipay" application is to switch it from running in the background to running in the foreground, the page currently displayed to the user is the last page used by the user, and this page may not be the payment page required by the user; if activating the "Alipay" application ” application is to start an application that is not running, and the page currently displayed to the user is the main page of the application. However, the user needs to use the "Alipay" application for payment, that is, what the user needs is the payment page in the "Alipay" application. For the convenience of the user, this embodiment may be further improved as follows. If the image obtained by the co-processor 302 is still as shown in FIG. 12a, and the identified image feature is "scanning code gun", then what the user may need is the page of "payment QR code", then the main processor 301 can activate the "QR code payment" page. After the Alipay” application, it can further open the “QR code payment” page for the user, and the page displayed by the electronic device is shown in Figure 12c; if the image obtained by the coprocessor 302 is shown in Figure 14a, the identified If the image category is "payment QR code", the main processor 301 can further open the "scan" page for the user after activating the "Alipay" application, as shown in Fig. 14b. This embodiment can further simplify user operations and improve user experience.

In some scenarios, the user's requirement may be to use a certain application program to perform information query. Therefore, after activating the corresponding application program, the main processor 301 may further perform a search based on the information extracted from the image. For example, the image obtained by the coprocessor 302 is shown in Figure 15a, the identified image feature is "bus stop sign", and the number "646" is extracted from the image, and the main processor 301 receives the transmission from the coprocessor 302. The identification information of the target application and the extracted feature information corresponding to the "bus stop sign", that is, the identification of the "Beijing real-time bus" application and the extracted number "646"; the main processor 301 activates the "Beijing real-time bus" application. bus” application, and search for “646” in the application, so as to display the current 646 bus information for the user, as shown in Figure 15b.

As mentioned above, the processor 280 in the mobile phone 200 may include one or more processors. Therefore, when the device 300 is applied to the mobile phone 200, the main processor 301 and the coprocessor 302 correspond to the processors in the mobile phone 200 280; the signal processing device 303 is respectively coupled with the image sensor in the mobile phone 200 and the other input device controller 271, so as to obtain data from the image sensor and enable the coprocessor 302 to obtain images. The signal processing device 303 may also be one of the processors in the processor 280 . Optionally, the aforementioned database may be stored in the memory of the electronic device. In addition, the memory can further store necessary software required for the operation of the main processor 301 and the coprocessor 302, such as driver software, operating system software or application software.

In a possible design, the above-mentioned device 300 may be one or more chips, for example, the above-mentioned main processor 301 , co-processor 302 and signal processing device 303 are integrated in one chip, and the chip is arranged in the electronic device . The electronic device provided with the chip is further provided with the above-mentioned image sensor, but the image sensor may not be provided on the same chip with the above-mentioned device 300 .

Exemplarily, the above-mentioned main processor 301 may be a central processing unit (central processing unit, CPU) or an application processor or a microprocessor of an electronic device, and is located in the processor 280. When the main processor 301 is working normally, the power consumption is larger than that when the co-processor 301 is working normally. Therefore, the co-processor 302 processes the acquired images, and when it is determined that an application program needs to be activated, notifies the main The processor performs the operation of activating the application.

Optionally, the main processor 301 may be in a low power consumption state before receiving information from the coprocessor 302 for indicating image features, such as first information/second information or receiving a semantic label, and the main processor 301 is in a low power consumption state. The power consumption in the power consumption state is lower than the power consumption when the coprocessor 302 is running a handler. For example, the low power state may be an idle state, a sleep state, or a sleep state. The main processor 301 can be woken up by the coprocessor 302 by receiving information or semantic tags for indicating image features, or the main processor 301 can be woken up by the coprocessor 302 in other ways, such as interrupt notification, and further Receiving information or semantic tags for indicating image features and a series of subsequent operations are not limited in this embodiment.

The above-mentioned coprocessor 302 may be a neural network processor (eg, NPU), a digital signal processor, an image processing unit, or a hardware accelerator, etc., and its power consumption during operation may be lower than that of the main processor 301 . The coprocessor 302 can be said to be inside or outside the processor 280, which is not limited in this embodiment. Since the power consumption of the coprocessor 302 is relatively low, even if the coprocessor is set as a normally-on processor, the power consumption of the electronic device will not be greatly affected. In order to further improve the user experience, the above embodiments of the present application can be applied to the case where the screen of the electronic device is off or the screen is locked and the screen is on. Normally, when the electronic device is in the screen off state, the main processor 301 is in a dormant state to reduce power consumption. power. At this time, due to the low power consumption of the coprocessor, even when the screen of the electronic device is locked or closed, it can still be in a working state without consuming a lot of power.

The above-mentioned signal processing device 303 can be an ISP, and can be used to process the data collected by the image sensor, for example, to perform automatic exposure control (automatic exposure control, AEC), automatic gain control (automatic gain control, AGC), automatic white balance ( Processing of functions such as automatic white balance, AWB), color correction, lens shading correction, gamma correction, and dead pixel removal. Alternatively, the above-mentioned signal processing device 303 may also be a sensor signal processor, and its function is similar to that of the ISP, and it processes the data collected by the image sensor. Alternatively, the ISP may be located within the processor 280 or alternatively, the ISP may be located outside the processor 280. Optionally, a shared memory may be deployed in the device 300, and the shared memory is used to store the data collected by the image sensor. The signal processing device 303 can also be an intelligent sensor hub (Sensor hub), which is used to obtain the data collected by the image sensor from the shared memory, and then transmit it to the coprocessor 302, so that after the coprocessor 302 performs image recognition, it will The identification result is sent to the main processor 301 .

In a possible design, the image sensor may be actively triggered by the user to collect data. In another possible design, the above-mentioned image sensor may be always on and collect data in real time; alternatively, the image sensor may also collect data according to a preset period, that is, automatically perform data collection at regular intervals. Therefore, the camera or lens carrying the image sensor can be considered to be in an Always On state, so as to recognize the surrounding environment in real time and activate the application program in time to improve the user experience. Optionally, the camera or lens carrying the image sensor may use a low-power camera. The process of collecting data by the image sensor does not need to be triggered by the user every time, but is automatically realized by the device. The preset period may be a fixed period, or multiple periods may be set for the image sensor, and different periods may be used according to different environments. For example, if the ambient light is strong, user needs may be generated at any time, and a preset period of 0.1s can be used; if the ambient light is dark, the possibility of user needs being generated is small, and a preset period of 1s can be used at this time. . In a possible design, the signal processing device 303 can also process the acquired data in real time or periodically, and process it, so that the coprocessor 302 can perform further identification. If both the image sensor and the signal processing device 303 are configured to collect data or process data according to a preset period, they may be configured with the same period or with different periods.

Further, an embodiment of the present application also provides a computer storage medium, where a software program is stored in the storage medium, and when the software program is read and executed by the coprocessor, the software program can implement the execution of the coprocessor in any of the foregoing embodiments. or, the software program, when read and executed by the host processor, can implement the functions executed by the host processor in any of the embodiments.

The embodiments of the present application also provide a computer program product, which, when running on the coprocessor, enables the coprocessor to perform the functions performed by the coprocessor in any of the foregoing embodiments; or, when the computer program product is executed on the main processor When running on the above, the main processor is made to perform the functions performed by the main processor in any of the above embodiments. The computer storage medium includes the memory 220 mentioned in the previous embodiment, and can store the relevant software data of the corresponding database in FIG. 3 , which is not repeated here. Therefore, the coprocessor and the running software can be used as a separate product to realize the function of image recognition mentioned in the previous embodiment to find the corresponding default application, so as to send the identification of the relevant default application to the coprocessor. to the main processor. Therefore, the software run by the coprocessor may include multiple software modules, including but not limited to a recognition module for recognizing an image and a search module for searching for a corresponding default application, which is not limited in this embodiment.

Based on the same concept, this embodiment of the present application provides another application identification device 1600. As shown in FIG. 16, the device 1600 includes: a main processing module 1601, configured to receive a user-triggered first instruction, the first instruction is used to instruct the first image to be associated with the first application; the co-processing module 1602 is used to identify the first image to obtain the first image feature, and send a message to the main processing module to instruct the The first information of the first image feature; the main processing module is further configured to associate the first image feature with the first application when it is determined that the first information is not included in the database, and store the first image feature in the database. The first information and the first application are stored in the database; wherein the database includes information for indicating at least one image feature and an application associated with each image feature. The above solutions support user customization to establish the relationship between image features and application programs. In subsequent image recognition, if an image feature associated with an application program is identified, the recognition device can automatically start the application program associated with the image feature, which not only simplifies the user's operation process, but also expands the activation process based on image recognition technology. type of application.

In an optional implementation manner, the device 1600 further includes a signal processing module 1603 for processing the first data from the image sensor to obtain the first image.

In an optional implementation manner, the signal processing module 1603 is further configured to process the second data from the image sensor to obtain a second image; the co-processing module 1602 is further configured to process the second data from the image sensor. The second image is identified to obtain the second image feature, and when it is determined that the database includes second information for indicating the second image feature, the second information is sent to the main processing module 1601 . The main processing module 1601 is further configured to determine the application program associated with the second image feature in the database as the target application program to be activated.

In an optional implementation manner, the co-processing module 1602 is further configured to, when the database does not include the second information for indicating the characteristics of the second image, to perform semantic analysis on the second image feature to obtain a semantic label; wherein, the semantic label includes text information used to characterize the second image or the second image feature; the main processing module 1601 is also used for the The semantic tag is received from the co-processing module 1602, and is further used to indicate to the user at least one application program associated with the semantic tag, and receive a second instruction triggered by the user, the second instruction is used to instruct activation a second application in the at least one application, and determining the second application as the target application to be activated.

In an optional implementation manner, after receiving the identifier of the target application from the co-processing module 1602, the main processing module 1601 is further configured to: determine whether the device is installed in the electronic device where the device is deployed The target application program is present; when the target application program is installed in the electronic device, the target application program is activated; or, when the target application program is not installed in the electronic device but a third When the application program is applied, activate the target application program through the target application program interface built in the third application program; or, when the target application program is not installed in the terminal, prompt the user to download the target application program .

In an optional implementation manner, the signal processing module 1603 is specifically configured to process the second data from the image sensor periodically or in real time to obtain the second image.

In an optional implementation manner, the co-processing module 1602 is specifically configured to recognize the first image or the second image by using a small sample image recognition algorithm.

Any one or more of the above multiple modules may be implemented by software, hardware or a combination of the two, and the functions of the modules implemented by hardware may refer to the description of the corresponding components in FIG. 3 . Modules implemented in software may include computer program instructions and be executed by corresponding components in FIG. 3 , which are not limited in this embodiment.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in one or more flows of the flowchart and/or one or more blocks of the block diagram, and the apparatus may be described with reference to FIG. 16 .

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the protection scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (10)

1. A device for identifying an application, comprising:

   a main processor, configured to receive a first instruction triggered by a user, where the first instruction is used to instruct to associate the first image with the first application;

   a coprocessor, configured to identify the first image to obtain a first image feature, and send first information for indicating the first image feature to the main processor;

   The main processor is further configured to associate the first image feature with the first application when it is determined that the database does not include the first information, and save the first information in the database and the first application; wherein the database includes information indicating at least one image feature and an application associated with each image feature.
2. The device of claim 1, further comprising a signal processing device for processing the first data from the image sensor to obtain the first image.
3. The apparatus of claim 2, wherein:

   The signal processing device is further configured to process the second data from the image sensor to obtain a second image;

   The coprocessor is further configured to identify the second image to obtain a second image feature, and when it is determined that the database includes second information for indicating the second image feature, convert the second information to the second image. sent to the host processor;

   The main processor is further configured to determine the application program associated with the second image feature in the database as the target application program to be activated.
4. The apparatus of claim 3, wherein,

   The coprocessor is further configured to perform semantic analysis on the second image or the second image feature when the database does not include the second information for indicating the second image feature, to obtain semantic label; wherein, the semantic label includes text information used to characterize the second image or the second image feature;

   The main processor is further configured to receive the semantic tag from the coprocessor, indicate to the user at least one application program associated with the semantic tag, and receive a second instruction triggered by the user, the second instruction It is used to instruct activation of a second application in the at least one application, and determine that the second application is the target application to be activated.
5. The device according to claim 3 or 4, wherein the main processor is further configured to:

   Determine whether the target application is installed in the electronic device where the device is deployed;

   When the target application is installed in the electronic device, activate the target application; or,

   When the target application program is not installed in the electronic device but a third application program is installed, activate the target application program through the target application program interface built in the third application program; or,

   When the target application is not installed in the terminal, the user is prompted to download the target application.
6. The device according to any one of claims 3-5, wherein the signal processing device is specifically configured to process the second data from the image sensor periodically or in real time to obtain the first Second image.
7. The device of any one of claims 2-6, wherein the signal processing device comprises at least one of an image signal processor ISP or a sensor processor.
8. The device according to any one of claims 1-7, wherein the coprocessor is specifically configured to recognize the first image or the second image by using a small sample image recognition algorithm.
9. The device of any one of claims 1-8, wherein the coprocessor comprises at least one of a neural network processor, a digital signal processor, an image processing unit or a hardware accelerator.
10. An electronic device, characterized by comprising the device according to any one of claims 1-9 and the image sensor, wherein the image sensor is used to collect data periodically or in real time.

Images