WO2022062722A1

WO2022062722A1 - Application installation processing method and apparatus, storage medium and electronic device

Info

Publication number: WO2022062722A1
Application number: PCT/CN2021/111176
Authority: WO
Inventors: 金越; 李文静; 郭彦东; 李亚乾; 杨林
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-09-23
Filing date: 2021-08-06
Publication date: 2022-03-31
Also published as: CN114253559A

Abstract

An application installation processing method, an application installation processing apparatus, a computer readable storage medium and an electronic device. The application installation processing method comprises: obtaining a desktop image of a first terminal (S310); identifying an application in the desktop image according to characters and/or icons in the desktop image (S320); and determining an application to be installed of a second terminal according to the application in the desktop image (S330). The method achieves quick migration of the application and reduces the leakage risk of sensitive information.

Description

Application installation processing method, device, storage medium and electronic device

This application claims the priority of the Chinese patent application with the application number of 202011009695.1, which was filed on September 23, 2020, and entitled "Application Installation and Processing Method, Device, Storage Medium and Electronic Equipment", all of the Chinese patent application The contents are incorporated herein by reference.

technical field

The present disclosure relates to the field of computer technologies, and in particular, to an application installation processing method, an application installation processing apparatus, a computer-readable storage medium, and an electronic device.

Background technique

When a user replaces a terminal device such as a smartphone or a tablet computer, it is usually necessary to install an original App (Application, application program) on the new device. If the user manually downloads and installs each app, it takes a lot of time and effort.

SUMMARY OF THE INVENTION

The present disclosure provides an application installation processing method, an application installation processing apparatus, a computer-readable storage medium, and an electronic device.

According to a first aspect of the present disclosure, there is provided an application installation processing method, including: acquiring a desktop image of a first terminal; identifying an application in the desktop image through text and/or icons in the desktop image ; Determine the application to be installed on the second terminal according to the application in the desktop image.

According to a second aspect of the present disclosure, there is provided an application installation processing device, comprising: a desktop image acquisition unit for acquiring a desktop image of a first terminal; and an application recognition unit for passing text in the desktop image and/or icon, identifying the application program in the desktop image; an application program determining unit, configured to determine the application program to be installed on the second terminal according to the application program in the desktop image.

According to a third aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the installation processing method of the application program of the first aspect and possible implementation thereof are implemented Way.

According to a fourth aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to Execute the installation processing method of the application program of the first aspect and a possible implementation manner thereof.

Description of drawings

FIG. 1 shows a schematic diagram of a system architecture in this exemplary embodiment;

FIG. 2 shows a structural diagram of an electronic device in this exemplary embodiment;

FIG. 3 shows a flowchart of an application installation processing method in this exemplary embodiment;

FIG. 4 shows a flowchart of a method for acquiring a desktop image in this exemplary embodiment;

FIG. 5 shows a flowchart of a method for determining a starting frame in this exemplary embodiment;

FIG. 6 shows a flowchart of another method for determining a starting frame in this exemplary embodiment;

FIG. 7 shows a schematic diagram of a desktop image in this exemplary embodiment;

FIG. 8 shows a flowchart of a method for determining an application program by a target to be identified in this exemplary embodiment;

FIG. 9 shows a flowchart of a method for determining an application program through text and icons in this exemplary embodiment;

FIG. 10 shows a flowchart of a method for determining an application program by icon matching in this exemplary embodiment;

FIG. 11 shows a flowchart of a method for determining an application to be installed in this exemplary embodiment;

FIG. 12 shows a schematic diagram of an installation processing method of an application program deployed on a server in this exemplary embodiment;

Fig. 13 shows a flow chart of a method for installing an application program executed by a second terminal in this exemplary embodiment;

FIG. 14 shows a structural diagram of an installation processing apparatus for an application in this exemplary embodiment;

FIG. 15 is a structural diagram of another application installation processing apparatus in this exemplary embodiment.

detailed description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the figures are functional entities that do not necessarily necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In a solution of the related art, when migrating an app between Android devices, it is necessary to install software for app migration on the new device, and establish a connection between the old device and the new device (such as connecting via Bluetooth, or connecting to the same wireless local area network) In the case of , select the files that need to be migrated, and transfer them after matching the dynamic code to realize the migration. This solution needs to establish a connection between two devices, and the operation process is relatively complicated, and the transmission is relatively time-consuming.

In another solution of the related art, when migrating an App between iOS devices, or between an iOS device and an Android device, it is necessary to use cloud storage services such as iCloud to upload the App-related data on the old device to the cloud, and then from the cloud. Download to new device for migration. This solution relies on cloud storage services, which usually limit the size of the file data to be migrated, and need to upload and download to the cloud twice, which is more time-consuming.

In view of the above problems, exemplary embodiments of the present disclosure provide an application installation processing method. FIG. 1 shows a schematic diagram of the system architecture of the operating environment of the method. As shown in FIG. 1 , the system architecture 100 may include: a first terminal 110 , a second terminal 120 and a server 130 . The first terminal 110 and the second terminal 120 may be terminal devices such as smart phones, tablet computers, personal computers, wearable devices, game consoles, etc. The first terminal 110 is an old device, and the second terminal 120 is a new device. The same or similar App as the first terminal 110 is installed on the second terminal 120 . The server 130 may be a background system that provides an App installation service. For example, the server 130 may be a server of an App migration software, providing a complete App migration service, or a platform end of an App store, providing an App search and download service. The server 130 forms a connection with at least the second terminal 120 through a network, and may also form a connection with the first terminal 110 at the same time.

In an optional implementation manner, the server 130 includes a business server 1301 and a basic server 1302 . The business server 1301 is responsible for providing App installation services to the second terminal 120, for example, providing the second terminal 120 with a list of apps to be installed; the basic server 1302 is responsible for providing basic services such as App search and installation package acquisition.

The application installation processing method in the exemplary embodiment of the present disclosure may be executed by the second terminal 120 or the server 130 described above. In order to implement the method, the exemplary embodiment of the present disclosure further provides an electronic device, which may be the second terminal 120 or the server 130 . Typically, an electronic device includes a processor and a memory. The memory is used to store executable instructions of the processor, and may also store application data, such as image data, game data, etc.; the processor is configured to execute the application installation processing method in this exemplary embodiment by executing the executable instructions.

The following takes the mobile terminal 200 in FIG. 2 as an example to illustrate the structure of the above electronic device. It will be understood by those skilled in the art that the configuration in Figure 2 can also be applied to stationary type devices, in addition to components specifically for mobile purposes.

As shown in FIG. 2, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a USB (Universal Serial Bus, Universal Serial Bus) interface 230, a charging management module 240, a power management module 241, Battery 242, Antenna 1, Antenna 2, Mobile Communication Module 250, Wireless Communication Module 260, Audio Module 270, Speaker 271, Receiver 272, Microphone 273, Headphone Interface 274, Sensor Module 280, Display Screen 290, Camera Module 291, Indication 292, a motor 293, a key 294, a Subscriber Identification Module (SIM) card interface 295, and the like.

The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), controller, encoder, decoder, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or neural network processor (Neural-Network Processing Unit, NPU), etc.

In some embodiments, the processor 210 may include one or more interfaces through which connections are formed with other components of the mobile terminal 200 .

Internal memory 221 may be used to store computer executable program code, which includes instructions. The internal memory 221 may include volatile memory, such as DRAM (Dynamic Random Access Memory, dynamic random access memory), SRAM (Static Random Access Memory, static random access memory), and may also include non-volatile memory, such as at least one disk storage device , Flash memory devices, etc. The external memory interface 222 can be used to connect an external memory, so as to expand the storage capacity of the mobile terminal 200 .

The USB interface 230 is an interface conforming to the USB standard specification, and can be used to connect a charger to charge the mobile terminal 200, and can also be connected to an earphone or other electronic devices.

The charging management module 240 is used to receive charging input from the charger. While charging the battery 242, the charging management module 240 can also supply power to the device through the power management module 241; the power management module 241 can also monitor the state of the battery.

The wireless communication function of the mobile terminal 200 may be implemented by the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modulation and demodulation processor, the baseband processor, and the like. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 250 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the mobile terminal 200 . The wireless communication module 260 can provide applications on the mobile terminal 200 including wireless local area networks (Wireless Local Area Networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) networks), Bluetooth (Bluetooth, BT), global navigation satellite Wireless communication solutions such as Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR).

The mobile terminal 200 can realize the display function through the GPU, the display screen 290 and the application processor, etc., can realize the shooting function through the ISP, the camera module 291, the encoder, the decoder, the GPU, the display screen 290 and the application processor, etc., and can also realize the shooting function. The audio function is realized by the audio module 270 , the speaker 271 , the receiver 272 , the microphone 273 , the headphone interface 274 and the application processor.

The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyro sensor 2803, an air pressure sensor 2804, and the like.

The indicator 292 can be an indicator light, which can be used to indicate the charging status, the change of power, and can also be used to indicate messages, missed calls, notifications, and the like. The motor 293 can generate vibration prompts, and can also be used for touch vibration feedback and the like. The keys 294 include a power-on key, a volume key, and the like.

The mobile terminal 200 may support one or more SIM card interfaces 295 for connecting SIM cards to realize voice calls and mobile communications.

FIG. 3 shows the flow of an application installation processing method in this exemplary embodiment, which may include:

Step S310, obtaining the desktop image of the first terminal;

Step S320, identifying the application program in the desktop image through the text and/or icon in the desktop image;

Step S330: Determine the application to be installed on the second terminal according to the application in the desktop image.

Through the above method, the application to be installed of the second terminal is determined according to the application in the desktop of the first terminal. On the one hand, when the user changes from the old first terminal to the new second terminal, the application to be installed on the second terminal is determined based on the recognition of the desktop image, without scanning and parsing the application data of the first terminal, Conducive to the rapid migration of applications. On the other hand, there is no need to read relevant data from the first terminal, thus reducing the risk of sensitive information leakage in the first terminal, and there is no need to establish a connection between the first terminal and the second terminal, making the solution applicable to more scenarios. On the other hand, the entire processing process can be realized through simple image acquisition operations such as taking pictures and scanning, which simplifies user operations and improves user experience.

Each step in FIG. 3 will be described in detail below.

In step S310, a desktop image of the first terminal is acquired.

The desktop of the first terminal may include the icon of the App installed on the first terminal and the name of the App below the icon, so the desktop image can reflect the App information of the first terminal. The desktop image may be obtained by taking a photo or a screenshot of the desktop of the first terminal. For example, after the first terminal takes a screenshot of the desktop, it sends the captured desktop image to the second terminal or the server.

In an optional embodiment, step 310 may include:

controlling the camera to be turned on to capture images of the desktop of the first terminal;

Acquire one or more of the captured desktop images.

For example, the second terminal turns on the camera, and captures images of the desktop of the first terminal by taking pictures, scanning, or the like. In an optional implementation manner, a "scan" function may be added to the system interface, the App store, or the App migration software. When the user selects this function, the camera is turned on and an image scanning interface is displayed at the same time. The interface may include a scanning frame. The user aligns the scanning frame with the desktop of the first terminal, and the scanned image collected is the desktop image. As a result, the desktop image can be obtained through simple operation and visualization, and the learning cost of the user using the related functions is reduced.

In the process of image collection, the actual collected images may include photographed images, scanned images, preview images, etc. The number of images is usually more than one frame. All the collected images can be used as desktop images, or a part of them can be selected as desktop images. .

In the process of collecting desktop images, useless images may be generated, such as images collected when the user is not aiming at the desktop of the first terminal, blurred images collected when the user's hand shakes, and the like. If these useless images are also used as desktop images and are identified in subsequent steps, system resources will be wasted. Based on this, in an optional implementation manner, as shown in FIG. 4 , the acquisition of the desktop image can be achieved through the following steps:

Step S410, in the process of image acquisition, determine the starting frame;

In step S420, starting from the initial frame, the captured image of one or more frames is used as the desktop image.

In the scene of taking pictures or scanning, after the camera is turned on, the user generally needs to adjust the position or angle of the camera to aim at the desktop of the first terminal, which takes a certain amount of time. Therefore, the first few frames of images collected are usually useless images. Starting from a certain frame, the camera has been accurately and stably aimed at the desktop of the first terminal, and this frame is the starting frame. The images collected thereafter can all be images with the desktop of the first terminal as the main image content, and can be used as desktop images. Therefore, it is equivalent to screening all the collected images, retaining the images that can accurately reflect the desktop content of the first terminal, reducing the processing of useless images, and improving the waste of system resources.

Several exemplary ways of determining the start frame are provided below:

(1) Referring to Fig. 5, step S410 can be realized by the following steps:

Step S510, in the process of image acquisition, determine the relative change amount of each frame of image;

Step S520 , when the relative change amounts of the consecutive N frames of images are all smaller than the change amount threshold, determine the start frame from the consecutive N frames of images.

Among them, the relative change refers to the change of the image relative to the previous frame of image, which can be characterized by the change of any one or more image parameters, including but not limited to image pixel value, color, brightness, gray degree, etc.

In an optional implementation manner, the relative change amount of the current frame image may be determined according to the change amount of the color distribution data of the current frame image relative to the color distribution data of the previous frame image. For example, divide the color space into multiple subspaces in advance, count the distribution of the pixels of the current frame image in different subspaces, calculate the pixel ratio value in each subspace, and arrange the pixel ratio values in the order of the subspaces to form the current frame. The color distribution vector of the image; process the previous frame image in the same way to obtain the color distribution vector of the previous frame image; calculate the distance between the two color distribution vectors (such as Euclidean distance, Mansfield distance, etc.), as relative change. Or, count the color histograms of the current frame image and the previous frame image, and calculate the difference between the two color histograms (eg, calculate the difference between the curves corresponding to the two color histograms) as the relative change.

In an optional embodiment, a feature vector can be extracted from the current frame image and the previous frame image through models such as CNN (Convolutional Neural Network, convolutional neural network), respectively, and the distance between the two feature vectors can be calculated, as a relative change.

When the user adjusts the camera, the image changes between different frames before and after are relatively large, and when the camera is adjusted and stably aimed at the desktop of the first terminal, the image changes between the different frames before and after are small. Therefore, when the relative changes of images of consecutive N frames are smaller than the preset change threshold, it can be determined that the camera has been stably aimed at the desktop of the first terminal, and then the starting frame can be determined.

N represents the required number of images for continuous stabilization, which is a positive integer not less than 2, which can be set according to experience or actual conditions. Generally, the higher the frequency of image acquisition, the larger the value of N.

The change threshold is a standardized threshold for measuring whether two consecutive frames of images are stable, and can be set according to experience or actual conditions. When using different image parameters to calculate the relative change, the change threshold can also be adjusted adaptively.

In an optional implementation manner, when a larger value is set for N, the variation threshold can be appropriately lowered, or when a higher variation threshold is set, the value of N can be appropriately lowered; and vice versa .

When it is detected that the relative changes of consecutive N frames of images are all smaller than the change threshold, any frame of the N frames of images can be used as the starting frame, for example, the first frame or the last frame of the N frames of images is used as the starting frame. start frame.

For example, set N to 10 and the change threshold to 0.2; in the process of image acquisition, every time a frame of image is collected, its color distribution data is analyzed and compared with the color distribution data of the previous frame image to calculate the relative change When 10 consecutive frames of images are detected, and the change in the color distribution data of each frame of images compared to the previous frame of images is less than 0.2, then the starting frame is determined in these 10 frames of images.

(2) Referring to Fig. 6, step S410 can be realized by the following steps:

Step S610, in the process of image acquisition, perform target detection on the acquired image;

Step S620, when a target of a preset category is detected, the detected image is determined as the starting frame.

Target detection refers to the detection of objects or specific objects in the image, which can be processed by target detection models such as CNN, RCNN (Recurrent CNN, recurrent convolutional neural network), SSD (Single Shot MultiBox Detector, single-step multi-box detection). The target detection detection model generally includes an input layer, a hidden layer and an output layer; the input layer is used to receive the input of the image; the hidden layer is used to process the input image; the output layer is used to output the result of the image processing. Information such as category, location, size, etc. of each detected target can be included.

In this exemplary embodiment, the preset category may be a category related to the desktop of the first terminal, such as a mobile terminal, a desktop, an icon, and the like. When a target is detected from the image and the target category is a preset category, it is determined that the camera is currently aimed at the desktop of the first terminal, so the image can be determined as the initial frame image.

(3) In the process of image acquisition, determine the optical flow between each frame of image and the previous frame of image, when the number of pixels in the optical flow of consecutive N frames of images is less than the number threshold, from the continuous N Determine the start frame in the frame image. Optical flow can be used to measure the change between two adjacent frames of images. Generally, the more pixels contained in the optical flow, the greater the change. Therefore, the number threshold of the number of pixels in the optical flow can be set according to experience or actual conditions. When the number of pixels in the optical flow is less than the number threshold, it means that the two frames of images before and after are basically the same; when the number of pixels in the optical flow of consecutive N frames of images is less than the number threshold, it means that the camera has stabilized, and the starting point can be determined from it. frame.

(4) When using the second terminal to scan the desktop image of the first terminal, in order to ensure that the second terminal does not shake violently, the pose data of the second terminal can be obtained, for example, through the built-in IMU (Inertial Measurement Unit, inertial measurement unit) Measure the pose data; then check whether the pose data is stable, such as judging that the difference between the pose data between the two adjacent timestamps is less than the preset pose difference threshold; keep multiple (such as N consecutive) pose data in a row Stabilization is one of the ways to judge the starting frame. Specifically, in the process of collecting images, the pose data can be measured synchronously, the pose data of each timestamp is formed into a pose vector, and the distance from the pose vector of the previous timestamp can be calculated to obtain the relative value of the current timestamp relative to the current timestamp. The pose data difference of the last timestamp is compared with the preset pose difference threshold. If the pose data difference is less than the threshold, it means that the pose data of the current timestamp is stable, and the second terminal does not experience significant jitter. , thus, when the pose data of the consecutive N time stamps are all kept stable, any frame image collected in the time period of the consecutive N time stamps can be determined as the starting frame.

In practical applications, any combination of the above methods can be carried out. For example, the combination of the above methods (1) and (2) can be used to perform the stability detection and target detection of the image color distribution data at the same time. When the color distribution data of consecutive N frames of images remain When it is stable and a target of the preset class is detected, the starting frame is determined.

In the process of scanning the desktop of the first terminal for image acquisition, the above method of determining the starting frame can obtain an effective desktop image. Compared with the method of photographing the desktop image of the first terminal, it eliminates the possibility of occurrence during photographing. Compared with the way of taking screenshots of the desktop of the first terminal, there is no need to obtain a screenshot image from the first terminal, and there is no need to establish a connection between the first terminal and the second terminal. , the desktop image can be obtained even when the first terminal has no network. For example, after taking out the SIM card from the first terminal, the first terminal has no network service. At this time, the second terminal can also be used to scan its desktop to obtain the desktop image. Therefore, this method is suitable for more scenarios.

In an optional implementation manner, the number M of desktop images may be preset, and after the start frame is determined, the start frame and subsequent M-1 frames may be acquired to obtain M frames of desktop images.

In an optional implementation manner, after the start frame is determined, the frame image of the start frame may be determined as a desktop image, and subsequent processing is performed. If the subsequent processing process is normal, there is no need to acquire more desktop images; if the subsequent processing process is abnormal, such as the desktop image cannot be identified, more desktop images can be selected after the initial frame and the subsequent processing is performed again.

Further, starting from the starting frame, only one frame of image can be selected as the desktop image at a time, and subsequent processing can be performed. If the subsequent processing process is normal, the process ends. If the subsequent processing process is abnormal, the currently selected desktop image After that, select another frame of image (for example, the next frame of image can be selected, or a frame of image can be selected after a certain number of frames) as the desktop image; this cycle is repeated until the subsequent processing process is normal, and the process ends. This further reduces the number of desktop images that need to be processed, increasing efficiency.

It should be noted that the first terminal may include multiple desktops, for example, the number of apps thereof is large, occupying 3 desktop pages. Then, image acquisition can be performed on each desktop separately, and each desktop image can be processed separately in the subsequent process.

Continuing to refer to FIG. 3 , in step S320 , the applications in the desktop image are identified by the text and/or icons in the desktop image.

FIG. 7 is a schematic diagram of a desktop image, and FIG. 7 includes text and icons. The text refers to the text of the App name, such as "AA News", "BB Music", etc.; the icon is the App icon, such as the icons above "AA News" and "BB Music". These two kinds of information are the main information in the desktop image, and through these two kinds of information, it is possible to identify which applications are in the desktop image, which is equivalent to identifying which applications are installed in the first terminal.

Generally, text and icons in the desktop image appear in pairs, that is, each App has a line of name text and a corresponding icon. Therefore, this part of information can be extracted separately for identification without processing other information (such as desktop background) in the desktop image. In an optional implementation manner, referring to FIG. 8 , step S320 may include:

Step S810, extracting one or more targets to be identified from the desktop image, each target to be identified includes a text and an icon;

Step S820, according to the text and/or icon in the to-be-recognized target, identify the application program corresponding to the to-be-recognized target.

The text and icons in the target to be recognized can be extracted separately.

Text extraction can be achieved by algorithms such as OCR (Optical Character Recognition, Optical Character Recognition). OCR is to determine the shape of the characters contained in the image by detecting the dark and bright, and then translate it into computer text by means of character recognition. Specifically, grayscale and binarization can be performed on the background image to distinguish the background and foreground of the image; then the image is optimized and corrected, such as noise reduction, tilt correction, etc.; character features are extracted from the foreground part; The characters in the desktop image are identified by comparison with the data in the character database. After the characters are recognized, they are organized into texts according to the different positions of the characters in the desktop image.

The extraction of icons can be realized by algorithms such as edge detection and shape detection. For example, detect the connected area in the desktop image, and determine the part where the pixel value suddenly changes as the edge, so as to obtain multiple independent connected areas; since the icon is generally a rounded square, the shape of the connected area is detected, and the rounded square is The connected area is extracted to get the icon. It is also possible to detect icons and their positions in desktop images through models such as CNN, so as to achieve icon extraction.

In an optional implementation manner, the extracted text or icons may also be filtered according to the border position of the first terminal in the desktop image and the grid arrangement rule of the desktop. For example, delete the recognized text or icon outside the border; the grid points on the desktop are arranged as 3*4, if there are 4 icons in a row, it means that one of them is a misrecognized object. Align icons in a row, delete an extra icon, and so on.

There is a fixed positional relationship between the text and the icon of each App. As shown in Figure 7, the text is generally located directly below the icon. Therefore, according to the extracted position information of the text and the icon, the text and the icon are associated, and one text is associated with an icon, and the two form a target to be recognized. For example, associate text and icons that meet the following conditions: the abscissa of the midpoint of the icon is the same as the abscissa of the midpoint of the text, and the ordinate of the lower border of the icon is the same as the ordinate of the upper border of the text (or the difference is less than a certain threshold). Then, the characters and icons in each target to be recognized are respectively recognized to determine which application program it is.

The corresponding application program can be identified by using any aspect of the information in the text and the icon, and the application program can also be identified by combining the two aspects of the information in the text and the icon, so as to improve the accuracy.

In an optional implementation manner, referring to FIG. 9 , step S320 may include:

Step S910: Match each text in the desktop image with the name of the application program, and determine the application program corresponding to the text information according to the matching result.

where app_name is the standardized name for each app in the app library. After the text is extracted from the desktop image, there may be some misrecognition, such as recognizing a certain character as other characters with similar glyphs, such as recognizing "AA News" as "AA Xinjian". The above misidentifications can be corrected by matching against standardized application names. Matching of text to application name, including but not limited to the following ways:

Character matching can be performed on the text and the application name, such as detecting whether the proportion of the same characters is higher than a certain threshold (such as 50%, which can be set according to experience or actual conditions), and detecting whether different characters are characters with similar glyphs , when these two conditions are met, it is judged that the text matches the name of the application program successfully, and it is determined that the text is the name of the application program, thereby determining the application program.

The same font can be used to draw the binarized image of the text and the application name, and then the sequence features of the image are extracted by models such as LSTM (Long Short-Term Memory, long short-term memory network), and the similarity of the two sequence features is calculated. When the similarity is higher than a certain threshold (eg, 80%, which can be set according to experience or actual conditions), it is judged that the text and the application name are successfully matched, and the application corresponding to the text is determined.

In an optional implementation manner, referring to the above shown in FIG. 9 , step S320 may further include:

Step S920, when the text and the application program name are not successfully matched, determine the application program corresponding to the text according to the icon corresponding to the text.

If the misrecognition of the text is serious and the deviation from the correct application name is large, it may be unsuccessful to match all the application names in the application library. In this case, the icon recognition method can be used to determine the application. .

In an optional implementation, a CNN model for image classification can be pre-trained, and the classification label is the application name. Therefore, inputting the icon into the CNN model can directly output the corresponding application name, which is very convenient.

In an optional implementation manner, a method of icon matching may also be used. Referring to Figure 10, step S920 may include:

Step S1010, forming the icon corresponding to the above text into a first icon image. For example, the part where the icon is located can be cut out from the desktop image, which is the first icon image.

Step S1020, respectively matching the first icon image with the second icon image formed by each application icon in the application library. For example, image retrieval is used to calculate the matching degree (or confidence, similarity, etc., basically the same meaning) between the first icon image and each second icon image, such as extracting the feature vector of the first icon image and the second icon respectively. The feature vectors of the image, and the similarity between the feature vectors is calculated.

Step S1030, when there is a second icon image whose matching degree with the first icon image is greater than the matching degree threshold, determine the application program corresponding to the second icon image as the application program corresponding to the above text. The matching degree threshold is a standard for measuring whether the first icon image and the second icon image match, and can be set according to experience or actual conditions, such as 80%. When the degree of matching between the first icon image and the second icon image is greater than the threshold, it is determined that the two match, so as to determine the application corresponding to the above text or icon.

It should be noted that there may be multiple second icon images whose matching degrees with the first icon image are all greater than the matching degree threshold, and the second icon image with the highest matching degree may be selected as the matching result, or all the first icon images may be selected as the matching result. The two icon images are presented to the user as a matching result, and the user selects one icon therefrom. It is also possible that no second icon image whose matching degree with the first icon image is greater than the matching degree threshold is not found, that is, if the matching is unsuccessful, an unsuccessful matching result can be presented, for example, the application corresponding to the above text is marked as "" ?", "Unknown", etc.

The flow shown in FIG. 9 is essentially that the character recognition application is preferentially used, and if the recognition is unsuccessful, the icon recognition application is used. Since the corresponding application program can be accurately identified through the text in most cases, the flow of FIG. 9 can reduce the processing amount and further improve the efficiency.

In an optional implementation, it can also be determined to use character recognition or icon recognition according to the characteristics of the environment and the scene. Generally, in an environment such as insufficient lighting and reflections, the effect of using character recognition is poor, and the use of text recognition can be used. Icon recognition. For example, the ambient light parameters can be detected by the ambient light sensor. If the light parameters are within the preset value range (indicating the range of good lighting, which can be set according to experience or actual conditions), it is judged that the light is sufficient and there is no reflection. Use text recognition, otherwise use icon recognition.

In an optional implementation manner, in order to improve the accuracy, characters and icons may also be recognized at the same time. Specifically, the text can be matched with the application program name to obtain the first matching result, and the icon can be matched with the application program icon to obtain the second matching result; then compare whether the first matching result and the second matching result are the same, if they are the same, Any matching result is used to determine the corresponding application, and if they are not the same, an unsuccessful matching result is returned; the first matching result and the second matching result with a higher degree of matching may also be selected to determine the corresponding application.

Through step S320, the corresponding application program can be determined for each text or icon in the desktop image, thereby determining the application program of the first terminal.

Continuing to refer to FIG. 3 , in step S330 , the application to be installed on the second terminal is determined according to the application in the desktop image.

If the same application program as that of the first terminal needs to be installed on the second terminal, the application programs in the desktop image are all application programs to be installed on the second terminal. Or, a part of application programs may have been installed on the second terminal, the installed application programs are excluded from the application programs of the desktop image, and the rest are the applications to be installed. Or, if the system of the second terminal is different from that of the first terminal, there may be some application programs, which use different versions or names in different systems, and the application programs in the desktop image can be adapted according to the system of the second terminal. sex conversion to obtain the application to be installed on the second terminal.

In an optional implementation manner, referring to FIG. 11 , step S330 may include:

Step S1110, generating an application list of the first terminal according to the application in the desktop image;

Step S1120, obtaining user feedback information for the application list of the first terminal;

Step S1130: Determine the application to be installed on the second terminal according to the user feedback information.

The application list of the first terminal may be presented in the form of a list or an image. The application manifest can be displayed to the user and allow the user to submit feedback, such as providing feedback on errors in the manifest, or changing or removing parts of the application. Further, according to the user feedback information, the above application list is adjusted to obtain the application to be installed on the second terminal.

In an optional implementation manner, in the application list, multiple recommended applications of the same type may be provided beside each application. For example, if “AA News” is installed on the first terminal, it can be recommended in the application list. "XX News", "YY News", and add relevant text descriptions, such as "70% of users have installed XX News", or show the ratings of recommended apps, etc. This disclosure does not limit this.

In an optional implementation manner, the location information of the application to be installed on the desktop of the second terminal may be determined according to the location information of each application in the desktop image. For example, when the screen size of the second terminal and the first terminal are the same and the resolution settings are the same, the desktop grid arrangement of the two is also the same, and the location information of each application program in the desktop image of the first terminal can be used as each application program. position information on the desktop of the second terminal; or, when the grid points of the desktop of the second terminal are different from those of the first terminal, corresponding conversion calculations can be performed. , Arrange each application program in order from top to bottom, and then rearrange it according to the desktop grid arrangement of the second terminal to determine the position of each application program. Thereby, the automatic layout of the desktop of the second terminal is realized.

In an optional implementation manner, the desktop preview image of the second terminal may also be generated according to the location information of the application to be installed on the desktop of the second terminal, which is based on the layout of the application to be installed on the desktop Preview allows users to visually see the layout effect. Users can also make adjustments in the desktop preview image, such as changing the location of the application, to determine the final desktop layout scheme.

The above-mentioned method for determining the application to be installed may be deployed on the server and executed. Referring to FIG. 12 , an API (Application Programming Interface) of the “scanning” service is deployed on the second terminal 120, and the user performs the “scanning” operation to obtain the desktop image of the first terminal and send it to the base Server 1302. The basic server 1302 provides an identification service, performs App identification, and provides a search service. In the search service, through the information of the application library obtained from the resource service, the App identification result is matched and corrected, and the finalized App information is sent to Business server 1301. The service server 1301 generates the desktop layout of the second terminal 120 and returns it to the second terminal 120 for displaying the results. Users can provide feedback on the desktop layout, such as corrections for misidentification, feedback icons and correct App names. The feedback information is sent to the resource service of the basic server 1302, and the resource is corrected and updated according to the icon and the correct App name, so as to improve the accuracy of subsequent App identification. Subsequently, the basic server 1302 provides App download through the resource service, and automatically downloads the App to the second terminal 120 and installs it. Through the method shown in Figure 12, the installation process is mainly deployed on the server side, which reduces the pressure on the terminal side, and realizes the decoupling between the scan service and the basic service, which is conducive to flexible deployment and expansion of related service logic.

FIG. 13 shows an exemplary flow of the installation processing method performed by the second terminal, including:

In step S1301, the user selects to enable the scan function, which can be deployed as a separate client, or can be integrated into the App store or system functions, and the camera is turned on at the same time.

Step S1302, the camera scans the desktop image of the first terminal, and determines whether the scanned image is the initial frame; if so, executes step S1303, and if not, continues to determine whether the next frame of image is the initial frame.

Step S1303 , starting from the starting frame, acquiring a desktop image, performing App positioning in combination with the text and icons in the desktop image, and extracting the targets to be identified corresponding to each App.

Step S1304, matching the text with the App name provided by the App resource service. The App resource service can be deployed on the server, or can be pre-deployed on the second terminal.

In step S1305, it is judged whether the text matching is successful; if yes, step S1306 is executed; if not, step S1307 is executed.

Step S1306: Determine the corresponding App according to the matching result.

Step S1307, matching the icon with the App icon provided by the App resource service.

In step S1308, it is judged whether the icon matching is successful; if yes, go to step S1306, if not, go to step S1309.

Step S1309, returning the result that the identification is unsuccessful.

Step S1310, when all the targets to be identified have obtained corresponding App identification results, including successful identification of the corresponding App and unsuccessful identification results, determine the App to be installed on the second terminal, download and install these Apps, and follow the steps of the first terminal. desktop layout.

The whole process ends, and the App of the first terminal is migrated to the second terminal.

Exemplary embodiments of the present disclosure also provide an installation processing apparatus for an application program. Referring to FIG. 14 , the application installation processing apparatus 1400 may include a processor 1410 and a memory 1420, wherein the memory 1420 stores the following program units:

The desktop image acquisition unit 1421 is configured to acquire the desktop image of the first terminal;

The application identification unit 1422 is configured to identify the application in the desktop image through the text and/or icons in the desktop image;

The application program determination unit 1423 is configured to determine the application program to be installed of the second terminal according to the application program in the desktop image;

The processor 1410 is used to execute the above program units.

In an optional implementation manner, the desktop image acquisition unit 1421 is configured to:

Acquire one or more of the captured desktop images.

In an optional implementation manner, the above-mentioned acquisition of one or more frames of desktop images collected includes:

In the process of image acquisition, determine the starting frame;

Starting from the starting frame, take the captured image of one or more frames as the desktop image.

In an optional implementation manner, in the process of image acquisition, the determination of the starting frame includes:

In the process of image acquisition, the relative change amount of each frame of image is determined, and the relative change amount is the change amount of each frame image relative to the previous frame image;

When the relative changes of consecutive N frames of images are all smaller than the threshold of change, the start frame is determined from the consecutive N frames of images; N is a positive integer not less than 2.

In an optional implementation manner, the above-mentioned determination of the relative change amount of each frame of image includes:

The relative change amount of the current frame image is determined according to the change amount of the color distribution data of the current frame image relative to the color distribution data of the previous frame image.

In the process of image acquisition, target detection is performed on the acquired images;

When an object of a preset category is detected, the detected image is determined as the starting frame.

In the process of image acquisition, the optical flow corresponding to each frame of image is determined, and the optical flow is the optical flow between each frame of image and the previous frame of image;

When the number of pixels in the optical flow corresponding to the consecutive N frames of images are all less than the number threshold, the starting frame is determined from the consecutive N frames of images; N is a positive integer not less than 2.

In an optional implementation manner, the application identification unit 1422 is configured to:

Extract one or more objects to be recognized from the desktop image, each object to be recognized includes a text and an icon;

Identify the application program corresponding to the target to be recognized according to the text and/or icon in the target to be recognized.

In an optional implementation manner, the application identification unit 1422 is further configured to:

After extracting one or more targets to be identified from the desktop image, the extracted text or icons are filtered according to the border position of the first terminal in the desktop image and the grid point arrangement rule of the first terminal.

In an optional implementation manner, the above-mentioned through the text and/or icon in the desktop image, identify the application program in the desktop image, including:

Match each text in the desktop image with the name of the application program, and determine the application program corresponding to the text information according to the matching result.

In an optional implementation manner, the above-mentioned identifying the application program in the desktop image through the text and/or icons in the desktop image further includes:

When the text and the application name are unsuccessfully matched, the application corresponding to the text is determined according to the icon corresponding to the text.

In an optional implementation manner, the above-mentioned determination of the application program corresponding to the text according to the icon corresponding to the text includes:

forming the icon corresponding to the text into a first icon image;

Matching the first icon image with the second icon image formed by each application icon in the application library;

When there is a second icon image whose matching degree with the first icon image is greater than the matching degree threshold, the application program corresponding to the second icon image is determined as the application program corresponding to the text.

In an optional implementation manner, the application program determination unit 1423 is configured to:

generating an application list of the first terminal according to the application in the desktop image;

obtaining user feedback information for the application list of the first terminal;

The application to be installed on the second terminal is determined according to the user feedback information.

In an optional implementation manner, the application program determination unit 1423 is further configured to:

According to the location information of each application in the desktop image, the location information of the application to be installed on the desktop of the second terminal is determined.

In an optional implementation manner, the above-mentioned determining the location information of the application to be installed on the desktop of the second terminal according to the location information of each application in the desktop image includes:

Determine the order of the applications to be installed according to the location information of each application in the desktop image;

The applications to be installed are rearranged according to the order of the grid point arrangement of the desktop of the second terminal and the applications to be installed, so as to determine the location information of the applications to be installed on the desktop of the second terminal.

A desktop preview image of the second terminal is generated according to the location information of the application to be installed on the desktop of the second terminal.

In response to the adjustment operation on the desktop preview image, the location information of the application to be installed in the desktop of the second terminal is changed.

Exemplary embodiments of the present disclosure also provide another application installation processing apparatus. Referring to Fig. 15, the installation processing apparatus 1500 of the application may include:

a desktop image acquiring unit 1510, configured to acquire a desktop image of the first terminal;

The application identification unit 1520 is configured to identify the application in the desktop image through the text and/or icons in the desktop image;

The application program determining unit 1530 is configured to determine the application program to be installed on the second terminal according to the application program in the desktop image.

In an optional implementation manner, the desktop image acquisition unit 1510 is configured to:

Acquire one or more of the captured desktop images.

In the process of image acquisition, determine the starting frame;

In an optional implementation manner, the application identification unit 1520 is configured to:

In an optional implementation manner, the application identification unit 1520 is further configured to:

In an optional implementation manner, the above-mentioned identifying the application program in the desktop image through the text and/or icons in the desktop image includes:

forming the icon corresponding to the text into a first icon image;

In an optional implementation manner, the application program determination unit 1530 is configured to:

In an optional implementation manner, the application program determining unit 1530 is further configured to:

The specific details of each part in the above-mentioned apparatus have been described in detail in the method part of the implementation, and thus will not be repeated.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium, which can be implemented in the form of a program product, including program code, which, when the program product runs on a terminal device, is used to cause the terminal device to execute this specification The steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section above may be performed, for example, by any one or more of the steps in FIGS. 3 to 6 or 8 to 13 . The program product can take the form of a portable compact disk read only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory, read only memory (ROM), erasable programmable Read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

A computer readable signal medium may include a propagated data signal in baseband or as part of a carrier wave with readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium can also be any readable medium, other than a readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming Language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).

As will be appreciated by one skilled in the art, various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", "module" or "system".

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and embodiments are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

A method for installing and processing an application, comprising:

Obtain the desktop image of the first terminal;

Identify applications in the desktop image through the text and/or icons in the desktop image;

The application to be installed on the second terminal is determined according to the application in the desktop image.
The method according to claim 1, wherein the acquiring the desktop image of the first terminal comprises:

Controlling the camera to be turned on to capture images of the desktop of the first terminal;

Acquire one or more of the captured desktop images.
The method according to claim 2, wherein the acquiring one or more frames of desktop images collected comprises:

In the process of image acquisition, determine the starting frame;

Starting from the starting frame, the captured image of one or more frames is used as the desktop image.
The method according to claim 3, wherein, in the process of image acquisition, determining the starting frame, comprising:

In the process of image acquisition, determine the relative change amount of each frame of image, where the relative change amount is the change amount of each frame image relative to the previous frame image;

When the relative change amounts of consecutive N frames of images are all smaller than the change amount threshold, the start frame is determined from the consecutive N frames of images; N is a positive integer not less than 2.
The method according to claim 4, wherein the determining the relative change amount of each frame of image comprises:

The relative change amount of the current frame image is determined according to the change amount of the color distribution data of the current frame image relative to the color distribution data of the previous frame image.
The method according to claim 3, wherein, in the process of image acquisition, determining the starting frame, comprising:

In the process of image acquisition, target detection is performed on the acquired images;

When a target of a preset category is detected, the detected image is determined as the starting frame.
The method according to claim 3, wherein, in the process of image acquisition, determining the starting frame, comprising:

In the process of image acquisition, determine the optical flow corresponding to each frame of image, and the optical flow is the optical flow between the each frame of image and the previous frame of image;

When the number of pixels in the optical flow corresponding to the consecutive N frames of images are all less than the number threshold, the starting frame is determined from the consecutive N frames of images; N is a positive integer not less than 2.
The method according to claim 1, wherein the identifying the application program in the desktop image through the text and/or icons in the desktop image comprises:

Extract one or more objects to be identified from the desktop image, each of the objects to be identified includes a text and an icon;

According to the text and/or the icon in the to-be-recognized target, the application program corresponding to the to-be-recognized target is identified.
The method according to claim 8, wherein after extracting one or more targets to be identified from the desktop image, the method further comprises:

The extracted text or icon is filtered according to the position of the frame of the first terminal in the desktop image and the arrangement rule of the desktop grid of the first terminal.
The method according to claim 1, wherein the identifying the application program in the desktop image through the text and/or icons in the desktop image comprises:

Matching each text in the desktop image with the application program name respectively, and determining the application program corresponding to the text information according to the matching result.
The method according to claim 10, wherein the identifying the application in the desktop image through the text and/or icons in the desktop image further comprises:

When the text and the application name are unsuccessfully matched, the application corresponding to the text is determined according to the icon corresponding to the text.
The method according to claim 11, wherein the determining the application program corresponding to the text according to the icon corresponding to the text comprises:

forming the icon corresponding to the text into a first icon image;

matching the first icon image with the second icon image formed by each application icon in the application library;

When there is a second icon image whose matching degree with the first icon image is greater than a matching degree threshold, the application program corresponding to the second icon image is determined as the application program corresponding to the text.
The method according to claim 1, wherein the determining the application to be installed on the second terminal according to the application in the desktop image comprises:

generating an application list of the first terminal according to the application in the desktop image;

obtaining user feedback information for the application list of the first terminal;

The application to be installed on the second terminal is determined according to the user feedback information.
The method according to claim 1, wherein the method further comprises:

The location information of the application to be installed on the desktop of the second terminal is determined according to the location information of each application in the desktop image.
The method according to claim 14, wherein the location information of the application to be installed in the desktop of the second terminal is determined according to the location information of each application in the desktop image, include:

determining the sequence of the applications to be installed according to the location information of the applications in the desktop image;

The to-be-installed applications are rearranged according to the order of the desktop grid arrangement of the second terminal and the to-be-installed applications to determine the position of the to-be-installed application on the desktop of the second terminal information.
The method of claim 14, wherein the method further comprises:

A desktop preview image of the second terminal is generated according to the location information of the application to be installed in the desktop of the second terminal.
The method of claim 16, wherein the method further comprises:

In response to the adjustment operation on the desktop preview image, the location information of the application to be installed in the desktop of the second terminal is changed.
An installation processing device for an application program, characterized in that it comprises a processor and a memory, and the processor is configured to execute the following program units stored in the memory:

a desktop image acquisition unit, configured to acquire the desktop image of the first terminal;

an application identification unit, configured to identify an application in the desktop image through text and/or icons in the desktop image;

The application program determination unit is configured to determine the application program to be installed on the second terminal according to the application program in the desktop image.
A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.
An electronic device, comprising:

processor; and

a memory for storing executable instructions for the processor;

wherein the processor is configured to perform the method of any one of claims 1 to 13 by executing the executable instructions.