WO2022057419A1

WO2022057419A1 - Method and apparatus for acquiring information related to subject, and storage medium and electronic device

Info

Publication number: WO2022057419A1
Application number: PCT/CN2021/106300
Authority: WO
Inventors: 金越; 李亚乾; 郭彦东; 杨林
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-09-21
Filing date: 2021-07-14
Publication date: 2022-03-24
Also published as: CN114253437A

Abstract

A method and apparatus for acquiring information related to a subject, and a computer-readable storage medium and an electronic device, which relate to the technical field of terminals. The method for acquiring information related to a subject comprises: acquiring an image of a target application program icon (S32); on the basis of the image of the target application program icon, detecting whether there is a target subject in a subject library, wherein the pattern of an application program icon of the target subject is consistent with the pattern of the target application program icon (S34); and upon detecting that there is the target subject in the subject library, acquiring information related to the target subject (S36). By means of the method, information related to a subject can be acquired conveniently and quickly.

Description

Subject-related information acquisition method and device, storage medium and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application with the application number of 202010997678.7 and the title of "Subject-related information acquisition method and device, storage medium and electronic device" filed on September 21, 2020. The entire content of the Chinese patent application is approved by Reference is incorporated herein in its entirety.

technical field

The present disclosure relates to the technical field of terminals, and in particular, to a subject-related information acquisition method, a subject-related information acquisition apparatus, a computer-readable storage medium, and an electronic device.

Background technique

The theme of the terminal device can provide users with different interface display effects, which is an important aspect of user presentation personalization. When a user wants to know the subject of the terminal device held by others, he often needs to ask others for the name of the subject, and then use his own terminal device to query the subject, so as to obtain relevant information of the subject.

However, users need to ask others and inquire on the terminal device. It can be seen that the current process of obtaining topic-related information is cumbersome, inconvenient, time-consuming and labor-intensive.

SUMMARY OF THE INVENTION

According to a first aspect of the present disclosure, there is provided a method for obtaining theme-related information, including: obtaining an image of a target application icon; detecting whether a target theme exists in a theme library based on the image of the target application icon, and the application The style of the icon is consistent with the style of the icon of the target application; when it is detected that the target theme exists in the theme library, the relevant information of the target theme is obtained.

According to a second aspect of the present disclosure, there is provided a subject-related information acquisition device, comprising: an image acquisition module configured to acquire an image of a target application icon; a subject detection module configured to be based on the image of the target application icon, Detecting whether the target theme exists in the theme library, and the style of the application icon of the target theme is consistent with the style of the target application icon; the first theme information obtaining module is configured to obtain the target theme when it is detected that the theme library exists. Information about the target topic.

According to a third aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the above-mentioned method for obtaining subject-related information.

According to a fourth aspect of the present disclosure, there is provided an electronic device, comprising a processor; a memory configured to store one or more programs, which, when executed by the processor, cause the processor to implement the above-mentioned How to get information about the subject.

Description of drawings

FIG. 1 shows a schematic diagram of a device involved in a subject-related information acquisition solution according to an embodiment of the present disclosure;

FIG. 2 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure;

FIG. 3 schematically shows a flowchart of a method for acquiring subject-related information according to an exemplary embodiment of the present disclosure;

Figure 4 shows a schematic diagram of scanning an image of an application icon in response to a user's operation;

Fig. 5 shows the schematic diagram of the relative positional relationship between the application image and the corresponding text;

Fig. 6 shows the schematic diagram of acquiring and displaying the download information of the target theme;

FIG. 7 schematically shows a flowchart of the entire subject-related information acquisition process according to an embodiment of the present disclosure;

FIG. 8 schematically shows a block diagram of an apparatus for obtaining subject-related information according to an exemplary embodiment of the present disclosure;

FIG. 9 schematically shows a block diagram of an apparatus for obtaining subject-related information according to another exemplary embodiment of the present disclosure;

Fig. 10 schematically shows a block diagram of a subject-related information acquisition apparatus according to still another exemplary embodiment of the present disclosure.

detailed description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the figures are functional entities that do not necessarily necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow charts shown in the figures are merely illustrative and do not necessarily include all steps. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation. In addition, all the following terms "first", "second" and "third" are for the purpose of distinction only, and should not be used as a limitation of the present disclosure.

FIG. 1 shows a schematic diagram of a device involved in a subject-related information acquisition solution according to an embodiment of the present disclosure.

Referring to FIG. 1 , the terminal device 1001 may be a device to acquire topic-related information, that is, the terminal device 1001 executes the topic-related information acquisition scheme described in the present disclosure. The terminal device 1001 may include, but is not limited to, a mobile phone, a tablet computer, a smart wearable device such as a smart watch, and the like.

The terminal device 1001 can be made to acquire the image of the second application icon through the terminal device 1002 or the server 1003 . The terminal device 1002 may be a mobile phone, a tablet computer, a personal computer, a smart wearable device such as a smart watch, or the like. In addition, the numbers of terminal devices and servers in FIG. 1 are only illustrative. According to actual needs, there may be any number of terminal devices and servers. For example, the server 1003 may be a server cluster composed of multiple servers.

In some embodiments of the present disclosure, the terminal device 1001 may scan the display interface of the terminal device 1002, and the display interface of the terminal device 1002 displays the icon of the target application. Thus, the terminal device 1001 can acquire the image of the target application icon. Next, the terminal device 1001 searches the theme library based on the image of the target application icon to detect whether the target theme exists in the theme library, wherein the style of the application icon of the target theme is consistent with the style of the target application icon. When it is detected that there is a target topic, relevant information of the target topic is acquired.

In other embodiments of the present disclosure, the terminal device 1001 may take pictures or videos of the display interface of the terminal device 1002, and the display interface of the terminal device 1002 displays the icon of the target application. Thus, the terminal device 1001 can acquire the image of the target application icon. Next, the terminal device 1001 searches the theme library based on the image of the target application icon to detect whether the target theme exists in the theme library, wherein the style of the application icon of the target theme is consistent with the style of the target application icon. When it is detected that there is a target topic, relevant information of the target topic is acquired.

In the above embodiment, there is no data communication between the terminal device 1001 and the terminal device 1002 .

In still other embodiments of the present disclosure, the terminal device 1002 may perform a screen capture or screen recording operation, and send the generated image or video to the terminal device 1001, so that the terminal device 1001 may obtain the image of the target application icon. Next, the terminal device 1001 searches the theme library based on the image of the target application icon to detect whether the target theme exists in the theme library, wherein the style of the application icon of the target theme is consistent with the style of the target application icon. When it is detected that there is a target topic, relevant information of the target topic is acquired.

In still other embodiments of the present disclosure, the terminal device 1001 may acquire the image of the target application icon from the server 1003 . For example, the terminal device 1001 may acquire the image of the target application icon from the web page. Next, the terminal device 1001 searches the theme library based on the image of the target application icon to detect whether the target theme exists in the theme library, wherein the style of the application icon of the target theme is consistent with the style of the target application icon. When it is detected that there is a target topic, relevant information of the target topic is acquired.

In addition, the relevant information of the target theme described in the present disclosure may be one or more of the information related to the download of the target theme, the introduction information of the target theme, and the installation program of the target theme. This is so that the user can view the topic-related information, or the terminal device 1001 automatically performs a topic switching operation, so as to switch the topic of the terminal device 1001 to the target topic.

It should be noted that the subject-related information acquisition method of the exemplary embodiment of the present disclosure is generally executed by the terminal device 1001 , and accordingly, the subject-related information acquisition apparatus described below is generally configured in the terminal device 1001 .

Figure 2 shows a schematic diagram of an electronic device suitable for use in implementing exemplary embodiments of the present disclosure. The terminal device described in the present disclosure (the first device described below) can be configured as shown in FIG. 2 . It should be noted that the electronic device shown in FIG. 2 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

The electronic device of the present disclosure includes at least a processor and a memory, and the memory is configured to store one or more programs, which, when executed by the processor, enable the processor to realize the subject-related information of the exemplary embodiments of the present disclosure get method.

Specifically, as shown in FIG. 2, the electronic device 200 may include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, and a power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, speaker 271, receiver 272, microphone 273, headphone jack 274, sensor module 280, display screen 290, camera module 291 , an indicator 292, a motor 293, a key 294, a Subscriber Identification Module (SIM) card interface 295, and the like. The sensor module 280 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 200 . In other embodiments of the present application, the electronic device 200 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or neural network processor (Neural-network Processing Unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. In addition, a memory may also be provided in the processor 210 for storing instructions and data.

The electronic device 200 implements a display function through a GPU, a display screen 290, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 290 and the application processor. The GPU is used for performing mathematical and geometric calculations and for graphics rendering. In the scenario where the present disclosure is used for implementing theme switching, the GPU can be used to realize the rendering of the theme icon style. Processor 210 may include one or more GPUs that execute program instructions to generate or alter display information.

The electronic device 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290, the application processor, and the like. In some embodiments, the electronic device 200 may include one or N camera modules 291 , where N is a positive integer greater than 1. If the electronic device 200 includes N cameras, one of the N cameras is the main camera.

For example, the display interface of another electronic device can be scanned or photographed by the camera module 291 to obtain an image of the icon of the target application, and then the process of obtaining the relevant information of the target theme can be realized based on the image.

Internal memory 221 may be used to store computer executable program code, which includes instructions. The internal memory 221 may include a storage program area and a storage data area. The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200.

The present application also provides a computer-readable storage medium. The computer-readable storage medium may be included in the electronic device described in the above-mentioned embodiments, or it may exist alone without being assembled into the electronic device.

The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

The computer-readable storage medium can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a computer-readable storage medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to implement the methods described in the following embodiments.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The units involved in the embodiments of the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.

The details of the solution will be described below by taking the first device implementing the subject-related information acquisition solution of the exemplary embodiment of the present disclosure as an example. The first device corresponds to the above-mentioned terminal device 1001, and can be configured in the form of the device shown in FIG. 2 .

FIG. 3 schematically shows a flowchart of a method for acquiring subject-related information according to an exemplary embodiment of the present disclosure. Referring to FIG. 3 , the method for obtaining the subject-related information may include the following steps:

S32. Obtain an image of the target application icon.

According to some embodiments of the present disclosure, the first device may perform a scanning operation on the interface of the second device to acquire the image of the icon of the target application. The image of the target application icon is displayed on the interface of the second device. It should be understood that the application program described in the present disclosure is the so-called APP.

FIG. 4 shows a schematic diagram of scanning an image of an application program icon in response to a user's operation.

Referring to FIG. 4 , first, the user can click the theme APP icon on the first device 41 to enter the theme APP. Next, in the theme APP, the scan operation can be implemented through the configured scan control, that is, the user can click the "scan" icon. Subsequently, during the scanning process, if the display interface of the second device 42 is scanned, and the display interface displays the image of the icon of the target application program, the first device 41 can obtain the image of the icon of the target application program.

In addition to the exemplary scenario shown in FIG. 4 , the user can also perform a scanning operation in, for example, a camera APP, etc., to scan the second device, and further obtain the image of the icon of the target application by scanning.

In view of the possibility of misidentification and waste of resources. The present disclosure may also include a solution in which the first device identifies the second device.

The process of recognizing the second device can also be called the process of detecting the second device. The detection method can include using a deep learning algorithm such as CNN (Convolutional Neural Network, convolutional neural network) to train a target detection model, according to the target. The detection model performs object detection on the scanned image. The target detection model generally includes an input layer, a hidden layer and an output layer. The input layer is used to receive the scanned image, the hidden layer is used to process the received image, and the output layer is used to output the result of the image processing. The position and size information of the second device in the screen is determined. Specifically, the target detection algorithm may include SSD, Faster-RCNN, etc. The purpose of target detection is to determine whether it is a terminal device.

Given that the scanned image of the target application icon may be small, it cannot be further identified.

In one embodiment, before detecting the second device and before acquiring the image of the target application icon, the first device determines the size of the scanning area that the image of the interface of the second device occupies, if the image of the interface of the second device occupies the entire size of the scanning area When the size of the scanning area exceeds the size threshold, the operation of acquiring the image of the icon of the target application is performed. If the size threshold is not exceeded, the user may be prompted on the scanning interface to adjust the position of the first device. The size threshold may be, for example, 2/3 of the scanning area of the first device. However, the present disclosure does not limit the specific value of the size threshold.

In another embodiment, before detecting the second device and before acquiring the image of the target application icon, the first device determines the size of the image of the target application icon on the interface of the second device, if the size of the image of the target application icon is If the size exceeds the icon size threshold, the operation of obtaining the image of the target application icon is performed. If the icon size threshold is not exceeded, the user may be prompted to adjust the position of the first device on the scanning interface. Similarly, the present disclosure does not limit the specific value of the icon size threshold.

In addition, in view of the fact that when the first device performs the scanning operation, the obtained image of the target application program icon may be blurred due to the shaking of the first device.

In yet another embodiment, the first device can detect whether the first device is in a state of stable position, and the stable position can be understood as that the position and posture of the mobile phone have not changed in several consecutive frames. For example, it can be determined by means of a color histogram whether the first device is in a positionally stable state.

Specifically, first, a color histogram corresponding to each frame of image can be generated by using multiple frames of images obtained based on the scanning operation; then, according to the color histogram corresponding to each frame of image, it is determined whether the first device is in a stable position , for example, determine whether the proportion of each color in the color histogram has changed within several consecutive frames (for example, 5 frames, 10 frames, etc.), if there is no change or the degree of change is less than a threshold, it is considered that the first device is in a stable position status. If the first device is not in a positionally stable state, the user may be prompted on the scanning interface.

In an embodiment in which both the size occupied by the image and the stability of the device are considered, the size of the scanning area occupied by the image of the interface of the second device may be determined, and if the size of the scanning area occupied by the image of the interface of the second device exceeds the size threshold, detection is performed. Whether the first device is in a stable position, if it is detected that the first device is in a stable position, the first device may perform an operation of acquiring the image of the target application icon.

In addition, it is also possible to first determine whether the first device is in a stable position, and then determine whether the size of the image of the interface of the second device occupying the scanning area exceeds the size threshold. Or judge these two separately, judge them separately, and then combine the results.

The above only describes the process of acquiring the image of the icon of the target application program by the first device using the scanning operation for the second device.

According to other embodiments of the present disclosure, the first device may acquire a video including an image of an icon of a target application. The video may come from the first device recording the display interface of the second device. It may also be a video generated after the second device itself performs a screen recording operation, and the second device sends the video to the first device. In addition, the video may be a video obtained by the first device from the server, which is not limited in the present disclosure.

After the first device obtains the video, the video frame may be identified and processed to obtain the image of the icon of the target application.

In addition to the video, the first device may also obtain a picture containing the icon of the target application. The picture may be generated by the second device taking a screenshot, in this case, the second device may send the picture to the first device. Alternatively, the first device may obtain the picture from the server, and after obtaining the picture, may perform identification processing on the picture to obtain the image of the target application icon.

It should be noted that, the number of images of the target application icon acquired in step S32 may be one or more. In the case of acquiring images of multiple target application icons, it is easy to understand that the styles of these target application icons are consistent. In addition, the style of the application icon may also be referred to as the style of the application icon.

S34. Based on the image of the target application program icon, detect whether the target theme exists in the theme library, and the style of the target theme application program icon is consistent with the style of the target application program icon.

In an exemplary embodiment of the present disclosure, the topic library refers to a collection of topics that can be queried by the first device. The themes in the theme library may generally include themes that come with the first device system and themes in the installed theme APP. In addition, the theme library can also be a collection of all themes including Internet resources. It can be understood that the process of detecting whether the target theme exists in the theme database is the process of searching for the target theme from the theme database.

It should be noted that the image, name and other information of the application icon of each theme in the theme library can be obtained directly.

According to some embodiments of the present disclosure, the first device may compare the similarity between the image of the target application icon and the image of the application icon of each theme in the theme library, and determine whether there is a theme library according to the result of the similarity comparison target topic.

The image of the target application icon is marked as image A, and the image of the application icon of a theme in the theme library is marked as image B. The following is an example of the similarity comparison process between image A and image B. The process of comparing the similarity between the image of the theme and the image of the application icon of each theme in the theme library will be explained.

First, the image features of image A can be extracted. The image features may include one or more of global features, local features, color features, shape features, and texture features of the image. Among them, global features are helpful for retrieval at the semantic level; local features can be used for local key point matching, which helps to find local identical points; color features can make images have richer representations; shape features and texture features can Helps to identify approximate differences between images.

The way of extracting image features includes, but is not limited to, using a neural network. For example, the shape feature can be the output of a convolutional layer in a convolutional neural network. The present disclosure does not limit the network structure and training method of the neural network.

Next, the image features of image B can be acquired. The features of image B can be extracted in the same way as image A is extracted. Alternatively, the image features of the image B have been stored in the storage unit corresponding to the theme library, and can be directly called when needed.

When calculating the similarity between the image feature of image A and the image feature of image B, the image feature of image A can be converted into a vector, and the image feature of image B can be converted into a vector, thus, the similarity of the two vectors can be calculated by calculating The degree of similarity between image A and image B is determined, and the similarity between two vectors can be calculated by cosine similarity, Euclidean distance, Manhattan distance, Mahalanobis distance, etc.

For the image of the application icon of each theme in the theme library, the above process of calculating the similarity with the image of the target application icon may be performed. Then, the maximum value of the similarity is determined, and if the maximum value is greater than the first similarity threshold, the subject of the icon image corresponding to the maximum similarity is determined as the target subject. For example, among the multiple calculated similarities, the similarity between image W and image A is the largest, and then it is determined whether the similarity between image W and image A is greater than the first similarity threshold, and if it is greater, the subject corresponding to image W is determined. for the target subject.

According to other embodiments of the present disclosure, whether the target theme exists in the theme library can also be detected in combination with the name of the application.

First, the first device may acquire the text image corresponding to the icon of the target application program.

As shown in Figure 5, on the interface, the text area corresponding to the application (ie the name of the application) is usually located below the icon of the application, and the upper border of the text area can be the lower border of the icon, and the lower border of the text area Usually in the middle of the icon's lower border and the lower icon's upper border. In addition, the left (right) border of the text area is usually located where the current chart extends a certain distance to the left (right).

Accordingly, the image can be cropped to obtain the text image corresponding to the icon of the target application.

Next, the first device may use the text image corresponding to the target application program icon and the image of the target application program icon to detect whether the target theme exists in the theme library.

Specifically, text recognition can be performed on the text image to determine the name of the target application. A candidate application whose name is the same as that of the target application is determined from the theme library, and a candidate application set is constructed, wherein one or more candidate applications exist in the candidate application set. Then obtain images of each candidate application program icon, compare the image of the target application program icon with the image of each candidate application program icon for similarity, and determine whether there is a target theme in the theme library according to the result of the similarity comparison.

For the process of determining whether there is a target theme in these embodiments, the maximum value of the similarity may be determined, and if the answer of the maximum value corresponds to the second similarity threshold, then the theme of the candidate application icon corresponding to the maximum similarity is determined as target topic. For example, among the calculated similarities, the similarity between the image V of the candidate application icon and the image A is the largest, then determine whether the similarity between the image V of the candidate application icon and the image A is greater than the second similarity threshold, if If it is greater than that, the theme corresponding to the candidate application icon is determined as the target theme.

As far as the above text recognition process is concerned, the method of OCR (Optical Character Recognition, Optical Character Recognition) can be used to realize. In this case, before the recognition, the first device may further preprocess the text image, and the preprocessing process includes binarization and noise reduction processing, so as to reduce the difficulty of the feature extraction algorithm and improve the recognition accuracy. In addition, the text line can also be detected to obtain the inclination direction of the text, so as to correct the inclination of the text.

For another example, the trained feature extraction network can be used to extract the text features. Here, a sequence network such as LSTM (Long Short-Term Memory, long short-term memory network) can be used to extract the sequence features of the image to form a feature vector. Then, the feature vector of the application name of each theme in the theme library can be obtained, and the comparison is carried out to determine the candidate application program whose name is the same as that of the target application program in the theme library, and then combined with the above-mentioned image similarity comparison process, Finalize the target theme.

Considering that the character recognition may be wrongly recognized and the number of application programs installed by the user is limited, the present disclosure also provides a solution for correcting the recognized characters.

First, the first device performs text recognition on the text image to obtain the first middle name;

Next, the first device compares the first intermediate name with the names in the application name library. The application name library may be pre-configured and may be stored in the first device or other devices capable of data communication with the first device.

If the first intermediate name is consistent with a name in the application name library, the first intermediate name may be determined as the name of the target application.

If the first intermediate name is inconsistent with the names in the application name library, the first intermediate name may be corrected to determine the name of the target application.

For the name correction process, the first device may determine the name with the greatest similarity to the first intermediate name in the application name library as the second intermediate name. This process can be understood as a character alignment process.

If the similarity between the first middle name and the second middle name is greater than the name similarity threshold, the second middle name may be used as the name of the target application.

For example, the result identified by OCR is "Wi-Fi only month key", which can be corrected to "WiFI master key" after being processed by this solution.

If the above conditions are not satisfied, that is, the name of the recognized application program is far from the name of the application program name library, the word processing process of the application program can be eliminated.

In addition, considering that when designing themes, designers usually only theme the system icons and commonly used icons. Therefore, in the embodiment of judging in combination with words, it is also possible to judge whether the target application is an icon modified during theme design in combination with words, and if so, execute the above-mentioned image similarity comparison process, and if not, terminate the target application. Analysis process of the program icon.

That is to say, before the above-mentioned determination of candidate applications, in still other embodiments of the present disclosure, the first device may further determine, according to the name of the target application, whether the icon of the target application is an icon modified during theme design, wherein , the icons modified during theme design refer to the system icons and commonly used icons designed by the person designing the theme. In the case where the icon of the target application is the icon modified at the time of theme design, the operation of determining the candidate application is performed.

S36. When it is detected that the target theme exists in the theme library, obtain relevant information of the target theme.

In an exemplary embodiment of the present disclosure, the relevant information of the target theme may be one or more of information related to the download of the target theme, introduction information of the target theme, and an installation program of the target theme.

According to an embodiment of the present disclosure, the relevant information of the target theme is the installation program of the target theme. After acquiring the installation program, the first device can automatically run the installation program to switch the current theme of the first device to the target theme. Specifically, the first device may obtain the installation program of the target theme from the server, for example, by means of a theme store.

The process of switching themes in this step does not require user participation, and is automatically executed by the first device.

According to another embodiment of the present disclosure, the relevant information of the target theme is download information of the target theme, so that a download interface corresponding to the download information is displayed. On the download interface, the target theme can be downloaded in response to the user's download operation.

As shown in FIG. 6 , the download information may include topic name, topic size, developer, topic source, topic popularity, and the like. This information is displayed on the download interface, which can be presented in the form of a pop-up window. On the download interface, you can configure the "Download" button. After the user clicks, the download of the target theme can be realized.

This scheme of downloading the theme based on the user's further operation provides the user with a confirmation stage to avoid resource consumption caused by wrong download.

In the solutions of the above embodiments, the target theme is detected, thereby obtaining theme-related information corresponding to the icon of the target application program, thereby helping the user to view the theme-related information and/or realize the switching of the theme. The process is simple and fast.

In addition, when the first device detects that the target theme does not exist in the theme library, the present disclosure also provides a solution for assisting in determining the theme.

First, the first device can acquire an image of the target wallpaper. The target wallpaper described in the exemplary embodiment of the present disclosure may be a lock screen wallpaper or a desktop wallpaper.

Similar to the process of acquiring the image of the target application program icon in the above step S32, the first device may scan or record the display interface of the second device, and the display interface of the second device displays the target wallpaper. Thereby, the first device can acquire the image of the target wallpaper. Alternatively, the second device can take screenshots or screen recordings of its own display interface, and send the pictures or videos to the first device, so that the first device can also acquire the image of the target wallpaper. In addition, the first device may also acquire the image of the target wallpaper from the server.

Next, the first device may detect whether there is a first candidate theme in the theme library based on the image of the target wallpaper, and the wallpaper of the first candidate theme is consistent with the target wallpaper.

Specifically, the similarity between the image of the target wallpaper and the wallpaper of each theme in the theme library may be compared, and according to the result of the similarity comparison, it is determined whether there is a first candidate theme in the theme library.

On the one hand, features of the target wallpaper can be extracted, the features include one or more of global features, local features, color features, shape features, and texture features of the wallpaper image. Then, the characteristics of the wallpaper corresponding to each theme in the theme database are obtained, and the comparison process of the feature similarity is used to determine whether there is a first candidate theme in the theme database. It can be understood that this method of image retrieval is helpful for the expansion of the wallpaper templates of each theme in the theme library, and it is only necessary to add wallpapers to the theme library and correspond to the theme.

On the other hand, the trained neural network can be used for classification processing to determine the first candidate theme whose wallpaper is consistent with the target wallpaper from the theme library.

For the process of specifically determining whether there is a first candidate theme in the theme library, the maximum value of the similarity can be determined, and if the maximum value is greater than the third similarity threshold, the theme of the wallpaper corresponding to the maximum similarity value is determined as the first a candidate subject. For example, among the multiple calculated similarities, the similarity between wallpaper y and the target wallpaper is the largest, then determine whether the similarity between wallpaper y and the target wallpaper is greater than the third similarity threshold, and if it is greater, determine the theme corresponding to wallpaper y. as the first candidate topic.

Subsequently, when it is detected that the first candidate topic exists in the topic database, relevant information of the first candidate topic is acquired. Similarly, the related information of the first candidate theme may be one or more of information related to the download of the first candidate theme, introduction information of the first candidate theme, and an installation program of the first candidate theme. This further helps the user to view the relevant information of the first candidate topic and/or switch the topic of the first device to the first candidate topic.

In addition, in the case where it is detected that the first candidate topic does not exist in the topic library, some embodiments of the present disclosure may also determine the second candidate topic from the topic library. The second candidate theme is the theme with the greatest similarity between the image of the application icon in the theme library and the image of the target application icon. Based on the process of detecting whether there is a target theme in step S34, the second candidate theme can be determined through each similarity calculated in this process. In this case, relevant information on the second candidate topic can be acquired. Similarly, the related information of the second candidate theme may be one or more of information related to the download of the second candidate theme, introduction information of the second candidate theme, and an installation program of the second candidate theme. This further helps the user to view the relevant information of the second candidate topic and/or switch the topic of the first device to the second candidate topic.

FIG. 7 schematically shows a flow chart of the entire subject-related information acquisition process according to an embodiment of the present disclosure.

Referring to FIG. 7 , in step S702, the first device performs APP icon positioning to determine the positions and sizes of all APP icons in the image, so as to facilitate subsequent text recognition and icon recognition. Specifically, a deep learning target detection method or an image edge detection method can be used to locate the APP icon.

In step S704, the first device may crop the APP icon and the text area, so as to perform the processing of text recognition and icon recognition respectively.

In step S706, the first device performs text recognition and correction; in step S708, the first device determines system icons and commonly used icons from the image based on the recognized text, and these icons are icons that need to be modified in style during theme design .

In step S710, the first device performs icon image comparison to determine the similarity between the APP icon in the image and the icon in the theme library. The similarity here may be the average similarity obtained after comparing the similarity between each APP icon in the image and the icon in the theme library, that is, a theme in the theme library corresponds to an average similarity.

In step S712, it is judged whether the target theme can be determined, if yes, then step S714 is executed; if not, step S716 is executed.

In step S714, the first device obtains relevant information of the target topic.

In step S716, a wallpaper image is obtained, and the wallpaper similarity is compared.

In step S718, it is judged whether the first candidate theme can be determined based on the comparison result of the similarity of the wallpapers, if yes, go to step S720; if not, go to step S722.

In step S720, the first device obtains relevant information of the first candidate topic.

In step S722, a second candidate theme is determined by using the similarity of the APP icons, and the second candidate theme is the theme with the greatest similarity between the icon and the obtained APP icon.

In step S724, the first device obtains relevant information of the second candidate topic.

To sum up, in the method for obtaining subject-related information in an exemplary embodiment of the present disclosure, based on the obtained image, the image is analyzed to determine the subject corresponding to the image, and the related information of the subject is obtained. Thereby, it is helpful for the user to view the relevant information of the topic and/or realize the switching of the topic, and the whole process is simple and fast.

It should be noted that although the various steps of the methods of the present disclosure are depicted in the figures in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps must be performed to achieve the desired the result of. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, and the like.

Further, an apparatus for acquiring topic-related information is also provided in this exemplary embodiment.

FIG. 8 schematically shows a block diagram of a subject-related information acquisition apparatus of an exemplary embodiment of the present disclosure. Referring to FIG. 8 , the subject-related information acquisition apparatus 8 according to an exemplary embodiment of the present disclosure may include an image acquisition module 81 , a subject detection module 83 , and a first subject information acquisition module 85 .

Specifically, the image acquisition module 81 may be configured to acquire the image of the target application icon; the theme detection module 83 may be configured to detect whether the target theme exists in the theme library based on the image of the target application icon, and the application icon of the target theme The style of the icon is consistent with the style of the target application icon; the first theme information obtaining module 85 may be configured to obtain relevant information of the target theme in the case of detecting that the target theme exists in the theme library.

According to an exemplary embodiment of the present disclosure, the theme detection module 83 may be configured to perform: compare the similarity between the image of the target application icon and the image of the application icon of each theme in the theme library; according to the result of the similarity comparison, Determine if the target theme exists in the theme library.

According to an exemplary embodiment of the present disclosure, the theme detection module 83 may also be configured to perform: acquiring a text image corresponding to the icon of the target application; detecting the theme library by using the text image corresponding to the icon of the target application and the image of the icon of the target application Whether the target topic exists in .

According to an exemplary embodiment of the present disclosure, the process of detecting whether a target theme exists in the theme library by the theme detection module 83 may also be configured to perform: character recognition on text images to determine the name of the target application; Determine a candidate application with the same name as the target application, build a candidate application set; obtain the image of each candidate application icon in the candidate application set; compare the image of the target application icon with the image of each candidate application icon Carry out similarity comparison; according to the result of similarity comparison, determine whether there is a target theme in the theme library.

According to an exemplary embodiment of the present disclosure, the process of performing text recognition on the text image by the subject detection module 83 to determine the name of the target application may be configured to perform: text recognition on the text image to obtain the first middle name; The first intermediate name is compared with the name in the application name library; if the first intermediate name is consistent with a name in the application name library, the first intermediate name is determined as the name of the target application; if the first intermediate name is consistent with a name in the application name library If the name is inconsistent with the name in the application name library, the first intermediate name is corrected to determine the name of the target application.

According to an exemplary embodiment of the present disclosure, the process in which the subject detection module 83 corrects the first intermediate name to determine the name of the target application program may be configured to perform: determining that the application program name library is most similar to the first intermediate name The name of the second middle name is used as the second middle name; if the similarity between the first middle name and the second middle name is greater than the name similarity threshold, the second middle name is used as the name of the target application.

According to an exemplary embodiment of the present disclosure, before determining the candidate application program, the theme detection module 83 may be further configured to execute: according to the name of the target application program, determine whether the icon of the target application program is an icon modified during theme design; wherein, In the case where the icon of the target application is the icon modified at the time of theme design, the operation of determining the candidate application is performed.

According to an exemplary embodiment of the present disclosure, the first theme information obtaining module 85 may be configured to perform: obtaining an installation program of the target theme, so as to run the installation program of the target theme, and switch the theme to the target theme.

According to an exemplary embodiment of the present disclosure, the first theme information obtaining module 85 may be further configured to perform: obtaining download information of a target theme so as to display a download interface corresponding to the download information, and downloading the target theme in response to a user's download operation.

According to an exemplary embodiment of the present disclosure, the apparatus for obtaining subject-related information is configured in the first device, and the image obtaining module 81 may be configured to perform: scanning the interface of the second device to obtain the image of the target application icon; The image of the target application icon is displayed on the interface of the second device.

According to an exemplary embodiment of the present disclosure, the image acquisition module 81 may be further configured to perform: before acquiring the image of the target application program icon, determine the size of the scan area that the image of the interface of the second device occupies; if the interface of the second device occupies the size of the scanning area; If the size of the image occupying the scanning area exceeds the size threshold, the image of the icon of the target application is obtained.

According to an exemplary embodiment of the present disclosure, the image acquisition module 81 may be further configured to perform: before acquiring the image of the target application program icon, determine the size of the scan area that the image of the interface of the second device occupies; if the interface of the second device occupies the size of the scanning area; If the size of the image in the scanning area exceeds the size threshold, it is detected whether the first device is in a stable position; if the first device is in a stable position, an image of the icon of the target application is acquired.

According to an exemplary embodiment of the present disclosure, the process in which the image acquisition module 81 detects whether the first device is in a positionally stable state may be configured to perform: generating a color histogram corresponding to each frame of images using multiple frames of images obtained based on the scanning operation Figure; according to the color histogram corresponding to each frame of image, it is determined whether the first device is in a state of stable position.

According to an exemplary embodiment of the present disclosure, the image acquisition module 81 may be further configured to perform: acquiring a video including an image of a target application icon; and acquiring an image of the target application icon from the video.

According to an exemplary embodiment of the present disclosure, referring to FIG. 9 , compared with the topic-related information acquiring apparatus 8 , the topic-related information acquiring apparatus 9 may further include a second topic information acquiring module 91 .

Specifically, the second theme information obtaining module 91 may be configured to execute: when it is detected that the target theme does not exist in the theme library, obtain the image of the target wallpaper; based on the image of the target wallpaper, detect whether the first theme exists in the theme library The candidate theme, the wallpaper of the first candidate theme is consistent with the target wallpaper; in the case of detecting that the first candidate theme exists in the theme library, relevant information of the first candidate theme is obtained.

According to an exemplary embodiment of the present disclosure, the process in which the second theme information acquisition module 91 detects whether the first candidate theme exists in the theme library based on the image of the target wallpaper may be configured to perform: compare the image of the target wallpaper with each theme in the theme library Similarity comparison is performed on the images of the wallpaper; according to the result of the similarity comparison, it is determined whether there is a first candidate theme in the theme library.

According to an exemplary embodiment of the present disclosure, referring to FIG. 10 , compared to the topic-related information acquisition apparatus 9 , the topic-related information acquisition apparatus 10 may further include a third topic-related information acquisition module 101 .

Specifically, the third topic information acquisition module 101 may be configured to perform: determining a second candidate topic from the topic library, where the second candidate topic is the image of the application icon in the topic library with the greatest similarity to the image of the target application icon topic; get information about the second candidate topic.

Since the respective functional modules of the apparatus for obtaining subject-related information in the embodiments of the present disclosure are the same as those in the above-mentioned method embodiments, they will not be repeated here.

From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to an embodiment of the present disclosure.

In addition, the above-mentioned figures are merely schematic illustrations of the processes included in the methods according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the chronological order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, in multiple modules.

It should be noted that although several modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.

Other embodiments of the present disclosure will readily suggest themselves to those skilled in the art upon consideration of the specification and practice of what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

A method for obtaining subject-related information, including:

Get the image of the target application icon;

Detecting whether a target theme exists in the theme library based on the image of the target application icon, and the style of the application icon of the target theme is consistent with the style of the target application icon;

When it is detected that the target theme exists in the theme library, relevant information of the target theme is acquired.
The method for obtaining theme-related information according to claim 1, wherein, based on the image of the target application program icon, detecting whether the target theme exists in the theme library comprises:

comparing the similarity between the image of the target application icon and the image of the application icon of each theme in the theme library;

According to the result of the similarity comparison, it is determined whether the target subject exists in the subject database.
The method for obtaining theme-related information according to claim 1, wherein, based on the image of the target application program icon, detecting whether the target theme exists in the theme library comprises:

obtaining the text image corresponding to the icon of the target application;

Using the text image corresponding to the target application program icon and the image of the target application program icon, it is detected whether there is a target theme in the theme library.
The method for obtaining theme-related information according to claim 3, wherein detecting whether the target theme exists in the theme library by using the text image corresponding to the target application program icon and the image of the target application program icon, comprising:

Perform text recognition on the text image to determine the name of the target application;

Determine a candidate application whose name is the same as that of the target application from the theme library, and construct a candidate application set;

acquiring the image of each candidate application icon in the candidate application set;

comparing the similarity between the image of the target application icon and the image of each candidate application icon;

According to the result of the similarity comparison, it is determined whether the target subject exists in the subject database.
The method for obtaining subject-related information according to claim 4, wherein character recognition is performed on the text image to determine the name of the target application, comprising:

performing text recognition on the text image to obtain a first middle name;

comparing the first intermediate name with the names in the application name library;

If the first intermediate name is consistent with a name in the application name library, determining the first intermediate name as the name of the target application;

If the first intermediate name is inconsistent with the name in the application name library, the first intermediate name is corrected to determine the name of the target application.
The method for obtaining subject-related information according to claim 5, wherein correcting the first intermediate name to determine the name of the target application program comprises:

determining the name with the greatest similarity to the first intermediate name in the application name library as the second intermediate name;

If the similarity between the first middle name and the second middle name is greater than a name similarity threshold, the second middle name is used as the name of the target application.
The method for acquiring topic-related information according to claim 4, wherein, before determining the candidate application program, the method for acquiring topic-related information further comprises:

According to the name of the target application, determine whether the icon of the target application is an icon modified during theme design;

Wherein, when the icon of the target application is an icon modified during theme design, the operation of determining the candidate application is performed.
The method for obtaining subject-related information according to claim 1, wherein obtaining the relevant information of the target subject comprises:

The installation program of the target theme is obtained, so as to run the installation program of the target theme and switch the theme to the target theme.
The method for obtaining subject-related information according to claim 1, wherein obtaining the relevant information of the target subject comprises:

The download information of the target theme is acquired, so as to display a download interface corresponding to the download information, and the target theme is downloaded in response to a user's download operation.
The method for obtaining theme-related information according to claim 1, wherein, applied to the first device, obtaining the image of the icon of the target application program comprises:

Scan the interface of the second device to obtain the image of the icon of the target application;

The image of the target application icon is displayed on the interface of the second device.
The method for acquiring topic-related information according to claim 10, wherein before acquiring the image of the target application icon, the method for acquiring topic-related information further comprises:

determining the size of the scanning area occupied by the image of the interface of the second device;

If the size of the image of the interface of the second device occupying the scanning area exceeds the size threshold, acquire the image of the icon of the target application.
The method for acquiring topic-related information according to claim 10, wherein before acquiring the image of the target application icon, the method for acquiring topic-related information further comprises:

determining the size of the scanning area occupied by the image of the interface of the second device;

If the size of the image of the interface of the second device occupying the scanning area exceeds a size threshold, detecting whether the first device is in a stable position;

If the first device is in a positionally stable state, an image of the icon of the target application is acquired.
The method for acquiring topic-related information according to claim 12, wherein detecting whether the first device is in a positionally stable state comprises:

Using the multi-frame images obtained based on the scanning operation, the color histogram corresponding to each frame image is generated;

According to the color histogram corresponding to each frame of image, it is determined whether the first device is in a state of stable position.
The method for obtaining theme-related information according to claim 1, wherein obtaining the image of the target application program icon comprises:

obtaining a video containing an image of the target application icon;

An image of the target application icon is obtained from the video.
The method for acquiring topic-related information according to any one of claims 1 to 14, wherein, in the case where it is detected that the target topic does not exist in the topic database, the method for acquiring topic-related information further comprises:

Get the image of the target wallpaper;

based on the image of the target wallpaper, detecting whether there is a first candidate theme in the theme library, and the wallpaper of the first candidate theme is consistent with the target wallpaper;

In the case of detecting that the first candidate topic exists in the topic database, obtain relevant information of the first candidate topic.
The method for obtaining theme-related information according to claim 15, wherein, based on the image of the target wallpaper, detecting whether there is a first candidate theme in the theme library comprises:

comparing the image of the target wallpaper with the images of the wallpapers of each theme in the theme library;

According to the result of the similarity comparison, it is determined whether the first candidate topic exists in the topic database.
The method for acquiring topic-related information according to claim 15, wherein, in the case where it is detected that the first candidate topic does not exist in the topic database, the method for acquiring topic-related information further comprises:

determining a second candidate theme from the theme library, where the second candidate theme is the theme with the greatest similarity between the image of the application icon in the theme library and the image of the target application icon;

Obtain relevant information of the second candidate topic.
A subject-related information acquisition device, comprising:

an image acquisition module, configured to acquire the image of the icon of the target application;

a theme detection module, configured to detect whether a target theme exists in the theme library based on the image of the target application icon, and the style of the application icon of the target theme is consistent with the style of the target application icon;

The first topic information acquisition module is configured to acquire the relevant information of the target topic when it is detected that the target topic exists in the topic library.
A computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the subject-related information acquisition method according to any one of claims 1 to 17.
An electronic device comprising:

processor;

a memory configured to store one or more programs which, when executed by the processor, cause the processor to implement subject-related information as claimed in any one of claims 1 to 17 get method.