WO2022001600A1 - Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement - Google Patents

Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement Download PDF

Info

Publication number
WO2022001600A1
WO2022001600A1 PCT/CN2021/098998 CN2021098998W WO2022001600A1 WO 2022001600 A1 WO2022001600 A1 WO 2022001600A1 CN 2021098998 W CN2021098998 W CN 2021098998W WO 2022001600 A1 WO2022001600 A1 WO 2022001600A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
application
preset
module
icon
Prior art date
Application number
PCT/CN2021/098998
Other languages
English (en)
Chinese (zh)
Inventor
李亚乾
侯振靖
黄超
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2022001600A1 publication Critical patent/WO2022001600A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/41—Analysis of document content
    • G06V30/418—Document matching, e.g. of document images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Definitions

  • the present application relates to the technical field of image processing, and relates to, but is not limited to, information analysis methods and devices, devices, and storage media.
  • Embodiments of the present application provide an information parsing method, apparatus, device, and storage medium.
  • an embodiment of the present application provides an information parsing method, the method comprising:
  • the information type includes a preset type
  • an embodiment of the present application provides an information parsing device, the device includes a determining module, an parsing module and a jumping module, wherein:
  • the determining module is configured to determine the information type of the information included in the object to be analyzed; wherein the object to be analyzed is an image related to an application program;
  • the parsing module is configured to parse the information corresponding to the preset type in the object to be parsed, and determine the identification information of the application when the information type includes a preset type;
  • the jumping module is configured to jump to the application store according to the identification information of the application in response to receiving the instruction to search for the application.
  • an embodiment of the present application provides an information parsing device, including a memory and a processor, where the memory stores a computer program that can be run on the processor, and the processor implements the above information parsing method when executing the program steps in .
  • an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the above information analysis method.
  • FIG. 1 is an optional schematic flowchart of an information analysis method provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a barcode and a two-dimensional code provided by an embodiment of the present application
  • FIG. 3 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application.
  • FIG. 4 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application.
  • FIG. 5 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application.
  • FIG. 6 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application.
  • FIG. 7 is a schematic diagram of an application poster provided by an embodiment of the present application.
  • 8A is a schematic flowchart of two-dimensional code detection and correction provided by an embodiment of the present application.
  • 8B is a schematic flowchart of poster cropping and correction provided by an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of information parsing of a two-dimensional code without an icon according to an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of information parsing of a two-dimensional code with an icon provided by an embodiment of the present application
  • FIG. 11 is a schematic flowchart of icon information analysis provided by an embodiment of the present application.
  • FIG. 13 is a schematic flowchart of a solution for single feature comparison based on image search provided by an embodiment of the present application
  • FIG. 15 is a schematic flowchart of a solution for single information recognition based on image recognition provided by an embodiment of the present application.
  • 16 is a schematic flowchart of a fusion solution for image recognition-based multi-information recognition provided by an embodiment of the present application
  • FIG. 17 is a schematic diagram of the composition and structure of an information analysis apparatus provided by an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a hardware entity of an information parsing device provided by an embodiment of the present application.
  • first ⁇ second ⁇ third involved in the embodiments of the present application is only to distinguish similar objects, and does not represent a specific ordering of objects. It is understandable that “first ⁇ second ⁇ third” "Where permitted, the specific order or sequence may be interchanged to enable the embodiments of the application described herein to be practiced in sequences other than those illustrated or described herein.
  • the embodiments of the present application provide an information parsing method, which is applied to a mobile terminal.
  • the mobile terminals include but are not limited to mobile phones, notebook computers, tablet computers and handheld Internet devices, multimedia devices, streaming media devices, mobile Internet devices, wearable devices or other types of terminal devices.
  • the functions implemented by the method can be implemented by calling the program code by the processor in the mobile terminal.
  • the program code can be stored in a computer storage medium. It can be seen that the mobile terminal includes at least a processor and a storage medium.
  • FIG. 1 is an optional schematic flowchart of an information analysis method provided by an embodiment of the present application. As shown in FIG. 1 , the method includes:
  • Step S110 determining the information type of the information included in the object to be parsed.
  • the object to be parsed is an image related to an application program.
  • information such as product icons, preset pictures, identification characters, product introductions, and link addresses.
  • the corresponding information types include pictures, texts, and comprehensive products.
  • a barcode/QR code for information such as name, date of manufacture, manufacturer and link address.
  • the information in the preview screen may only contain one of the above-mentioned information or a combination of certain information.
  • the information type of the information represents an information encoding method, and different information encoding methods are adopted according to different information carried, thereby generating different information types such as text, sound, video, and pictures.
  • the barcode 21 and the two-dimensional code 22 are both common image carriers used for information encoding and parsing. effective means of collecting data. Compared with the traditional barcode 21, the two-dimensional code 22 can store more information, can also represent more types of data, and has a strong error correction ability.
  • Mainstream QR code applications include information acquisition, website redirection, mobile payment, account login, etc.
  • Step S120 in the case that the information type includes a preset type, parse the information corresponding to the preset type in the object to be parsed, and determine the identification information of the application program.
  • the preset type of information is information representing the core content of the application or the corresponding product, and includes at least one of the following preset types of information: a two-dimensional code, an application icon, and text information.
  • QR code centers include application icons, while some QR codes only use black and white pixel matrix coding; application icons include application icons and/or application posters at the center of the QR code.
  • the identification information of the application program may be the name of the application program or the corresponding product name, or may be an ID number.
  • the information of each preset type is parsed through visual algorithms such as image search technology or image recognition technology, so as to quickly obtain the name/ID of the application or product when scanning the image related to the application.
  • Step S130 in response to receiving the instruction to search for the application, jump to the application store according to the identification information of the application.
  • the jump from the current scanning program to the application store program is realized by designing the logic of the terminal itself.
  • the application program name or product name can be used as input, and the corresponding application program software can be searched in the application program store to download and install.
  • the information type of the information included in the object to be parsed is determined; wherein, the object to be parsed is an image related to an application program; then, when the information type includes a preset type , parse the information corresponding to the preset type in the object to be parsed, and determine the identification information of the application; finally, in response to receiving an instruction to search for the application, jump according to the identification information of the application Go to the application store; in this way, by parsing the information of the preset type in the object to be parsed, the obtained object to be parsed is quickly mapped to the identification information of the application and jumped to the application store, so as to solve the need for users to quickly obtain the application identification. information, but the QR code is damaged or the icon cannot be parsed correctly when the icon is updated. It also applies to scenarios without QR code.
  • FIG. 3 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application.
  • the object to be parsed is an application poster
  • the information corresponding to the preset type is a file containing an application icon.
  • Two-dimensional code the method includes at least the following steps:
  • Step S310 determining the information type of the information included in the application poster.
  • Step S320 when the application icon is located in the two-dimensional code, intercept the image area corresponding to the application icon from the two-dimensional code containing the application icon.
  • the location and area of the application icon can be located through the location of the two-dimensional code.
  • the location and area of the icon can be obtained through an object detection algorithm, an image segmentation algorithm, and the like.
  • Step S330 analyze the image area to determine the identification information of the application program.
  • the identification information of the application program is determined by performing image recognition or image search on the image area corresponding to the icon.
  • the identification information of the application program may be the name of the application program or the corresponding product name, or may be an ID.
  • Step S340 in response to receiving the instruction to search for the application, jump to the application store according to the identification information of the application.
  • Step S350 in response to the instruction to download the application, download the application through an application store.
  • the identification information of the application such as the product name
  • the application can be downloaded through the application store.
  • the area corresponding to the application icon in the two-dimensional code is intercepted for analysis, so that the information of the two-dimensional code itself can be ignored when scanning, and the direct Detecting and identifying the icon can avoid the situation where the QR code is damaged and cannot obtain the correct analysis result, and improve the user experience.
  • the information of the application identification information obtained by the analysis is diverted and directly jumped to the application store, which is convenient for users to download or purchase.
  • FIG. 4 is an optional schematic flowchart of the information parsing method provided by this embodiment of the present application.
  • the object to be parsed is an application poster, and the information types include two or more preset types.
  • the method includes at least the following steps:
  • Step S410 determining the information type of the information included in the application poster.
  • Step S420 in the case that the information type includes two or more preset types, preprocess the application poster, and determine information of each preset type from the application poster.
  • the information types include two or more preset types
  • first preprocess the application poster scanned by the camera including the alignment and normalization of the acquired poster image information and the standard image, as well as the promotion content and two QR code positioning, cropping, correction, etc., application icon positioning, detection and segmentation, etc., so as to extract information such as QR code, application icon and promotional content from the application poster.
  • Step S430 Prioritize the information corresponding to each preset type.
  • step S440 target information is determined according to the sorting result.
  • the priority of the target information is higher than the priority of any other information in the information of each preset type.
  • the two or more preset types of information include a two-dimensional code without the application icon
  • the method further includes: determining the two-dimensional code without the application icon The priority level is lower than the priority level of other preset types of information.
  • the QR code may be damaged and the correct information cannot be obtained by parsing.
  • Priority level so that target information is selected from other preset types of information for priority analysis.
  • Step S450 analyze the target information to determine the identification information of the application program.
  • the target information can be input into the trained feature extraction model to extract the features contained in the target information, so as to obtain the identifier of the application program contained in the target information; the target information can also be input into the trained information In the identification model, the identification of the application program contained in the target information is directly obtained.
  • the above-mentioned step S450 may be implemented through the following steps: “Perform feature extraction on the target information to obtain a target feature point set; perform feature comparison between the target feature point set and a preset sample feature point set. Yes, determine the identification information of the application”.
  • the above step S450 may be implemented through the following steps to "perform image recognition on the target information to obtain the identification information of the application program".
  • Step S460 in response to receiving the instruction to search for the application, jump to the application store according to the identification information of the application.
  • the embodiment of the present application determines the target information by prioritizing different preset types of information in the object to be analyzed, so as to quickly analyze the most important information content at present based on the feature extraction and feature comparison algorithm or based on the image recognition algorithm, Map the images related to the application to the introduction information of the corresponding application for further processing such as downloading or purchasing.
  • FIG. 5 is an optional schematic flowchart of the information analysis method provided by the embodiment of the present application. As shown in FIG. 5 , the object to be parsed is an application poster, and the method includes at least the following steps:
  • Step S510 determining the information type of the information included in the application poster.
  • Step S520 in the case that the information type includes two or more preset types, call a parser corresponding to each of the preset types, and perform feature extraction on the information of each of the preset types to obtain at least one feature point collection.
  • the application poster to be parsed may also include information such as QR code, application icon, text, etc., and feature extraction can be performed on each preset type of information, that is, each preset type can be obtained through single-mode feature extraction. Set the feature point set corresponding to the type of information.
  • the feature extraction model can be obtained through the manual shooting of poster and icon data of various applications, and joint training with standard application poster and icon data. Therefore, the feature extraction model is used to perform feature extraction on each of the preset types of information. At the same time, new application information or existing application update poster pictures are dynamically added to perform real-time training and update of the feature extraction model.
  • Step S530 Perform feature fusion on the at least one feature point set to obtain a target feature point set.
  • the feature fusion is to generate new features from different extracted features by a certain method.
  • feature fusion is performed on the feature point set extracted from the two-dimensional code and the feature point set extracted from the application program icon, so as to obtain the comprehensive feature point set included in the application program poster. It is worth noting that the fusion process can be performed at three levels: input, feature, and output
  • Step S540 compare the features of the target feature point set with a preset sample feature point set to determine the identification information of the application program.
  • the preset sample feature point set is obtained by performing feature extraction on a sample application poster.
  • the process of feature comparison between the standard feature point set and the preset sample feature point set is essentially the comparison of the application poster to be scanned and the sample application poster in a high-dimensional feature space. Identifying information for the final application is determined by matching with the characteristics of the sample application poster.
  • Step S550 in response to receiving the instruction to search for the application, jump to the application store according to the identification information of the application.
  • feature extraction is performed on information of different preset types in the application poster, respectively, to obtain a feature point set corresponding to each preset type of information, and then feature fusion is performed to obtain a target feature point set, and finally the target feature point set is obtained.
  • the feature comparison is performed with the preset sample feature point set corresponding to the sample application poster to determine the identification information of the application.
  • FIG. 6 is an optional schematic flowchart of the information parsing method provided by the embodiment of the present application. As shown in FIG. 6 , the object to be parsed is an application poster, and the method at least includes the following steps:
  • Step S610 determining the information type of the information included in the application poster.
  • Step S620 in the case that the information type includes two or more preset types, call the parser corresponding to each of the preset types, and perform information identification on the information of each of the preset types to obtain at least one set of identification data.
  • identifying each of the preset types of information refers to performing image recognition on the information area included in the application poster, so as to identify various types of information content. For example, perform image recognition on the QR code in the application poster, and obtain the information content contained in the QR code to obtain a set of identification data; perform image recognition on the icon in the application poster, and obtain the information content contained in the icon, namely Obtain another set of identification data, etc.
  • Step S630 Perform data fusion on the at least one set of identification data according to the preset identification confidence, and determine the identification information of the application program.
  • the preset recognition confidence level represents the probability that the difference between the recognized mean value and the overall real situation is smaller than a specific threshold.
  • the so-called confidence level also known as the confidence level, refers to the degree to which a specific individual believes in the truth of a specific proposition.
  • data fusion is performed on the identified identification data, that is, the content contained in the information of each preset type of information collected by the application poster is fused, so as to determine the identification information and Download address information.
  • Step S640 in response to receiving the instruction to search for the application, jump to the application store according to the identification information of the application.
  • image recognition is performed on different preset types of information in the application poster, to obtain a plurality of information related to the application, and then the information output level is fused according to the recognition confidence, and finally the identification information and Download address information.
  • the embodiment of the present application proposes a technical solution capable of parsing poster information through a visual algorithm, so that the application poster image corresponds to the name/ID of the application or product in one-to-one correspondence.
  • the technical process of the information analysis method mainly includes two parts: one part is the extraction and preprocessing of different types of information in the application poster; the other part is to identify and analyze different types of information respectively.
  • the core information that may be included in the application poster such as QR code, application icon, promotional content, and text information, which are four categories of application-related information, basically cover the information that users may encounter in various scanning scenes.
  • Type of information In actual use, the scene in the scanned image may contain only one of the above-mentioned information or a combination of certain information.
  • FIG. 7 is a schematic diagram of an application poster provided by an embodiment of the present application.
  • the application poster includes a two-dimensional code 71 , promotional content 72 , an icon 73 at the center of the two-dimensional code 71 , and the upper left corner of the promotional content 72 73 , the application name 74 and the slogan 75 at the center of the promotional content 72 .
  • the icon 73 in the center of the two-dimensional code 71 and the icon 73 in the upper left corner of the promotional content 72 are both application icons, and the application name 74 and the slogan 75 in the center of the promotional content 72 are text information.
  • the two-dimensional code 71 is the key information scanned by the camera. Most of the two-dimensional code 71 of the application program contains the icon 73 in the center, but there are cases where the icon is not included. There are many technical means to detect two-dimensional code 71. You can choose special algorithm for two-dimensional code detection, target detection and image post-processing, semantic segmentation, key point detection, text area detection (such as regression irregular quadrilateral idea) and other methods to obtain two-dimensional code. Dimension code area.
  • the information contained in the QR code, icon, and promotional content can basically be regarded as being distributed on a two-dimensional plane.
  • the photographed picture is often subjected to inverse affine transformation, and the conventional rectangle/square becomes an irregular quadrilateral in the picture.
  • FIG. 8A is a schematic flowchart of two-dimensional code detection and correction provided by an embodiment of the application. As shown in FIG. 8A , the two-dimensional code area 81 detected in the viewfinder is generally an irregular quadrilateral, and the two-dimensional code area 81 can be After correction, a conventional rectangular two-dimensional code area 82 is obtained.
  • FIG. 8B is a schematic flowchart of poster clipping and correction provided by an embodiment of the present application. As shown in FIG.
  • the clipped promotional content is a trapezoid 83
  • the affine transformation is calculated through four vertices A, B, C, and D. matrix, and affine transformation is performed on the trapezoid 83 to obtain a regular rectangle 84.
  • the icon 73 will appear in two different scenarios: the first one is at the same time as the two-dimensional code 71 and is located in the center of the two-dimensional code 71; the second is independent of the two-dimensional code 71, for example, it may appear in the application
  • the top of the program poster may also appear on the desktop interface of the terminal device, or on the screen of the download address.
  • the correction of the promotional content 72 or the two-dimensional code 71 has been completed by default.
  • the position and area of the icon 73 can be located by the position of the two-dimensional code 71, or a separate algorithm can be designed to detect the two-dimensional code 71 and the icon 73 at the same time.
  • This multi-task model that combines two different features can often improve the detection accuracy of each.
  • the position and area of the icon 73 can be obtained through a target detection algorithm, an image segmentation algorithm, and the like.
  • the text can be extracted by a common Optical Character Recognition (Optical Character Recognition, OCR) technology, and matched with the existing application name in the database to determine the final application name.
  • OCR Optical Character Recognition
  • Decoding of application information can be achieved by taking screenshots.
  • FIG. 9 is a schematic flowchart of information parsing of a two-dimensional code without an icon provided by an embodiment of the present application. As shown in FIG. 8 , it at least includes: step S901 two-dimensional code detection; step S902 information decoding; Step S903 screenshots; step S904 icon identification and name identification; finally obtain the application name/ID.
  • the QR code contains the application icon, the information of the QR code itself can be ignored when scanning, and the icon can be detected and recognized directly.
  • FIG. 10 is a schematic flowchart of information extraction of a two-dimensional code with an icon provided by an embodiment of the present application, as shown in FIG. 10 , including solution 1 and solution 2.
  • scheme 1 it includes step S1001 two-dimensional code detection; step S1002 icon positioning; for scheme 2, it includes step S1003 two-dimensional code and icon multi-task positioning detection; finally scheme 1 and scheme 2 go through step S1004 icon recognition, and obtain the final application name/ID.
  • FIG. 11 is a schematic flowchart of icon information analysis provided by the embodiment of the present application. As shown in FIG. 11 , for the first case, it includes step S1101 icon positioning and step S1102 icon identification, and finally obtains the application name/ID; The two cases include step S1103 text detection, step S1104 OCR character recognition and step S1105 character matching, and finally obtain the application name or ID.
  • the identification of the propaganda content 72 can be performed independently, or the identification of multi-dimensional information fusion can be performed.
  • Single-task and multi-task models can be established, such as feature fusion or cross-modal feature fusion for the identified icon 73 features and the identified promotional content 72 features.
  • Fig. 12 is a schematic flowchart of the information analysis of publicity content provided by the embodiment of the present application, as shown in Fig. 12, including step S1201 cropping and correction of publicity content, step S1202 publicity content identification; step S1203 icon positioning, step S1204 icon identification; step S1203 S1205 text detection, step S1206 OCR recognition and text matching, and finally obtain the application name/ID.
  • the first two steps are to realize information extraction preprocessing and information identification and analysis, and then build the overall technical framework of poster information analysis.
  • Four different schemes will be introduced here, as shown in Figures 13 to 16.
  • the first two schemes are based on image search technology and are suitable for scenarios with a large number of application/product categories, frequent data updates, and a small number of samples; the latter two
  • This solution is based on image recognition technology and is suitable for scenarios with a small number of application/product categories, slow data updates, and many standard and natural environment shooting samples.
  • FIG. 13 is a schematic flowchart of a solution for single-feature comparison based on image search provided by an embodiment of the present application. As shown in FIG. 13 , it includes the following steps: Step S1301 obtains a poster picture; Step S1302 Information extraction and preprocessing; Step S1303 Priority sorting Step S1304 feature extraction; Step S1305 online real-time training feature extraction model; Step S1306 real-time update standard poster image and APP information database; Step S1307 feature extraction; Step S1308 update feature database; Step S1309 feature comparison; Step S1310 output application name /ID. Among them, step S1304 is to perform feature extraction on the information with the highest priority, and step S1307 is to perform feature extraction on the corresponding information in the standard poster image.
  • the image search-based multi-feature fusion solution provided by the embodiment of the present application is to perform feature extraction on all the extracted information, and then perform feature fusion and comparison.
  • Fig. 14 is a schematic flowchart of a solution for multi-feature fusion and comparison based on image search provided by an embodiment of the application, as shown in Fig. 14 , including the following steps: step S1401 obtaining a poster picture; step S1402 information extraction and preprocessing; step S1403 feature Extraction; step S1404 feature fusion; step S1405 online real-time training feature extraction model; step S1406 real-time update standard poster image and application information database; step S1407 feature extraction; step S1408 update fusion feature database; step S1409 feature comparison; step S1410 Output application name/ID.
  • step S1404 is to perform feature extraction on different information in the acquired poster picture respectively
  • step S1407 is to perform feature extraction on corresponding information in the standard poster picture.
  • the solution based on single-information image recognition determines the most important information through information priority after information extraction and preprocessing, and then directly outputs the parsed application name or ID information through the recognition model.
  • Fig. 15 is a schematic flowchart of a solution for single information recognition based on image recognition provided by an embodiment of the present application, as shown in Fig. 15 , including the following steps: step S1501 obtaining a poster picture; step S1502 information extraction and preprocessing; step S1503 information identification; Step S1504 update the standard and the photographed application poster in real time; step S1505 update the application information database; step S1506 online real-time training information recognition model; step S1507 information fusion; step S1508 output the application name/ID.
  • steps S1504 to S1506 are the process of training the information recognition model.
  • the fusion scheme based on multi-information image recognition identifies all the extracted information separately, obtains multiple application program names and ID information, and then performs fusion at the information output level according to the recognition confidence.
  • step S1601 obtaining a poster picture
  • step S1602 information extraction and preprocessing
  • step S1603 information recognition Step S1604 real-time update standard and photographed application poster
  • step S1605 update application information database
  • step S1606 online real-time training information recognition model
  • step S1607 information fusion step S1608 output application name/ID.
  • steps S1604 to S1606 are the process of training the information recognition model.
  • the images scanned by the camera are first preprocessed, including the positioning, cropping, and correction of posters and QR codes, the positioning, detection, and segmentation of icons, and the alignment and normalization of the extracted poster image information and standard images. change, etc.
  • the first two schemes are based on image search, which is essentially the comparison of images in high-dimensional feature space.
  • the search-based scheme is obtained through manual shooting of posters and icon data from various applications, and joint training with standard poster and icon data. Feature extraction model.
  • the latter two schemes are based on image recognition, using a large number of photographed poster and icon images, and using manual annotation as training data to train an information recognition model to identify posters, icons and other information.
  • the feature extraction model and recognition model in the four schemes can be single-task or multi-task learning models.
  • Different information in the application poster can be prioritized according to the importance of the information contained, such as the icon in the QR code, the icon in the promotional content, the promotional content, the application name/product name/slogan in descending order of priority .
  • the scanned image may contain QR code, icon, text and other information at the same time, which can be extracted and compared through single-mode feature extraction, or can be compared through multi-mode feature fusion. It should be noted that the fusion process can be performed at three levels: input, feature, and output.
  • This embodiment of the present application can implement the resolution of the application or product name/ID of the poster.
  • Posters/icons are no longer simple pictures, but can be used as carriers for small information storage, such as storing application or product names and corresponding IDs, etc.
  • the corresponding information can be parsed from the input poster pictures, Can be used as input for application downloads, product purchases. It can also perform terminal scanning based on name and ID to realize augmented reality advertisements and enrich poster content.
  • the embodiment of the present application can expand the scanning application scenario, and improve the download probability of the application store application.
  • users can easily and quickly use the local smart scan function, which expands the application scenarios of the scan function, whether it is for a QR code containing an icon, an application icon or a poster, in various user scanning scenarios.
  • You can jump to the local app store to download the app. This convenience triggers the opportunity for more app downloads, increasing the probability that users will download apps.
  • the embodiment of the present application maps various scanned information to the corresponding application program name/ID, and directly jumps to the download address page of the application store.
  • the embodiments of the present application first propose a method for extracting and preprocessing information such as icons, QR codes, promotional content, and text that may exist in posters, and then build an overall technical architecture based on traditional and deep learning visual algorithms.
  • the core innovations are mainly reflected in the technical architecture of four different poster comprehensive information analysis and the application of image search/recognition technology in poster information analysis scenarios.
  • An embodiment of the present application provides an information parsing device, the device includes each module included and each submodule included in each module, which can be implemented by a processor in a terminal; of course, it can also be implemented by a specific logic circuit;
  • the processor can be a central processing unit (Central Rrocessing Unit, CPU), a microprocessor (Micro Rrocessing Unit, MPU), a digital signal processor (Digital Signal Processor, DSP) or a field programmable gate array ( Field Programmable Gate Array, FPGA), etc.
  • CPU Central Rrocessing Unit
  • MPU microprocessor
  • DSP Digital Signal Processor
  • FPGA Field Programmable Gate Array
  • FIG. 17 is a schematic structural diagram of an information parsing apparatus provided by an embodiment of the present application. As shown in FIG. 17 , the apparatus 1700 includes a determination module 1701, an analysis module 1702, and a jump module 1703, wherein:
  • the determining module 1701 is configured to determine the information type of the information included in the object to be parsed; wherein the object to be parsed is an image related to an application;
  • the parsing module 1702 is configured to parse the information corresponding to the preset type in the object to be parsed, and determine the identification information of the application when the information type includes a preset type;
  • the jumping module 1703 is configured to jump to the application store according to the identification information of the application in response to receiving the instruction to search for the application.
  • the object to be parsed is an application poster
  • the preset type of information in the application poster includes at least one of the following: a two-dimensional code, an application icon, and text information; wherein the An application icon is located on the layout of the application poster, and/or, on the QR code.
  • the apparatus 1700 further includes a download module configured to download the application program through an application store in response to the instruction to download the application program.
  • the parsing module 1702 includes an intercepting sub-module and a first parsing sub-module, wherein: the intercepting sub-module is configured to be configured when the application icon is located in the two-dimensional code , intercept the image area corresponding to the application icon from the two-dimensional code containing the application icon; the first parsing submodule is configured to analyze the image area and determine the identification information of the application .
  • the information types include two or more preset types
  • the parsing module 1702 includes a priority sorting sub-module, a determination sub-module and a second parsing sub-module, wherein: the priority sorting sub-module a module, configured to prioritize the information corresponding to the two or more preset types when the information types include more than two preset types; the determining sub-module is configured to determine according to the sorting result target information; wherein, the priority of the target information is higher than the priority of other preset types of information in the object to be parsed; the second parsing sub-module is configured to parse the target information and determine the target information. the identification information of the application.
  • the two or more preset types of information include a two-dimensional code without the application icon, and the priority sorting sub-module is further configured to determine that the application icon does not exist
  • the priority level of the QR code is lower than the priority level of other preset types of information.
  • the second parsing sub-module is further configured to perform feature extraction on the target information to obtain a target feature point set; perform feature extraction on the target feature point set and a preset sample feature point set By comparison, the identification information of the application is determined.
  • the second parsing submodule is further configured to perform image recognition on the target information to obtain identification information of the application.
  • the information types include two or more preset types
  • the parsing module 1702 includes a third parsing sub-module and a feature fusion sub-module, wherein: the third parsing sub-module is configured to be respectively Analyze the information of each preset type to obtain the analysis result of each of the information; the feature fusion sub-module is configured to perform feature fusion on the analysis result of each of the information to obtain the application program identification information.
  • the above-mentioned information analysis method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application may be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to make
  • the automatic test line of the device including the storage medium executes all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), magnetic disk or optical disk and other media that can store program codes.
  • FIG. 18 is a schematic diagram of a hardware entity of the information parsing device provided by an embodiment of the present application.
  • the hardware entity of the device 1800 includes: a processor 1801, a communication interface 1802 and memory 1803, where
  • the processor 1801 generally controls the overall operation of the device 1800.
  • the communication interface 1802 may enable the device 1800 to communicate with other terminals or servers over a network.
  • the memory 1803 is configured to store instructions and applications executable by the processor 1801, and can also cache data (eg, image data) to be processed or processed by the processor 1801 and each module in the device 1800, which can be stored in flash memory (FLASH) or random Access memory (Random Access Memory, RAM) implementation.
  • data eg, image data
  • FLASH flash memory
  • RAM random Access Memory
  • the embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the information analysis method provided in the foregoing embodiments.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.
  • the unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.
  • each functional unit in each embodiment of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.
  • the above-mentioned integrated units of the present application are implemented in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application may be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to make The automatic test line of the device performs all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.
  • the embodiment of the present application parses the information of the preset type in the object to be parsed, quickly maps the acquired object to be parsed to the identification information of the application program and jumps to the application store, so as to solve the problem that the user needs to quickly obtain the identification information of the application program, but
  • the problem that the QR code is damaged or the icon cannot be correctly parsed when the icon is updated is also applicable to scenarios without QR code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement. Le procédé consiste à : déterminer un type d'informations d'informations comprises dans un objet à analyser (S110), ledit objet étant une image associée à un programme d'application ; sous réserve que le type d'informations comprenne un type prédéfini, analyser des informations correspondant au type prédéfini dans ledit objet, et déterminer des informations d'identifiant du programme d'application (S120) ; et, en réponse à la réception d'une instruction de recherche du programme d'application, sauter à une mémoire de programme d'application selon les informations d'identifiant du programme d'application (S130).
PCT/CN2021/098998 2020-07-03 2021-06-08 Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement WO2022001600A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010637185.2A CN111860232A (zh) 2020-07-03 2020-07-03 信息解析方法及装置、设备、存储介质
CN202010637185.2 2020-07-03

Publications (1)

Publication Number Publication Date
WO2022001600A1 true WO2022001600A1 (fr) 2022-01-06

Family

ID=73152127

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/098998 WO2022001600A1 (fr) 2020-07-03 2021-06-08 Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement

Country Status (2)

Country Link
CN (1) CN111860232A (fr)
WO (1) WO2022001600A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111860232A (zh) * 2020-07-03 2020-10-30 Oppo广东移动通信有限公司 信息解析方法及装置、设备、存储介质
CN112749543B (zh) * 2020-12-22 2022-08-05 浙江吉利控股集团有限公司 一种信息解析过程的匹配方法、装置、设备及存储介质
CN112989864B (zh) * 2021-03-11 2024-06-07 北京骑胜科技有限公司 图形码损坏的识别方法、设备、存储介质、程序产品
CN113159246B (zh) * 2021-04-15 2022-03-08 中物(北京)物流信息服务有限公司 基于二维码标签的钢厂货物识别方法、装置及计算机设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103279477A (zh) * 2013-04-16 2013-09-04 百度在线网络技术（北京）有限公司 应用程序的搜索方法、装置和系统
CN103501463A (zh) * 2013-10-25 2014-01-08 乐视网信息技术（北京）股份有限公司 一种数据获取方法及电子设备
US20160275327A1 (en) * 2013-12-26 2016-09-22 Tencent Technology (Shenzhen) Co., Ltd. Graphical code processing method and apparatus
CN106610835A (zh) * 2016-12-23 2017-05-03 广东欧珀移动通信有限公司 识别码处理方法、装置和计算机设备
CN109472179A (zh) * 2018-10-23 2019-03-15 努比亚技术有限公司 二维码识别方法、终端及计算机可读存储介质
CN111221447A (zh) * 2018-11-27 2020-06-02 阿里巴巴集团控股有限公司 下载应用的方法、装置、电子设备及计算机存储介质
CN111860232A (zh) * 2020-07-03 2020-10-30 Oppo广东移动通信有限公司 信息解析方法及装置、设备、存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103970576A (zh) * 2014-05-23 2014-08-06 小米科技有限责任公司 安装信息展示方法、获取方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103279477A (zh) * 2013-04-16 2013-09-04 百度在线网络技术（北京）有限公司 应用程序的搜索方法、装置和系统
CN103501463A (zh) * 2013-10-25 2014-01-08 乐视网信息技术（北京）股份有限公司 一种数据获取方法及电子设备
US20160275327A1 (en) * 2013-12-26 2016-09-22 Tencent Technology (Shenzhen) Co., Ltd. Graphical code processing method and apparatus
CN106610835A (zh) * 2016-12-23 2017-05-03 广东欧珀移动通信有限公司 识别码处理方法、装置和计算机设备
CN109472179A (zh) * 2018-10-23 2019-03-15 努比亚技术有限公司 二维码识别方法、终端及计算机可读存储介质
CN111221447A (zh) * 2018-11-27 2020-06-02 阿里巴巴集团控股有限公司 下载应用的方法、装置、电子设备及计算机存储介质
CN111860232A (zh) * 2020-07-03 2020-10-30 Oppo广东移动通信有限公司 信息解析方法及装置、设备、存储介质

Also Published As

Publication number Publication date
CN111860232A (zh) 2020-10-30

Similar Documents

Publication Publication Date Title
WO2022001600A1 (fr) Procédé, appareil et dispositif d'analyse d'informations, et support d'enregistrement
US9934254B2 (en) Terminal apparatus, information processing system, and information processing method
CN103686344B (zh) 增强视频系统及方法
US9881084B1 (en) Image match based video search
US12015807B2 (en) System and method for providing image-based video service
CN103050025A (zh) 一种移动终端的学习方法及其学习系统
US9361135B2 (en) System and method for outputting and selecting processed content information
CN107590267B (zh) 基于图片的信息推送方法及装置、终端和可读存储介质
TWI781554B (zh) 物品名稱確定方法、裝置、電腦設備及儲存媒體
US20170337222A1 (en) Image searching method and apparatus, an apparatus and non-volatile computer storage medium
KR20130022491A (ko) 증강 현실 데이터를 이용할 수 있는 어플리케이션 자동 추천 장치 및 방법
CN111309200B (zh) 一种扩展阅读内容的确定方法、装置、设备及存储介质
US20150189384A1 (en) Presenting information based on a video
CN111491187A (zh) 视频的推荐方法、装置、设备及存储介质
CN109213397B (zh) 数据处理方法、装置和用户端
WO2022193911A1 (fr) Procédé et appareil d'acquisition d'informations d'instruction, support de stockage lisible et dispositif électronique
CN111741321A (zh) 一种直播控制方法、装置、设备及计算机存储介质
CN110309324A (zh) 一种搜索方法及相关装置
JP2010205121A (ja) 情報処理装置および携帯端末
US20230298073A1 (en) Media processing techniques for enhancing content
KR20140068302A (ko) 자연영상 텍스트 인식을 이용한 콘텐츠 서비스 시스템 및 방법
CN116049490A (zh) 素材搜索方法、装置和电子设备
CN115860829A (zh) 一种智能广告图像生成方法及装置
CN113377975B (zh) 多媒体资源的处理方法、装置、计算机设备及存储介质
US11120840B2 (en) Information processing method and electronic device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21833362

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21833362

Country of ref document: EP

Kind code of ref document: A1