CN116563864B

CN116563864B - Page number recognition method and device, electronic equipment and readable storage medium

Info

Publication number: CN116563864B
Application number: CN202310829754.7A
Authority: CN
Inventors: 林立伟; 罗英群; 肖飞秋; 罗铁
Original assignee: Shenzhen Skyworth Smart Technology Co ltd
Current assignee: Shenzhen Skyworth Smart Technology Co ltd
Priority date: 2023-07-07
Filing date: 2023-07-07
Publication date: 2023-10-24
Anticipated expiration: 2043-07-07
Also published as: CN116563864A

Abstract

The application discloses a page number identification method, a page number identification device, electronic equipment and a readable storage medium, which are applied to the technical field of image data processing, wherein the page number identification method comprises the following steps: acquiring a current complete image of a target object; removing the interference information of the current complete image to obtain an interference-free image; and identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image. The application solves the technical problem of low recognition accuracy of page recognition.

Description

Page number recognition method and device, electronic equipment and readable storage medium

Technical Field

The present application relates to the field of image data processing technologies, and in particular, to a method and apparatus for identifying a page, an electronic device, and a readable storage medium.

Background

With the continuous development of image recognition technology, more and more fields are applied to the technology, and one of the fields of education and learning terminals is that in the field, students can shoot problem pages through learning terminal equipment, and then the standard problem answers under the page numbers in a database are obtained through recognizing the photo page numbers.

At present, when the image page number of the problem page is identified, a shot image is usually used for searching and matching a standard image, and the corresponding problem page is queried in a background database by the page number mapped by the standard image, so that relevant problem information is finally obtained, however, because different problem books exist in page number setting areas, and interference information such as student answer notes, teacher annotation content and the like on the problem page influences the feature extraction effect, a larger error exists when the images are matched, and the condition that the standard page number image is easily matched and mistakes occur is caused, the identification accuracy of currently carrying out page number identification is low.

Disclosure of Invention

The application mainly aims to provide a page number identification method, a page number identification device, electronic equipment and a readable storage medium, and aims to solve the technical problem that identification accuracy of page number identification is low in the prior art.

In order to achieve the above object, the present application provides a page number recognition method including:

acquiring a current complete image of a target object;

removing the interference information of the current complete image to obtain an interference-free image;

and identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image.

Optionally, the step of identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image includes:

selecting a target standard image matched with the interference-free image from a preset standard image library according to the interference-free image characteristics of the interference-free image;

inquiring a standard page number corresponding to the target standard image in a preset image page number mapping table;

and identifying the page number to be identified carried by the target object as the standard page number.

Optionally, the preset standard image library includes at least one standard image, and the step of selecting the target standard image matched with the interference-free image in the preset standard image library according to the interference-free image characteristics of the interference-free image includes:

for any standard image, extracting standard image features of the standard image, and calculating first feature similarity between the interference-free image features of the interference-free image and the standard image features;

and selecting the target standard image from the standard images by comparing the first feature similarity.

Optionally, the step of selecting the target standard image matched with the interference-free image from the preset standard image library according to the interference-free image characteristics of the interference-free image includes:

Respectively calculating second feature similarity between the interference-free image features and standard image features of at least one standard image in the preset standard image library;

according to the magnitude relation between the second feature similarity and a first preset similarity threshold, screening a similar standard image set consisting of at least one similar standard image from the preset standard image library;

respectively calculating third feature similarity between the interference-free image features and similar standard image features of each similar standard image;

and selecting the target standard image from the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold.

Optionally, the step of selecting the target standard image in the similar standard image set according to the magnitude relation between each third feature similarity and a second preset similarity threshold value includes:

screening at least one candidate standard image in the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold;

for any candidate standard image, acquiring a first similarity weight corresponding to second feature similarity of the candidate standard image, and acquiring a second similarity weight corresponding to third feature similarity of the candidate standard image, and calculating to obtain total feature similarity of the candidate standard image according to the second feature similarity, the first similarity weight, the third feature similarity and the second similarity weight;

And selecting the target standard image from the candidate standard images according to the total feature similarity.

Optionally, the step of removing the interference information of the current complete image to obtain an interference removed image includes:

performing binarization processing on the current complete image to obtain a binarized image;

adjusting a model structure of a preset image processing model to obtain an adjusted image processing model;

and the interference information is removed by inputting the binarized image into the adjusted image processing model, so that the interference-free image is obtained.

Optionally, before the step of adjusting the model structure of the preset image processing model to obtain the adjusted image processing model, the page number identification method further includes:

acquiring a sample image set and an image processing model to be trained, wherein the sample image set comprises at least one sample image;

for any sample image, removing interference information of the sample image to obtain positive sample label data and negative sample label data, and respectively enhancing the positive sample label data and the negative sample label data to obtain a sample enhanced image;

Obtaining a sample training image set by carrying out space transformation on each sample enhanced image;

and carrying out iterative training on the image processing model to be trained according to the sample training image set to obtain a preset image processing model.

In order to achieve the above object, the present application provides a page number recognition apparatus including:

the acquisition module is used for acquiring a current complete image of the target object;

the processing module is used for removing the interference information of the current complete image to obtain an interference-removed image;

and the identification module is used for identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image.

Optionally, the identification module is further configured to:

Optionally, the preset standard image library includes at least one standard image, and the identification module is further configured to:

selecting the target standard image from the standard images by comparing the first feature similarity

Optionally, the identification module is further configured to:

selecting the target standard image from the candidate standard images according to the total feature similarity

Optionally, the processing module is further configured to:

Optionally, the page number recognition device is further configured to:

The application also provides an electronic device comprising: the program of the page number recognition method can realize the steps of the page number recognition method when being executed by a processor.

The present application also provides a computer-readable storage medium having stored thereon a program for implementing a page number recognition method, which when executed by a processor implements the steps of the page number recognition method as described above.

The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the page number identification method as described above.

The application provides a page number identification method, a page number identification device, electronic equipment and a readable storage medium, namely, a current complete image of a target object is obtained; removing the interference information of the current complete image to obtain an interference-free image; and identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image. The method comprises the steps of obtaining a current complete image carrying a page number to be identified, and further avoiding the relative position relation between the page number and the image, meanwhile, after the current complete image is obtained, removing interference information of the current complete image, and carrying out standard image matching based on the interference removed image, so that a page number identification result of a target object is finally obtained, namely, the purpose of avoiding influence of the interference information on feature extraction effect and further influencing standard image matching accuracy can be achieved. But is not limited by the relative positional relationship between the page and the image and the influence of interference information on the image in the page recognition process. Therefore, the technical defects that the problem books are different in page number setting areas, the characteristic extraction effect is affected by the interference information such as student answer notes and teacher annotation content on the problem pages, and the image matching is caused to have larger errors, so that the condition that the standard page number image matching is wrong easily occurs are overcome, and the recognition accuracy of page number recognition is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.

FIG. 1 is a flow chart of a page number recognition method according to a first embodiment of the present application;

FIG. 2 is a schematic diagram of a page number recognition process of a problem page in a page number recognition method according to an embodiment of the present application;

FIG. 3 is a flow chart of a page number recognition method according to a second embodiment of the present application;

fig. 4 is a schematic structural diagram of a page number recognition device according to a third embodiment of the present application;

fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.

The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

In order to make the above objects, features and advantages of the present application more comprehensible, the following description of the embodiments accompanied with the accompanying drawings will be given in detail. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In the first embodiment of the present application, it should be understood that, at first, when recognizing the image page, feature matching is usually performed between the photographed page area image and the standard page image in the image library, so as to match the standard page image with high similarity, and the corresponding problem page is queried in the background database by using the page mapped by the standard page image, so as to finally obtain relevant problem information, while in the process of inputting problem book information in advance through the page recognition associated background database, in order to avoid the situation that the similarity of text content on different problem pages is high and affects the accuracy of page recognition, the feature matching points of images are increased, however, the efficiency of page recognition is reduced due to the increase of the feature matching points of images, and a great amount of time is generally consumed when the feature points of images are matched, for example, taking a problem book of 100 pages of a book as an example, when a problem page with a size of a page A4 is searched and matched in the problem book of 100 pages, a Brute-Force algorithm is generally adopted to match based on 5000 feature points, which generally needs to take 25 seconds, besides the problem of recognition efficiency, the current page recognition mode has certain limitations, for example, corresponding two-dimensional codes are set on different pages, related information is obtained by scanning the two-dimensional codes, the positions of the two-dimensional codes are relatively fixed in the mode, so that the recognition flexibility is low, the preset page position area is also recognized by an ORC (optical character recognition, OCR) technology, the mode depends on the position range of manually calibrated pages, the page distribution is required to have regularity, the recognition effect of the ORC technology is easily influenced by factors such as illumination, resolution, text angle and the like, so that the accuracy of page recognition is low, there is also a way of automatically locating the page area to replace manual calibration of the page range, but, the page positions of different page styles are different, for example, some books are marked at the upper left corner and some books are marked at the lower right corner, so that the recognition effect and recognition efficiency of the page are lower in the process of page recognition, and once the situation of page recognition errors occurs in the process of page recognition, the use experience of users on related products is reduced, so that a method for improving the accuracy of page recognition is needed at present.

In a first embodiment of the present application, referring to fig. 1, a method for identifying a page number includes:

step S10, acquiring a current complete image of a target object;

step S20, removing the interference information of the current complete image to obtain an interference-removed image;

and step S30, identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image.

In this embodiment, it should be noted that, the page recognition method is applied to a page recognition device, the page recognition device may be a terminal device of a learning machine, a mobile phone, a personal PC, or the like, the page to be recognized may be a page to be recognized, specifically, page 1, page 2, page 3, or the like, the target object may be a text carrier carrying the page to be recognized and having interference information, specifically, a problem page, a reading page, or a drawing page, where the interference information is used to represent a handwriting for interfering with page recognition, the handwriting may be a handwriting repeatedly printed during printing, a writing written by a user, or a lot of handwriting, etc., the current complete image is used to represent a current image covering the complete image information, the current image may be acquired by a photographing device installed on the page recognition device, or may be uploaded to the page recognition device after photographing the target object by other photographing devices, for example, in one embodiment, if the target object is a certain page of A4, all the information contained in the page is assumed to be represented in the current image, and the complete image is used to remove the interference information.

Additionally, it should be noted that, the page number recognition device includes an obtaining module, a processing module and a recognition module, where the processing module includes a feature extraction module, and the recognition module includes a feature matching module, where the feature extraction module is provided with a preset image processing model, where the preset image processing model is used to remove interference information of an input image, for example, in an implementation manner, if the interference information is handwriting written by a user on a page number, the preset image processing model is an erase net neural network model, the erase net neural network model may distinguish a handwriting font on a target object from a print font, where the erase net neural network model may be obtained by pre-training based on a data set marked by a large document, and may specifically include a branch and a discriminator branch of the neural network based on GAN (Generative Adversarial Nets), and when the discriminator outputs a prediction result, a noise-removing image for enabling the erase net neural network model to output a real feedback removal handwriting may be combined with a plurality of different stages, and may include a coarse-noise loss function, a coarse-noise loss function may be generated, and a noise-loss function may be generated, where the loss function may include a coarse-noise loss function.

In addition, it should be noted that, before the interference information of the current complete image is removed by the preset image processing model, the current complete image needs to be preprocessed to lay a foundation for the image processing of the preset image processing model, for example, in an implementation manner, the current complete image may be subjected to binarization processing based on an adaptive threshold function of Opencv (Open Source Computer Vision Library ), where the adaptive threshold of the binarization processing is automatically dynamically adjusted according to the pixel distribution of the set area, so as to obtain a binarized image retaining most font edges in the current complete image, color information unnecessary in the model training and reasoning process may be removed after the image binarization is performed, and meanwhile, in order to prevent the related algorithm for extracting feature points in the page recognition process from being excessively sensitive to high-frequency noise points, the binarized image may be further processed in a smooth manner by a gaussian filter function.

In addition, it should be noted that, before performing page recognition, the target standard image is an initial complete image of the target object, where no interference information is generated, and before performing page recognition, for example, in an implementation manner, if the target object is a problem page, the database background having an association relationship with the problem page will record the problem book containing the problem page in advance, and then when a student performs a answer with respect to the problem page, the problem book is updated synchronously and stored in the database background, where the interference information is generated when the student performs the answer, and the matching process of the interference-free image and the target standard image may be performed through Brute-Force based on the search matching of the image feature points between them, and the page recognition result is used to represent the actual page value of the page to be recognized, that is, the page value carried by the target standard image may be specifically 100, 105 or 120.

As an example, steps S10 to S30 include: shooting a target object according to a page number identification instruction input by a user to obtain a current complete image, wherein the page number identification instruction can be triggered by the user clicking a 'page number identification function' button; performing binarization processing on the current complete image to obtain a binarized image, performing filtering processing on the binarized image according to a preset Gaussian filter function to obtain a filtered binarized image, and inputting the filtered binarized image into a preset image processing model to obtain a de-interference image; and matching a target standard image for the interference-free image based on a preset feature point matching algorithm and a preset number of image feature points extracted from the interference-free image, and identifying the page number to be identified carried by the target object according to the target standard image. According to the embodiment of the application, after the current complete image of the target object is shot, binarization processing is carried out on the current complete image to obtain a binarized image, and high-frequency noise points in the binarized image are removed to obtain a binarized image after filtering processing, so that interference information of the binarized image after filtering processing is removed based on a preset image processing model to obtain an interference-free image, then a target standard image matched with the interference-free image is searched through a characteristic point matching algorithm, finally, the page to be identified carried by the target object is identified, and therefore, the effect of removing the influence of the interference information in the shot image on the characteristic effect of image extraction in the page identification process can be realized while the page identification flexibility is improved, and the identification accuracy of page identification is improved.

The step of identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image comprises the following steps:

step A10, selecting a target standard image matched with the interference-free image from a preset standard image library according to the interference-free image characteristics of the interference-free image;

step A20, inquiring a standard page number corresponding to the target standard image in a preset image page number mapping table;

and step A30, identifying the page number to be identified carried by the target object as the standard page number.

In this embodiment, it should be noted that, when the target standard image is matched with the de-interference image, if the searching and matching are performed based on the massive images of the background database, a great amount of time is consumed for searching and matching, and then the matching amount of the images can be reduced through the preset standard image library during searching and matching, for example, in an implementation manner, A, B and three exercise books are stored in the background of the database, different exercise books are respectively provided with the corresponding preset standard image library, the preset standard image library corresponding to the de-interference image can be indexed through the image identifier of the de-interference image, then the target standard image is matched in the preset standard image library based on the de-interference image characteristics of the de-interference image, meanwhile, after the target standard image is obtained, if the page carried by the target standard image is obtained, the page recognition efficiency is further reduced, and then when the standard image is recorded in advance in the background database, the preset image mapping table can be established according to the mapping relation between the standard image and the page carried by the standard image, so that the target standard image can be obtained in the subsequent step, namely, the target standard image carrying page characterization is used for representing the target standard page.

As an example, steps a10 to a30 include: according to the image identification of the interference elimination image, matching a preset standard image library for the interference elimination image, extracting image characteristics of the interference elimination image to obtain interference elimination image characteristics, and selecting a target standard image matched with the interference elimination image from the preset standard image library based on the interference elimination image characteristics and a preset characteristic point matching algorithm; inquiring a corresponding standard page number in a preset image page number mapping table by taking the target standard image as an index; and identifying the page number to be identified carried by the target object as the standard page number. The matching quantity of the images is reduced in the matching process of the feature points, namely, the feature points are matched in a preset standard image library corresponding to the interference-free images, and meanwhile, the mapping relation between the target standard image and the standard page number is recorded in the background of the database in advance, so that the aim of obtaining the standard page number on the premise of not carrying out secondary page number identification on the target standard image can be fulfilled, and the identification efficiency of page number identification is improved.

The step of selecting the target standard image matched with the interference-free image from the preset standard image library according to the interference-free image characteristics of the interference-free image comprises the following steps:

Step B10, for any standard image, extracting standard image characteristics of the standard image, and calculating first characteristic similarity between the interference-free image characteristics of the interference-free image and the standard image characteristics;

and step B20, selecting the target standard image from the standard images by comparing the similarity of the first features.

In this embodiment, it should be noted that, in the process of page number identification, if feature point matching is performed based on a conventional feature point matching algorithm, because the extraction amount of feature points is large, the matching efficiency between images still cannot reach the expectations, and further, a FLANN (Fast Library for Approximate Nearest Neighbors, a fast library of approximate fastest neighbors) algorithm may be used to rapidly perform feature similarity calculation between images, so as to determine the matching degree between images based on the feature similarity, where before performing feature similarity calculation between images, an ORB (Oriented FAST and Rotated BRIEF, fast feature point extraction and description) algorithm may be used to detect key feature points on images, and calculate a corresponding feature vector for each feature point, where the feature vector represents an intensity pattern around the key point, so that a feature vector table composed of feature point feature vectors of multiple feature points can be used to characterize a specific area in an image, and a standard image is used to characterize an initial complete image that does not generate interference information, for example, in one embodiment, it is assumed that a target object is 100 pages in total, and before performing feature similarity calculation between images, a problem page is recorded as a standard exercise page for each page of an exercise book.

As an example, steps B10 to B20 include: for any standard image, extracting image features of the standard image through an image feature extraction model, and eliminating first feature similarity between the interference image features and the standard image features according to a FLANN algorithm; and comparing the first feature similarities by one, and taking the standard image corresponding to the maximum value of the feature similarities in the first feature similarities as the target standard image.

The step of selecting the target standard image matched with the interference-free image from a preset standard image library according to the interference-free image characteristics of the interference-free image comprises the following steps:

step C10, respectively calculating second feature similarity between the interference-free image features and standard image features of at least one standard image in the preset standard image library;

step C20, screening a similar standard image set consisting of at least one similar standard image from the preset standard image library according to the magnitude relation between the second feature similarity and a first preset similarity threshold;

step C30, calculating third feature similarity between the interference-free image features and similar standard image features of the similar standard images respectively;

And C40, selecting the target standard image from the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold.

In this embodiment, it should be noted that, when calculating the feature similarity, the algorithm characteristics of different similarity algorithms are different, for example, the brutreflex_hammin algorithm focuses on the calculation efficiency, the FLANN algorithm focuses on the calculation accuracy, and further, by combining two feature similarity algorithms, the calculation efficiency and accuracy can be considered when calculating the feature similarity, and both the first preset similarity threshold and the second similarity threshold can be set according to the actual requirement, where the similarity standard image is used to represent a standard image whose feature similarity with the interference-removed image is greater than the first preset similarity threshold in the preset standard image library, for example, in an implementation manner, a set of similarity standard images with higher similarity ranks is obtained by first screening the FLANN algorithm, in the process of primary screening, a container v1 with k being the size of a minimum pile data structure for primary screening can be set, k image characteristic values with the minimum distance and corresponding image indexes are stored in the container v1, the data structure is characterized through element ancestors (the image indexes are I, the distances are D), wherein the smaller the distances are distances calculated through a FLANN algorithm, the higher the characteristic similarity is, the hamming distance can be specifically obtained, the hamming distance is calculated, the hamming distance between k images in the container v1 and images to be queried is calculated through a BRUTEFORCE_HAMMIN algorithm, the image indexes and the hamming distance are reserved in a container v2 with another pile structure, and then a target standard image is selected in v 2.

As an example, steps C10 to C40 include: according to a FLANN algorithm, respectively calculating second feature similarity between the interference-free image features and standard image features of at least one standard image in the preset standard image library; comparing the second feature similarity of the interference-free image with the standard image with a first preset similarity threshold value, and taking the standard image with the second feature similarity larger than the first preset similarity threshold value as a similar standard image until the standard images in the preset standard image library are completely compared, so as to obtain a similar standard image set consisting of at least one similar standard image; according to a BRUTEFORCE_HAMMIN algorithm, calculating third feature similarity between the interference-free image features and similar standard image features of the similar standard images respectively; and comparing the third feature similarity of the interference-free image with the similar standard image with a second preset similarity threshold value, taking the similar standard image with the third feature similarity larger than the second preset similarity threshold value as a high-similarity image until the similar standard images in the similar standard image set are compared, taking the high-similarity image as the target standard image if the high-similarity image is one, and taking the high-similarity image corresponding to the largest third feature similarity in the third feature similarity as the target standard image if the high-similarity image is a plurality of high-similarity images.

The step of selecting the target standard image in the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold value includes:

step D10, screening at least one candidate standard image in the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold;

step D20, for any candidate standard image, acquiring a first similarity weight corresponding to a second feature similarity of the candidate standard image, and acquiring a second similarity weight corresponding to a third feature similarity of the candidate standard image, and calculating to obtain a total feature similarity of the candidate standard image according to the second feature similarity, the first similarity weight, the third feature similarity and the second similarity weight;

and D30, selecting the target standard image from the candidate standard images according to the total feature similarity.

In this embodiment, it should be noted that, in combination with two feature similarity calculation methods, the calculation efficiency and the calculation accuracy can be both considered, but if the feature similarity algorithm relying on the secondary screening when selecting the target standard image affects the selection accuracy of the target standard image to some extent, after the second feature similarity and the third feature similarity are calculated by different algorithms, different similarity weights may be assigned to the two similarities, for example, in one implementation, the second feature similarity is represented by a hamming distance v1 (i, distance), the third feature similarity is represented by a hamming distance v2 (i, distance), the similarity weight of the second feature similarity is w1, and the similarity weight of the third feature similarity is w2, so that the total feature similarity The sum of w1 and w2 is 100%, w1 can be 80%, w2 can be 20%, v1 represents a container for storing similar standard images, v2 represents a container for storing candidate standard images, in the practical application process, the accuracy of the target standard images selected after setting the similarity weight is improved compared with the nearest neighbor algorithm only adopting FLANN, the recognition speed is improved by two times compared with the nearest neighbor algorithm only adopting BRUTEFORCE_HAMMIN algorithm, and the recognition accuracy can reach 98% through the total feature similarity.

As an example, steps D10 to D30 include: for any similar standard image, comparing the third feature similarity of the interference-free image and the similar standard image with a second preset similarity threshold, and taking the similar standard image with the third feature similarity larger than the second preset similarity threshold as a candidate standard image until the similar standard images in the similar standard image set are compared; for any candidate standard image, acquiring a first similarity weight corresponding to second feature similarity of the candidate standard image, acquiring a second similarity weight corresponding to third feature similarity of the candidate standard image, multiplying the second feature similarity by the first similarity weight, and taking the sum of the products of the third feature similarity and the second similarity weight as the total feature similarity of the candidate standard image; and carrying out pairwise comparison on the total feature similarity, selecting the total feature similarity with the maximum value, and taking the candidate standard image corresponding to the total feature similarity with the maximum value as the target standard image.

In an embodiment, referring to fig. 2, fig. 2 is a schematic flow chart showing page number recognition of a problem page, firstly recording problem book integral information, converting the problem book integral information into a standard image, extracting to obtain standard image features, matching the feature matching module with interference-free image features corresponding to the current problem page, and finally obtaining a standard image with the highest similarity degree with the current integral image, so that the problem information is obtained by indexing in a background database according to image indexes of the standard image.

The embodiment of the application provides a page number identification method, namely, a current complete image of a target object is acquired; removing the interference information of the current complete image to obtain an interference-free image; and identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image. The method comprises the steps of obtaining a current complete image carrying a page number to be identified, and further avoiding the relative position relation between the page number and the image, meanwhile, after the current complete image is obtained, removing interference information of the current complete image, and carrying out standard image matching based on the interference removed image, so that a page number identification result of a target object is finally obtained, namely, the purpose of avoiding influence of the interference information on feature extraction effect and further influencing standard image matching accuracy can be achieved. But is not limited by the relative positional relationship between the page and the image and the influence of interference information on the image in the page recognition process. Therefore, the technical defects that the problem books are different in page number setting areas, the characteristic extraction effect is affected by the interference information such as student answer notes and teacher annotation content on the problem pages, and the image matching is caused to have larger errors, so that the condition that the standard page number image matching is wrong easily occurs are overcome, and the recognition accuracy of page number recognition is improved.

Further, referring to fig. 3, in the second embodiment of the present application, the same or similar contents as those of the first embodiment can be referred to the above description, and the description thereof will be omitted. On the basis, the step of removing the interference information of the current complete image to obtain an interference-removed image comprises the following steps:

e10, performing binarization processing on the current complete image to obtain a binarized image;

step E20, adjusting a model structure of a preset image processing model to obtain an adjusted image processing model;

and E30, inputting the binarized image into the adjusted image processing model, and removing the interference information to obtain the interference-removed image.

In this embodiment, it should be noted that, in case that the model structure of the existing erase net model is complex, if the model is directly adopted to process the current complete image, a great amount of time will be consumed in the model training and reasoning process, so the model structure can be adjusted to improve the model processing efficiency, for example, in an implementation manner, a calculation module capable of removing the branches of the discriminator, the style loss and the content loss only retains the loss generated by the branches of the oarse-erase sub-network and the definition sub-network.

As an example, steps E10 to E30 include: performing binarization processing on the current complete image to obtain a binarized image; removing calculation modules of the branches, the style loss and the content loss of a discriminator of the EraseNet model to obtain a processed EraseNet model; and the interference information is removed by inputting the binarized image into the processed EraseNet model, so that the interference-free image is obtained.

Before the step of adjusting the model structure of the preset image processing model to obtain the adjusted image processing model, the page number identification method further includes:

step F10, acquiring a sample image set and an image processing model to be trained, wherein the sample image set comprises at least one sample image;

step F20, for any sample image, removing interference information of the sample image to obtain positive sample label data and negative sample label data, and respectively enhancing the positive sample label data and the negative sample label data to obtain a sample enhanced image;

step F30, obtaining a sample training image set by performing spatial transformation on each sample enhanced image;

And step F40, performing iterative training on the image processing model to be trained according to the sample training image set to obtain a preset image processing model.

In this embodiment, it should be noted that, in general, the preset image processing model obtained by training under the conventional training set may obtain a certain effect on removing the interference information of the image, but the processing effect under some feature scenarios is not good, for example, on removing the interference information of the problem page, because the content of the problem image is relatively special, there are correction symbols such as hook forks, wavy lines and underlines, and there are complex subject graphics that cannot appear in the training set, and there are conditions such as image distortion and concave-convex deformation in the middle of the page on acquiring the image by using the photographed image, so that the training set may be adjusted to improve the image processing capability of the image processing model.

Additionally, it should be noted that the sample image set may be obtained by photographing with a photographing device, where there is interference information in the sample image set, for example, in one embodiment, the sample image has handwriting and print handwriting, and the image processing model to be trained is used to characterize the image processing model to be trained.

As an example, steps F10 to F40 include: acquiring a sample image set and an image processing model to be trained, wherein the sample image set comprises at least one sample image; for any sample image, negative sample tag data is obtained by removing interference information of the sample image, positive sample tag data is obtained by removing non-interference information of the sample image, a data set to be processed is generated by random sampling, and data enhancement is performed on any data of the data set to be processed, wherein the removal of the interference information can be performed by image processing software, and the data enhancement modes comprise, but are not limited to, horizontal/vertical overturn images, rotation images, image color change and the like; obtaining a sample training image set by carrying out space transformation on each sample enhanced image, wherein the space transformation mode can be perspective transformation, affine transformation, transformation inclination angle and the like; and carrying out iterative training on the image processing model to be trained according to the sample training image set to obtain a preset image processing model, wherein a mode of migration training and model reasoning is adopted in the iterative training process.

In one embodiment, during the process of processing the sample image, probability values may be defined randomly, and the above-defined image enhancement processing manner may be combined, in each iteration round, each input image is first transformed after combination to obtain a new image, during migration training, shallow model parameters in model weight parameters provided by an original manufacturer may be retained, only model parameters of the last layer of the model may be fine-tuned to obtain a better training effect, in order to make the model learn to fit an application scene, model parameters of a frozen coarse-feature sub-network may be frozen, gradient update of only the last 3 up-sampling convolution layers is retained, and these 3 are gradient updates of the convolution layers used for restoring the original image of the clean background from the feature image sampled before, fine-tuning the layers makes the model obtain model parameters fit to the application scene, and meanwhile, without updating model parameters of the shallow layer in a large amount, makes the model converge and be more stable

The embodiment of the application provides an interference information removing method. Namely, binarizing the current complete image to obtain a binarized image; adjusting a model structure of a preset image processing model to obtain an adjusted image processing model; and the interference information is removed by inputting the binarized image into the adjusted image processing model, so that the interference-free image is obtained. According to the embodiment of the application, the model structure of the preset image processing model is adjusted, so that the model training and reasoning time of the preset image processing model in the process of processing the binary image is reduced, and meanwhile, the preset image processing model still maintains the function of removing the image interference information, so that the purpose of obtaining the interference-free image meeting the requirements after the image is processed through the preset image processing model can be realized, and the efficiency of removing the image interference information is improved on the premise of ensuring the interference information removing capability.

The third embodiment of the present application further provides a page number recognition device, referring to fig. 4, where the page number recognition device includes:

an acquiring module 101, configured to acquire a current complete image of a target object;

the processing module 102 is configured to remove the interference information of the current complete image to obtain an interference-removed image;

and the identifying module 103 is used for identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image.

Optionally, the identification module 103 is further configured to:

Optionally, the preset standard image library includes at least one standard image, and the identifying module 103 is further configured to:

Optionally, the identification module 103 is further configured to:

Optionally, the processing module 102 is further configured to:

Optionally, the page number recognition device is further configured to:

The page number recognition device provided by the invention solves the technical problem of low recognition accuracy of page number recognition by adopting the page number recognition method in the embodiment. Compared with the prior art, the beneficial effects of the page recognition device provided by the embodiment of the invention are the same as those of the page recognition method provided by the embodiment, and other technical features of the page recognition device are the same as those disclosed by the method of the embodiment, so that the description is omitted herein.

A fourth embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page number identification method of the above-described embodiments.

Referring now to fig. 5, a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 5, the electronic device may include a processing apparatus 1001 (e.g., a central processing unit, a graphics processor, etc.), which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage apparatus 1003 into a Random Access Memory (RAM) 1004. In the RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, the ROM1002, and the RAM1004 are connected to each other by a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus.

In general, the following systems may be connected to the I/O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, and the like; an output device 1008 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage device 1003 including, for example, a magnetic tape, a hard disk, and the like; and communication means 1009. The communication means may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While electronic devices having various systems are shown in the figures, it should be understood that not all of the illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 1009, or installed from the storage device 1003, or installed from the ROM 1002. The above-described functions defined in the method of the embodiment of the present disclosure are performed when the computer program is executed by the processing device 1001.

The electronic equipment provided by the invention adopts the page number identification method in the embodiment, and solves the technical problem of low identification accuracy of page number identification. Compared with the prior art, the electronic device provided by the embodiment of the invention has the same beneficial effects as the page number identification method provided by the embodiment, and other technical features in the electronic device are the same as the features disclosed by the method of the embodiment, and are not repeated here.

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

The fifth embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon for performing the page number recognition method in the above-described embodiment.

The computer readable storage medium according to the embodiments of the present invention may be, for example, a usb disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this embodiment, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The above-described computer-readable storage medium may be contained in an electronic device; or may exist alone without being assembled into an electronic device.

The computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquiring a current complete image of a target object; removing the interference information of the current complete image to obtain an interference-free image; and identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented in software or hardware. Wherein the name of the module does not constitute a limitation of the unit itself in some cases.

The computer readable storage medium provided by the application stores the computer readable program instructions for executing the page number identification method, and solves the technical problem of low identification accuracy of page number identification. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the embodiment of the application are the same as those of the page number identification method provided by the above embodiment, and are not described in detail herein.

A sixth embodiment of the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the page number identification method as described above.

The computer program product provided by the application solves the technical problem of low recognition accuracy in page recognition. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the page number identification method provided by the above embodiment, and are not described herein.

The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the application, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein, or any application, directly or indirectly, within the scope of the application.

Claims

1. A page number recognition method, characterized in that the page number recognition method comprises:

acquiring a current complete image of a target object;

removing the interference information of the current complete image to obtain an interference-free image, wherein the interference information is used for representing handwriting for identifying an interference page number;

identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image, wherein the step of identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image comprises the following steps:

identifying the page number to be identified carried by the target object as the standard page number, wherein the step of selecting the target standard image matched with the interference-free image from a preset standard image library according to the interference-free image characteristics of the interference-free image comprises the following steps:

respectively calculating second feature similarity between the interference-free image features of the interference-free image and standard image features of at least one standard image in the preset standard image library;

respectively calculating third feature similarities between the interference-free image features and similar standard image features of the similar standard images, wherein the second feature similarities and the third feature similarities are feature similarities calculated by adopting different similarity algorithms;

selecting the target standard image in the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold, wherein the step of selecting the target standard image in the similar standard image set according to the magnitude relation between the third feature similarity and the second preset similarity threshold comprises the following steps:

Selecting the target standard image from the candidate standard images according to the total feature similarity, wherein the total feature similarityThe second feature similarity is represented by a hamming distance v1 (i, distance), the third feature similarity is represented by a hamming distance v2 (i, distance), the second feature similarity weight is w1, the third feature similarity weight is w2, the sum of w1 and w2 is 100%, v1 represents a container storing similar standard images, and v2 represents a container storing candidate standard images.

2. A page number recognition method as defined in claim 1, wherein said library of preset standard images includes at least one standard image,

3. The page number identification method as claimed in claim 1, wherein said step of removing the interference information of the current complete image to obtain a de-interference image comprises:

4. A page number recognition method as defined in claim 3, wherein, before said step of adjusting the model structure of the preset image processing model to obtain an adjusted image processing model, said page number recognition method further comprises:

5. A page number recognition apparatus, characterized by comprising:

the processing module is used for removing the interference information of the current complete image to obtain an interference-free image, wherein the interference information is used for representing handwriting for identifying an interference page number;

the identification module is used for identifying the page number to be identified carried by the target object according to the target standard image matched with the interference-free image, wherein the identification module is also used for selecting the target standard image matched with the interference-free image from a preset standard image library according to the characteristics of the interference-free image; inquiring a standard page number corresponding to the target standard image in a preset image page number mapping table; identifying the page number to be identified carried by the target object as the standard page number, wherein the identification module is further configured to:

Respectively calculating second feature similarity between the interference-free image features of the interference-free image and standard image features of at least one standard image in the preset standard image library; according to the magnitude relation between the second feature similarity and a first preset similarity threshold, screening a similar standard image set consisting of at least one similar standard image from the preset standard image library; respectively calculating third feature similarities between the interference-free image features and similar standard image features of the similar standard images, wherein the second feature similarities and the third feature similarities are feature similarities calculated by adopting different similarity algorithms; selecting the target standard image from the similar standard image set according to the magnitude relation between the third feature similarity and a second preset similarity threshold, wherein the identification module is further configured to:

6. An electronic device, the electronic device comprising:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the page number identification method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program that implements a page number recognition method, the program implementing the page number recognition method being executed by a processor to implement the steps of the page number recognition method as recited in any one of claims 1 to 4.