WO2023272656A1

WO2023272656A1 - Picture book recognition method and apparatus, family education machine, and storage medium

Info

Publication number: WO2023272656A1
Application number: PCT/CN2021/103859
Authority: WO
Inventors: 张明云
Original assignee: 东莞市小精灵教育软件有限公司
Priority date: 2021-06-30
Filing date: 2021-06-30
Publication date: 2023-01-05

Abstract

Disclosed in embodiments of the present application is a picture book recognition method, comprising: acquiring a standard picture library of a picture book; upon detecting a click operation of a user on the picture book, collecting a current picture book page obtained by capturing; retrieving in the standard picture library, and determining a standard picture book page corresponding to the current picture book page as a target picture book page; according to a click position corresponding to the click operation, determining a click area by using a fingertip positioning method; converting the click area into a target area consistent with a coordinate system corresponding to a standard coordinate; searching for a standard block comprising the target area, and determining same as a target block; and determining the picture book recognition result on the basis of the target block. By introducing the coordinate positioning and transformation methods, positioning is performed according to the click area, so that accurate positioning of the picture book is realized, and thus the corresponding area of the standard picture book is recognized, the recognition of a far picture is avoided, and the text recognition rate in the picture book is greatly improved. In addition, also provided are a picture book recognition apparatus, a family education machine, and a storage medium.

Description

Picture book identification method, device, tutoring machine and storage medium

technical field

The present application relates to the technical field of image processing, and in particular to a picture book recognition method, device, tutoring machine and storage medium.

Background technique

The tutor machine is an Android tablet that provides high-quality educational resources. The user first places the picture book in front of the tablet, uses the camera device in the tablet to take pictures of the picture book, and then uses the picture book APP in the tablet to identify the cover of the picture book and confirm the picture book to be read. The user turns the page Picture book, finger points to the text in the picture book page, and the picture book app in the tablet instantly displays the text, audio and video content of the corresponding text to realize the function of picture book literacy, assisting users to understand the content in the picture book, and realizing assisted reading through electronic devices such as tablets The effect of picture books reduces the difficulty of children reading picture books, and at the same time liberates parents from repeated guidance and assisting children in reading picture books.

technical problem

However, the current picture book recognition scheme for tutoring machines has the following defects: First, the main technical principle of picture book literacy is to recognize the text at the position clicked by the user through OCR, but there are various styles of artistic characters in picture books, and general OCR cannot recognize such text. As a result, artistic characters cannot be recognized; secondly, limited by the shape of the device, the picture book pictures taken are trapezoidal pictures with a large top and a small bottom. The lower it is, the poorer the picture book literacy effect at the far end. Therefore, it is urgent to provide a picture book recognition method that can improve the character recognition rate in picture books.

technical solution

Based on this, it is necessary to address the above problems and propose a picture book recognition method, device, tutoring machine and storage medium that can improve the character recognition rate in picture books.

A picture book recognition method, said method comprising:

Acquiring a standard picture book library, the standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, each of the standard blocks is marked with different standard coordinates, wherein, The standard block is obtained by dividing each standard picture book page according to preset division rules;

When the user's click operation on the picture book is detected, the current picture book page obtained by shooting is collected, and the shooting pixels of the current picture book page are obtained;

Searching in the standard picture book page, determining the standard picture book page corresponding to the current picture book page as the target picture book page, and obtaining the standard pixels of the target picture book page;

According to the click position corresponding to the click operation, a fingertip positioning method is used to determine the click area;

converting the clicked area into a target area consistent with a coordinate system corresponding to the standard coordinates based on the photographing pixels and the standard pixels;

Finding the standard block containing the target area from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page is determined as the target block;

A picture book recognition result is determined based on the target block.

A picture book recognition device, said device comprising:

The obtaining module is used to obtain a standard library of picture books. The standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages. Each of the standard blocks is marked with a different Standard coordinates, wherein the standard block is obtained by dividing each standard picture book page according to preset division rules;

The collection module is used to collect the captured current picture book page and obtain the shooting pixels of the current picture book page when the click operation of the picture book by the user is detected;

A retrieval module, configured to search in the standard library, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page;

A positioning module, configured to determine the click area by using a fingertip positioning method according to the click position corresponding to the click operation;

A conversion module, configured to convert the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates based on the shooting pixels and the standard pixels;

A search module, configured to search for a standard block containing the target area from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page and determine it as the target block;

A recognition module, configured to determine a picture book recognition result based on the target block.

A tutoring machine includes a memory and a processor, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the following steps:

A picture book recognition result is determined based on the target block.

A computer-readable medium, storing computer-readable instructions, which, when executed by a processor, cause the processor to perform the following steps:

A picture book recognition result is determined based on the target block.

Beneficial effect

The above-mentioned picture book identification method, device, tutoring machine, and storage medium, by obtaining the standard picture book library, the standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, each The standard blocks are marked with different standard coordinates, wherein the standard block is obtained by dividing each standard picture book page according to the preset division rules; when the click operation of the picture book by the user is detected, the photographing the obtained current picture book page, and obtaining the shooting pixels of the current picture book page; searching in the standard library, determining the standard picture book page corresponding to the current picture book page as the target picture book page, and obtaining the standard picture book page of the target pixels; according to the click position corresponding to the click operation, the click area is determined by using the fingertip positioning method; based on the shooting pixels and the standard pixels, the click area is converted into a coordinate system consistent with the standard coordinates The target area; from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page, search for the standard block containing the target area and determine it as the target block; determine the picture book recognition result based on the target block, by The introduction of coordinate positioning and transformation methods, positioning according to the clicked area, realizes the precise positioning of the picture book page, and then recognizes the corresponding area of the standard picture book page, avoiding the recognition of remote pictures, and greatly improving the text recognition rate in picture books .

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

in:

Fig. 1 is the flowchart of picture book recognition method in an embodiment;

Fig. 2 is the flowchart of picture book identification method in another embodiment;

Fig. 3 is a flow chart of determining picture book recognition results in an embodiment;

Fig. 4 is a flowchart of a method for determining a target area in an embodiment;

FIG. 5 is a flow chart of a method for determining a click area in an embodiment;

Fig. 6 is a structural block diagram of a picture book recognition device in an embodiment;

Fig. 7 is a structural block diagram of the tutoring machine in one embodiment.

Embodiments of the present invention

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

As shown in FIG. 1 , in one embodiment, a method for identifying picture books is provided, and the method for identifying picture books can be applied to both a terminal and a server. This embodiment uses the application to a server as an example for illustration. The picture book identification method specifically includes the following steps:

Step 102, obtain the picture book standard library, the standard library contains a plurality of standard picture book pages and a plurality of standard blocks corresponding to each standard picture book page, each standard block is marked with a different standard coordinates, wherein the standard block It is obtained by dividing each standard picture book page according to preset division rules.

Wherein, the standard block refers to an area in a standard picture book page in the picture book. A standard picture book page can be a picture scanned by a scanning device. The preset dividing rule can be divided according to the content of a standard picture book page, and the corresponding standard block contains at least pictures in a character area, a picture area or an artistic word area, wherein the character area can be an area containing a text, or Can be an area containing multiple text. Specifically, each standard picture book page can be identified based on feature extraction, divided according to the identified content to obtain standard blocks, and the corresponding standard coordinates for each standard block. It can be understood that in this embodiment, each standard picture book is also divided in advance and the standard coordinates are marked, so as to realize the positioning of each standard block, so that further processing can be performed based on the standard block.

Step 104, when the user's click operation on the picture book is detected, the captured current picture book page is captured, and the shooting pixels of the current picture book page are acquired.

Wherein, the current picture book page refers to the picture of the picture book currently clicked by the user that needs to be identified. Shooting pixels refer to the pixel information of the current picture book page. Specifically, when the server detects the user's click operation on the picture book, the current picture book page is captured by the camera device, and the pixel information of the current picture book page is acquired.

Step 106, search in the standard library, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page.

Wherein, the target picture book page refers to a standard picture book page that is consistent with the content of the current picture book page. Standard pixels refer to the pixel information of the target picture book page. Specifically, image comparison methods can be used, for example, to extract the image features of the current picture book page and each standard picture book page, determine the standard picture book page that matches the image features of the current picture book page as the target picture book page, and obtain the target picture book page standard pixels of the page.

Step 108, according to the click position corresponding to the click operation, use the fingertip positioning method to determine the click area.

Among them, the fingertip positioning method refers to a positioning method that detects the position of the hand in the image and locates the coordinate information of the fingertip. The fingertip positioning method can be an image recognition positioning method based on deep learning, or a positioning based on feature extraction. method. As a preference of this embodiment, in order to improve the efficiency of fingertip positioning, a positioning method based on feature extraction is selected to avoid the cumbersome image recognition of deep learning and the time-consuming increase of fingertip positioning.

Step 110, based on the captured pixels and the standard pixels, transform the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates.

Specifically, the proportional relationship between the shooting pixel and the standard pixel may be calculated, and then according to the proportional relationship between the two, the clicked area is transformed into a target area consistent with the coordinate system corresponding to the standard coordinate. Understandably, since the current picture book page is captured by a camera device, the shape and quality of the current picture book page captured are affected by the limited shape of the camera device. For example, the current picture book page may be large at the top and small at the bottom. Trapezoid picture. In order to improve the accuracy of the subsequent recognition of the clicked area, the clicked area is converted into a target area that is consistent with the coordinate system corresponding to the standard coordinates, to ensure the accuracy of the clicked area, and then to ensure the accuracy of the corresponding target area, so that the subsequent target picture book page The target area for picture book recognition.

Step 112, searching for a standard block containing the target area from the standard coordinates of the multiple standard blocks corresponding to the target picture book page and determining it as the target block.

Wherein, the target block refers to an area of a standard picture book page that requires picture book identification. Specifically, after the target area is determined, according to the coordinates of the target area and the standard coordinates of the multiple standard blocks corresponding to the target picture book page, search the standard coordinates of the multiple standard blocks corresponding to the target picture book page that contains The standard block of the target area is used to obtain the target area for subsequent efficient identification based on the target block.

Step 114, determine the picture book recognition result based on the target block.

Specifically, screenshot the picture according to the target area in the target block, perform OCR recognition on the intercepted area, and obtain the picture book recognition result. It can be understood that in this embodiment, by using the target area to identify the standard block of the standard picture book page, since the quality of the standard picture book page is higher than the quality of the captured picture, the identification of the remote picture is avoided. , greatly improving the recognition efficiency of picture books.

The above-mentioned picture book identification method obtains the standard picture book library, which contains a plurality of standard picture book pages and a plurality of standard blocks corresponding to each standard picture book page, and each standard block is marked with a different standard coordinate, wherein, The standard block is obtained by dividing each standard picture book page according to the preset division rules; when the user's click operation on the picture book is detected, the current picture book page obtained by shooting is collected, and the shooting pixels of the current picture book page are obtained; Search in the standard library, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page; according to the click position corresponding to the click operation, use the fingertip positioning method to determine the click area; based on the shooting pixels and standard pixels, convert the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates; find the standard block containing the target area from each standard coordinate of the multiple standard blocks corresponding to the target picture book page and determine it as the target area block; based on the target block to determine the recognition result of the picture book, by introducing the coordinate positioning and transformation method, and positioning according to the clicked area, the precise positioning of the picture book page is realized, and then the corresponding area of the standard picture book page is recognized, avoiding the remote The recognition of pictures has greatly improved the recognition rate of Chinese characters in picture books.

As shown in Figure 2, in one embodiment, before determining the picture book recognition result based on the target block, it also includes:

Step 116, based on the click area, intercept the first picture book page from the current picture book page;

Step 118, respectively extracting the first text information of the first picture book page and the second text information of the target block;

Step 120, judging whether the first text information matches the second text information;

Step 122, if not matching, determine that the recognition result is that the clicked area is a blank area.

In this embodiment, the first picture book page is intercepted according to the clicked area in the current picture book page, OCR is performed on the first picture book page and the target block respectively, and the first text information of the first picture book page and the second text information of the target block are obtained. Text information, judging whether the first text information matches the second text information, if they do not match, it indicates that there is no picture book information that matches the clicked area, therefore, it is determined that the clicked area is a blank area as a result of the recognition. Further, after determining that the clicked area is blank After the area, you can continue to obtain new click areas for picture book recognition, or reposition for picture book recognition to improve the efficiency of picture book recognition.

As shown in Figure 3, in one embodiment, determining the picture book recognition result based on the target block includes:

Step 114A, based on the target area, intercept the second picture book page from the target block;

Step 114B, identify the second picture book page, and obtain the picture book recognition result.

Specifically, intercept the second picture book page according to the target area in the target block, perform OCR recognition on the second picture book page, and obtain the picture book recognition result. Understandably, since the second picture book page is an area of a standard picture book page, its picture Compared with the current picture book page, the quality is higher, therefore, the recognition accuracy rate of the second picture book page is greatly improved, and the accuracy of the picture book recognition result is guaranteed.

As shown in Figure 4, in one embodiment, based on the captured pixels and the standard pixels, the clicked area is transformed into a target area consistent with the coordinate system corresponding to the standard coordinates, including:

Step 110A, calculating the mapping transformation matrix based on the captured pixels and the standard pixels;

In step 110B, the clicked area is subjected to coordinate transformation processing according to the mapping transformation matrix to obtain the target area.

In this embodiment, the affine transformation matrix between the current picture book page and the standard picture book page can be calculated according to the mapping relationship between the captured pixels and the standard pixels as the mapping transformation matrix; the coordinates of the clicked area are transformed and calculated according to the affine transformation matrix to obtain target area. It can be understood that in this embodiment, the accuracy of the target area is guaranteed by performing affine transformation on the coordinates of the clicked area.

In one embodiment, the method further includes: respectively identifying and semantically analyzing each standard block in the standard library, and generating a picture book interpretation information mapping table, each standard block corresponding to a piece of picture book interpretation information.

In this embodiment, each standard block can be identified and semantically analyzed in advance. For example, text content, picture content, or artistic word content can be voice analyzed to generate picture book interpretation information corresponding to each standard block, and the picture book The paraphrase information mapping table is stored in the server.

In one embodiment, the method further includes: obtaining the target picture book interpretation information corresponding to the target block from the picture book interpretation information mapping table; and displaying the target picture book interpretation information.

Specifically, the target picture book interpretation information corresponding to the target block is obtained from the picture book interpretation information mapping table, and the target picture book interpretation information is displayed. Recognition greatly improves the user experience in the picture book reading process and the user's ability to understand the picture book content during the picture book reading process.

As shown in FIG. 5, in one embodiment, according to the click position corresponding to the click operation, the click area is determined by using the fingertip positioning method, including:

Step 108A, acquiring the clicked image including the click operation performed by the finger;

Step 108B, performing edge detection on the clicked image to obtain finger contour features;

In step 108C, the click area is determined based on the contour features of the finger.

In this embodiment, at first, obtain the click image that includes the finger to perform the click operation; edge detection is performed on the click image to obtain the finger contour features, and the edge detection methods include but are not limited to Sobel operator, Laplacian operator, Canny The operator locates the fingertip of the index finger based on the contour features of the finger, returns the coordinate information of the hand and the fingertip of the index finger, and determines the click area; it can also locate the middle joint of the index finger and the root of the index finger based on the finger contour feature and detection of the fingertip of the index finger , the middle joint of the middle finger, and the coordinate information of the root of the middle finger to determine the click area. In this embodiment, the precise positioning of the clicked area is realized through the method of edge detection, and the efficiency of fingertip positioning is improved.

As shown in Figure 6, in one embodiment, a picture book recognition device is proposed, the device includes:

The acquiring module 602 is used to acquire a standard picture book library, the standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, and each of the standard blocks is marked with a different standard coordinates, wherein the standard block is obtained by dividing each standard picture book page according to preset division rules;

The collection module 604 is used to collect the current picture book page obtained by shooting when the user's click operation on the picture book is detected, and obtain the shooting pixels of the current picture book page;

A retrieval module 606, configured to search in the standard library, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page;

A positioning module 608, configured to determine the click area by using a fingertip positioning method according to the click position corresponding to the click operation;

A conversion module 610, configured to convert the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates based on the photographing pixels and the standard pixels;

A search module 612, configured to search for a standard block containing the target area from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page and determine it as the target block;

A recognition module 614, configured to determine a picture book recognition result based on the target block.

In one embodiment, the device also includes:

An intercepting module, configured to intercept a first picture book page from the current picture book page based on the click area;

An extraction module, configured to extract the first text information of the first picture book page and the second text information of the target block respectively;

A matching module, configured to determine whether the first text information matches the second text information;

A determining module, configured to determine that the clicked area is a blank area as a result of the recognition if there is no match.

In one embodiment, the recognition module includes:

An intercepting unit, configured to intercept a second picture book page from the target block based on the target area;

The identification unit is configured to identify the second picture book page to obtain the picture book identification result.

In one embodiment, the conversion module includes:

a calculation unit, configured to calculate a mapping transformation matrix based on the captured pixels and the standard pixels;

The transformation unit is configured to perform coordinate transformation processing on the clicked area according to the mapping transformation matrix to obtain the target area.

In one embodiment, the device also includes:

A search unit, configured to obtain the target picture book interpretation information corresponding to the target block from the picture book interpretation information mapping table;

A display unit, configured to display the target picture book interpretation information.

In one embodiment, the positioning module includes:

An acquisition unit, configured to acquire a click image including a finger click operation;

An extraction unit, configured to perform edge detection on the clicked image to obtain finger contour features;

A determining unit, configured to determine the click area based on the outline features.

Fig. 7 shows the internal structure diagram of the tutoring machine in one embodiment. The tutoring machine may specifically be a server, and the server includes but is not limited to a high-performance computer and a cluster of high-performance computers. As shown in FIG. 7, the tutoring machine includes a processor, a memory and a network interface connected through a system bus. Wherein, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the tutoring machine stores an operating system and also stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can realize the method for identifying picture books. Computer-readable instructions may also be stored in the internal memory, and when the computer-readable instructions are executed by the processor, the processor may execute the picture book recognition method. Those skilled in the art can understand that the structure shown in Figure 7 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation to the tutoring machine on which the solution of this application is applied. The specific tutoring machine can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, the picture book recognition method provided in this application can be implemented in the form of a computer-readable instruction, and the computer-readable instruction can be run on the tutoring machine as shown in FIG. 7 . Various program templates constituting the picture book recognition device can be stored in the memory of the tutoring machine. For example, an acquisition module 602 , a collection module 604 , a retrieval module 606 , a positioning module 608 , a conversion module 610 , a search module 612 , and an identification module 614 .

A tutoring machine, comprising a memory, a processor, and computer-readable instructions stored in the memory and operable on the processor. When the processor executes the computer-readable instructions, the following steps are implemented: acquiring a picture book The standard library, the standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, each of the standard blocks is marked with a different standard coordinates, wherein the The standard block is obtained by dividing each standard picture book page according to the preset division rules; when the user's click operation on the picture book is detected, the current picture book page obtained by shooting is collected, and the shooting pixels of the current picture book page are obtained; Retrieve in the standard picture book page, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page; according to the click position corresponding to the click operation, use fingertips to locate The method determines the click area; based on the shooting pixels and the standard pixels, convert the click area into a target area consistent with the coordinate system corresponding to the standard coordinates; from the multiple standard blocks corresponding to the target picture book page Find the standard block containing the target area in each of the standard coordinates and determine it as the target block; determine the picture book recognition result based on the target block.

In one embodiment, before determining the picture book recognition result based on the target block, it further includes: intercepting the first picture book page from the current picture book page based on the click area; extracting the first picture book page respectively the first text information of the target block and the second text information of the target block; determine whether the first text information matches the second text information; if not, determine that the recognition result is that the clicked area is a blank area.

In one embodiment, the determining the picture book recognition result based on the target block includes: intercepting a second picture book page from the target block based on the target area; identifying the second picture book page to obtain The picture book recognition result.

In one embodiment, converting the click area into a target area consistent with the coordinate system corresponding to the standard coordinates based on the shooting pixels and the standard pixels includes: based on the shooting pixels and the A mapping transformation matrix is calculated for standard pixels; coordinate transformation processing is performed on the clicked area according to the mapping transformation matrix to obtain the target area.

In one embodiment, according to the click position corresponding to the click operation, using the fingertip positioning method to determine the click area includes: acquiring a click image that includes a finger performing a click operation; performing edge detection on the click image to obtain Finger contour features; determining the click area based on the contour features.

A computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, characterized in that, when the computer-readable instructions are executed by a processor, the following steps are implemented: obtaining a standard library of picture books, the standard The gallery contains a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, and each of the standard blocks is marked with a different standard coordinate, wherein the standard block is obtained by placing each A standard picture book page is obtained by dividing according to the preset division rules; when the click operation of the picture book by the user is detected, the current picture book page obtained by shooting is collected, and the shooting pixels of the current picture book page are obtained; Retrieve, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page; according to the click position corresponding to the click operation, use the fingertip positioning method to determine the click area; based on the The shooting pixels and the standard pixels are used to convert the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates; from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page Searching for a standard block containing the target area and determining it as the target block; determining the picture book recognition result based on the target block.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer-readable instructions to instruct related hardware, and the program can be stored in a non-volatile computer-readable In the storage medium, when the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims

A picture book identification method, characterized in that the method comprises:

Acquiring a standard picture book library, the standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages, each of the standard blocks is marked with different standard coordinates, wherein, The standard block is obtained by dividing each standard picture book page according to preset division rules;

When the user's click operation on the picture book is detected, the current picture book page obtained by shooting is collected, and the shooting pixels of the current picture book page are obtained;

Searching in the standard picture book page, determining the standard picture book page corresponding to the current picture book page as the target picture book page, and obtaining the standard pixels of the target picture book page;

According to the click position corresponding to the click operation, a fingertip positioning method is used to determine the click area;

converting the clicked area into a target area consistent with a coordinate system corresponding to the standard coordinates based on the photographing pixels and the standard pixels;

Finding the standard block containing the target area from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page is determined as the target block;

A picture book recognition result is determined based on the target block.
The picture book recognition method according to claim 1, wherein, before determining the picture book recognition result based on the target block, further comprising:

Based on the click area, intercepting a first picture book page from the current picture book page;

respectively extracting the first text information of the first picture book page and the second text information of the target block;

judging whether the first text information matches the second text information;

If not, it is determined that the recognition result is that the clicked area is a blank area.
The picture book recognition method according to claim 1, wherein said determining the picture book recognition result based on the target block comprises:

Based on the target area, intercepting a second picture book page from the target block;

Recognize the second picture book page to obtain the picture book recognition result.
The picture book recognition method according to claim 1, characterized in that, based on the shooting pixels and the standard pixels, converting the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates includes :

calculating a mapping transformation matrix based on the captured pixels and the standard pixels;

and performing coordinate transformation processing on the clicked area according to the mapping transformation matrix to obtain the target area.
The picture book identification method according to claim 1, wherein the method further comprises:

Each standard block in the standard library is identified and semantically analyzed to generate a picture book interpretation information mapping table, and each standard block corresponds to a piece of picture book interpretation information.
The picture book identification method according to claim 5, wherein the method further comprises:

Obtain the target picture book interpretation information corresponding to the target block from the picture book interpretation information mapping table;

The paraphrase information of the target picture book is displayed.
The picture book recognition method according to claim 1, characterized in that, according to the click position corresponding to the click operation, using a fingertip positioning method to determine the click area includes:

Obtain the click image that contains the click operation of the finger;

Carry out edge detection to described click image, obtain finger outline feature;

The click area is determined based on the contour feature.
A picture book recognition device, characterized in that the picture book recognition device comprises:

The obtaining module is used to obtain a standard library of picture books. The standard library includes a plurality of standard picture book pages and a plurality of standard blocks corresponding to each of the standard picture book pages. Each of the standard blocks is marked with a different Standard coordinates, wherein the standard block is obtained by dividing each standard picture book page according to preset division rules;

The collection module is used to collect the captured current picture book page and obtain the shooting pixels of the current picture book page when the click operation of the picture book by the user is detected;

A retrieval module, configured to search in the standard library, determine the standard picture book page corresponding to the current picture book page as the target picture book page, and obtain the standard pixels of the target picture book page;

A positioning module, configured to determine the click area by using a fingertip positioning method according to the click position corresponding to the click operation;

A conversion module, configured to convert the clicked area into a target area consistent with the coordinate system corresponding to the standard coordinates based on the shooting pixels and the standard pixels;

A search module, configured to search for a standard block containing the target area from each of the standard coordinates of the plurality of standard blocks corresponding to the target picture book page and determine it as the target block;

A recognition module, configured to determine a picture book recognition result based on the target block.
A tutoring machine, characterized in that it includes a memory, a processor, and computer-readable instructions stored in the memory and operable on the processor, characterized in that the processor executes the computer-readable The step of realizing the picture book recognition method according to any one of claims 1 to 7 when the instruction is given.
A computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, the picture book according to any one of claims 1 to 7 is realized. Identify the steps of the method.