CN110929684B

CN110929684B - Content identification method and device for picture book

Info

Publication number: CN110929684B
Application number: CN201911253237.XA
Authority: CN
Inventors: 贾志强; 俞晓君; 俞志晨; 陆羽皓
Original assignee: Beijing Guangnian Infinite Technology Co ltd
Current assignee: Beijing Guangnian Infinite Technology Co ltd
Priority date: 2019-12-09
Filing date: 2019-12-09
Publication date: 2023-04-18
Anticipated expiration: 2039-12-09
Also published as: CN110929684A

Abstract

The invention provides a content identification method for a picture book, which comprises the following steps: the method comprises the following steps: acquiring images of the picture book to obtain cover image data containing the cover information of the picture book and inner page image data containing the inner page information of the picture book; step two: performing content identification on the front cover image data, and recording corresponding correction angle information for representing the arrangement angle of the picture book when the content identification is successful; step three: during the process of identifying the inner page, the correction angle information is called, and the inner page image data is subjected to inner page angle conversion to obtain converted inner page image data; step four: and performing content identification on the converted inner page image data. The invention can identify the contents of the front cover and the inner page of the picture book under the support of the intelligent equipment special for children, and assist the children in reading the picture book; moreover, when the content of the drawn book is identified, the cover can be identified at multiple angles, and the influence of the inclined drawn book placing angle on the content identification is overcome.

Description

Content identification method and device for picture book

Technical Field

The invention relates to the field of artificial intelligence, in particular to a content identification method and device for a picture book.

Background

The picture book originates from Europe and America in the second half of the 19 th century, is a preferred book for children in families in developed countries, and is internationally acknowledged as a book most suitable for children to read, and is rich in form and large in detail, so that the picture book becomes a new trend of enlightening education for children. This refers to a kind of book with a few characters and mainly used for painting. The picture book not only tells stories and learns knowledge, but also can comprehensively help children construct a mental world and cultivate multivariate intelligence. When children are small, the contact of the picture book is faster to read than the direct contact of the picture book with various characters, and the reading of the picture book is also a preferred mode for cultivating the reading habit of the children.

However, in practical use, when a child reads a picture book, the picture book is not usually placed on a desktop but is often moved or obliquely placed on the desktop, so that the oblique picture book angle has a large influence on the device for identifying the picture book content when the picture book content is identified, and the success rate of picture book identification is reduced.

Therefore, the invention provides a content identification method and device for a picture book.

Disclosure of Invention

In order to solve the above problem, the present invention provides a content identification method for a picture book, the method comprising the following steps:

the method comprises the following steps: acquiring images of the picture book to obtain cover image data containing the cover information of the picture book and inner page image data containing the inner page information of the picture book;

step two: performing content identification on the cover image data, and recording corresponding correction angle information for representing the placement angle of the picture book when the content identification is successful;

step three: during the process of identifying the inner page, calling the correction angle information, and performing inner page angle conversion on the inner page image data to obtain converted inner page image data;

step four: and performing content identification on the converted inner page image data.

According to an embodiment of the present invention, the first step further includes: and carrying out duplication elimination processing on the collected front cover image data and the collected inner page image data, and deleting repeated images to obtain the duplicate eliminated front cover image data and inner page image data.

According to an embodiment of the present invention, the second step further includes: and uploading the cover image data to a cloud end, and performing image retrieval to identify contents through the cloud end.

According to an embodiment of the present invention, the second step further includes:

sequentially adopting a plurality of correction angles to carry out content recognition on the cover image data in a full-view angle recognition mode, and determining corresponding correction angle information when the content recognition is successful;

and binding the currently obtained correction angle information with the current unique identification code of the intelligent equipment special for the children, and storing the information in an angle database.

According to one embodiment of the present invention, the step three further comprises the following steps:

during the process of identifying the inner page, identifying the unique identification code of the intelligent equipment special for the child, searching in the angle database, and determining the matched correction angle information;

and performing angle rotation on the inner page image contained in the inner page image data based on the matched correction angle information to obtain converted inner page image data.

According to an embodiment of the present invention, the fourth step further comprises: uploading the converted inner page image data to the cloud end, and performing content recognition through the cloud end to obtain multi-mode output data.

According to an embodiment of the invention, the method further comprises:

acquiring identity characteristic information of a current user, judging user attributes of the current user, and determining the category of the current user, wherein the category of the user comprises: a child user.

According to another aspect of the invention, there is also provided a program product containing a series of instructions for carrying out the steps of the method according to any one of the above.

According to another aspect of the present invention, there is also provided a content recognition apparatus for a picture book, the apparatus including:

a first module: the system comprises a picture book, a front cover image data acquisition module, a back cover image data acquisition module, a front cover image data acquisition module and a back cover image data acquisition module, wherein the picture book is used for carrying out image acquisition on the picture book, and acquiring front cover image data containing picture book front cover information and back page image data containing picture book back page information;

a second module: the correction angle information is used for carrying out content identification on the front cover image data and recording corresponding correction angle information used for representing the layout angle of the picture book when the content identification is successful;

a third module: the correction angle information is called when the inner page identification process is carried out, and the inner page angle conversion is carried out on the inner page image data to obtain the converted inner page image data;

a fourth module: the method is used for identifying the content of the converted inner page image data.

According to another aspect of the invention there is also provided a child-specific smart device for carrying out a series of instructions of the method steps as described in any one of the above.

The content identification method and the content identification device for the picture book can identify the content of the front cover and the inner page of the picture book under the support of intelligent equipment special for children, and assist the children in reading the picture book; and, when drawing this content recognition, can carry out the front cover discernment of multi-angle, overcome crooked this that draws and put the influence of angle to content recognition, greatly promoted the efficiency and the success rate of drawing this reading, improved user's the experience of drawing this reading.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

FIG. 1 shows a flow diagram of a content recognition method for a picture book according to one embodiment of the invention;

FIG. 2 shows a flow chart of content recognition in a content recognition method for a picture book according to an embodiment of the invention;

FIG. 3 is a block diagram of a content recognition apparatus for a picture book according to an embodiment of the present invention;

FIG. 4 shows a block diagram of a child-specific smart device, in accordance with one embodiment of the present invention;

FIG. 5 shows an interaction diagram of a content recognition method for a picture book according to one embodiment of the invention;

FIG. 6 is a schematic diagram of three-party communication between a user, a child-specific smart device, and a cloud according to an embodiment of the invention; and

fig. 7 shows a flow chart of a content recognition method for a picture book according to another embodiment of the invention.

Detailed Description

To make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention are described in detail below with reference to the accompanying drawings.

For clarity, the following description is required before the examples:

the intelligent equipment special for children supports multi-mode man-machine interaction, and has AI capabilities of natural language understanding, visual perception, language voice output, emotion expression action output and the like; the social attributes, personality attributes, character skills and the like can be configured, so that the user can enjoy intelligent and personalized smooth experience. In a specific embodiment, the intelligent device special for children refers specifically to a book drawing accompanying intelligent device, which can be an intelligent table lamp, a book drawing robot, an intelligent tablet, a accompanying watch, a story machine capable of identifying the book drawing and the like.

The children special intelligent device acquires multi-modal data (including a picture book content identification instruction) of a user, and performs semantic understanding, visual identification, cognitive computation and emotion computation on the multi-modal data under the support of the cloud capability to complete a decision output process. The special intelligent device for children can be provided with apps of android, ios, linux and windows platforms.

The cloud terminal is a terminal which provides the processing capability of the children special intelligent device for performing semantic understanding (image recognition, language semantic understanding, action semantic understanding, visual recognition, emotion calculation and cognitive calculation) on interaction requirements (including drawing book recognition wishes) of the user, interaction with the user is achieved, and the children special intelligent device is made to output multi-mode data (including content recognition data).

Various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Fig. 1 shows a flow chart of a content identification method for a sketch according to an embodiment of the invention.

As shown in fig. 1, in step S101, a drawing book is subjected to image acquisition, and cover image data including cover information of the drawing book and inner page image data including inner page information of the drawing book are acquired.

Particularly, carry out image acquisition to drawing the book through the equipment that possesses the image acquisition ability, can carry out image acquisition to drawing this content through the camera that sets up on children's special smart machine, also can carry out image acquisition to drawing this content through the camera that does not set up on children's special smart machine, later with image data transmission to children's special smart machine that will gather.

In one embodiment, in step S101, the collected front cover image data and inner page image data may be further subjected to a duplicate removal process, and duplicate images are deleted to obtain the duplicate-removed front cover image data and inner page image data.

As shown in fig. 1, in step S102, content recognition is performed on the front cover image data, and correction angle information for representing the placement angle of the picture book is recorded when the content recognition is successful.

In one embodiment, in step S102, the cover image data may be uploaded to the cloud, and image retrieval may be performed for content identification through the cloud.

Specifically, as shown in fig. 2, the front cover multi-angle recognition may be performed first, that is, the cloud performs content recognition on the front cover image data by sequentially adopting a plurality of correction angles through a full-view recognition mode, and determines corresponding correction angle information when the content recognition is successful. Specifically, the cover image data can be identified by using a plurality of angles, the identification accuracy and success rate can be identified, and the correction angle information when the content identification succeeds can be found.

Further, can be with drawing the contained angle definition between this benchmark axis and the desk lamp camera discernment datum line for the angle that needs the correction, the contained angle scope is 0 degree angle to 180 degrees, and when the contained angle was zero, the discernment angle that need not correct and draw this, when carrying out front cover angle discernment, can begin from 0 degree angle in proper order, discerns front cover content in proper order, and the angle when confirming the discernment is successful is as correction angle information.

Then, as shown in fig. 2, the correction angle information is recorded, and the currently obtained correction angle information is bound with the current unique identification code of the intelligent device dedicated to the child, and is stored in the angle database. Specifically, each child-specific intelligent device has a Unique Identifier, and the Identifier may be a UUID (universal Unique Identifier), so that all elements in the distributed system can have Unique identification information.

As shown in fig. 1, in step S103, when the inner page identification process is performed, the correction angle information is retrieved, and the inner page image data is subjected to inner page angle conversion to obtain converted inner page image data.

In one embodiment, as shown in fig. 2, when the inner page identification process is performed, the unique identification code of the child-specific intelligent device is identified and searched in the angle database to determine the matched correction angle information.

Particularly, can set up the angle database on the high in the clouds, be used for depositing specially and correct angle information, correct angle information in the angle database corresponds with special smart machine of children, after the only identification code of the special smart machine of children is discerned, can confirm the correction angle information that corresponds with it. Because the skew angle of the picture book is not fixed and can change along with time, the correction angle information corresponding to the unique identification code is also variable and can be updated at any time.

Then, as shown in fig. 2, the intra-page angle conversion is performed, and intra-page angle rotation is performed on the intra-page image included in the intra-page image data based on the matched correction angle information, so that converted intra-page image data is obtained. In order to improve the identification efficiency and success rate of the inner page image, the previously collected inner page image data needs to be rotated according to the determined correction angle information, generally, the inner page needs to be rotated to an angle of 0 degree, that is, the angle between the drawing reference axis and the identification reference line of the table lamp camera is 0 degree, and the accuracy rate of image identification is theoretically the highest.

As shown in fig. 1, in step S104, content recognition is performed on the converted inner page image data.

In one embodiment, as shown in fig. 2, the converted image data of the inner page may be uploaded to a cloud, and the cloud performs content recognition to obtain multi-modal output data. Generally, the multimodal output data is content recognition data, which includes voice data, for example, the content recognition data may be voice story data corresponding to an inner page of a story book (e.g., snow white princess voice data).

In the invention, the intelligent equipment special for children is provided with image acquisition (scanning) equipment which can acquire image information under the current visual field. Specifically, a camera can be arranged on the intelligent equipment special for the children, the intelligent equipment enters into the picture book content recognition under the instruction of the user, the current image information is collected through the camera, the current image information can be a story book which the user is looking at, and the camera takes pictures (scans) of the story book to acquire the image information.

Then, the image information is identified, the content in the inner page is determined, and the content is played through the intelligent device special for children (specifically, the intelligent device special for children displays the content identification data through a loudspeaker, a display screen and the like). In addition, the content identification data may also be stored in the child-dedicated smart device or in the local storage device, and the storage location of the content identification data may be adjusted according to the actual application, which is not limited in the present invention.

According to one embodiment of the present invention, identity characteristic information of a current user is acquired, a user attribute of the current user is judged, and a category of the current user is determined, wherein the category of the user includes: a child user. The user group to which the invention is directed is mainly a child user, so the identity attribute of the user needs to be determined. There are many ways to determine the identity of the user, and generally, the identity of the user can be identified through a facial recognition function or a fingerprint recognition method. Other ways of determining the identity of the user may be applied to the present invention, and the present invention is not limited thereto.

Fig. 3 is a block diagram illustrating a content recognition apparatus for a picture book according to an embodiment of the present invention.

As shown in fig. 3, the apparatus comprises a first module 301, a second module 302, a third module 303 and a fourth module 304. The first module 301 includes an acquisition unit 3011 and a deduplication unit 3012. The second module 302 includes an angle unit 3021 and a recording unit 3022. The third module 303 includes a retrieving unit 3031 and a correcting unit 3032. The fourth module 304 includes a recognition unit 3041 and an output unit 3042.

The first module 301 is configured to perform image acquisition on the picture book, and acquire cover image data including the cover information of the picture book and inner page image data including the inner page information of the picture book. The acquisition unit 3011 is configured to acquire cover image data including the drawing book cover information and inner page image data including the drawing book inner page information. The deduplication unit 3012 is configured to perform deduplication processing on the collected front cover image data and inner page image data, delete duplicate images, and obtain deduplicated front cover image data and inner page image data.

The second module 302 is configured to perform content identification on the cover image data, and record corresponding correction angle information for representing the placement angle of the picture book when the content identification is successful. The angle unit 3021 is configured to perform content identification on the cover image data by sequentially using a plurality of correction angles through a full-view identification method, and determine corresponding correction angle information when the content identification is successful. The recording unit 3022 is configured to bind the currently obtained correction angle information with the current unique identifier of the smart device dedicated for children, and store the information in the angle database.

The third module 303 is configured to, during the process of identifying the inner page, retrieve correction angle information, and perform inner page angle conversion on the inner page image data to obtain converted inner page image data. The calling unit 3031 is configured to, during the process of identifying the inside page, identify the unique identification code of the intelligent device dedicated to the child and search the angle database to determine the matched correction angle information. The correction unit 3032 is configured to perform angle rotation on an inner page image included in the inner page image data based on the matched correction angle information to obtain converted inner page image data.

The fourth module 304 is used for performing content identification on the converted inner page image data. The recognition unit 3041 is configured to perform content recognition on the converted inner page image data to obtain multi-modal output data. The output unit 3042 is configured to output multi-modal output data, where the multi-modal output data includes content recognition data.

FIG. 4 shows a block diagram of a child-specific smart device architecture, according to an embodiment of the present invention. As shown in fig. 4, the child-specific smart device 400 includes a signal acquisition device 401, a display screen 402, a signal output device 403, and a central processor 404.

The intelligent device 400 for children comprises intelligent devices supporting input and output modules such as vision, perception and control, can access the internet, such as an intelligent desk lamp, a robot, an intelligent tablet, a watch and a recognizable story machine for drawing books, and has a function of drawing book content recognition interaction, can receive a book drawing content recognition instruction input by a user, and outputs content recognition data on the intelligent device for children.

The special intelligent device for children 400 can comprise a client dedicated to the drawing content identification interaction process, the client can be loaded in an android system environment, and the special intelligent device for children can be an android system child watch with 4G or even 5G communication capacity.

The signal acquisition device 401 is used to acquire image information and instructions of a user. Examples of signal acquisition devices 401 include microphones, scanners, cameras, sensory devices for voice operation, such as using visible or invisible wavelengths of radiation, signals, environmental data, and so forth. Data of a plurality of modes (including image information and an instruction of a user) can be acquired through the above-mentioned input device. The multimodal data may include one or more of text, audio, visual, and perceptual data, and the present invention is not limited thereto.

The display 402 can show the current sketch content, and can also display the corresponding matching data (corresponding illustration when playing a story) when playing the sketch content data.

The signal output device 403 is used to output audio data. The signal output device 403 may be a device capable of outputting audio data, such as a power amplifier and a speaker. The signal output device 403 may also be a device supporting other types of data, and the invention is not limited thereto.

The central processor 404 is used to process the data generated during the finger-read interaction. The Processor may be a data Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center of the terminal, and various interfaces and lines connecting the various parts of the overall terminal.

The child-specific smart device 400 includes a memory, which mainly includes a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like; the storage data area may store data (such as audio data, browsing history, and the like) created according to the use of the child-specific smart device 400, and may also store click-to-read data corresponding to a click-to-read printed matter, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

In addition, the content recognition system for picture books provided by the present invention may also be incorporated into a program product comprising a series of instructions for performing the steps of the content recognition method for picture books. The program product is capable of executing computer instructions comprising computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc.

The program product may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like.

It should be noted that the program product may include content that is appropriately increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, the program product does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

Fig. 5 shows an interaction diagram of a content recognition method for a picture book according to an embodiment of the present invention. Referring to fig. 5, a schematic diagram of the content recognition of the user-drawn book is shown, in which the intelligent device 400 dedicated for children uses an intelligent desk lamp, and the drawn book 502 is a storybook.

As shown in fig. 5, the user 501 may receive a drawing content recognition instruction sent by the user 501 in a voice manner, and start a drawing content recognition process, at this time, a camera on the intelligent desk lamp aims at the storybook, the camera collects front cover image data and inner page image data, after the front cover image data is successfully recognized, angle conversion is performed on the inner page image data according to the correction angle information, and then the converted inner page image data is recognized. For example, the current inner page image data is "two mice in the story", at this time, the table lamp starts to play the story of the two mice through the speaker, and the user 501 can control the pause and the play of the story through the touchable virtual keys on the intelligent table lamp. The process of playing two pony mouse stories is as follows:

the desk lamp is played: in the mouse kingdom, such a message prevails: there is one apple tree in the nine-row mountain. The apple tree can bear large red apple, has rich nutrition and brain nourishing effect, and can not be apprehended by cat.

The desk lamp is played: the bite and when heard, the mind: with such good things! Then we start to eat the apple in the morning! Then, when the time agreed by the sting and the current date is up, the two immediately lift the bamboo pole to go to the nine-row mountain. (at this time, the user inputs the intention of 'pause', the desk lamp pauses playing, and the desk lamp continues playing after the user inputs the intention of 'start')

The desk lamp is played: when the person bites or slips, the person is not easy to climb to the cliff when the person lifts the bamboo pole, the apple tree is near to a short distance, but the apple tree is on the top of a mountain opposite to the cliff, and the person is a dead chicken and meditation is. Finally, the jingle thinks of a solution, say: "I have done so! But, do i smart than you, press the head of the bamboo pole, walk to the head of the bamboo pole, do i go to pick apples, do you go? "the bite or sting actually means that the person can eat the apple in an early stage. When the busy point agrees. Thus, they place the bamboo pole on the ground towards the apple tree, with one end in the cliff, and the other end extending out of the cliff to be close to the apple tree when sitting down to press. Walking on a bamboo pole with a stung and stinging carelessly, like walking on a steel wire in a performance, the stuttering with him is not a natural and chatting: "Hold down, hold down … …" when the nod is struck.

The desk lamp is played: the sting picks the apple at all, and when the sting is worried about eating the apple alone, the sting is not shared. When a sting: "give me apples quickly! "Ding is considered as not heard. When the user is in a hurry, the user stands up to walk to the place for robbery. At this time, when a very important thing is forgotten-two people woollen on the bamboo poles of the cliff edge! However, if people only hear the 'so' calls with different weights, the bamboo pole is out of balance, the people bite or bite, the bamboo pole and the apple cannot see at once, only the healthy and strong apple tree which still stands opposite to the cliff is seen, and countless red and large apples are still hung on the tree. ( At this time, the user inputs the intention of "next" (voice, gesture, finger reading instruction, etc.), and the desk lamp prompts "determine to play next? And if the user inputs a 'no' intention, the desk lamp continues to play. )

The desk lamp is played: really means that the hot bean curd cannot be eaten in sudden heart attack!

As shown in fig. 5, since the desk lamp has a natural learning attribute, the desk lamp with the learning enhancement function can be mounted to double the learning effect with little effort. According to the intelligent desk lamp, the scheme of the intelligent desk lamp is provided with the camera, and the recognition effect of image recognition in the scene is better optimized due to the light supplement function of the desk lamp.

Fig. 6 shows a three-party communication diagram of a user, a child-specific smart device, and a cloud according to an embodiment of the invention.

In order to enable the child-specific smart device 400 to support the user 501 for the identification of the content of the drawing, a communication connection needs to be established between the user 501, the child-specific smart device 400, and the cloud 601. The communication connection should be real-time and unobstructed to ensure that the interaction is not affected.

In order to complete the interaction, some conditions or preconditions need to be met. These conditions or preconditions include the hardware facilities of the child-specific smart device 400 that have the vision, perception and control functions.

After the preparation in the earlier stage is completed, the child-specific smart device 400 starts to perform interaction with the user 501, and first, the child-specific smart device 400 receives a drawing content recognition instruction input by the user 501.

The picture book content recognition instruction can be voice data, visual data, tactile data, and can also be that a user presses a physical key. The child-specific intelligent device 400 is configured with a corresponding device for receiving the command for identifying the content of the sketch, and is configured to receive the command for identifying the content of the sketch sent by the user 501. At this time, the child-specific smart device 400 and the user 501 are both parties of the communication, and the direction of data transfer is from the user 501 to the child-specific smart device 400.

Then, the special smart machine 400 for children performs image acquisition on the painted book, acquires cover image data containing the cover information of the painted book and inner page image data containing the inner page information of the painted book, and uploads the data to the cloud 601 after the duplication elimination processing. At this time, two parties for data transmission are the child-specific smart device 400 and the cloud 601, and the data transmission direction is from the child-specific smart device 400 to the cloud 601.

The method comprises the steps that multi-angle content identification can be carried out on cover image data at the cloud 601, corresponding correction angle information used for representing the placement angle of a picture book is recorded when the content identification is successful, the correction angle information is called in the process of inner page identification, inner page angle conversion is carried out on inner page image data to obtain converted inner page image data, then content identification is carried out on the converted inner page image to obtain content identification data.

Cloud 601 then returns content identification data to child-specific smart device 400. The cloud 601 returns corresponding content identification data according to the request of the child-specific smart device 400. At this time, the cloud 601 and the child-specific smart device 400 are two parties of the communication, and the data is transmitted from the cloud 601 to the child-specific smart device 601.

The child-specific smart device 400 then returns the content identification data to the user 501, and typically the child-specific smart device 400 may play the voice data through a speaker and the video data through a display device.

As shown in fig. 7, in step S701, the child-dedicated smart device 400 issues a request to the cloud 601. Thereafter, in step S702, the child-dedicated smart device 400 is in a state of waiting for the cloud 601 to reply. During the waiting period, the child-specific smart device 400 times the time it takes to return data.

In step S703, if the returned response data is not obtained for a long time, for example, the predetermined time length is more than 5S, the child-dedicated smart device 400 selects to perform local reply, and generates local general response data. Then, in step S704, the local common response is output, and the voice playing device is called to perform voice playing.

In conclusion, the content identification method and device for the picture book provided by the invention can identify the content of the front cover and the inner page of the picture book under the support of the intelligent equipment special for children, and assist the children in reading the picture book; and, when drawing this content recognition, can carry out the front cover discernment of multi-angle, overcome crooked this that draws and put the influence of angle to content recognition, greatly promoted the efficiency and the success rate of drawing this reading, improved user's the experience of drawing this reading.

It is to be understood that the disclosed embodiments of the invention are not limited to the particular structures, process steps, or materials disclosed herein but are extended to equivalents thereof as would be understood by those ordinarily skilled in the relevant arts. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Thus, the appearances of the phrase "one embodiment" or "an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment.

Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for identifying content of a picture book, the method comprising:

step four: performing content identification on the converted inner page image data;

the second step further comprises: sequentially adopting a plurality of correction angles to perform content recognition on the cover image data in a full-view recognition mode, and determining corresponding correction angle information when the content recognition is successful; and binding the currently obtained correction angle information with the current unique identification code of the intelligent equipment special for the children, and storing the information in an angle database.

2. The method of claim 1, wherein step one further comprises: and performing duplicate removal processing on the collected front cover image data and the collected inner page image data, and deleting repeated images to obtain the duplicate-removed front cover image data and inner page image data.

3. The method of claim 1, wherein step two further comprises: and uploading the cover image data to a cloud end, and performing image retrieval to identify the content through the cloud end.

4. The method of claim 1, wherein the third step further comprises the steps of:

and carrying out angle rotation on the inner page image contained in the inner page image data based on the matched correction angle information to obtain the converted inner page image data.

5. The method of claim 1, wherein said step four further comprises: and uploading the converted inner page image data to a cloud end, and performing content recognition through the cloud end to obtain multi-modal output data.

6. The method of any one of claims 1-5, further comprising:

7. A computer readable storage medium containing a series of instructions for performing the method steps of any of claims 1-6.

8. A content recognition apparatus for a picture book, characterized in that the method according to any of claims 1-6 is performed, the apparatus comprising:

a fourth module: which is used for identifying the content of the converted inner page image data.

9. A child-specific smart device characterized by a series of instructions for carrying out the method steps of any one of claims 1-6.