CN115700847A - Drawing book reading method and related equipment - Google Patents

Abstract

The application discloses a picture book reading method and related equipment, wherein in the user reading process, the electronic equipment can record abnormal behaviors (such as page jumping, long-time non-page turning and the like) of a user, segment the time of the whole reading process, output the concentration degree rating of the user in a corresponding time period based on the times of the abnormal behaviors of the user in each time period, and further calculate the value of the concentration degree of the user in the reading process at this time and the concentration degree rating. Therefore, under the condition that hardware cost and algorithm cost are not additionally increased, according to the page turning record of the user, the concentration degree of the user in the reading process can be output, and the user experience is improved. In addition, in the reading process of the user, if the user has abnormal behaviors (for example, behaviors of continuously skipping pages for 2 times within 1 minute and having skipped pages reaching 3 pages and the like) in a short time, the electronic device can remind the user to concentrate on the attention or remind the user to finish the reading, so that the interactive experience is improved.

Description

Picture book reading methods and related equipment

technical field

The present application relates to the field of terminal technology, and in particular to a picture book reading method and related equipment.

Background technique

In the stage of early childhood education, picture books are an important way for children to understand the world. However, traditional paper picture books often encounter many difficulties in the use process. For example, many parents do not have enough time to accompany their children to read picture books. Therefore, electronic devices such as point reading pens and point readers have appeared on the market. Accompany the child to read picture books, but these electronic devices cannot calculate the child's concentration in the process of reading picture books. When parents cannot accompany their children to read picture books, that is, when the child reads picture books independently, parents cannot know that the child is reading picture books this time. Concentration in the process and poor user experience.

Contents of the invention

The embodiment of the present application provides a picture book reading method and related equipment, which can calculate the user's concentration during the reading process.

In the first aspect, the embodiment of the present application provides a method for reading a picture book, which is applied to an electronic device including a camera. When the page number of the picture book page changes, a page turning record is generated; the electronic device judges whether the user has abnormal behavior based on the page turning record, and if so, the electronic device records the abnormal behavior; the electronic device determines the user's concentration rating based on the abnormal behavior.

By implementing the method provided in the first aspect, the embodiment of the present application can output the user's concentration in the reading process according to the user's page-turning record without additional hardware and algorithm costs, and improve user experience.

In a possible implementation, the electronic device determines the user's concentration rating based on the abnormal behavior, which specifically includes: the electronic device records the total time of the user's entire reading process of picture books, and divides the total time into M time periods, where M is a positive integer; the electronic device records the number of occurrences of abnormal behaviors in M time periods, and based on the number of occurrences of abnormal behaviors in M time periods, respectively determines the concentration rating of the user corresponding to M time periods; wherein, the number of abnormal behaviors The higher the number of occurrences, the lower the rating of concentration; the fewer the number of occurrences of abnormal behavior, the higher the rating of concentration. In this way, the concentration rating of the user at various time periods during the reading process can be determined.

In a possible implementation, the method further includes: if the electronic device determines that the concentration rating of the user corresponding to the mth time period among the M time periods is not the highest, and based on the mth time period among the M time periods, The number of occurrences of abnormal behaviors in the +1 time period determines that the concentration rating of the user corresponding to the m+1th time period is A, where the concentration rating A is neither the highest nor the lowest, then the electronic device determines that the first The concentration rating of the user corresponding to the m+1 time period is adjusted from concentration rating A to concentration rating B, where concentration rating B is one level lower than concentration rating A, and m is a positive integer and less than M. In this way, it can be determined that the user may have a long period of inattention.

In a possible implementation, after the electronic device determines the user's concentration rating based on the abnormal behavior, the method further includes: the electronic device determines the user's concentration rating based on the duration of M time periods and the user's concentration level corresponding to the M time periods. The concentration rating of the user throughout the process of reading the picture book; wherein, the higher the user's concentration rating corresponding to the M time periods, and/or, the longer the duration corresponding to the time period with the higher concentration rating, the user's entire reading picture book The higher the concentration rating of the process. In this way, the user's concentration rating throughout the reading process can be determined.

In a possible implementation, the abnormal behavior includes one or more of the following: "page skipping" behavior, "page not turned for a long time" behavior, "turning page before finishing reading" behavior, "no response to question" behavior . In this way, it can be determined which behaviors the abnormal behaviors specifically include.

In a possible implementation, the page turning record includes one or more of the following: the page number the user turned to, the start time, the proportion of the page read, the end time of the playing content, the question time, whether the question was answered, and the end time . In this way, it can be determined which contents are specifically included in the page turning records.

In a possible implementation, the electronic device judges whether the user has abnormal behavior based on the page-turning record, specifically including: if the electronic device detects that the page number before the user turns the page is not continuous with the page number that the user turned to in the page-turning record, Then the electronic device determines that the user has a "page jumping" behavior. In this way, it can be determined whether the user has abnormal behavior based on the page number turned by the user in the page turning record.

In a possible implementation manner, the electronic device judges whether the user has abnormal behavior based on the page-turning record, which specifically includes: if the electronic device detects that the end time in the page-turning record is delayed by more than the first time threshold from the end time of the playing content, then The electronic device determines that the user exhibits "long time no page turning" behavior. In this way, it can be determined whether the user has abnormal behavior based on the end time in the page turning record and the end time of the playing content.

In a possible implementation, the electronic device judges whether the user has abnormal behavior based on the page-turning record, which specifically includes: if the electronic device detects that the proportion of this page in the page-turning record is less than 1, the electronic device determines that the user has " Turning the page before finishing reading” behavior. In this way, it can be determined whether the user has abnormal behavior based on the reading ratio of the current page in the page turning record.

In a possible implementation, the electronic device judges whether the user has abnormal behavior based on the page-turning record, which specifically includes: the electronic device initiates a question to the user and records the time of the question; if the electronic device is within the second time threshold after initiating the question If no response from the user is received, the electronic device determines that the behavior of "no response to the question" occurs by the user. In this way, it can be determined whether the user has abnormal behavior based on the questioning time in the page-turning record.

In a possible implementation manner, after the electronic device determines that the user has an abnormal behavior, the method further includes: the electronic device outputs a prompt corresponding to the abnormal behavior through an output device. In this way, the user can be reminded to concentrate or stop reading when the user has abnormal behavior.

In a second aspect, an embodiment of the present application provides an electronic device, which includes one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and one or more The multiple memories are used to store computer program codes, and the computer program codes include computer instructions. When one or more processors execute the computer instructions, the electronic device executes the method in any possible implementation manner of the first aspect above.

In a third aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are run on the electronic device, the electronic device is made to perform any of the above-mentioned first aspects. Method in one possible implementation.

In a fourth aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is run on a computer, causes the computer to execute the method in any possible implementation manner of the first aspect above.

Description of drawings

Fig. 1 is a schematic flow chart of a picture book recognition method provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a picture book recognition scene provided by an embodiment of the present application;

Fig. 3 is a schematic flow chart of a picture book reading method provided in the embodiment of the present application in the reading stage;

Fig. 4 is a schematic flow chart of a picture book reading method provided in the embodiment of the present application in the statistical stage;

Fig. 5 is a schematic diagram of a change trend of a user's concentration in a reading process provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of another electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of this application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in the text is only a description of associated objects The association relationship indicates that there may be three kinds of relationships, for example, A and/or B, which may indicate: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiment of the present application , "plurality" means two or more than two.

It should be understood that the terms "first" and "second" in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses.

Reference in this application to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

In the stage of early childhood education, picture books are an important way for children to understand the world. However, traditional paper picture books often encounter many difficulties in the use process. For example, many parents do not have enough time to accompany their children to read picture books. Therefore, electronic devices such as point reading pens and point readers have appeared on the market. Accompany the child to read picture books, but these electronic devices cannot calculate the child's concentration in the process of reading picture books. When parents cannot accompany their children to read picture books, that is, when the child reads picture books independently, parents cannot know that the child is reading picture books this time. Concentration in the process and poor user experience.

At present, more electronic devices capable of reading picture books (also called intelligent terminal devices for reading picture books) appear on the market.

For example, some electronic devices can read the content of picture books. When the user places the picture book within a certain range in front of the electronic device, the electronic device can recognize the picture book through the camera. When the user flips the picture book to a certain page, the electronic device can recognize the content of the current page of the picture book. And read out the text content on the picture book (that is, play the audio corresponding to the text content on the picture book), but these electronic devices still cannot evaluate and calculate whether the user is concentrating during the reading process.

For another example, some electronic devices can generate corresponding push content to push to the user according to the number of times the user browses the content of the trial chapters and the full text of the book. However, these electronic devices have few behavioral characteristics and It is simple, and it is impossible to evaluate and calculate whether the user is concentrating during the reading process.

For another example, some electronic devices observe the user's behavior through the camera during the user's reading process, and judge the user's reading status (whether to focus on reading). interaction with users. However, these electronic devices not only need a camera for recognizing picture books, but also need an external camera for recognizing the user's face, relying on vision-related algorithms to recognize the user's status, the cost and threshold are high, and it is easy to involve user privacy issues.

The embodiment of the present application provides a picture book reading method. During the user's reading process, the electronic device can record the user's abnormal behavior (such as page skipping, page not turning for a long time, etc.), and record the time of the entire reading process. Segmentation, after that, output the user's concentration rating in the corresponding time period based on the number of abnormal behaviors of the user in each period, and further calculate the value and concentration rating of the user's overall concentration in the reading process. In this way, without additional hardware cost and algorithm cost, according to the user's page-turning record, the user's concentration during the reading process can be output to improve user experience. In addition, during the user's reading process, if the user has abnormal behaviors in a short period of time (such as skipping pages twice within 1 minute, and the number of pages skipped reaches 3 pages, etc.), the electronic device can remind The user can concentrate or remind the user to end the reading, so as to improve the interactive experience.

Firstly, a flow of a picture book recognition method provided by the embodiment of the present application is introduced.

Fig. 1 exemplarily shows a process flow of a picture book recognition method provided by an embodiment of the present application.

As shown in FIG. 1 , the method can be applied to an electronic device 100 capable of reading picture books. The specific steps of this method are described in detail below:

S101. The electronic device 100 collects images.

Specifically, the electronic device 100 may use its own camera to collect images in real time in the image collection area, and the real-time collected images may be used in subsequent steps for picture book recognition. Wherein, the image acquisition area may be an area that can be photographed by the camera.

As shown in FIG. 2 , when reading a picture book, the user can first place the picture book within a certain range in front of the electronic device 100 (that is, within the image collection area of the camera of the electronic device 100 ), so that the electronic device 100 can use the camera to complete Completely or nearly completely capture the picture book. Further, the electronic device 100 may use the camera to collect images including picture books in real time.

Optionally, the electronic device 100 may start to execute step S101 after receiving the user's first input, where the first input may include but not limited to the user's voice command input, click operation and so on.

S102. The electronic device 100 executes an image recognition algorithm to recognize picture books.

In the embodiment of the present application, the electronic device 100 may execute an image recognition algorithm in real time to recognize the above-mentioned images collected in real time.

Specifically, the electronic device 100 may input the images collected in real time into a pre-trained picture book recognition model to perform picture book recognition.

A large number of picture book images may be pre-stored in the template library, and the picture book image may be an image of a cover page of a picture book or an image of a content page of a picture book. There may be an association relationship between the picture book image and the picture book to which it belongs, and the association relationship may also be stored in the template library. Exemplarily, the specific implementation of establishing the association relationship may include: establishing an association relationship between the picture book image and the corresponding picture book name, Wherein, the name of the picture book may indicate which picture book the picture book is.

Wherein, the electronic device 100 may use an image recognition technology based on visual AI capabilities for picture book recognition, and the embodiment of the present application does not limit the image recognition algorithm used in the picture book recognition process.

Wherein, the template library and the picture book recognition model can be stored locally or in the cloud, which is not limited in this embodiment of the present application.

S103. The electronic device 100 judges whether the picture book can be recognized.

Specifically, if the electronic device 100 determines that the image collected in real time is successfully matched with one of the picture book images prestored in the template library, the electronic device 100 determines that the picture book can be recognized.

Among them, the picture book images in the template library are all trained picture book images, that is, these images have undergone processes such as feature extraction and model training. The electronic device 100 can recognize the picture book only when the image collected by the camera in real time includes the above-mentioned trained picture book image.

S104. The electronic device 100 determines whether the current page of the picture book is a cover page of the picture book.

Specifically, when the electronic device 100 performs picture book recognition, it may firstly match the images captured by the camera in real time with the images of all picture book cover pages stored in advance. If the image of the cover page matches successfully), the electronic device 100 can determine that the image captured by the camera in real time includes the image of the cover page of the picture book, that is, the electronic device 100 has recognized the cover page of the picture book. The image collection area is the cover page of the picture book.

Further, after determining that the current page of the picture book is the cover page of the picture book, the electronic device 100 may execute step S105.

S105. The electronic device 100 plays the audio for switching picture books, and switches the alternative content index page at the same time.

Specifically, if the electronic device 100 recognizes the cover page of the picture book, it may be considered that the user has switched the picture book. Further, the electronic device 100 may play an audio for switching the picture book, and the audio is used to prompt the user that the picture book has been switched. Optionally, the audio content may also include the name of the picture book switched to by the user, for example, the audio may be "You have switched the picture book, and the name of the picture book is XXX". At the same time, the electronic device 100 may switch the candidate content index page, that is, the electronic device 100 may use the content page of the picture book corresponding to the identified picture book as the current candidate content index page.

S106. The electronic device 100 determines whether the current page of the picture book matches the candidate content index page.

Specifically, after the electronic device 100 determines that the current page of the picture book is not the cover page of the picture book, the electronic device 100 can match the image of the current page of the picture book captured by the camera in real time with the image of the current candidate content index page one by one. If successful (that is, the image of the current page of the picture book captured by the camera in real time matches one of the images of the current candidate content index page), the electronic device 100 can determine that the current page of the picture book matches the candidate content index page, that is, The electronic device 100 has identified the current page of the picture book. Otherwise, this recognition fails.

Further, after the electronic device 100 identifies the current page of the picture book, the electronic device 100 may execute step S107.

Wherein, the current candidate content index page is the content page of the picture book corresponding to the picture book identified by the electronic device 100 in step S105.

In the embodiment of the present application, the feature matching method used in the picture book recognition process can refer to the feature matching method disclosed in the patent publication number CN111695419A, and the full text of CN111695419A is incorporated here. Not limited thereto, the picture book recognition process may also use other feature matching methods in the prior art, which is not limited in this embodiment of the present application.

It is easy to understand that when switching picture books, the user needs to first place the cover page of the picture book within a certain range in front of the electronic device 100, so that the electronic device 100 can recognize the cover page of the picture book and switch the alternative content index page, so that The electronic device 100 may be made to perform subsequent steps based on the candidate content index page.

In the embodiment of the present application, the electronic device 100 can periodically perform picture book recognition. For example, the electronic device 100 can acquire the latest recognition result output by the picture book recognition model every 200 ms, so that it can recognize the picture book pages in the entire reading process of the user. changes.

S107. The electronic device 100 determines whether the user turns to a new page.

Specifically, after the electronic device 100 recognizes the current page of the picture book, since the electronic device 100 can periodically identify the picture book, the electronic device 100 can determine whether the current page of the picture book recognized this time is different from the current page of the picture book recognized last time. Whether it is the same page, if so, it means that the page of the picture book has not changed, that is, the current page of the picture book recognized this time is the current page of the picture book recognized last time, then the electronic device 100 can determine that the user has not turned to a new page; if not , it means that the page of the picture book has changed, that is, the current page of the picture book recognized this time is not the current page of the picture book recognized last time, then the electronic device 100 can determine that the user has turned to a new page.

Wherein, the electronic device 100 may determine whether the current page of the picture book recognized this time is the same page as the current page of the picture book recognized last time may include but not limited to the following two methods:

Implementation method 1. Determine whether the content of the current page of the picture book recognized this time is the same as that of the current page of the picture book recognized last time. If so, then the current page of the picture book recognized this time is equal to the current page of the picture book recognized last time. same page.

Implementation method 2. Determine whether the page number of the current page of the picture book recognized this time is the same as the page number of the current page of the picture book recognized last time. If so, then the current page of the picture book recognized this time and the current page of the picture book recognized last time are same page.

S108. The electronic device 100 starts to record the reading information of the new page and starts to play the audio corresponding to the content of the new page.

Specifically, after the electronic device 100 determines that the user has turned to a new page, the electronic device 100 may start recording the reading information of the new page, and at the same time, the electronic device 100 may switch the played audio and start playing the audio corresponding to the content of the new page.

Among them, the new page reading information may include information such as the time when the user turns the page, the proportion of the current page read before the page is turned, and whether the question is answered or not. These information will be introduced in detail in the following embodiments, and will not be expanded here.

The specific execution process of the picture book reading method provided in the embodiment of the present application is introduced below.

The method can be applied to the electronic device 100 capable of reading picture books. The specific execution process of this method is divided into two stages: the reading stage and the statistical stage. The following is a detailed introduction:

1. Reading stage

The electronic device 100 can enter the reading stage after the picture book recognition is successful. Since the electronic device 100 can periodically use the image recognition algorithm to carry out picture book recognition (for example, once every 200ms), and recognize the content on the picture book page, therefore, in the reading stage, When the user turns a page (that is, the user turns from the current page to a new page), the electronic device 100 can recognize the new page that the user turns to, and can record the time point when the user turns the page this time, and the page turned to. Page number, and information such as whether the current page has been read before turning the page. Optionally, if the electronic device 100 interacts with the user on a question during the reading process, the electronic device 100 may also record information about whether the user responds to the question. Further, the electronic device 100 may extract information such as the user's abnormal behavior (such as page skipping, page not turning for a long time, etc.) and the start and end time corresponding to the abnormal behavior from the information recorded during the reading process. Further, when the electronic device 100 recognizes that the user has several abnormal behaviors within a short period of time, the electronic device 100 may confirm with the user whether to continue reading.

The following describes in detail the specific flow of the picture book reading method provided in the embodiment of the present application at the reading stage.

Fig. 3 exemplarily shows the specific flow of the picture book reading method provided in the embodiment of the present application in the reading stage. The specific steps of the method in the reading phase are described in detail below:

S301. The electronic device 100 records information such as the time point when the user turns the page, the proportion of the current page read before the page is turned, and whether the question is answered or not.

See Table 1 below. Table 1 exemplarily shows information such as the user's page turning time, the proportion of the current page read before turning the page, and whether the question is answered or not recorded by the electronic device 100 during a complete reading process of the user.

Wherein, the time points in Table 1 represent the minutes and seconds at the beginning of a reading process, for example, 1:00 represents the first minute and the 0th second at the beginning of a reading process. One row in Table 1 represents a page-turning record, and the "sequence number" in Table 1 represents the sequence number of each page-turning record. A page turning record can include but not limited to the following information: "page number", "start time", "end time of playing content", "reading ratio of this page" (that is, the reading ratio of the current page before turning the page), "question time", "response to the question", "end time".

When the user turns a page (that is, turns from the current page to a new page), the electronic device 100 can detect the user's page turning behavior (for example, the electronic device 100 can use the image recognition algorithm in the aforementioned picture book recognition process to recognize that the picture book page changes) , at this time, the page number of the new page (i.e. the "page number" in Table 1 or the page number turned by the user) can be recorded, and at the same time, the page turning time point (i.e. the "end time" of the current page in Table 1) can also be recorded ", the "start time" of the new page).

When the user turns to a new page, the electronic device 100 can start playing the audio corresponding to the content of this page, and when the electronic device 100 finishes playing the audio corresponding to the content of this page, it can record the time point when the audio corresponding to the content of this page ends ( That is, the "end time of playing content" in Table 1), and at the same time, the proportion of users who have finished reading this page (that is, the "proportion of reading this page" in Table 1) can also be recorded.

Wherein, the proportion of users who finish reading the page can be determined based on the duration of the played audio corresponding to the content of the page and the total duration of the audio corresponding to the content of the page.

For example, when the electronic device 100 finishes playing the audio corresponding to the content of this page, if the audio corresponding to the content of this page has been played completely (that is, the electronic device 100 does not detect that the user appears before the audio corresponding to the content of this page is completely played, page-turning behavior), then the electronic device can record that the reading ratio of this page is 1.

For another example, when the electronic device 100 finishes playing the audio corresponding to the content of this page, if the audio corresponding to the content of this page has not been played completely (that is, when the electronic device 100 only plays a part of the audio corresponding to the content of this page), it will detect until the user has a page-turning behavior), then the electronic device 100 can calculate the ratio of the duration of the audio corresponding to the content of this page that has been played to the total duration of the audio corresponding to the content of this page, and record this ratio as the user finished reading this page. page ratio.

When the user turns to a new page, when the content of the new page contains question interaction, the electronic device 100 can record the time point of the question (that is, the "question time" in Table 1). In addition, after the electronic device 100 initiates the question, it also It may be recorded whether the user's voice response to the question is received (that is, "whether the question is answered or not" in Table 1).

It should be noted that in Table 1, only one question is initiated on the same page as an example to record the question result. In some embodiments, the electronic device 100 can adjust the recording method for multiple questions initiated on the same page. For example, the electronic device 100 can record the question result once per line. Check whether the page numbers of adjacent rows are consistent to combine multiple query results on the same page.

Table 1

S302. The electronic device 100 extracts information such as start and end times corresponding to events such as user skipping pages, not turning pages for a long time, turning pages before finishing reading, and asking questions without responding.

Referring to Table 2 below, Table 2 exemplarily shows possible abnormal behaviors of the user during a complete reading process and the manner in which the electronic device 100 recognizes the abnormal behaviors.

Among them, the abnormal behaviors that may occur during a complete reading process of the user may include but are not limited to the following four types: page skipping, not turning pages for a long time, turning pages before finishing reading, and not responding to questions.

Abnormal behavior 1. Page jumping:

"Page skipping" may mean that the user turns multiple pages at a time during the reading process, skips some unread pages, and does not turn back within a short time.

A possible implementation manner for the electronic device 100 to recognize the behavior of "jumping pages":

(1) The electronic device 100 can compare the corresponding page numbers in two adjacent page-turning records, and determine that the page number corresponding to the second page-turning record (ie, the next page-turning record) is higher than that of the first page-turning record (ie, the previous page-turning record). The page number corresponding to page record) is greater than 1 page, that is, the difference between the page number corresponding to the second page-turning record and the page number corresponding to the first page-turning record is greater than 1, that is to say, the electronic device 100 detects that the user The page number before turning the page is not continuous with the page number the user turned to.

Exemplarily, as shown in Table 1, the page-turning record corresponding to sequence number 3 may be the second page-turning record, and the page-turning record corresponding to sequence number 2 may be the first page-turning record. It is easy to see that the second page-turning record The page number corresponding to the page record is page number 4, the page number corresponding to the first page turning record is page number 2, and the difference between page number 4 and page number 2 is 2, and 2 is greater than 1. Therefore, the electronic device 100 can determine that the user is at time point 1: At 50 o'clock, a "page jump" behavior occurred, that is, from page 2 to page 4, the user turned 2 pages at a time, and the number of page jumps was 1 page.

In the same way, it is easy to see from Table 1 that at the time point of 5:40, the user had the behavior of "jumping pages", that is, from page 6 to page 8, the user turned 2 pages at a time, and the page skipped The number of pages is 1 page.

(2) The electronic device 100 can determine that the page numbers corresponding to the unread pages are included between the page numbers corresponding to the two page turning records, and the electronic device 100 can determine that the user has not turned back to the first page within a short period of time (for example, within 10 seconds). The next page of the page number corresponding to a page turning record.

For example, assuming that the page number corresponding to the first page-turning record is 2, and the page number corresponding to the second page-turning record is 4, if the electronic device 100 detects that there is no page-turning page containing page number 3 between the two page-turning records record, the electronic device 100 may determine that the page number corresponding to the page that the user has not read is 3, so as to determine that the page numbers corresponding to the unread pages are contained between the page numbers corresponding to the above two page turning records. Further, if the electronic device 100 also detects that the user has not turned back to the page corresponding to the above page number 3 within a short period of time (for example, within 10 seconds), the electronic device 100 can determine that the user has not turned back to the page corresponding to the page number 3 within a short period of time. The next page of the page number corresponding to a page turning record. Therefore, the electronic device 100 can determine that the user has "jumped pages", that is, the user has jumped from the second page to the fourth page.

In some embodiments, the electronic device 100 may also combine the two possible implementations of (1) and (2) above to comprehensively determine whether the user has a "page skipping" behavior. For example, the electronic device 100 can use the above-mentioned method (1) to preliminarily determine whether the user has "jumped pages". Behavior, if so, the electronic device 100 can comprehensively determine that the user has a "page skipping" behavior. In this way, it is possible to avoid misidentification of the electronic device 100 .

Abnormal behavior 2. The page has not been turned for a long time:

"No page turning for a long time" may mean that the user does not perform page turning behavior within a period of time (for example, within 1 minute) after the electronic device 100 finishes playing all the audios corresponding to the contents of the current page.

A possible implementation manner for the electronic device 100 to identify the behavior of "no page turning for a long time":

The electronic device 100 can find two adjacent page-turning records, and determine that the start time corresponding to the second page-turning record (ie, the next page-turning record) (ie, the "end time" in the first page-turning record) is shorter than the first page-turning record. The end time delay of the playback content corresponding to a page-turning record (ie, the previous page-turning record) exceeds a preset time threshold (for example, 1 minute). It is easy to understand that the electronic device 100 can also compare the "end time" and "the end time of the playing content" in the same page-turning record, and determine that the "end time" is delayed by more than the preset time threshold ( eg 1 minute).

Wherein, the aforementioned preset time threshold may also be referred to as a first time threshold.

Exemplarily, as shown in Table 1, the page-turning record corresponding to sequence number 5 may be the second page-turning record, and the page-turning record corresponding to sequence number 4 may be the first page-turning record, assuming that the above preset time threshold is 1 Minutes, it is easy to see that the start time corresponding to the second page-turning record is 1 minute and 30 seconds later than the end time of the playback content corresponding to the first page-turning record, which exceeds the preset time threshold by 1 minute. Therefore, the electronic device 100 It can be determined that the behavior of "the user has not turned the page for a long time" occurred at the time point of 5:00.

Abnormal behavior 3. Turning the page before finishing reading:

"Turn the page before finishing reading" may mean that the user turns the page when the electronic device 100 only plays a part of the audio corresponding to the content of the current page.

A possible implementation manner for the electronic device 100 to recognize the behavior of "turning the page before finishing reading":

When the electronic device 100 detects the user's page-turning behavior, it can start to generate a page-turning record corresponding to a new page (that is, a new page-turning record), and at the same time, the electronic device 100 can find a page-turning record corresponding to a new page. The previous page-turning record, and it is determined that the page-reading ratio corresponding to the page-turning record is less than 1, that is, when a new page-turning record occurs, the content of the previous page has not been read.

For example, as shown in Table 1, the electronic device 100 can detect the user's page-turning behavior at the time point 5:40, and start to generate the page-turning record corresponding to the serial number 6. At the same time, the electronic device 100 can find the page-turning record corresponding to the serial number 5. Since the page-turning record corresponding to the page-turning record is 0.8, and 0.8 is less than 1, the electronic device 100 can determine that the user has the behavior of "turning the page before finishing reading" at the time point of 5:40 .

It is easy to understand that if the electronic device 100 detects that the "ratio of unread this page" in a certain page-turning record is less than 1, the electronic device 100 can determine that the user has the behavior of "turning the page before finishing reading".

Abnormal behavior 4. No response to questions:

"No response to the question" may mean that after the electronic device 100 initiates the question, the user does not make a voice response to the question.

A possible implementation manner for the electronic device 100 to identify the behavior of "no response to the question":

When the picture book content includes question-like interactions, that is, for pages with interactions, the electronic device 100 can initiate a question to the user, and record the question time, within a preset time threshold (for example, within 10 seconds) after the electronic device 100 initiates the question. , the electronic device 100 does not receive the user's voice response to this question.

Wherein, the aforementioned preset time threshold may also be referred to as a second time threshold.

Exemplarily, as shown in Table 1, it can be seen from the page-turning record of the serial number 3 that the question initiated by the electronic device 100 at the time point 2:40, assuming that the preset time threshold is 10 seconds, if the electronic device 100 If the voice response to the question from the user has not been received within 10 seconds after initiating the question, that is, the electronic device 100 has not received the voice response to the question from the user at the time point 2:50, then the electronic device 100 It can be determined that the user has "asked no response" behavior.

Table 2

See Table 3 below. Table 3 exemplarily shows the user's page skipping, long-time page turning, and page turning before finishing reading, which are extracted by the electronic device 100 based on the information recorded in Table 1. , Inquiry information such as the start and end time corresponding to the unresponsive event.

Among them, Table 3 includes the abnormal behaviors that may occur during a complete reading process of the user, and the information corresponding to the abnormal behaviors, that is, the start time, end time, number of skipped pages, percentage of finished reading, and duration of unturned pages.

table 3

It is easy to understand that the process of extracting the user's abnormal behavior in the above step S302 can be executed after generating a page-turning record in step S301, rather than after generating all the page-turning records in a complete reading process .

S303. The electronic device 100 determines whether to remind the user.

Specifically, during the user's reading process, the electronic device 100 may first determine whether the user has abnormal behavior in a short period of time, and further determine whether to remind the user.

If the electronic device 100 determines that the user has relatively abnormal behavior in a short period of time, the electronic device 100 may determine that it is necessary to remind the user, and further, the electronic device 100 may execute step S304 to remind the user.

If the electronic device 100 determines that the user has no abnormal behavior within a short period of time, the electronic device 100 may determine that there is no need to remind the user, and further, the electronic device 100 may not make a reminder.

Wherein, the electronic device 100 may determine whether the user has relatively abnormal behavior in a short period of time based on the information corresponding to the abnormal behavior extracted in step S302 (for example, the information in Table 3).

See Table 4 below. Table 4 exemplarily shows relatively abnormal behaviors that may occur in a short period of time during the reading process, corresponding prompting methods, and feedback processing methods.

Table 4

Among them, the relatively abnormal behaviors that may occur during the reading process of the user may include but are not limited to the following four types:

Abnormal behavior 1. Within a certain period of time (for example, within 1 minute), consecutive page skipping reaches a preset threshold (for example, 2 times), and the number of skipped pages reaches a preset threshold (for example, 3 Page).

Taking "page skipping reaches 2 consecutive times within 1 minute, and the number of pages skipped reaches 3 pages" in Table 4 as an example, the electronic device 100 may determine based on the information corresponding to the "page skipping" behavior extracted in step S302. It is judged whether the user skips pages continuously for 2 times within 1 minute, and the number of pages skipped reaches 3 pages.

Among them, the electronic device 100 can determine whether the difference between the start time (or end time) corresponding to two adjacent "page skipping" behaviors is greater than 1 minute, and if so, the electronic device 100 determines that the user continuously skips pages within 1 minute. page reached 2 times. Further, the electronic device 100 may determine whether the sum of the number of pages skipped corresponding to the above two adjacent "page skipping" behaviors is greater than or equal to 3 pages, and if so, the electronic device 100 determines that the number of pages skipped by the user reaches 3 pages .

Abnormal behavior 2. The page-turning time is too long, and the duration of a single non-turning page reaches a preset time threshold (for example, 2 minutes).

Taking "the page-turning time is too long, and the single non-turning time reaches 2 minutes" in Table 4 as an example, the electronic device 100 can base on the non-page-turning time corresponding to the "long time no page-turning" behavior extracted in step S302 To determine whether the user does not turn the page once for more than or equal to 2 minutes, if so, the electronic device 100 determines that the user does not turn the page for a single time for 2 minutes.

Abnormal Behavior 3. Continuous reading reaches a preset threshold of pages (for example, 3 pages) and is not finished, and the percentage of completion of reading is less than a preset ratio threshold (for example, 80%).

Taking "Consecutive reading reaches 3 pages without finishing reading, and the reading completion ratio is less than 80%" in Table 4 as an example, the electronic device 100 may base on the reading behavior corresponding to the behavior of "turning pages before finishing reading" extracted in step S302. The completion ratio is used to judge whether the user has not finished reading up to 3 pages, and the completion ratio is less than 80%.

Among them, the electronic device 100 can determine whether the number of times the user has continuously performed the behavior of "turning the page before finishing reading" is greater than or equal to 3 times, and if so, the electronic device 100 can determine that the user has not finished reading 3 pages in a row. Further, the electronic device 100 can determine whether the completion rate corresponding to the above-mentioned 3 consecutive times of "turning the page before finishing reading" is less than 80%. The reading percentage is less than 80%.

Relatively abnormal behavior 4. The picture book is not put into the recognizable range for a period of time (for example, within 1 minute).

Taking "the picture book is not within the recognizable range for 1 minute" in Table 4 as an example, the electronic device 100 can use the image recognition algorithm in the aforementioned picture book recognition process to identify the picture book, if the electronic device 100 cannot recognize the picture book, and If the picture book cannot be recognized for longer than or equal to 1 minute, the electronic device 100 may determine that the user has not put the picture book within the recognizable range for 1 minute.

S304. The electronic device 100 reminds the user.

Specifically, when the electronic device 100 determines that it is necessary to remind the user, the electronic device 100 may use a corresponding prompt method to remind the user to concentrate or remind the user to end the current reading and so on.

Wherein, the corresponding prompt method used to remind the user may be the prompt method shown in Table 4 as an example:

For the above-mentioned relatively abnormal behavior 1, the electronic device 100 may output the prompt "I see that you have been turning pages, do you not like this content?" as shown in Table 4 exemplarily.

Further, the electronic device 100 may receive the user's voice response to the prompt, and perform corresponding feedback processing on the voice response. For example, if the electronic device 100 detects that the user's voice response represents an intention of "dislike", the electronic device 100 may prompt the user to change a book. For another example, if the electronic device 100 detects that the user's voice response represents the intention of "like", the electronic device 100 can continue to perform the reading task (such as playing the audio corresponding to the content of this page), and ignore the subsequent reading in the current reading process. The relatively abnormal behavior 1 that may occur, that is, if the electronic device 100 detects that the abnormal behavior 1 occurs again, the electronic device 100 may not give a reminder.

For the above abnormal behavior 2, the electronic device 100 may output the prompt "Do you want to continue reading picture books? Do you want to take a break?" as shown in Table 4 exemplarily.

Further, the electronic device 100 may receive the user's voice response to the prompt, and perform corresponding feedback processing on the voice response. For example, if the electronic device 100 detects that the user's voice response represents the intention of "not continuing", the electronic device 100 may stop executing the picture book reading task (ie exit the picture book). For another example, if the electronic device 100 detects that the user's voice response represents the intention of "continue", the electronic device 100 can continue to perform the reading task (such as playing the audio corresponding to the content of this page), and ignore the subsequent reading tasks in this reading process. The relatively abnormal behavior 2 that may occur, that is, if the electronic device 100 detects that the abnormal behavior 2 occurs again, the electronic device 100 may not give a reminder.

For the above-mentioned relatively abnormal behavior 3, the electronic device 100 may output the prompt "I haven't finished reading the first few pages, please read it patiently" as shown in Table 4 exemplarily.

Further, the electronic device 100 can continue to perform the reading task (such as playing the audio corresponding to the content of this page), and ignore the relatively abnormal behavior 3 that may occur later in the current reading process, that is, if the electronic device 100 detects the abnormal behavior When 3 appears again, the electronic device 100 may not make a reminder.

For the above abnormal behavior 4, the electronic device 100 may output the prompt "I can't see the picture book now, please put the picture book in front of me" as exemplarily shown in Table 4.

Further, the electronic device 100 may continue to use the image recognition algorithm in the aforementioned picture book recognition process for picture book recognition. If the electronic device 100 fails to recognize the picture book for a period of time (for example, 1 minute) after prompting, the electronic device 100 may stop. Execute the picture book reading task (that is, exit the picture book). If the electronic device 100 recognizes the picture book after prompting, the electronic device 100 continues to execute the picture book reading task (for example, playing the audio corresponding to the content of this page).

Wherein, the feedback processing of the electronic device 100 requires the electronic device 100 to recognize the user's voice and understand the user's intention. The electronic device 100 may implement speech recognition and intent understanding by utilizing automatic speech recognition (Automatic Speech Recognition, ASR) capabilities and natural language processing (Natural Language Processing, NLP) capabilities of the cloud.

Wherein, the prompt words exemplarily shown in Table 4 may be output by the electronic device 100 through an output device (such as a loudspeaker, a display screen, etc.).

It should be noted that the relatively abnormal behaviors, corresponding prompts and feedback processing methods shown in Table 4 that may occur in a short period of time during the reading process of the user are only exemplary and should not be construed as limiting the present application.

By executing the method shown in FIG. 3 , during the reading phase, the electronic device 100 can find out in real time whether the user has abnormal behaviors, and if so, the electronic device 100 can remind the user to pay attention during the reading process, or remind the user to end the book. reads.

In some embodiments, the electronic device 100 does not need to remind the user after it is determined that the user has an abnormal behavior, but after it is determined that the user has an abnormal behavior, it can promptly remind the user to concentrate, or end reading, etc. .

2. Statistical stage

After the electronic device 100 completes the reading stage (that is, after finishing the current reading), the electronic device 100 can enter the statistical stage. In the statistical stage, the electronic device 100 can record the total duration of the user's entire picture book reading process this time, and segment the total duration ( For example, the total duration can be divided into M time periods, where M is a positive integer), and the number of abnormal behaviors of users in each time period is recorded, and then determined based on the number of abnormal behaviors of users in each time period And output the concentration rating of the user in the corresponding time period. Further, the electronic device 100 may calculate the value of the user's overall concentration in the entire process of reading the picture book this time based on the duration of each time period of the current reading process and the user's concentration rating in the corresponding time period, and determine the current The user's concentration rating during the entire process of reading picture books.

In the embodiment of the present application, the entry of the electronic device 100 into the counting phase may be triggered by an instruction (such as a voice instruction) issued by the user. For example, the electronic device 100 may enter the counting phase after receiving the user's voice instruction "I have finished this reading".

In some embodiments, the electronic device 100 may also automatically enter the counting phase after detecting that the content of the last page of the picture book is played.

The specific flow of the picture book reading method provided in the embodiment of the present application in the statistical stage is introduced in detail below.

Fig. 4 exemplarily shows the specific flow of the picture book reading method provided by the embodiment of the present application in the statistical stage. The specific steps of the method in the statistical stage are described in detail below:

S401. The electronic device 100 divides the total duration of one reading process by the user into segments.

Specifically, after the aforementioned reading stage ends, the electronic device 100 may segment the total duration of one reading process of the user according to a certain segmentation rule. For example, the electronic device 100 may divide the total duration into multiple time periods, and if a certain division time point is just between the time periods when an abnormal behavior occurs, the time period may be extended so that the time period includes the occurrence of the abnormal behavior. The time period of the abnormal behavior, and the subsequent time period can also be extended in sequence.

Exemplarily, the electronic device 100 may segment the total duration of one reading process by the user in units of 1 minute. Taking Table 1 as an example, refer to Table 1. It can be seen from Table 1 that the total time of a user's reading process is 10 minutes. If it is segmented in units of 1 minute, it can be divided into the following 10 time periods: 0 :00-1:00; 1:00-2:00; 2:00-3:00; 3:00-4:00; 4:00-5:00; 5:00-6:00; 6:00 -7:00; 7:00-8:00; 8:00-9:00; 9:00-10:00. However, the split time point 4:00 of the time period 3:00-4:00 is just between the time period 3:30-5:00 when the behavior of "not turning the page for a long time" occurs, so the electronic device 100 can extend the The time period is 3:00-4:00, that is, the time period 3:00-4:00 and the time period 4:00-5:00 can be combined into one time period 3:00-5:00. It is easy to understand that the electronic device 100 can finally divide the total duration of a user's reading process in Table 1 into the following nine time periods: 0:00-1:00; 1:00-2:00; 2:00-3:00 ;3:00-5:00;5:00-6:00;6:00-7:00;7:00-8:00;8:00-9:00;9:00-10:00.

It should be noted that the segmentation rules for the electronic device 100 to divide the total time of one reading process by the user into segments may also be other, which is not limited in this embodiment of the present application.

S402. The electronic device 100 records the number of abnormal behaviors of the user in each time period, and grades the concentration of the user in each time period based on the recorded number of abnormal behaviors of the user in each time period.

Specifically, after the electronic device 100 divides the total duration of one reading process of the user into segments, the electronic device 100 may record the number of abnormal behaviors of the user in each time segment. Wherein, there is a certain mapping relationship between the number of abnormal behaviors of the user in a single time period and the concentration rating, and the mapping relationship can be preset. After the electronic device 100 records the number of abnormal behaviors of the user in a single time period Then the corresponding concentration rating can be obtained based on the above mapping relationship.

Referring to Table 5 below, Table 5 exemplarily shows the mapping relationship between the number of abnormal behaviors of users in a single time period and the concentration rating. It can be seen from Table 5 that the electronic device 100 divides the number of abnormal behaviors of the user in a single time period into four situations: 0 times, 1 time, 2 times, and 3 times. The electronic device 100 divides the degree of concentration into four levels, and the four levels are in order from high to low: focused, relatively focused, not very focused, and not focused. When the number of abnormal behaviors of the user in a single time period is 0, the concentration rating is concentrated; when the number of abnormal behaviors of the user in a single time period is 1 time, the concentration rating is relatively focused; the user is abnormal in a single time period When the number of behaviors is 2 times, the degree of concentration is rated as not very focused; when the number of abnormal behaviors of the user in a single time period is 3 times, the degree of concentration is rated as inattentive. It is easy to see that the more occurrences of abnormal behaviors, the lower the concentration rating, and the fewer the occurrences of abnormal behaviors, the higher the concentration rating.

It should be noted that the mapping relationship between the number of abnormal behaviors of the user in a single time period and the concentration rating shown in Table 5 is only exemplary, and should not be construed as limiting the present application.

In some embodiments, the mapping relationship between the number of abnormal behaviors of the user in a single time period and the concentration rating may also be other. Concentration can also be divided into more grades or fewer grades, and the number of abnormal behaviors of users in a single time period corresponding to each concentration rating can also be other single values or intervals.

In some other embodiments, when determining the concentration rating of the user in each time period, the concentration rating can be further distinguished in more detail in combination with the degree of abnormality of the user's abnormal behavior. Wherein, the degree of abnormality of the user's abnormal behavior can be determined based on the length of time the user has not turned pages for a long time, the number of pages skipped, and the like.

The number of times a user experienced unusual behavior in a single time period Concentration Rating 0 concentrate on 1 relatively focused 2 less focused 3 inattentive

table 5

The electronic device 100 may obtain the concentration rating of the user in each time period based on the recorded number of abnormal behaviors of the user in each time period.

Further, the electronic device 100 can also compare the concentration ratings of users in two adjacent time periods (ie, the current time period and the previous time period of the current time period), if the current time period and the previous time period of the current time period, If the user's concentration ratings are all non-focused (for example, relatively focused, or not very focused, or not focused), the concentration rating of the current time period can be adjusted, for example, on the basis of the original concentration rating Up and down one level to get an adjusted concentration rating for the current time period. In this way, under the condition that it is judged that the user may have been inattentive for a long period of time, the rating of the user's concentration during this period may be lowered.

Referring to Table 6 below, Table 6 exemplarily shows the concentration rating of the previous time period of the current time period, the original concentration rating of the current time period, and the adjusted concentration rating of the current time period.

It can be seen from Table 6 that when the concentration rating of time period 1 is relatively focused/not very focused/not focused, and the original concentration rating of time period 2 is not very focused, due to the two time periods The concentration ratings of users in this period are non-focused, so the original concentration ratings of time period 2 need to be adjusted to obtain the adjusted concentration ratings of time period 2. The adjusted concentration rating of time period 2 can be obtained by dropping one level from the originally deserved concentration rating of time period 2, that is, not being very focused is reduced by one level to non-focused. Therefore, the adjusted time period 2 is rated as inattentive.

In some embodiments, when the concentration rating of time period 1 is more focused/not very focused/not focused, and the original concentration rating of time period 2 is non-focused, because non-focus is the focus of Table 5 Therefore, in this case, the original concentration rating of time period 2 does not need to be adjusted, that is, the original concentration rating of time period 2 is the same as the adjusted Concentration ratings are consistent.

In the other three cases in Table 6, since the user's concentration ratings in these two time periods did not appear to be non-focused, the original concentration ratings of time period 2 do not need to be adjusted, that is, the time period 2 would have deserved a concentration rating consistent with the adjusted concentration rating for time period 2.

Table 6

Based on the information in Table 5 and Table 6, it is easy to understand that if the electronic device 100 divides the total duration of the user's current picture book reading process into M time periods, where M is a positive integer, the first time period in the M time periods The concentration rating of the user corresponding to the m time periods is not the highest (for example, the concentration rating is relatively focused/not very focused/not focused), and the electronic device 100 is based on the m+1th time in the M time periods The number of occurrences of abnormal behavior in the segment determines that the concentration rating of the user corresponding to the m+1th time period is A, where the concentration rating A is neither the highest nor the lowest (for example, the concentration rating A is relatively focused/not very focus), then the electronic device 100 can further determine that the concentration rating of the user corresponding to the m+1th time period is adjusted from the concentration rating A to the concentration rating B, wherein the concentration rating B is higher than the concentration rating A One level lower (for example, if the concentration rating A is more focused, then the concentration rating B is less focused), m is a positive integer and less than M.

Based on the information in Table 1 and Table 3, the electronic device 100 can obtain the analysis results shown in Table 7 below by performing Step S401 and Step S402.

See Table 7, as can be seen from Table 7, in the time period 0:00-1:00, 6:00-7:00, 7:00-8:00, 8:00-9:00, 9:00 In -10:00, no abnormal behavior occurred in the user, that is, the number of abnormal behaviors of the user was 0. According to Table 5, the concentration ratings of these 5 time periods are all focused.

In the time period 1:00-2:00, the user has "jumping page" behavior, that is, the number of times the user has abnormal behavior is 1. According to Table 5, the concentration rating that should have been given during this time period is relatively focused, and Since the concentration rating of the previous time period 0:00-1:00 in this time period is focused, the concentration rating of this time period is still the original concentration rating of this time period, that is, relatively focused.

In the time period 2:00-3:00, the user has the behavior of "no response to the question", that is, the number of times the user has abnormal behavior is 1. According to Table 5, the concentration rating that should have been in this time period is relatively focused. However, since the concentration rating of the previous time period 1:00-2:00 in this time period is relatively concentrated, the concentration rating of this time period is lowered by one level based on the original concentration rating of this time period , that is, less focused.

In the time period 3:00-5:00, the user has the behavior of "not turning the page for a long time", that is, the number of times the user has abnormal behavior is 1. According to Table 5, it can be seen that the concentration rating that this time period should have been compared Concentration, but since the concentration rating of the previous time period 2:00-3:00 in this time period is not very focused, the concentration rating of this time period is based on the original concentration rating of this time period Down one level, that is, less focused.

In the time period 5:00-6:00, the user has the behavior of "turning the page before finishing reading" and "jumping the page", that is, the number of abnormal behaviors of the user is 2. According to Table 5, this time period should have The concentration rating obtained is not too focused, but because the concentration rating of the previous time period 3:00-5:00 in this time period is not very focused, the concentration rating of this time period is originally in this time period The deserving concentration rating is lowered by one level, that is, inattention.

Table 7

In some embodiments, the concentration ratings of each time period in Table 7 can also be the concentration ratings that each time period should have, that is, the electronic device 100 may not have the original concentration of each time period. If any adjustments are made to the rating, the original concentration rating for each time period will be directly used as the final concentration rating for each time period.

Further, based on the information in Table 7, the schematic diagram of the change trend of the user's concentration in a reading process as shown in FIG. 5 can be obtained.

It can be seen from Figure 5 that the user's concentration during a reading process shows a trend of first decreasing and then increasing, that is, within the time period of 0-6 minutes, the user's concentration rating is in the order of concentration (0-1 minute) , relatively focused (1-2 minutes), not very focused (2-5 minutes), not focused (5-6 minutes), that is, the concentration shows a decreasing trend; in the period of 6-10 minutes, the user's concentration The degrees of concentration are all concentrated (6-10 minutes), that is, the degree of concentration shows a trend of improvement compared with that before 6 minutes.

Since the abnormal behavior of the user during the reading process is marked in Fig. 5, it can also be seen from Fig. 5 in which time period the user has the abnormal behavior. As shown in Figure 5, the user has the behavior of "jumping pages" in the time period 1:00-2:00 and the time period 5:00-6:00, and the user has the behavior of "asking questions" in the time period 2:00-3:00 "Not responding" behavior, the user has the behavior of "not turning the page for a long time" in the time period 3:00-5:00, and the user has the behavior of "turning the page before finishing reading" in the time period 5:00-6:00.

S403. The electronic device 100 calculates the user's concentration value for this reading based on the duration of each time period and the concentration rating.

Specifically, the electronic device 100 may use multiple methods to calculate the user's concentration value for this reading based on the duration of each time period and the concentration rating. The following uses linear weighting as an example to illustrate the specific calculation process for the electronic device 100 to calculate the user's concentration value for this reading based on the duration of each time period and the concentration rating:

In a possible implementation, the value V of the concentration of the user's current reading can be calculated using the following formula 1:

Among them, T ₁ , T ₂ , T ₃ , T ₄ are the duration of the aforementioned four types of focus in turn: focus, relatively focus, less focus, and non-focus duration; C ₃ , C ₄ are less focus, not focus The number of times (or called less focused, unfocused time periods); T is the total duration of the user's current reading (that is, the time interval between the start time and the end time of a reading process); T _p is the current The total duration of the pause in the reading process (that is, the sum of the time interval between each time the user pauses to read the picture book in the middle of a reading process and the user returns to continue reading the picture book); C _p is the number of pauses in the reading process; P is the total number of pages turned (skipped pages are not counted); P _f is the total number of pages actually read by the user (that is, the total number of pages turned minus the number of pages turned before they are finished).

In this embodiment of the application, the electronic device 100 may calculate the duration of each pause in the middle of a reading process by the electronic device 100. When the electronic device 100 receives an instruction (such as a voice instruction) to suspend reading from the user, it can be recorded. After the time t1, the electronic device 100 may record the time t2 when receiving an instruction (such as a voice command) from the user to continue reading again, and calculate the difference between the time t2 and the time t1, and the difference is The length of time the user pauses once.

比较专注的持续时长占比

可以依次给予正向加权a₁＝1,a₂＝0。对于不太专注的持续时长占比

不专注的持续时长占比

暂停的时长占比

可以依次给予负向加权a₃＝-2,a₄＝-5,a_p＝-1。对于用户读完的页数的比例

可以给予正向加权a_f＝2，且可以设置thr_p＝0.9，如果

超过0.9，则

可以为正向的值(即正值)。Wherein, the values of the parameters a ₁ , a ₂ , a ₃ , a ₄ , a _p , a _f , and thr _p in the above formula 1 can all be preset, and users can also set them independently according to their own standards. For example, for the percentage of duration of concentration

The proportion of duration that is more focused

The forward weights a ₁ =1, a ₂ =0 can be given sequentially. Percentage of less focused duration

Percentage of inattentive duration

The percentage of pause time

Negative weights a ₃ =-2, a ₄ =-5, a _p =-1 can be given sequentially. Percentage of pages read by the user

The forward weight a _f =2 can be given, and thr _p =0.9 can be set, if

more than 0.9, then

Can be positive values (i.e. positive values).

Wherein, the penalty value Penalty is determined based on the number of periods of inattention and inattention. Exemplarily, the penalty value Penalty can be calculated by the following formula 2:

相乘。It can be seen from the above formula 2 that the more the number of less focused time periods C ₃ and the number of inattentive time periods C ₄ are, the closer the value of Penalty is to 0. In formula 1, the penalty value Penalty can be compared with

multiplied.

Referring to Table 8, Table 8 exemplarily shows the mapping relationship between the concentration value V and the concentration rating.

It can be seen from Table 8 that if the value V of concentration is greater than 0.5, it means that the user is focused on reading this time; if the value V of concentration is between -0.1 and 0.5, it means that the user is more focused on reading this time If the value V of concentration is between -3 and -0.1, it means that the user is not very focused on this reading; if the value V of concentration is less than -3, it means that the user is not focused on this reading.

The value of concentration V Concentration Rating V>0.5 concentrate on -0.1≤V≤0.5 relatively focused -3≤V<-0.1 less focused V<-3 inattentive

Table 8

It should be noted that the mapping relationship between the concentration value V and the concentration rating shown in Table 8 is only exemplary, and should not be construed as limiting the present application. In some embodiments, the three critical values 0.5, -0.1, -3 of the concentration value V in Table 8 can also be set as other values to distinguish different concentration ratings.

Based on the information in Table 1 to Table 8, the electronic device 100 can use the above formula 1 and formula 2 to calculate the concentration value of the user's current reading.

Specifically, it can be drawn from Table 1, Table 3, and Table 7 that _T1 is 5 minutes (that is, the sum of the duration of each time period when the concentration rating in Table 7 is "focused"), and T2 is ₁ Minutes (that is, the sum of the corresponding time periods when the concentration rating in Table 7 is "comparatively focused"), _T3 is 3 minutes (that is, the corresponding time periods when the concentration rating in Table 7 is "not very focused" The sum of the duration of the segment), _T4 is 1 minute (that is, the sum of the duration of each time segment when the concentration rating in Table 7 is "focused"), and T is 10 minutes (that is, the first page turning in Table 2 Record the corresponding start time to the time interval between the end time corresponding to the last page-turning record), P is 10 pages (that is, the total number of rows corresponding to the "page number" column in Table 2), and P _f is 9 pages (that is, Table 2 The total number of rows corresponding to the "page number" column minus the total number of rows corresponding to the "reading ratio of this page" column is less than 1), C ₃ is 2 times (the concentration rating in Table 7 is "not very focused ” times), C ₄ was 1 time (the number of times the degree of concentration was rated as “not focused” in Table 7). In this calculation process, it is assumed that T _p is 0 minutes and C _p is 0 times.

Further, based on the determined specific values of each parameter in the above formula 1 and the above formula 2, the electronic device 100 can use the above formula 1 and formula 2 to calculate and obtain the value of the concentration of the user for this reading is about -0.7.

Further, based on the mapping relationship between the concentration value V and the concentration rating in Table 8, the electronic device 100 can determine that the user is not very focused on reading this time.

In the case of a ₁ =1, a ₂ =0, a ₃ =-2, a ₄ =-5, a _p =-1, a _f =2, thr _p =0.9, the electronic device 100 uses the above formula 1 and The maximum value of the user's concentration for one reading calculated by the above formula 2 is about 1.2, and the minimum value is about -5.

)。Among them, the maximum value of the above-mentioned concentration is calculated when the user has no abnormal behavior during the entire reading process, that is, the user's concentration during the entire reading process is "concentration" (that is, T ₁ =T, T ₂ =T ₃ =T ₄ =0), no pause in the middle (ie T _p =0, C _p =0), no unfinished pages (ie

).

中途暂停次数C_p≥3、用户读完的页数的比例

Among them, the minimum value of the above-mentioned concentration is calculated under the following circumstances: the user's concentration is "not focused" throughout the reading process, that is, T ₄ =T, T ₁ =T ₂ =T ₃ =0, halfway The ratio of the total pause time to the total reading time of this session

The number of pauses C _p ≥ 3, the ratio of the number of pages the user has finished reading

It is easy to see that the higher the concentration rating of the user corresponding to each time period, and/or, the longer the duration corresponding to the time period with the higher concentration rating, the higher the user's concentration rating in the entire process of reading picture books. high.

It should be noted that the electronic device 100 may also use other weighting methods to calculate the value of the user's current reading concentration, which is not limited in this embodiment of the present application.

By executing the method shown in FIG. 4, the electronic device 100 can output the period-by-period results of the statistical phase (such as the analysis results shown in Table 7) and the final calculation results (that is, the value of the degree of concentration and the degree of concentration corresponding to the value of the degree of concentration). rating). The segmented result and the final calculation result can be used to evaluate the reading situation of the child user during the reading process, for parents' reference. In addition, the schematic diagram of the change trend of concentration marked with abnormal behaviors output by the electronic device 100 (for example, the schematic diagram of the change trend of the user's concentration in a reading process as shown in FIG. 5 ) can be used as the details of the child user's reading situation for parents to view.

In some embodiments, the electronic device 100 may determine whether the user is interested in this type of picture book or content based on the calculated concentration value and the concentration rating corresponding to the concentration value. The higher the concentration of , it can indicate that the user is more interested in this type of picture book or content. If the electronic device 100 determines that the user is interested in this type of picture book or content, the electronic device 100 may output some picture books of the same type for recommendation to the user.

In some other embodiments, each parameter in the above formula 1 and formula 2 can be set independently. In this way, some control can be given to the parent of the child user. For example, the specific meaning and importance of each parameter in the above-mentioned formula 1 and the above-mentioned formula 2 can be presented to the parents, which is convenient for the parents to understand. If the parents think that a certain parameter in the above-mentioned formula 1 and/or formula 2 is not important, the parents This parameter can be deleted, or the value of this parameter can be adjusted independently.

The electronic device 100 in the embodiment of the present application is an electronic device capable of reading picture books, and may also be called a smart terminal device for picture book reading. The electronic device 100 may be a picture book reading robot, a picture book reading mobile phone, a picture book reading tablet, and the like. The embodiment of the present application does not limit the type, physical form, and size of the electronic device 100 .

In some embodiments, when the electronic device 100 is a picture book reading robot, the electronic device 100 can establish a communication connection (such as a Bluetooth communication connection, etc.) The analysis results output by the aforementioned statistical stage (such as the aforementioned time-period results, the final calculation results, the schematic diagram of the change trend of the degree of concentration marked with abnormal behavior, etc.) are sent to the electronic device 200, and the electronic device 200 can save the results. The display on the display screen of the electronic device 200 is used for viewing by the parents of the child user, so as to know the reading situation of the child user during the reading process. Optionally, each parameter in the above formula 1 and formula 2 may also be set independently by parents on the electronic device 200 .

The structure of an electronic device 100 provided in the embodiment of the present application is introduced below.

FIG. 6 exemplarily shows the structure of an electronic device 100 provided in the embodiment of the present application.

As shown in FIG. 6 , the electronic device 100 may include: a processor 601 , a memory 602 , a speaker 603 , a microphone 604 , a bus 605 , a camera 606 , and a power supply 607 . These components may be connected by a bus 605 . in:

The processor 601 can be used to read and execute computer-readable instructions, including one or more processing cores. The processor 601 executes various functional applications and information processing by running software programs and modules. In a specific implementation, the processor 601 may mainly include a controller, an arithmetic unit, and a register. Among them, the controller is mainly responsible for instruction decoding, and sends out control signals for the operations corresponding to the instructions. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, logic operations, etc., and can also perform address operations and conversions. The register is mainly responsible for saving the register operands and intermediate operation results temporarily stored during the execution of the instruction. In a specific implementation, the hardware architecture of the processor 601 may be an application specific integrated circuit (Application Specific Integrated Circuits, ASIC) architecture, MIPS architecture, ARM architecture, or NP architecture, and so on.

The memory 602 is connected to the processor 601 through a bus 605 . Memory 602 may be used to store various software programs and/or sets of program instructions. The processor 601 is configured to execute at least one program instruction, so as to implement the technical solutions of the foregoing embodiments. Its implementation principle and technical effect are similar to those of the related embodiments of the method above, and will not be repeated here.

The speaker 603 (also referred to as an audio playback device or an audio playback device) is connected to the processor 601 through a bus 605, and can be used to convert audio electrical signals into sound signals. In the embodiment of the present application, the electronic device 100 may play the audio corresponding to the content of the picture book through the speaker 603 .

The microphone 604 is connected to the processor 601 through a bus 605 and can be used to convert sound signals into electrical signals. In the embodiment of the present application, the electronic device 100 may receive a voice signal from the user through the microphone 604, and may implement voice interaction with the user. Optionally, when the electronic device 100 does not need to implement the function of voice interaction with the user, the structure of the electronic device 100 may not include the microphone 604 .

The camera 606 is connected to the processor 601 through the bus 605 and can be used to capture still images or videos. In the embodiment of the present application, the camera 606 can be used to capture the image of the cover page of the picture book and/or the image of the content page of the picture book. Further, the electronic device 100 can use the image captured by the camera 606 (for example, a 3456×3456 camera) to Identify picture books.

The power supply 607 can be used to supply power to internal components such as the processor 601 , the memory 602 , the speaker 603 , the microphone 604 , and the camera 606 .

Optionally, the structure of the electronic device 100 may also include a steering gear, which can be used to adjust the attitude of the electronic device 100, so that the camera 606 of the electronic device 100 can capture picture books. For example, in the case where the electronic device 100 is a picture book reading robot, when the picture book reading robot performs the picture book reading task, the steering gear can control the picture book reading robot to lower its head (the camera can be installed on the head, such as the forehead), so that the picture book robot can take pictures. Picture books at a certain distance (for example, about 30 centimeters) in front of you.

The structure of the electronic device 100 may also include a display screen (not shown in the figure), which may be used to output graphic image information. In the embodiment of the present application, the display screen can display the analysis result of the user's concentration in a reading process output by the electronic device 100, for example, the analysis results shown in Table 7, the value of the calculated concentration and the value of the concentration. The concentration rating corresponding to the value, the schematic diagram of the concentration change trend marked with abnormal behavior, etc.

The structure of the electronic device 100 may also include a communication module (not shown in the figure), which may be used for the electronic device 100 to establish a communication connection with other electronic devices, and communicate with other electronic devices through the communication connection.

The structure of the electronic device 100 may also include a timer (not shown in the figure), which may be used for the electronic device 100 to record the user's page-turning time, question time, end time of playing content, and so on.

In this embodiment of the application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or Execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

In the embodiment of the present application, the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SS), or a volatile memory (volatile memory), such as random-access memory (RAM). A memory is, without limitation, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

The memory in the embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, and is used for storing program instructions and/or data.

It should be understood that the electronic device 100 shown in FIG. 6 is merely an example, and that the electronic device 100 may have more or fewer components than those shown in FIG. 6, two or more components may be combined, or Different component configurations are possible. The various components shown in Figure 6 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

The structure of another electronic device 100 provided in the embodiment of the present application is introduced below.

FIG. 7 exemplarily shows the structure of another electronic device 100 provided in the embodiment of the present application.

As shown in FIG. 7, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. . Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

Wherein, the controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or A universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (displayserial interface, DSI), and the like. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other terminal devices, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device 100 through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (codedivision multiple access, CDMA), wideband code Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou satellite navigation system (beidounavigation satellite system, BDS), a quasi-zenith satellite system (quasi- zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light emitting diode). AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device 100, and can be applied to applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also known as "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

It should be understood that the electronic device 100 shown in FIG. 7 is only an example, and that the electronic device 100 may have more or fewer components than those shown in FIG. 7, two or more components may be combined, or Different component configurations are possible. The various components shown in Figure 7 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

Wherein, the structure of the electronic device 200 may be partly or completely the same as the structure of the electronic device 100 shown in FIG. 7 , and will not be repeated here.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), and the like.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.

The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still apply to the foregoing embodiments Modifications are made to the recorded technical solutions, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of each embodiment of the application.

Claims (13)

1. A picture book reading method, characterized in that it is applied to an electronic device comprising a camera, and the method comprises: The electronic device collects a picture book image by using the camera, and recognizes a picture book page based on the picture book image; When the electronic device detects that the page number of the picture book page changes, it generates a page turning record; The electronic device judges whether the user has abnormal behavior based on the page turning record, and if so, the electronic device records the abnormal behavior; The electronic device determines an attentiveness rating of the user based on the abnormal behavior. 2. The method according to claim 1, wherein the electronic device determines the concentration rating of the user based on the abnormal behavior, specifically comprising: The electronic device records the total duration of the user's entire process of reading picture books, and divides the total duration into M time periods, where M is a positive integer; The electronic device records the occurrence times of the abnormal behavior in the M time periods, and respectively determines the user corresponding to the M time periods based on the occurrence times of the abnormal behavior in the M time periods concentration rating for Wherein, the more occurrences of the abnormal behaviors, the lower the concentration rating; the fewer the occurrences of the abnormal behaviors, the higher the concentration rating. 3. The method according to claim 2, wherein the method further comprises: If the electronic device determines that the concentration rating of the user corresponding to the mth time period in the M time periods is not the highest, and based on the m+1th time period in the M time periods The number of occurrences of the abnormal behavior determines that the concentration rating of the user corresponding to the m+1th time period is A, wherein the concentration rating A is neither the highest nor the lowest, then the electronic The device determines to adjust the concentration rating of the user corresponding to the m+1th time period from the concentration rating A to the concentration rating B, wherein the concentration rating B is higher than the concentration rating Rating A is one level lower, and m is a positive integer less than M. 4. The method according to claim 2 or 3, characterized in that, after the electronic device determines the concentration rating of the user based on the abnormal behavior, the method further comprises: The electronic device determines the concentration rating of the user's entire process of reading picture books based on the duration of the M time periods and the concentration rating of the user corresponding to the M time periods; Wherein, the higher the concentration rating of the user corresponding to the M time periods, and/or, the longer the duration corresponding to the higher time period of the concentration rating, the user's entire reading picture book process The higher the concentration rating. 5. The method according to any one of claims 1-4, wherein the abnormal behavior includes one or more of the following: "page skipping" behavior, "page not turned for a long time" behavior, "unread "Turn the page when you're done" behavior, "Question not responded to" behavior. 6. The method according to any one of claims 1-5, wherein the page turning record includes one or more of the following: the page number the user turned to, the start time, the percentage of the page being read, the playback Content end time, question time, question response or not, end time. 7. The method according to claim 5 or 6, wherein the electronic device determines whether the user has abnormal behavior based on the page-turning record, specifically comprising: If the electronic device detects that the page number before the user turns the page is not continuous with the page number turned by the user in the page turning record, the electronic device determines that the user has the "page skipping" behavior. 8. The method according to any one of claims 5-7, wherein the electronic device determines whether the user has abnormal behavior based on the page-turning record, specifically including: If the electronic device detects that the end time in the page turning record is delayed by more than the first time threshold than the end time of the playing content, the electronic device determines that the user has the "long time no page turning" "Behavior. 9. The method according to any one of claims 5-8, wherein the electronic device determines whether the user has abnormal behavior based on the page-turning record, specifically including: If the electronic device detects that the reading ratio of the current page in the page turning record is less than 1, the electronic device determines that the user has the behavior of "turning the page before finishing reading". 10. The method according to any one of claims 5-9, wherein the electronic device determines whether the user has abnormal behavior based on the page-turning record, specifically including: The electronic device initiates a question to the user, and records the time of the question, and if the electronic device does not receive a response from the user within a second time threshold after initiating the question, the electronic device determines that the user appears The "ask no response" behavior. 11. The method according to any one of claims 1-9, characterized in that, after the electronic device determines that the user has the abnormal behavior, the method further comprises: The electronic device outputs a prompt corresponding to the abnormal behavior through an output device. 12. An electronic device, characterized in that the electronic device comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, the The one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device is configured to perform the functions described in claims 1-11. any one of the methods described. 13. A computer storage medium, characterized in that the computer storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are run on the electronic equipment, the electronic equipment executes the The method described in any one of claims 1-11.

Images