CN111027358B

CN111027358B - Dictation and newspaper reading method based on writing progress and electronic equipment

Info

Publication number: CN111027358B
Application number: CN201910322545.7A
Authority: CN
Inventors: 魏誉荧
Original assignee: Guangdong Genius Technology Co Ltd
Current assignee: Guangdong Genius Technology Co Ltd
Priority date: 2019-04-19
Filing date: 2019-04-19
Publication date: 2023-06-30
Anticipated expiration: 2039-04-19
Also published as: CN111027358A

Abstract

A dictation and newspaper reading method based on writing progress and an electronic device are provided, wherein the method comprises the following steps: after the current dictation content is reported and read, the writing content and writing state information of a user aiming at the current dictation content are monitored; wherein, the writing state information at least comprises writing time length in writing content; judging whether the actual writing order of the writing content is matched with the standard writing order of the current dictation content; if the writing time length of the user aiming at the writing content is not longer than the preset standard writing time length, judging whether the writing time length of the user aiming at the writing content is longer than the preset standard writing time length according to the writing state information; if the content is larger than the preset value, the electronic equipment is controlled to switch and read the next dictation content. By implementing the embodiment of the invention, the efficiency and the intelligent degree of dictation, newspaper and reading can be improved.

Description

Dictation and newspaper reading method based on writing progress and electronic equipment

Technical Field

The invention relates to the technical field of education, in particular to a dictation and reading method based on writing progress and electronic equipment.

Background

Dictation is a common form of work in primary school languages. For most mathematics students, the pronunciation work of the new words in the dictation process is mainly completed by parents, namely, the parents report and read the new words to students in a manual pronunciation mode so as to assist the students to complete corresponding dictation tasks. The manual reading mode needs to be matched with the idle time of parents, however, the parents often do not have much time to assist students to dictate because of busy work, and the dictation task of the students cannot be completed in time.

In order to solve the problem that students cannot complete dictation tasks on time, dictation, newspaper and read applications are generated. Students can select the content which needs to be dictated on the electronic equipment provided with the dictation and newspaper reading application, and then the dictation content is newspaper-read through a dictation and newspaper reading application preset newspaper reading mode, or the dictation and newspaper reading are performed through manual control of the electronic equipment.

When the dictation content is submitted and read through a dictation and newspaper reading application preset newspaper reading mode, the dictation content is switched by means of the interval duration set by default, so that after the current dictation content is submitted and written, students usually need to wait for a certain duration until the electronic equipment reports and reads the next dictation content, and therefore the dictation and newspaper reading efficiency is lower. And by adopting a manual control mode, students are required to click corresponding operation buttons on the electronic equipment to realize the switching of dictation contents, and the intelligent degree of dictation and newspaper reading is low. Therefore, the current dictation and newspaper reading method has low dictation and newspaper reading efficiency and lacks intelligence.

Disclosure of Invention

The embodiment of the invention discloses a dictation, newspaper and read method based on writing progress and electronic equipment, which can improve the efficiency and the intelligent degree of dictation, newspaper and read.

The first aspect of the embodiment of the invention discloses a dictation and newspaper reading method based on writing progress, which comprises the following steps:

after the electronic equipment reads the current dictation content, monitoring the writing content and writing state information of a user aiming at the current dictation content; wherein the writing state information at least comprises writing duration in the writing content;

judging whether the actual writing order of the writing content is matched with the standard writing order of the current dictation content;

if the writing time length of the user aiming at the writing content is not longer than the preset standard writing time length, judging whether the writing time length of the user aiming at the writing content is longer than the preset standard writing time length according to the writing state information;

if the content is larger than the preset value, the electronic equipment is controlled to switch and read the next dictation content.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, if the writing time period is longer than the standard writing time period, the method further includes:

judging whether the current writing speed of the writing content is smaller than a preset standard writing speed or not;

and if the current writing speed is smaller than the standard writing speed, executing the control electronic equipment to switch and report the next dictation content.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, if the actual writing order matches the standard writing order, the method further includes:

Judging whether the actual writing stroke order is the ending stroke order of the writing content;

and if the actual writing stroke order is the ending stroke order of the writing content, executing the control electronic equipment to switch and report the next dictation content.

In a first aspect of the embodiment of the present invention, after the electronic device reads the current dictation, the method monitors writing content and writing status information of the user for the current dictation, including:

after the electronic equipment reads the current dictation content, the learning desktop where the electronic equipment is positioned is identified through an infrared sensor of the electronic equipment so as to obtain human infrared characteristic information;

determining a writing area of a user according to the human infrared characteristic information;

adjusting the shooting direction of a first camera of the electronic equipment so that the shooting direction of the first camera faces the writing area, and shooting a plurality of frames of writing images of the writing content written by a user in the writing area;

and analyzing the multi-frame writing image to obtain writing content and writing state information of the user aiming at the current dictation content.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, a second camera of the electronic device is controlled to capture multiple frames of facial images of a user;

Intercepting eye images from the multi-frame facial images, and analyzing the eye images to obtain first facial feature information;

a lip image is intercepted from the multi-frame facial image, and second facial feature information is obtained by analyzing the lip image;

calculating according to the first facial feature information and the second facial feature information to determine a fatigue index of a user, and judging whether the fatigue index is higher than a preset fatigue index;

and if the fatigue index is higher than the preset fatigue index, controlling the electronic equipment to pause reporting the next dictation content.

A second aspect of an embodiment of the present invention discloses an electronic device, including:

the monitoring unit is used for monitoring writing content and writing state information of a user aiming at the current dictation content after the electronic equipment reads the current dictation content; wherein the writing state information at least comprises writing duration in the writing content;

the judging unit is used for judging whether the actual writing stroke order of the writing content is matched with the standard writing stroke order of the current dictation content or not, and judging whether the writing time length of a user for the writing content is longer than the preset standard writing time length according to the writing state information when the actual writing stroke order is not matched with the standard writing stroke order;

And the newspaper reading unit is used for controlling the electronic equipment to switch and newspaper and read the next dictation content when the judging unit judges that the writing time length is longer than the standard writing time length.

In a second aspect of the embodiment of the present invention, the determining unit is further configured to determine, when it is determined that the current writing time period is longer than the standard writing time period, whether the current writing speed of the writing content is less than a preset standard writing speed;

the newspaper reading unit is specifically configured to control the electronic device to switch and newspaper to read the next dictation content when the judging unit judges that the current writing time is longer than the standard writing time and the current writing speed is lower than the standard writing speed.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the determining unit is further configured to determine, when it is determined that the actual writing order matches the standard writing order, whether the actual writing order is an ending order of the writing content;

and the newspaper reading unit is also used for controlling the electronic equipment to switch and newspaper and read the next dictation content when the judging unit judges that the actual writing stroke order is the ending stroke order of the writing content.

As an alternative implementation manner, in a second aspect of the embodiment of the present invention, the monitoring unit includes:

the recognition subunit is used for controlling the infrared sensor of the electronic equipment to recognize the learning desktop where the electronic equipment is positioned after the electronic equipment reads the current dictation content so as to obtain the infrared characteristic information of the human body;

a determining subunit, configured to determine a writing area of a user according to the human infrared feature information;

an adjusting subunit, configured to adjust a shooting direction of a first camera of the electronic device, so that the shooting direction of the first camera faces the writing area;

a shooting subunit, configured to shoot a multi-frame writing image of the writing content written in the writing area by a user;

and the analysis subunit is used for analyzing the multi-frame writing image so as to obtain writing content and writing state information of a user aiming at the current dictation content.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the electronic device further includes:

the control unit is used for controlling the second camera of the electronic equipment to shoot a plurality of frames of facial images of the user;

the intercepting and analyzing unit is used for intercepting eye images from the multi-frame facial images, obtaining first facial feature information by analyzing the eye images, intercepting lip images from the multi-frame facial images, and obtaining second facial feature information by analyzing the lip images;

A calculating unit for calculating according to the first facial feature information and the second facial feature information to determine a fatigue index of the user;

the judging unit is further used for judging whether the fatigue index is higher than a preset fatigue index;

and the pause unit is used for controlling the electronic equipment to pause the newspaper reading of the next dictation content.

A third aspect of an embodiment of the present invention discloses an electronic device, including:

a memory storing executable program code;

a processor coupled to the memory;

the processor invokes the executable program code stored in the memory to perform any of the methods disclosed in the first aspect of the embodiments of the present invention.

A fourth aspect of the invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to perform any of the methods disclosed in the first aspect of the embodiments of the invention.

A fifth aspect of an embodiment of the invention discloses a computer program product which, when run on a computer, causes the computer to perform any of the methods disclosed in the first aspect of the embodiment of the invention.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

After the electronic equipment reads the current dictation content, the writing content and writing state information of a user aiming at the current dictation content are monitored; wherein, the writing state information at least comprises writing time length in writing content; judging whether the actual writing order of the writing content is matched with the standard writing order of the current dictation content; if the writing time length of the user aiming at the writing content is not longer than the preset standard writing time length, judging whether the writing time length of the user aiming at the writing content is longer than the preset standard writing time length according to the writing state information; if the content is larger than the preset value, the electronic equipment is controlled to switch and read the next dictation content. Therefore, by implementing the embodiment of the invention, the grasping degree of the user on the current dictation content can be judged by sequentially judging the writing order and the writing time of the user on the current dictation content, and the electronic equipment is immediately controlled to switch and read the next dictation content when the user is judged not to grasp the dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the dictation efficiency is improved, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a dictation and newspaper reading method based on writing progress according to an embodiment of the invention;

FIG. 2 is a schematic flow chart of another dictation and newspaper reading method based on writing progress according to the embodiment of the invention;

FIG. 3 is a flow chart of a dictation and newspaper reading method based on writing progress according to an embodiment of the invention;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of another electronic device according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of another electronic device according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of still another electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a dictation, newspaper and read method based on writing progress and electronic equipment, which can improve the efficiency and the intelligent degree of dictation, newspaper and read. The following will describe in detail.

Example 1

Referring to fig. 1, fig. 1 is a flow chart of a dictation and newspaper reading method based on writing progress according to an embodiment of the invention. The method for correcting the dictation operation described in fig. 1 is applicable to electronic devices such as home education machines, learning boards, and the like, and the embodiment of the invention is not limited. The operating system of the electronic device may include, but is not limited to, an Android operating system, an IOS operating system, a Symbian operating system, a Black Berry operating system, a Windows Phone8 operating system, and the like, which are not limited to the embodiments of the present invention. As shown in fig. 1, the dictation and newspaper reading method based on the writing progress may include the following steps:

101. after the electronic equipment reads the current dictation content, the writing content and writing state information of a user aiming at the current dictation content are monitored; wherein the writing state information at least comprises writing time in writing content.

In the embodiment of the invention, the electronic equipment can provide dictation and newspaper reading service for the user through the pre-installed dictation and newspaper reading application. A parent or teacher can set a dictation task for a student user at a learning management and control terminal (such as a mobile phone, an intelligent tablet or a notebook computer and the like) bound with the electronic equipment, and the dictation task is sent to a dictation and newspaper reading application of the electronic equipment through the learning management and control terminal, so that the dictation and newspaper reading application of the electronic equipment provides dictation and newspaper reading service for the student user according to the dictation task. The dictation task may include dictation content (word generation, text, etc.), dictation execution time, etc., which is not limited by the embodiment of the present invention.

In an embodiment of the present invention, optionally, the writing state information may further include a writing action feature of the user; further optionally, the electronic device may determine whether the writing action feature matches a preset standard writing action feature; triggering execution of step 102 if the writing action feature matches a standard writing action feature; if the writing action feature does not match the standard writing action feature, a prompt message is output to prompt the user to pay attention to the writing action specification. The writing action feature of the user may include a pen holding gesture feature of the user, a sitting gesture feature of the user, and the like, which is not limited by the embodiment of the present invention. Still further alternatively, the electronic device may acquire the writing action characteristics of the user through an action sensing sensor (e.g., a pressure sensor, etc.) associated therewith, and the action sensing sensor may be disposed on a writing pen (a general pencil, a water-based pen, a smart pen, or an electronic writing pen, etc.) used by the user to complete the dictation task, and a seat on which the user sits at the end of the dictation task, which embodiments of the present invention are not limited.

102. The electronic equipment judges whether the actual writing order of the writing content is matched with the standard writing order of the current dictation content, and if the actual writing order of the writing content is not matched with the standard writing order of the current dictation content, the step 103 is triggered and executed; if so, ending the flow.

As an alternative implementation, if the actual writing order matches the standard writing order, the electronic device may also determine whether the actual writing order is the ending order of the writing content; if the actual writing order is the ending order of the writing, execution is triggered 104.

In the embodiment of the invention, the dictation content can be a raw word, so that the dictation content can be seen as consisting of a plurality of single words, each single word can be formed by a plurality of strokes, and the writing sequence of the strokes of each single word is the writing order. In general, each writing order has a preset order rule, for example, left-right, up-down, inside-out, and the like, and the standard writing order can be determined according to the preset rule. Alternatively, the standard writing strokes of the current dictation content may be stored in a preset stroke library of the electronic device, which is not limited by the embodiment of the present invention.

103. The electronic equipment judges whether the writing time length of the user aiming at the writing content is longer than the preset standard writing time length according to the writing state information, and if the writing time length is longer than the preset standard writing time length, the step 104 is triggered and executed; if the writing duration is not greater than the preset standard writing duration, continuing to execute step 103.

As an alternative embodiment, the writing duration may include a single word writing duration and an interval duration between two adjacent consecutive words; further, the electronic device determining, according to the writing state information, whether the writing duration of the user for the writing content is greater than a preset standard duration may include:

the electronic equipment judges whether the single-word writing time length in the writing content is longer than the preset standard single-word writing time length; if the single character writing time length is longer than the preset standard single character writing time length, judging whether the interval time length between two continuous characters in the content is longer than the preset standard interval time length or not; if the interval time length is longer than the preset standard interval time length, determining that the writing time length of the user aiming at the writing content is longer than the preset standard time length.

104. The electronic device controls the electronic device to switch and read the next dictation content.

In the embodiment of the present invention, the dictation task may include a plurality of dictation contents, so after the electronic device performs step 104, the electronic device may further go to step 101 until all the dictation tasks included in the dictation task are completely reported and read, and then the process is ended.

It can be seen that, in the method described in fig. 1, the mastering degree of the user on the current dictation content can be determined by sequentially determining the writing order and the writing duration of the user on the current dictation content, and when it is determined that the user does not mastering the dictation content, the electronic device is immediately controlled to switch and report the next dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the dictation efficiency is improved, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high.

Example two

Referring to fig. 2, fig. 2 is a flow chart of another dictation and newspaper reading method based on writing progress according to an embodiment of the invention. As shown in fig. 2, the dictation and newspaper reading method based on the writing progress comprises the following steps:

in the embodiment of the invention, the dictation and newspaper reading method based on the writing progress comprises steps 201 to 202, and for the description of the steps 201 to 202, please refer to the detailed description of the steps 101 to 102 in the first embodiment. In step 202, the electronic device determines whether the actual writing order of the writing content matches the standard writing order of the current dictation content, and if not, triggers execution of step 203; if so, ending the flow.

203. The electronic equipment judges whether the writing time of the user aiming at the writing content is longer than the preset standard writing time according to the writing state information; if the writing time is longer than the preset standard writing time, triggering and executing step 204; if not, continuing to execute step 203.

As an optional implementation manner, if the writing time length is longer than the preset standard writing time length, the electronic device may further determine whether the current writing speed of the written content is less than the preset standard writing speed, and if the current writing speed is less than the standard writing speed, control the electronic device to switch and report the next dictation content; and if the current writing speed is not less than the standard writing speed, switching and reading the next dictation content after the preset switching time.

In the embodiment of the present invention, the standard writing duration and the standard writing speed may be determined according to a pre-established writing model, where the writing model may be obtained by training collected sample data (the sample data may include writing speed data, data of a writing duration of a single word, and data of an interval time of a continuous word) for reflecting a writing process of a user on an electronic device or a book, and the writing model may be a neural network model.

204. The electronic device controls a second camera of the electronic device to shoot multiple frames of facial images of the user.

205. The electronic equipment intercepts eye images from a plurality of frames of facial images, and obtains first facial feature information by analyzing the eye images; and intercepting a lip image from the multi-frame facial image, and obtaining second facial feature information by analyzing the lip image.

In the embodiment of the invention, because the fatigue state of the person can be reflected on the facial features of the user, the electronic equipment can judge the fatigue state of the user by taking the closing times or closing degree (area) of eyes and opening and closing degree (area) of a mouth in a certain time period as sampling features.

206. The electronic device calculates according to the first facial feature information and the second facial feature information to determine the fatigue index of the user.

In the embodiment of the invention, the electronic equipment can calculate the fatigue index of the user according to the weight value corresponding to each facial feature information and the parameter value corresponding to each facial feature information which is determined in advance. Specifically, the electronic device may determine a parameter value corresponding to each piece of facial feature information according to a comparison result of each piece of facial feature information and standard facial feature information for the piece of facial feature information; further, calculating the product of the weight value corresponding to each facial feature information and the corresponding parameter value, and calculating the sum of all the products; the fatigue index of the user is determined from the sum of all the products.

In the scheme, the facial feature information comprises first facial feature information and second facial feature information, and preset weight values corresponding to the facial feature information are a first weight value and a second weight value; further, comparing the first facial feature information with first standard facial feature information aiming at the first facial feature information, and determining a first parameter value according to an obtained first comparison result; meanwhile, comparing the second facial feature information with first standard facial feature information aiming at the second facial feature information, and determining a second parameter value according to an obtained second comparison result; calculating the product of the first weight value and the first parameter value to obtain a first product result, and calculating the product of the second weight value and the second parameter value to obtain a second product result; and carrying out weighted sum calculation on the first product result and the second product result, and determining the fatigue index of the user according to the weighted sum calculation result. Comparing the facial feature information with standard facial feature information aiming at the facial feature information, and determining a parameter value according to a comparison result, wherein specifically, the facial feature information can be compared with the standard facial feature information aiming at the facial feature information to obtain the similarity of the facial feature information and the corresponding standard facial feature information; judging whether the similarity is higher than a threshold value of the corresponding preset similarity, and if so, judging that the parameter value is A; if not, the parameter value is B. Wherein, a and B may be any preset values (e.g. 0, 1, etc.), which are not limited in the embodiment of the present invention. Therefore, according to the embodiment of the invention, the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, so that the speed of determining the fatigue index is improved, and meanwhile, compared with a single factor, the accuracy and the reliability of fatigue judgment are higher.

207. The electronic device judges whether the fatigue index is higher than a preset fatigue index, and if the fatigue index is higher than the preset fatigue index, the step 208 is triggered and executed; if the fatigue index is not higher than the preset fatigue index, execution of step 209 is triggered.

As an optional implementation manner, if the fatigue index is not higher than the preset fatigue index, the electronic device may further acquire log information issued by the user on each social application platform in a preset period of time (for example, in a week, in three days, etc.), and identify the log information, so as to obtain the emotion type of the user. Triggering execution of step 209 when the emotion type is a positive emotion type; when the emotion type is a negative emotion type, execution of step 208 is triggered.

In the embodiment of the invention, optionally, the social application platform can include but is not limited to QQ, microblog and blog; the log information may include text, still pictures, moving pictures, voice, video, and the like. Taking the log information as an example, the electronic device identifies the log information to obtain the emotion type of the user, which may include:

the electronic equipment performs word segmentation processing on the log information to obtain a plurality of word segments to be matched;

Matching a plurality of word segments to be matched with each preset word segment set;

determining a target word segmentation set which is matched with the word segmentation to be matched successfully most, and determining a target emotion type corresponding to the target word segmentation set as an emotion type of a user; among other types of emotions for the user may include a positive type of emotion and a negative type of emotion.

In this scheme, word segmentation sets corresponding to each emotion type may be preset. The following description will be made by way of specific examples: the "say" content of the user in the QQ release ("say" is a functional module of the QQ space, that is, space dynamics, and the user can record own daily dynamics in the form of words or the like) is: today, restless and hydrokinetic. The electronic device can perform word segmentation processing on the speaking content, extract the word of vexation from the word, and determine that the emotion type of the user is a negative emotion type according to the matching of the word and each preset word segmentation set.

Therefore, according to the embodiment of the invention, the emotion recognition can be carried out on the user based on the log information issued by the user on the social application platform, so that the emotion recognition result is more in line with the actual situation of the user, and the recognition result is more accurate.

In addition, the electronic equipment can also collect physical sign data of the user and determine the emotion type of the user through the physical sign data of the user. Specifically, the electronic device may receive each type of sign data sent by the sign sensor associated in advance, compare each type of sign data with the standard sign data corresponding to the type of sign data, and determine a score corresponding to the type of sign data according to a comparison result; adding the scores corresponding to all types of physical sign data to obtain an added total score; and respectively calculating the difference value of the total score and the standard score corresponding to each emotion type, and determining the emotion type corresponding to the standard score with the smallest difference value of the total score as the emotion type of the user. For example, the preset emotion types include a positive emotion type, a normal emotion type and a negative emotion type, wherein the standard score corresponding to the positive emotion type is 80 points, and the standard score corresponding to the negative emotion type is 40 points; a score of 60 points corresponding to normal emotion types; and the total score is 30 points; the difference between the total score and the negative emotion type is the smallest, so that the emotion type of the user can be determined as the negative emotion type.

The physical sign sensor is positioned on a mobile device worn by a user and used for detecting human physiological characteristic data such as heart rate, blood pressure and body temperature of the user; thus, the sign sensor may include, but is not limited to, a pulse sensor, a human body temperature sensor, and the like.

Therefore, the embodiment of the invention can identify the current emotion type of the user according to the current sign data of the user, so that the identification result of the emotion type is more in line with the actual situation of the user, and the identification result is more accurate.

208. The electronic device controls to pause the newspaper from reading the next dictation.

209. And the electronic equipment switches and reads the next dictation content after the preset switching time.

In the embodiment of the present invention, the dictation task may include a plurality of dictation contents, so after the electronic device performs step 209, the electronic device may further go to step 201 until all the dictation tasks included in the dictation task are completely reported and read, and then the process is ended.

Based on the above embodiment, the current fatigue index of the user is determined from the facial feature information (such as eyes and mouth) of the user, and then the corresponding operation (switching or suspending the newspaper reading) is performed on the dictation content according to the magnitude of the fatigue index. The intelligent control method for the dictation content enables the switching of the dictation content to be more in line with the actual physical condition of the user, and further can improve the dictation effect of the user.

It can be seen that, in the method shown in fig. 2, the mastering degree of the user on the current dictation content can be determined by sequentially determining the writing order and the writing duration of the user on the current dictation content, and when it is determined that the user does not mastering the dictation content, the electronic device is immediately controlled to switch and read the next dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the method improves the dictation efficiency, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high; and the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, and compared with a single factor, the fatigue judgment accuracy and reliability are higher; in addition, the current emotion type of the user can be identified according to the current sign data of the user or the log information of the user, so that the identification result of the emotion type is more in line with the actual situation of the user, and the identification result is more accurate.

Example III

Referring to fig. 3, fig. 3 is a flow chart of a dictation and newspaper reading method based on writing progress according to an embodiment of the invention. As shown in fig. 3, the dictation and newspaper reading method based on the writing progress comprises the following steps:

301. After the electronic equipment reads the current dictation content, the electronic equipment recognizes the learning desktop where the electronic equipment is located through an infrared sensor of the electronic equipment so as to obtain the infrared characteristic information of the human body.

It should be noted that, the learning desktop where the default electronic device is located in the scheme is a desktop where the user writes the writing content according to the current dictation content.

302. And the electronic equipment determines the writing area of the user according to the human infrared characteristic information.

In the embodiment of the present invention, the writing area refers to an active area of the palm and the arm (including at least the forearm) of the user on the learning desktop.

303. The electronic device adjusts the shooting direction of a first camera of the electronic device so that the shooting direction of the first camera faces the writing area, and shoots a plurality of frames of writing images of the writing contents written by a user in the writing area.

In the embodiment of the invention, based on the content in the embodiment, the electronic equipment can identify the learning desktop where the electronic equipment is located through the built-in infrared sensor after reporting and reading the current dictation content, and at the moment, if the user dictates on the learning desktop, the electronic equipment can identify the arm of the user on the learning desktop through the infrared sensor so as to obtain the human infrared characteristic information; further, according to the human body infrared characteristic information, the moving range of the user on the learning desktop can be determined, the moving range is the writing area of the user, and then the shooting direction of the first camera is adjusted, so that the shooting direction aims at the writing area to shoot, and a multi-frame writing image of the writing content written by the user in the writing area is obtained. Therefore, the writing area of the user can be determined based on the human body infrared characteristic information obtained through the identification of the infrared sensor, and the shooting direction of the first camera is adjusted according to the writing area, so that the first camera shoots the process of writing the writing content of the user towards the writing area, a plurality of frames of writing images are obtained, the first camera is prevented from shooting excessive useless images, the shooting content of the camera is more targeted, the workload of the subsequent writing image identification is reduced, and the identification accuracy of the writing images is improved.

304. The electronic equipment analyzes the multi-frame writing image to obtain writing content and writing state information of a user aiming at the current dictation content; wherein the writing state information at least comprises writing time in writing content.

In the embodiment of the invention, it can be understood that the writing area can include a writing surface (such as a book, a paper surface of a homework book or an electronic writing surface) for writing the writing content by a user, so that the electronic device can obtain the writing content of the current dictation content by analyzing a plurality of frames of writing images, and can determine the writing time of the writing content by analyzing the difference of the writing content in each frame of writing images.

The dictation and newspaper reading method based on the writing progress includes steps 305 to 312, and for the description of steps 305 to 312, please refer to the detailed description of steps 202 to 209 in the second embodiment.

It can be seen that, in the method described in fig. 3, the mastering degree of the user on the current dictation content can be determined by sequentially determining the writing order and the writing duration of the user on the current dictation content, and the electronic device is immediately controlled to switch and read the next dictation content when the user is determined not to mastering the dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the method improves the dictation efficiency, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high; and the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, and compared with a single factor, the fatigue judgment accuracy and reliability are higher; the current emotion type of the user can be identified according to the current sign data of the user or the log information of the user, so that the identification result of the emotion type is more in line with the actual situation of the user, and the identification result is more accurate; in addition, the condition that the first camera shoots too many useless images is avoided, so that shooting contents of the camera are more targeted, the workload of the follow-up writing image recognition is reduced, and the recognition accuracy of the writing image is improved.

Example IV

Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the invention. As shown in fig. 4, the electronic device may include:

a monitoring unit 401, configured to monitor writing content and writing status information of a user for the current dictation content after the current dictation content is read by the electronic device, and provide the writing content and writing status information to the judging unit 402; wherein the writing status information may include at least a writing duration in the writing content.

The judging unit 402 is configured to judge whether an actual writing order of the writing content matches a standard writing order of the current dictation content, and when it is judged that the actual writing order does not match the standard writing order, judge whether a writing time length of the user for the writing content is longer than a preset standard writing time length according to the writing state information, and provide a judging result to the newspaper reading unit 403.

In an embodiment of the present invention, optionally, the writing state information may further include a writing action feature of the user; further optionally, the determining unit 402 may be further configured to determine whether the writing action feature matches a preset standard writing action feature; if the writing action feature is matched with the standard writing action feature, further judging whether the actual writing order of the writing content is matched with the standard writing order of the current dictation content, judging whether the writing time length of the user for the writing content is longer than the preset standard writing time length according to the writing state information when the actual writing order is not matched with the standard writing order, and providing the judging result to the newspaper reading unit 403; if the writing action feature does not match the standard writing action feature, an output unit (not shown in fig. 4) is controlled to output a prompt message to prompt the user to pay attention to the writing action specification. The writing action feature of the user may include a pen holding gesture feature of the user, a sitting gesture feature of the user, and the like, which is not limited by the embodiment of the present invention. Still further alternatively, the above-mentioned monitoring unit 401 may also obtain the writing action characteristics of the user through an action sensing sensor (for example, a pressure sensor, etc.) associated with the monitoring unit, and the action sensing sensor may be disposed on a writing pen (a common pencil, a water-based pen, a smart pen, or an electronic writing pen, etc.) used by the user to complete the dictation task, and a seat on which the user sits to complete the dictation task, which is not limited by the embodiment of the present invention.

As an alternative embodiment, the writing duration may include a single word writing duration and an interval duration between two adjacent consecutive words; then, further, the determining unit 402 may specifically determine, according to the writing status information, whether the writing duration of the writing content is greater than the preset standard duration, by using the writing status information:

judging whether the single-word writing time length in the writing content is longer than the preset standard single-word writing time length; if the single character writing time length is longer than the preset standard single character writing time length, judging whether the interval time length between two continuous characters in the content is longer than the preset standard interval time length or not; if the interval time length is longer than the preset standard interval time length, determining that the writing time length of the user aiming at the writing content is longer than the preset standard time length.

And the newspaper reading unit 403 is configured to control the electronic device to switch and newspaper and read the next dictation content when the determining unit 402 determines that the writing time period is longer than the standard writing time period.

It can be seen that, implementing the electronic device shown in fig. 4, the user's mastering degree of the current dictation content can be determined by sequentially determining the writing order and writing duration of the user with respect to the current dictation content, and when it is determined that the user does not mastering the dictation content, the electronic device is immediately controlled to switch and read the next dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the dictation efficiency is improved, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high.

Example five

Referring to fig. 5, fig. 5 is a schematic structural diagram of another electronic device according to an embodiment of the invention. The electronic device shown in fig. 5 is obtained by optimizing the electronic device shown in fig. 4. As shown in fig. 5, the electronic device may further include:

a control unit 404, configured to control the second camera of the electronic device to capture multiple frames of facial images of the user, and provide the multiple frames of facial images to the interception analysis unit 405.

Optionally, the control unit 404 may be configured to control the second camera of the electronic device to capture a multi-frame facial image of the user after the electronic device reads the current dictation content, and provide the multi-frame facial image to the interception analysis unit 405.

The clipping analysis unit 405 is configured to clip an eye image from a plurality of frames of facial images, and obtain first facial feature information by analyzing the eye image, clip a lip image from the plurality of frames of facial images, and obtain second facial feature information by analyzing the lip image, and provide the first facial feature information and the second facial feature information to the calculation unit 406.

A calculating unit 406 for calculating to determine a fatigue index of the user based on the first facial feature information and the second facial feature information, and providing the fatigue index to the judging unit 402.

In the embodiment of the present invention, the calculating unit 406 may calculate the fatigue index of the user according to the weight value corresponding to each facial feature information and the parameter value corresponding to each facial feature information determined in advance. Specifically, the calculation unit 406 may determine a parameter value corresponding to each piece of facial feature information according to a result of comparison of each piece of facial feature information with standard facial feature information for the piece of facial feature information; further, calculating the product of the weight value corresponding to each facial feature information and the corresponding parameter value, and calculating the sum of all the products; the fatigue index of the user is determined from the sum of all the products.

In the scheme, the facial feature information comprises first facial feature information and second facial feature information, and preset weight values corresponding to the facial feature information are a first weight value and a second weight value; further, comparing the first facial feature information with first standard facial feature information aiming at the first facial feature information, and determining a first parameter value according to an obtained first comparison result; meanwhile, comparing the second facial feature information with first standard facial feature information aiming at the second facial feature information, and determining a second parameter value according to an obtained second comparison result; calculating the product of the first weight value and the first parameter value to obtain a first product result, and calculating the product of the second weight value and the second parameter value to obtain a second product result; and carrying out weighted sum calculation on the first product result and the second product result, and determining the fatigue index of the user according to the weighted sum calculation result. Comparing the facial feature information with standard facial feature information aiming at the facial feature information, and determining a parameter value according to a comparison result, wherein specifically, the facial feature information can be compared with the standard facial feature information aiming at the facial feature information to obtain the similarity of the facial feature information and the corresponding standard facial feature information; judging whether the similarity is higher than a threshold value of the corresponding preset similarity, and if so, judging that the parameter value is A; if not, the parameter value is B. Wherein, a and B may be any preset values (e.g. 0, 1, etc.), which are not limited in the embodiment of the present invention. It can be seen that, in the embodiment of the present invention, the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, so that the speed of determining the fatigue index is increased,

Meanwhile, compared with a single factor, the method has higher accuracy and reliability for fatigue judgment.

The above-mentioned judging unit 402 is further configured to judge whether the fatigue index is higher than a preset fatigue index, and provide the judging result to the suspending unit 407.

A suspension unit 407, configured to control the electronic device to suspend reporting and reading the next dictation.

It can be seen that, by implementing the electronic device shown in fig. 5, the user's mastering degree of the current dictation content can be determined by sequentially determining the writing order and writing duration of the user with respect to the current dictation content, and when it is determined that the user does not mastering the dictation content, the electronic device is immediately controlled to switch and read the next dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the method improves the dictation efficiency, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high; and the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, and compared with a single factor, the fatigue judgment accuracy and reliability are higher; in addition, the current emotion type of the user can be identified according to the current sign data of the user or the log information of the user, so that the identification result of the emotion type is more in line with the actual situation of the user, and the identification result is more accurate.

Example six

Referring to fig. 6, fig. 6 is a schematic structural diagram of another electronic device according to an embodiment of the present invention. The electronic device shown in fig. 6 is obtained by optimizing the electronic device shown in fig. 6. As shown in fig. 6, in the electronic device:

the above-mentioned judging unit 402 is further configured to, when judging that the current writing time period is longer than the standard writing time period, judge whether the current writing speed of the writing content is smaller than the preset standard writing speed, and provide the judgment result to the newspaper reading unit 403.

The newspaper reading unit 403 is specifically configured to control the electronic device to switch and newspaper to read the next dictation content when the judging unit 402 judges that the current writing time is longer than the standard writing time and the current writing speed is less than the standard writing speed.

As an alternative embodiment, as shown in fig. 6, the above-mentioned determining unit 402 is further configured to, when determining that the actual writing order matches the standard writing order, determine whether the actual writing order is the ending order of the writing content, and provide the determination result to the newspaper reading unit 403.

The newspaper reading unit 403 is further configured to control the electronic device to switch and newspaper the next dictation content when the determining unit 402 determines that the actual writing order is the ending order of the writing content.

As another alternative embodiment, as shown in fig. 6, the monitoring unit 401 may include:

the recognition subunit 4011 is configured to control an infrared sensor of the electronic device to recognize a learning desktop where the electronic device is located after the electronic device reads the current dictation content, so as to obtain human infrared feature information, and provide the human infrared feature information to the determination subunit 4012.

The determining subunit 4012 is configured to determine a writing area of the user according to the human infrared characteristic information, and provide the writing area to the adjusting subunit 4013.

The adjusting subunit 4013 is configured to adjust a shooting direction of the first camera of the electronic device, so that the shooting direction of the first camera faces the writing area, and trigger the shooting subunit 4014 to start.

The photographing subunit 4014 is configured to photograph a plurality of frames of writing images of the writing contents written by the user in the writing area, and provide the plurality of frames of writing images to the analysis subunit 4015.

The analysis subunit 4015 is configured to analyze the multi-frame writing image to obtain writing content and writing status information of the user for the current dictation content.

It can be seen that, by implementing the electronic device shown in fig. 6, the user's mastering degree of the current dictation content can be determined by sequentially determining the writing order and writing duration of the user with respect to the current dictation content, and when it is determined that the user does not mastering the dictation content, the electronic device is immediately controlled to switch and read the next dictation content. Compared with the prior art that the dictation content is switched by adopting a preset dictation mode or a manual control mode, the method improves the dictation efficiency, and meanwhile, the dictation process is switched without manual intervention, so that the intelligent degree is high; and the fatigue index of the user can be determined through the weighted calculation results of the first facial feature information and the second facial feature information, and compared with a single factor, the fatigue judgment accuracy and reliability are higher; the current emotion type of the user can be identified according to the current sign data of the user or the log information of the user, so that the identification result of the emotion type is more in line with the actual situation of the user, and the identification result is more accurate; in addition, the condition that the first camera shoots too many useless images is avoided, so that shooting contents of the camera are more targeted, the workload of the follow-up writing image recognition is reduced, and the recognition accuracy of the writing image is improved.

Example seven

Referring to fig. 7, fig. 7 is a schematic structural diagram of another electronic device according to an embodiment of the invention. As shown in fig. 7, the electronic device may include:

a memory 901 storing executable program code;

a processor 902 coupled to the memory 901;

the processor 902 calls executable program codes stored in the memory 901, and executes any of the dictation and newspaper reading methods shown in fig. 1 to 3 based on the writing progress.

It should be noted that, the electronic device shown in fig. 7 may further include components not shown, such as a power supply, an input key, a speaker, a microphone, a screen, an RF circuit, a Wi-Fi module, a bluetooth module, a sensor, etc., which are not described in detail in this embodiment.

The embodiment of the invention discloses a computer readable storage medium which stores a computer program, wherein the computer program enables a computer to execute any of dictation and newspaper reading methods based on writing progress shown in fig. 1-3.

Embodiments of the present invention disclose a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform any of the dictation and reading methods based on writing progress shown in fig. 1-3.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art will also appreciate that the embodiments described in the specification are alternative embodiments and that the acts and modules referred to are not necessarily required for the present invention.

In various embodiments of the present invention, it should be understood that the sequence numbers of the foregoing processes do not imply that the execution sequences of the processes should be determined by the functions and internal logic of the processes, and should not be construed as limiting the implementation of the embodiments of the present invention.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, or a part contributing to the prior art or all or part of the technical solution, may be embodied in the form of a software product stored in a memory, comprising several requests for a computer device (which may be a personal computer, a server or a network device, etc., in particular may be a processor in a computer device) to execute some or all of the steps of the above-mentioned method of the various embodiments of the present invention.

Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program that instructs associated hardware, the program may be stored in a computer readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disk Memory, magnetic disk Memory, tape Memory, or any other medium that can be used for carrying or storing data that is readable by a computer.

The foregoing describes in detail a dictation and reading method and an electronic device based on writing progress, and specific examples are applied to illustrate the principles and embodiments of the present invention, and the description of the foregoing examples is only for helping to understand the method and core idea of the present invention. Meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. A dictation and newspaper reading method based on writing progress is characterized by comprising the following steps:

2. The method of claim 1, wherein if the writing time period is longer than the standard writing time period, the method further comprises:

3. The method of claim 1, wherein if the actual writing order matches the standard writing order, the method further comprises:

4. A method according to any one of claims 1 to 3, wherein after the electronic device reads the current dictation, monitoring the writing content and writing status information of the user for the current dictation, and comprising:

5. The method according to claim 1, wherein the method further comprises:

controlling a second camera of the electronic equipment to shoot a plurality of frames of facial images of a user;

6. An electronic device, the electronic device comprising:

7. The electronic device according to claim 6, wherein the judging unit is further configured to judge whether a current writing speed of the writing content is less than a preset standard writing speed when judging that a current writing time period is longer than the standard writing time period;

8. The electronic device according to claim 6, wherein the judging unit is further configured to judge whether the actual writing order is an ending order of the writing content when judging that the actual writing order matches the standard writing order;

9. The electronic device of any one of claims 6-8, wherein the monitoring unit comprises:

10. The electronic device of claim 6, wherein the electronic device further comprises: