CN111077990B - Method for determining content to be read on spot and learning equipment - Google Patents

Abstract

The invention relates to the technical field of education, and discloses a method for determining content to be read on demand and learning equipment, which comprises the following steps: when the shielding object is detected to be in contact with the learning page and a click-to-read mode starting instruction is received, controlling the learning device to start a click-to-read mode, and acquiring a first image containing the shielding object and the learning page through the image acquisition device in the click-to-read mode; acquiring position information of a shielding object in a first image; removing the image corresponding to the shielding object from the first image to obtain a second image; selecting a target electronic page matched with the second image; and determining the content to be clicked from the target electronic page according to the position information. By implementing the embodiment of the invention, the complete content to be read can be identified from the complete electronic page, and the comprehensiveness of identifying the content to be read is improved.

Description

Method for determining content to be read on spot and learning equipment

Technical Field

The invention relates to the technical field of education, in particular to a method for determining content to be read on demand and learning equipment.

Background

With the rapid development of learning devices, more and more learning devices can meet the reading demands of students. At present, the manner in which the learning device performs click-to-read is generally: the learning device detects what the student clicks on the book to click on, and then plays audio matching the content. However, in practice, it is found that the user typically clicks on the book with a finger or a stylus, and therefore, the finger or the stylus of the user may cover a part of the content to be read, which results in that the learning device cannot identify the whole content to be read, and thus the identified content to be read is not comprehensive.

Disclosure of Invention

The embodiment of the invention discloses a method for determining contents to be read and learning equipment, which can improve the comprehensiveness of identifying the contents to be read by determining a complete image to be identified.

The first aspect of the embodiment of the invention discloses a method for determining content to be read, which comprises the following steps:

when a shielding object is detected to be in contact with a learning page and a click-to-read mode starting instruction is received, controlling learning equipment to start a click-to-read mode, and acquiring a first image containing the shielding object and the learning page through image acquisition equipment in the click-to-read mode;

acquiring position information of the shielding object in the first image;

removing the image corresponding to the shielding object from the first image to obtain a second image;

selecting a target electronic page matched with the second image;

and determining the content to be clicked from the target electronic page according to the position information.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, after the determining, according to the location information, content to be clicked from the target electronic page, the method further includes:

acquiring standard voice corresponding to the content to be read;

Outputting the standard voice through a loudspeaker;

and, after the outputting of the standard speech through the speaker, the method further comprises:

when an error instruction input by a user is detected, the loudspeaker is controlled to stop outputting the standard voice, and prompt information is output, wherein the prompt information is used for prompting the user to control the shielding object to contact with the learning page again, and prompting the user to input the click-to-read mode starting instruction again so as to enable the learning device to recognize the content to be clicked again.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, the acquiring location information of the obstruction in the first image includes:

acquiring a first coordinate system corresponding to the first image;

determining a first coordinate of the shielding object in the first image according to the first coordinate system;

determining the first coordinate as position information of the shielding object;

and determining the content to be clicked from the target electronic page according to the position information, including:

acquiring a second coordinate system corresponding to the target electronic page;

calculating the conversion relation between the first coordinate system and the second coordinate system;

Converting the first coordinate in the position information into a second coordinate of the second coordinate system according to the conversion relation;

and determining the content to be clicked corresponding to the second coordinate from the target electronic page.

In an optional implementation manner, in a first aspect of the embodiment of the present invention, the removing, from the first image, an image corresponding to the occlusion object to obtain a second image includes:

identifying a target sub-image corresponding to the shielding object from the first image, and identifying the background color of the first image;

and adjusting the color of the target sub-image in the first image to be the background color, and determining the adjusted first image to be a second image.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, before the controlling the learning device to start the click-to-read mode when the shielding object is detected to be in contact with the learning page and the click-to-read mode start instruction is received, the method further includes:

detecting whether a shielding object is in contact with a learning page or not through image acquisition equipment;

if the shielding object is in contact with the learning page, collecting sound in the environment where the learning equipment is located in the process that the shielding object is in contact with the learning page;

Identifying whether the voice contains voice information matched with a click-to-read mode starting instruction;

if so, determining that the shielding object is in contact with the learning page and receiving the click-to-read mode starting instruction.

A second aspect of an embodiment of the present invention discloses a learning apparatus, including:

the first acquisition unit is used for controlling the learning equipment to start a click-to-read mode when detecting that a shielding object is in contact with a learning page and receiving a click-to-read mode starting instruction, and acquiring a first image containing the shielding object and the learning page through the image acquisition equipment in the click-to-read mode;

the first acquisition unit is used for acquiring the position information of the shielding object in the first image;

the rejecting unit is used for rejecting the image corresponding to the shielding object from the first image to obtain a second image;

a selecting unit, configured to select a target electronic page that matches the second image;

and the first determining unit is used for determining the content to be clicked from the target electronic page according to the position information.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the learning device further includes:

the second acquisition unit is used for acquiring standard voice corresponding to the content to be clicked after the first determination unit determines the content to be clicked from the target electronic page according to the position information;

A first output unit for outputting the standard voice through a speaker;

and, the learning apparatus further includes:

the second output unit is used for controlling the loudspeaker to stop outputting the standard voice and outputting prompt information after the first output unit outputs the standard voice through the loudspeaker and when an error instruction input by a user is detected, wherein the prompt information is used for prompting the user to control the shielding object to contact with the learning page again and prompting the user to input the click-to-read mode starting instruction again so as to enable the learning device to recognize the content to be clicked again.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the first obtaining unit includes:

the first acquisition subunit is used for acquiring a first coordinate system corresponding to the first image;

a first determining subunit, configured to determine, based on the first coordinate system, a first coordinate of the occlusion object in the first image;

a second determining subunit, configured to determine the first coordinate as position information of the obstruction;

and, the first determination unit includes:

the second acquisition subunit is used for acquiring a second coordinate system corresponding to the target electronic page;

A calculating subunit, configured to calculate a conversion relationship between the first coordinate system and the second coordinate system;

a conversion subunit, configured to convert the first coordinate in the location information into a second coordinate of the second coordinate system based on the conversion relationship;

and the third determining subunit is used for determining the content to be clicked corresponding to the second coordinate from the target electronic page.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the rejection unit includes:

the identification subunit is used for identifying a target sub-image corresponding to the shielding object from the first image and identifying the background color of the first image;

and the adjustment subunit is used for adjusting the color of the target sub-image in the first image to the background color and determining the adjusted first image as a second image.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the learning device further includes:

the detection unit is used for detecting whether the shielding object is in contact with the learning page or not through the image acquisition device before controlling the learning device to start the point reading mode when the first acquisition unit detects that the shielding object is in contact with the learning page and receives a point reading mode starting instruction;

The second acquisition unit is used for acquiring sound in the environment where the learning equipment is located in the process that the shielding object is in contact with the learning page when the detection result of the detection unit is yes;

the identifying unit is used for identifying whether the voice contains voice information matched with the click-to-read mode starting instruction;

and the second determining unit is used for determining that the shielding object is contacted with the learning page and receiving the click-to-read mode starting instruction when the identification result of the identifying unit is yes.

A third aspect of an embodiment of the present invention discloses an electronic device, including:

a memory storing executable program code;

a processor coupled to the memory;

the processor invokes the executable program code stored in the memory to perform part or all of the steps of any one of the methods of the first aspect.

A fourth aspect of the embodiments of the present invention discloses a computer-readable storage medium storing program code, wherein the program code comprises instructions for performing part or all of the steps of any one of the methods of the first aspect.

A fifth aspect of the embodiments of the present invention discloses a computer program product which, when run on a computer, causes the computer to perform part or all of the steps of any one of the methods of the first aspect.

A sixth aspect of the embodiments of the present invention discloses an application publishing platform for publishing a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform part or all of the steps of any one of the methods of the first aspect.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, when the contact of the shielding object and the learning page is detected and a click-to-read mode starting instruction is received, the learning device is controlled to start the click-to-read mode, and a first image containing the shielding object and the learning page is acquired through the image acquisition device in the click-to-read mode; acquiring position information of a shielding object in a first image; removing the image corresponding to the shielding object from the first image to obtain a second image; selecting a target electronic page matched with the second image; and determining the content to be clicked from the target electronic page according to the position information. Therefore, by implementing the embodiment of the invention, the image of the shielding object can be removed from the acquired image, the image of which the shielding object is removed is compared with the electronic page, the target electronic page matched with the image of which the shielding object is removed is found, and as the content in the electronic page is complete, the content to be read identified from the target image is complete, so that the comprehensiveness of identifying the content to be read is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a method for determining content to be read through according to an embodiment of the present invention;

FIG. 2 is a flow chart of another method for determining content to be read through according to an embodiment of the present invention;

FIG. 3 is a flow chart of another method for determining content to be read through according to an embodiment of the present invention;

fig. 4 is a schematic structural view of a learning apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural view of another learning apparatus disclosed in an embodiment of the present invention;

fig. 6 is a schematic structural view of another learning apparatus disclosed in an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a method for determining contents to be read and learning equipment, which can identify complete contents to be read from a complete electronic page and improve the comprehensiveness of identifying the contents to be read. The following will describe in detail.

Example 1

Referring to fig. 1, fig. 1 is a flowchart illustrating a method for determining contents to be read through according to an embodiment of the present invention. As shown in fig. 1, the method for determining the content to be read-on may include the following steps:

101. when the shielding object is detected to be in contact with the learning page and a click-to-read mode starting instruction is received, the learning device controls the learning device to start the click-to-read mode, and a first image containing the shielding object and the learning page is acquired through the image acquisition device in the click-to-read mode.

In the embodiment of the invention, the learning equipment can be a home teaching machine, a learning tablet and the like. The contact between the shielding object and the learning page can be the operation of pressing, moving or smearing the learning page by the shielding object, and the shielding object can be the finger, the touch pen or the capacitance pen of the user. The click mode opening instruction may be triggered by the user by inputting a voice including the click mode opening instruction, or may be input by the user performing an operation corresponding to the click mode opening instruction on the learning device, which is not limited in this embodiment of the present invention. In addition, the learning page may be a book or paper placed in an image acquisition area of the learning device, or the like.

In the embodiment of the invention, the image acquisition equipment can be a camera, a camera and the like, can be arranged at any position on the learning equipment and can also be arranged at a place other than the learning equipment, and can be in communication connection with the learning equipment in a wireless or wired mode so that the image acquisition equipment can transmit the acquired image to the learning equipment.

102. The learning device obtains position information of the obstruction in the first image.

As an alternative embodiment, the manner in which the learning device acquires the position information of the obstruction in the first image may include the following steps:

the learning device can identify a first mark point in the first image and determine a first main direction of the first mark point in the first image, wherein the first mark point can be any mark point existing in the learning page and the electronic page, and the main directions of the mark points in the learning page and the electronic page are the same;

the learning device determines a relative distance of the shield with respect to the first landmark and determines a relative angle of the shield with respect to a first main direction of the first landmark;

the learning device determines the relative distance and the relative angle as position information of the obstruction in the first image.

In the implementation of the embodiment, the mark point can be determined in the first image, and the relative distance and the relative angle between the shielding object and the mark point are calculated, so that the position information of the shielding object in the first image is determined, and the correlation degree of the position information of the shielding object and the first image is improved.

103. The learning equipment eliminates the image corresponding to the shielding object from the first image to obtain a second image.

In the embodiment of the invention, the learning device can delete the shielding object in the first image so that the shielding object does not exist in the obtained second image, and therefore, the shielding object cannot become an interference element matched with the target electronic page.

104. The learning device selects a target electronic page that matches the second image.

In the embodiment of the invention, the target electronic page can be one of a plurality of electronic pages stored in advance by the learning device, the learning device can shoot each page image of the teaching material learned by the user, the learning device can store the acquired image of the teaching material as the electronic page of the teaching material into a memory of the learning device or a server associated with the learning device, so that the learning device can select the target electronic page matched with the learning page clicked by the user from the plurality of electronic pages stored in advance, thereby the learning device acquires the complete content of the learning page clicked by the user, and the identified content to be clicked is complete.

105. And the learning equipment determines the content to be read from the target electronic page according to the position information.

In the embodiment of the invention, when the user controls the shielding object to execute the clicking operation in the learning page, the determined position information can be identified as any point in the learning page, and when the user controls the shielding object to move in the learning page, the determined position information can be identified as a line in the learning page, and the identified line can be a line segment or a curve. If the position information is any point in the learning page, the learning device can determine the acquired text information corresponding to the point as the content to be clicked, and if the position information is a line in the learning page, the learning device can acquire the area covered by the line in the learning page, can acquire the text information contained in the covered area from the learning page, and can determine the acquired text information as the content to be clicked.

As an alternative embodiment, the manner of determining the content to be clicked by the learning device from the target electronic page according to the location information may include the following steps:

the learning equipment identifies a second mark point in the target electronic page and determines a second main direction of the second mark point in the target electronic page;

The learning device determines a target position in the second image, the target position being at a distance from the second marker point and at a relative angle to the second main direction, based on the relative angle and the relative distance contained in the position information;

and the learning equipment determines the content to be read corresponding to the target position from the target electronic page.

By implementing the embodiment, the target position corresponding to the position information can be determined from the target electronic page, so that the content to be clicked corresponding to the target position in the target electronic page is determined, and the position of the content to be clicked determined from the target electronic page is more accurate.

In the method described in fig. 1, the complete content to be read can be identified from the complete electronic page, so that the comprehensiveness of identifying the content to be read is improved. In addition, by implementing the method described in fig. 1, the correlation between the position information of the shielding object and the first image can be improved. In addition, by implementing the method described in fig. 1, the position of the content to be clicked, which is determined from the target electronic page, can be more accurate.

Example two

Referring to fig. 2, fig. 2 is a flowchart illustrating another method for determining contents to be read through according to an embodiment of the present invention. As shown in fig. 2, the method for determining the content to be read-on may include the following steps:

201. When the shielding object is detected to be in contact with the learning page and a click-to-read mode starting instruction is received, the learning device controls the learning device to start the click-to-read mode, and a first image containing the shielding object and the learning page is acquired through the image acquisition device in the click-to-read mode.

202. The learning device acquires a first coordinate system corresponding to the first image.

In the embodiment of the invention, the first coordinate system may be a plane rectangular coordinate system, the origin of the plane rectangular coordinate system may be any point in the first image, and the unit distance in the plane rectangular coordinate system may be calculated according to a standard unit distance stored in advance by the learning device.

Alternatively, the unit distance of the first coordinate system may be calculated by: the learning equipment acquires a standard unit distance; and calculating an average distance between two adjacent characters from the first image; calculating the target proportion of the standard unit distance to the average distance; and reducing the standard unit distance by the target proportion to obtain a target unit distance, wherein the learning equipment can determine the target unit distance as the unit distance of the first coordinate system, so that the proportion of the coordinate system of the first image acquired by the learning equipment is matched with the standard proportion stored in advance by the learning equipment, and the accuracy of the learning equipment in identifying the target electronic page is improved.

203. The learning device determines a first coordinate of the occlusion in the first image based on the first coordinate system.

As an alternative embodiment, the learning device may determine the first coordinates of the obstruction in the first image based on the first coordinate system, including the following steps:

the learning equipment determines a target image corresponding to the shielding object from the first image;

the learning equipment identifies the target image and determines the outline of the target image;

the learning equipment analyzes the outline of the target to determine the top end of the shielding object corresponding to the target image;

the learning device is used for determining target coordinates of the top end of the shielding object according to the first coordinate system, and determining the target coordinates as first coordinates of the shielding object in the first image.

In the implementation of the embodiment, the top end of the shielding object can be determined through an image recognition method, and the top end clicking of the shielding object is usually used when a user clicks the content to be read through the shielding object, so that the accuracy of determining the clicking position of the shielding object can be improved through determining the coordinates of the top end of the shielding object.

204. The learning device determines the first coordinates as position information of the obstruction.

In the embodiment of the present invention, the steps 202 to 204 are implemented, so that a coordinate system can be established in the first image, so that the learning device can determine the coordinates corresponding to any object in the first image, and the learning device can quickly determine the position of the shielding object in the first image, thereby improving the accuracy of identifying the position of the shielding object.

205. The learning equipment eliminates the image corresponding to the shielding object from the first image to obtain a second image.

206. The learning device selects a target electronic page that matches the second image.

207. And the learning equipment acquires a second coordinate system corresponding to the target electronic page.

In the embodiment of the invention, the second coordinate system may be a plane rectangular coordinate system, the origin of the plane rectangular coordinate system may be any point in the target electronic page, and the unit distance in the plane rectangular coordinate system may be a standard unit distance prestored by the learning device.

208. The learning device calculates a conversion relationship between the first coordinate system and the second coordinate system.

As an alternative embodiment, the manner in which the learning device calculates the conversion relation between the first coordinate system and the second coordinate system may include the steps of:

the learning equipment compares the second image with the target electronic page, determines the same characteristic points in the second image and the target electronic page, determines the characteristic points in the second image as first characteristic points, and determines the characteristic points in the target electronic page as second characteristic points;

The learning device determines a third coordinate of the first feature point in the first coordinate system and a fourth coordinate of the second feature point in the second coordinate system;

the learning device obtains a conversion relation from the third coordinate to the fourth coordinate by calculating a difference value between the third coordinate and the fourth coordinate;

the learning device determines the conversion relationship as a conversion relationship of the first coordinate system and the second coordinate system.

According to the embodiment, the same characteristic points can be determined from the second image and the target electronic page, the coordinates of the characteristic points under different coordinate systems are obtained, the conversion relation between the different coordinates is calculated, the conversion relation between the coordinates is determined to be the conversion relation of the different coordinate systems, and therefore the calculation mode of the conversion relation between the first coordinate system and the second coordinate system is simplified.

209. The learning device converts the first coordinates in the position information into second coordinates of the second coordinate system based on the conversion relation.

210. And the learning equipment determines the content to be clicked corresponding to the second coordinate from the target electronic page.

In the embodiment of the present invention, the above steps 207 to 210 are implemented, so that the conversion relationship between the first coordinate system and the second coordinate system can be determined, and the coordinates of the shielding object in the first image are converted into the second coordinate system of the second image, and then the content to be clicked is determined from the second image according to the coordinates of the shielding object in the second coordinate system, so as to avoid the unidentified content in the content to be clicked.

211. The learning device acquires standard voice corresponding to the content to be clicked.

In the embodiment of the invention, the learning equipment can pre-establish a voice library or can pre-establish connection with a third-party voice library, so that the learning equipment can quickly acquire standard voice corresponding to the content to be read.

212. The learning device outputs standard speech through a speaker.

213. When an error instruction input by a user is detected, the learning device controls the loudspeaker to stop outputting standard voice and outputs prompt information, wherein the prompt information is used for prompting the user to control the shielding object to contact with the learning page again and prompting the user to input a click-to-read mode starting instruction again so as to enable the learning device to recognize the content to be clicked again.

In the embodiment of the invention, a user can input an error instruction by clicking a virtual key or a physical key on the learning device, and can output voice containing the error instruction in a voice mode, so that the learning device can detect the error instruction input by the user, wherein the error instruction can prompt the standard pronunciation currently output by the learning device to be inconsistent with the content to be clicked by the user, and the learning device can be considered to recognize that the content to be clicked is wrong, so that the learning device is required to recognize the content to be clicked selected by the user again, and the learning device can prompt the user to execute the click operation again in a mode of outputting prompt information, so that the recognition of the learning device again is realized.

In the embodiment of the present invention, the steps 211 to 213 are implemented, so that the standard pronunciation of the content to be read selected by the user can be output, the user can also identify the correctness of the standard pronunciation output by the learning device, if the learning device outputs an incorrect pronunciation, the user can feed back to the learning device, so that the learning device identifies the content to be read selected by the user again according to the feedback, thereby the question number learning device outputs the accuracy of the standard pronunciation of the content to be read.

In the method described in fig. 2, the complete content to be read can be identified from the complete electronic page, so that the comprehensiveness of identifying the content to be read is improved. In addition, the method described in fig. 2 is implemented, so that the accuracy of identifying the target electronic page by the learning device is improved. Furthermore, the accuracy of determining the position of the shutter point may be improved by implementing the method described in fig. 2. In addition, the accuracy of the position recognition of the shielding object is improved by implementing the method described in fig. 2. In addition, the method described in fig. 2 is implemented, so that the calculation mode of the conversion relation between the first coordinate system and the second coordinate system is simplified. Furthermore, the method described in fig. 2 is implemented to avoid the presence of unrecognized content in the content to be read.

Example III

Referring to fig. 3, fig. 3 is a flowchart illustrating another method for determining contents to be read through according to an embodiment of the present invention. As shown in fig. 3, the method for determining the content to be read-on may include the following steps:

301. the learning device detects whether a shielding object is in contact with the learning page through the image acquisition device, and if so, the steps 302-303 are executed; if not, the process is ended.

In the embodiment of the invention, since the learning page is usually a paper page, the learning page is deformed when the shielding object contacts with the paper learning page, so that the learning device can determine whether the shielding object contacts with the learning page by detecting whether the learning page is deformed or not through the image acquisition device, thereby improving the reliability of the image acquisition device for detecting the contact of the shielding object with the learning page.

302. The learning device collects sounds in the environment where the learning device is located in the process that the shielding object is in contact with the learning page.

In the embodiment of the invention, the learning equipment can collect the sound in the environment where the learning equipment is located through the audio collection equipment (such as a microphone and the like), and furthermore, the learning equipment can also perform operations such as noise reduction, voice extraction and the like on the collected sound so as to make the voice in the finally obtained sound clearer.

303. The learning device identifies whether the voice contains voice information matched with the click mode starting instruction, and if so, the learning device executes steps 304-310; if not, the process is ended.

304. The learning device determines that the obstruction is in contact with the learning page and receives a click-to-read mode activation instruction.

In the embodiment of the invention, the steps 301 to 304 are implemented, so that whether the sound in the environment where the learning equipment is located triggers the click-to-read mode to be started or not can be further detected under the condition that the shielding object is detected to be in contact with the learning page, and the content to be clicked is determined only under the condition that the click-to-read mode is determined to be started, thereby avoiding that the learning equipment recognizes other instructions as the instruction for acquiring the content to be clicked, and improving the intelligence of the learning equipment.

305. When the shielding object is detected to be in contact with the learning page and a click-to-read mode starting instruction is received, the learning device controls the learning device to start the click-to-read mode, and a first image containing the shielding object and the learning page is acquired through the image acquisition device in the click-to-read mode.

306. The learning device obtains position information of the obstruction in the first image.

307. The learning device identifies a target sub-image corresponding to the occlusion from the first image and identifies a background color of the first image.

308. The learning device adjusts the color of the target sub-image in the first image to the background color, and determines the adjusted first image as the second image.

In the embodiment of the present invention, the steps 307 to 308 are implemented, so that the color of the shielding object portion in the first image can be adjusted to the background color of the first image, so that the elements interfering with the identification of the target electronic page are avoided in the adjusted first image, thereby improving the identification accuracy of the target electronic page.

309. The learning device selects a target electronic page that matches the second image.

310. And the learning equipment determines the content to be read from the target electronic page according to the position information.

In the method described in fig. 3, the complete content to be read can be identified from the complete electronic page, so that the comprehensiveness of identifying the content to be read is improved. In addition, the method described in fig. 3 is implemented, so that the intelligence of the learning device is improved. In addition, the method described in FIG. 3 is implemented, so that the identification accuracy of the target electronic page is improved.

Example IV

Referring to fig. 4, fig. 4 is a schematic structural diagram of a learning device according to an embodiment of the present invention. As shown in fig. 4, the learning device may include:

The first acquisition unit 401 is configured to control the learning device to start the click-to-read mode when detecting that the shielding object contacts the learning page and receiving a click-to-read mode start instruction, and acquire a first image including the shielding object and the learning page through the image acquisition device in the click-to-read mode.

A first acquiring unit 402, configured to acquire position information of the obstruction in the first image acquired by the first acquiring unit 401.

As an optional implementation manner, the manner in which the first obtaining unit 402 obtains the position information of the obstruction in the first image may specifically be:

identifying a first mark point in a first image, and determining a first main direction of the first mark point in the first image, wherein the first mark point can be any mark point existing in a learning page and an electronic page, and the main directions of the mark points in the learning page and the electronic page are the same;

determining a relative distance of the shield with respect to the first landmark and determining a relative angle of the shield with respect to a first main direction of the first landmark;

the relative distance and the relative angle are determined as position information of the obstruction in the first image.

In the implementation of the embodiment, the mark point can be determined in the first image, and the relative distance and the relative angle between the shielding object and the mark point are calculated, so that the position information of the shielding object in the first image is determined, and the correlation degree of the position information of the shielding object and the first image is improved.

And a rejecting unit 403, configured to reject an image corresponding to the occlusion object from the first image acquired by the first acquiring unit 401, so as to obtain a second image.

A selecting unit 404, configured to select a target electronic page that matches the second image obtained by the rejecting unit 403.

The first determining unit 405 is configured to determine, according to the location information acquired by the first acquiring unit 402, the content to be clicked from the target electronic page selected by the selecting unit 404.

As an optional implementation manner, the manner in which the first determining unit 405 determines the content to be clicked from the target electronic page according to the location information may specifically be:

identifying a second mark point in the target electronic page, and determining a second main direction of the second mark point in the target electronic page;

determining a target position in the second image, the distance from the second mark point being the relative distance and the angle with the second main direction being the relative angle, based on the relative angle and the relative distance contained in the position information;

and determining the content to be read corresponding to the target position from the target electronic page.

By implementing the embodiment, the target position corresponding to the position information can be determined from the target electronic page, so that the content to be clicked corresponding to the target position in the target electronic page is determined, and the position of the content to be clicked determined from the target electronic page is more accurate.

Therefore, the learning device described in fig. 4 can identify the complete content to be read from the complete electronic page, so as to promote the comprehensiveness of identifying the content to be read. In addition, implementing the learning apparatus described in fig. 4 can improve the correlation of the position information of the obstruction with the first image. In addition, the learning device described in fig. 4 is implemented, so that the position of the content to be clicked, which is determined from the target electronic page, can be more accurate.

Example five

Referring to fig. 5, fig. 5 is a schematic structural diagram of another learning device according to an embodiment of the present invention. The learning device shown in fig. 5 is optimized by the learning device shown in fig. 4. The learning apparatus shown in fig. 5 may further include:

the second obtaining unit 406 is configured to obtain, after the first determining unit 405 determines the content to be clicked from the target electronic page according to the location information, a standard voice corresponding to the content to be clicked.

A first output unit 407 for outputting the standard voice acquired by the second acquisition unit 406 through a speaker.

The second output unit 408 is configured to control the speaker to stop outputting the standard voice after the first output unit 407 outputs the standard voice through the speaker and when an error command input by the user is detected, and output a prompt message, where the prompt message is used to prompt the user to control the shielding object to contact with the learning page again, and prompt the user to input a click-to-read mode start command again, so that the learning device re-identifies the content to be clicked.

In the embodiment of the invention, the standard pronunciation of the content to be read selected by the user can be output, the user can also correctly identify the standard pronunciation output by the learning device, if the learning device outputs the wrong pronunciation, the user can feed back to the learning device, so that the learning device can again identify the content to be read selected by the user according to the feedback, and the question number learning device can output the accuracy of the standard pronunciation of the content to be read.

As an alternative embodiment, the first acquisition unit 402 of the learning apparatus shown in fig. 5 may include:

a first obtaining subunit 4021, configured to obtain a first coordinate system corresponding to the first image collected by the first collecting unit 401;

a first determining subunit 4022, configured to determine a first coordinate of the obstruction in the first image based on the first coordinate system acquired by the first acquiring subunit 4021;

a second determining subunit 4023 configured to determine the first coordinate determined by the first determining subunit 4022 as position information of the obstruction;

by implementing the embodiment, the coordinate system can be established in the first image, so that the learning device can determine the coordinates corresponding to any object in the first image, the learning device can quickly determine the position of the shielding object in the first image, and the accuracy of identifying the position of the shielding object is improved.

As an alternative embodiment, the first determining subunit 4022 may specifically determine, based on the first coordinate system, the first coordinate of the occlusion object in the first image by:

determining a target image corresponding to the shielding object from the first image;

identifying a target image and determining the outline of the target image;

the method comprises the steps of determining the top end of a shielding object corresponding to a target image through analyzing the outline of the target;

and determining the target coordinate of the top end of the shielding object according to the first coordinate system, and determining the target coordinate as the first coordinate of the shielding object in the first image.

In the implementation of the embodiment, the top end of the shielding object can be determined through an image recognition method, and the top end clicking of the shielding object is usually used when a user clicks the content to be read through the shielding object, so that the accuracy of determining the clicking position of the shielding object can be improved through determining the coordinates of the top end of the shielding object.

Alternatively, the calculation manner of the unit distance of the first coordinate system acquired by the first acquiring subunit 4021 may specifically be: obtaining a standard unit distance; and calculating an average distance between two adjacent characters from the first image; calculating the target proportion of the standard unit distance to the average distance; and reducing the standard unit distance according to the target proportion to obtain a target unit distance, and determining the target unit distance as the unit distance of the first coordinate system so as to enable the proportion of the coordinate system of the acquired first image to be matched with the standard proportion prestored by the learning equipment, thereby improving the accuracy of the learning equipment in identifying the target electronic page.

As an alternative embodiment, the first determining unit 405 of the learning device shown in fig. 5 may include:

a second obtaining subunit 4051, configured to obtain a second coordinate system corresponding to the target electronic page selected by the selecting unit 404;

a calculation subunit 4052, configured to calculate a conversion relationship between the first coordinate system acquired by the first acquisition subunit 4021 and the second coordinate system acquired by the second acquisition subunit 4051;

a conversion subunit 4053, configured to convert the first coordinate in the position information determined by the second determination subunit 4023 into a second coordinate of the second coordinate system based on the conversion relationship obtained by the calculation subunit 4052;

the third determining subunit 4054 is configured to determine, from the target electronic page selected by the selecting unit 404, the content to be clicked corresponding to the second coordinate obtained by the converting subunit 4053.

The embodiment can determine the conversion relation between the first coordinate system and the second coordinate system, so that the coordinates of the shielding object in the first image are converted into the second coordinate system of the second image, and the content to be clicked is determined from the second image according to the coordinates of the shielding object in the second coordinate system, so that the content to be clicked is prevented from being unidentified.

As an alternative embodiment, the calculating subunit 4052 may specifically calculate the conversion relationship between the first coordinate system and the second coordinate system by:

comparing the second image with the target electronic page, determining the same characteristic points in the second image and the target electronic page, determining the characteristic points in the second image as first characteristic points, and determining the characteristic points in the target electronic page as second characteristic points;

determining a third coordinate of the first feature point in the first coordinate system, and determining a fourth coordinate of the second feature point in the second coordinate system;

obtaining a conversion relation from the third coordinate to the fourth coordinate by calculating a difference value between the third coordinate and the fourth coordinate;

the conversion relation is determined as a conversion relation of the first coordinate system and the second coordinate system.

According to the embodiment, the same characteristic points can be determined from the second image and the target electronic page, the coordinates of the characteristic points under different coordinate systems are obtained, the conversion relation between the different coordinates is calculated, the conversion relation between the coordinates is determined to be the conversion relation of the different coordinate systems, and therefore the calculation mode of the conversion relation between the first coordinate system and the second coordinate system is simplified.

Therefore, the learning device described in fig. 5 can identify the complete content to be read from the complete electronic page, so as to promote the comprehensiveness of identifying the content to be read. In addition, the learning device described in fig. 5 is implemented, so that the accuracy of the learning device in identifying the target electronic page is improved. Furthermore, implementing the learning device described in fig. 5 may improve the accuracy of determining the position of the obstacle click. In addition, the learning device described in fig. 5 is implemented, so that accuracy of identifying the position of the shielding object is improved. In addition, the learning device described in fig. 5 is implemented, and the calculation manner of the conversion relation between the first coordinate system and the second coordinate system is simplified. Furthermore, the learning device described in fig. 5 is implemented to avoid the presence of unidentified content in the content to be read-on.

Example six

Referring to fig. 6, fig. 6 is a schematic structural diagram of another learning device according to an embodiment of the present invention. The learning device shown in fig. 6 is optimized by the learning device shown in fig. 5. The culling unit 403 of the learning apparatus shown in fig. 6 may include:

the identifying subunit 4031 is configured to identify, from the first image acquired by the first acquisition unit 401, a target sub-image corresponding to the occlusion object, and identify a background color of the first image.

An adjustment subunit 4032 for adjusting the color of the target sub-image in the first image identified by the identification subunit 4031 to the background color, and determining the adjusted first image as the second image.

In the embodiment of the invention, the color of the shielding object part in the first image can be adjusted to the background color of the first image, so that the elements interfering with the identification of the target electronic page are avoided in the adjusted first image, and the identification accuracy of the target electronic page is improved.

As an alternative embodiment, the learning device shown in fig. 6 may further include:

a detection unit 409, configured to detect, by using the image acquisition device, whether there is a contact between the shielding object and the learning page before controlling the learning device to start the click-to-read mode when the first acquisition unit 401 detects that the shielding object is in contact with the learning page and receives a click-to-read mode start instruction;

a second collecting unit 410, configured to collect, when the detection result of the detecting unit 409 is yes, sound in an environment where the learning device is located in a process that the shielding object contacts the learning page;

a recognition unit 411, configured to recognize whether the voice collected by the second collection unit 410 includes voice information that matches the click-to-read mode on instruction;

The second determining unit 412 is configured to determine that the obstruction is in contact with the learning page and receive the click-to-read mode start instruction when the recognition result of the recognizing unit 411 is yes.

According to the embodiment, whether the sound in the environment where the learning equipment is located triggers the click-to-read mode to be started or not can be further detected under the condition that the shielding object is detected to be in contact with the learning page, and the content to be clicked is determined only under the condition that the click-to-read mode is determined to be started, so that the learning equipment is prevented from recognizing other instructions as instructions for acquiring the content to be clicked, and the intelligence of the learning equipment is improved.

Therefore, the learning device described in fig. 6 can identify the complete content to be read from the complete electronic page, so as to promote the comprehensiveness of identifying the content to be read. In addition, the learning device described in fig. 6 is implemented, and the intelligence of the learning device is improved. In addition, the learning device described in fig. 6 is implemented, so that the recognition accuracy of the target electronic page is improved.

Example seven

Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the invention. As shown in fig. 7, the electronic device may include:

a memory 701 storing executable program code;

A processor 702 coupled with the memory 701;

wherein the processor 702 invokes executable program code stored in the memory 701 to perform some or all of the steps of the methods in the method embodiments above.

The embodiment of the invention also discloses a computer readable storage medium, wherein the computer readable storage medium stores program code, and the program code comprises instructions for executing part or all of the steps of the method in the above method embodiments.

The embodiments of the present invention also disclose a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform some or all of the steps of the method as in the method embodiments above.

The embodiment of the invention also discloses an application release platform, wherein the application release platform is used for releasing a computer program product, and the computer program product is used for enabling the computer to execute part or all of the steps of the method in the method embodiments.

It should be appreciated that reference throughout this specification to "an embodiment of the invention" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in an embodiment of the invention" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art will also appreciate that the embodiments described in the specification are alternative embodiments and that the acts and modules referred to are not necessarily required for the present invention.

In various embodiments of the present invention, it should be understood that the sequence numbers of the foregoing processes do not imply that the execution sequences of the processes should be determined by the functions and internal logic of the processes, and should not be construed as limiting the implementation of the embodiments of the present invention.

In addition, the terms "system" and "network" are often used interchangeably herein. It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B, and may mean: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

In the embodiments provided herein, it should be understood that "B corresponding to a" means that B is associated with a, from which B can be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information.

Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program that instructs associated hardware, the program may be stored in a computer readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disk Memory, magnetic disk Memory, tape Memory, or any other medium that can be used for carrying or storing data that is readable by a computer.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, or a part contributing to the prior art or all or part of the technical solution, may be embodied in the form of a software product stored in a memory, comprising several requests for a computer device (which may be a personal computer, a server or a network device, etc., in particular may be a processor in a computer device) to execute some or all of the steps of the above-mentioned method of the various embodiments of the present invention.

The above describes in detail a method for determining content to be read-through and learning equipment disclosed in the embodiments of the present invention, and specific examples are applied to describe the principles and embodiments of the present invention, where the description of the above embodiments is only used to help understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (10)

1. A method for determining content to be read-on, the method comprising:

when a shielding object is detected to be in contact with a learning page and a click-to-read mode starting instruction is received, controlling learning equipment to start a click-to-read mode, and acquiring a first image containing the shielding object and the learning page through image acquisition equipment in the click-to-read mode;

acquiring position information of the shielding object in the first image, wherein the position information comprises a relative distance of the shielding object relative to a preset first mark point, and determining a relative angle of the shielding object relative to a first main direction of the first mark point;

Removing the image corresponding to the shielding object from the first image to obtain a second image;

selecting a target electronic page matched with the second image;

determining the content to be read from the target electronic page according to the position information;

the determining the content to be clicked from the target electronic page according to the position information comprises the following steps:

identifying a second mark point in the target electronic page, and determining a second main direction of the second mark point in the target electronic page;

determining a target position in the second image, the distance from the second mark point being the relative distance and the angle with the second main direction being the relative angle, based on the relative angle and the relative distance contained in the position information;

and determining the content to be clicked corresponding to the target position from the target electronic page.

2. The method of claim 1, wherein after determining the content to be clicked from the target electronic page according to the location information, the method further comprises:

acquiring standard voice corresponding to the content to be read;

outputting the standard voice through a loudspeaker;

And, after the outputting of the standard speech through the speaker, the method further comprises:

when an error instruction input by a user is detected, the loudspeaker is controlled to stop outputting the standard voice, and prompt information is output, wherein the prompt information is used for prompting the user to control the shielding object to contact with the learning page again, and prompting the user to input the click-to-read mode starting instruction again so as to enable the learning device to recognize the content to be clicked again.

3. The method according to claim 1 or 2, wherein the acquiring the position information of the obstruction in the first image comprises:

acquiring a first coordinate system corresponding to the first image;

determining a first coordinate of the shielding object in the first image according to the first coordinate system;

determining the first coordinate as position information of the shielding object;

and determining the content to be clicked from the target electronic page according to the position information, including:

acquiring a second coordinate system corresponding to the target electronic page;

calculating the conversion relation between the first coordinate system and the second coordinate system;

converting the first coordinate in the position information into a second coordinate of the second coordinate system according to the conversion relation;

And determining the content to be clicked corresponding to the second coordinate from the target electronic page.

4. A method according to any one of claims 1 to 3, wherein said removing the image corresponding to the occlusion from the first image to obtain a second image comprises:

identifying a target sub-image corresponding to the shielding object from the first image, and identifying the background color of the first image;

and adjusting the color of the target sub-image in the first image to be the background color, and determining the adjusted first image to be a second image.

5. The method according to any one of claims 1 to 4, wherein when the obstacle is detected to be in contact with the learning page and a click-to-read mode on instruction is received, the method further comprises, before controlling the learning device to turn on the click-to-read mode:

detecting whether a shielding object is in contact with a learning page or not through image acquisition equipment;

if the shielding object is in contact with the learning page, collecting sound in the environment where the learning equipment is located in the process that the shielding object is in contact with the learning page;

identifying whether the voice contains voice information matched with a click-to-read mode starting instruction;

If so, determining that the shielding object is in contact with the learning page and receiving the click-to-read mode starting instruction.

6. A learning apparatus, characterized by comprising:

the first acquisition unit is used for controlling the learning equipment to start a click-to-read mode when detecting that a shielding object is in contact with a learning page and receiving a click-to-read mode starting instruction, and acquiring a first image containing the shielding object and the learning page through the image acquisition equipment in the click-to-read mode;

a first obtaining unit, configured to obtain position information of the shielding object in the first image, where the position information includes a relative distance between the shielding object and a preset first mark point, and determine a relative angle between the shielding object and a first main direction of the first mark point;

the rejecting unit is used for rejecting the image corresponding to the shielding object from the first image to obtain a second image;

a selecting unit, configured to select a target electronic page that matches the second image;

the first determining unit is used for determining the content to be clicked from the target electronic page according to the position information;

the first determining unit is specifically configured to:

Identifying a second mark point in the target electronic page, and determining a second main direction of the second mark point in the target electronic page;

determining a target position in the second image, the distance from the second mark point being the relative distance and the angle with the second main direction being the relative angle, based on the relative angle and the relative distance contained in the position information;

and determining the content to be clicked corresponding to the target position from the target electronic page.

7. The learning device of claim 6, wherein the learning device further comprises:

the second acquisition unit is used for acquiring standard voice corresponding to the content to be clicked after the first determination unit determines the content to be clicked from the target electronic page according to the position information;

a first output unit for outputting the standard voice through a speaker;

and, the learning apparatus further includes:

the second output unit is used for controlling the loudspeaker to stop outputting the standard voice and outputting prompt information after the first output unit outputs the standard voice through the loudspeaker and when an error instruction input by a user is detected, wherein the prompt information is used for prompting the user to control the shielding object to contact with the learning page again and prompting the user to input the click-to-read mode starting instruction again so as to enable the learning device to recognize the content to be clicked again.

8. The learning apparatus according to claim 6 or 7, characterized in that the first acquisition unit includes:

the first acquisition subunit is used for acquiring a first coordinate system corresponding to the first image;

a first determining subunit, configured to determine, based on the first coordinate system, a first coordinate of the occlusion object in the first image;

a second determining subunit, configured to determine the first coordinate as position information of the obstruction;

and, the first determination unit includes:

the second acquisition subunit is used for acquiring a second coordinate system corresponding to the target electronic page;

a calculating subunit, configured to calculate a conversion relationship between the first coordinate system and the second coordinate system;

a conversion subunit, configured to convert the first coordinate in the location information into a second coordinate of the second coordinate system based on the conversion relationship;

and the third determining subunit is used for determining the content to be clicked corresponding to the second coordinate from the target electronic page.

9. The learning apparatus as claimed in any one of claims 6 to 8, characterized in that the culling unit includes:

the identification subunit is used for identifying a target sub-image corresponding to the shielding object from the first image and identifying the background color of the first image;

And the adjustment subunit is used for adjusting the color of the target sub-image in the first image to the background color and determining the adjusted first image as a second image.

10. The learning apparatus as claimed in any one of claims 6 to 9, characterized in that the learning apparatus further includes:

the detection unit is used for detecting whether the shielding object is in contact with the learning page or not through the image acquisition device before controlling the learning device to start the point reading mode when the first acquisition unit detects that the shielding object is in contact with the learning page and receives a point reading mode starting instruction;

the second acquisition unit is used for acquiring sound in the environment where the learning equipment is located in the process that the shielding object is in contact with the learning page when the detection result of the detection unit is yes;

the identifying unit is used for identifying whether the voice contains voice information matched with the click-to-read mode starting instruction;

and the second determining unit is used for determining that the shielding object is contacted with the learning page and receiving the click-to-read mode starting instruction when the identification result of the identifying unit is yes.