CN111563497A

CN111563497A - Frame question method and device based on movement track, electronic equipment and storage medium

Info

Publication number: CN111563497A
Application number: CN202010366625.5A
Authority: CN
Inventors: 曾菲
Original assignee: Guangdong Genius Technology Co Ltd
Current assignee: Guangdong Genius Technology Co Ltd
Priority date: 2020-04-30
Filing date: 2020-04-30
Publication date: 2020-08-21
Anticipated expiration: 2040-04-30
Also published as: CN111563497B

Abstract

The embodiment of the invention relates to the technical field of intelligent equipment, and discloses a frame question method and device based on a movement track, electronic equipment and a storage medium. The method comprises the following steps: when a frame question instruction is received, starting a camera to identify the moving track of an operating body on a bearing body; photographing the bearing body by using the moving track and a preset rule to obtain a coarse frame image; when the moving track is a continuous track, determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image, and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image; when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line, respectively constructing a first straight line and a second straight line based on the starting point coordinate and the end point coordinate; and constructing a selection frame and acquiring the picture of the characters in the selection frame. By implementing the embodiment of the invention, the selection frame can be obtained by only constructing the first straight line and the second straight line based on the starting point coordinate and the end point coordinate, so that the operation is more convenient.

Description

Frame question method and device based on movement track, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of intelligent equipment, in particular to a method and a device for framing a question based on a moving track, electronic equipment and a storage medium.

Background

In order to solve the difficult problems of the homework of primary and secondary schools, a plurality of application programs for searching questions or gathering wrong questions for solving the difficult problems of the homework of students appear on the market at present, and all the application programs shoot the question pictures through a camera to be stored as wrong questions or used for searching corresponding answers. The implementation methods thereof are roughly divided into three types:

the first method is to select the question desired by the user in a mode of manually dragging the user to a cutting frame, and the cutting frame is required to be operated for multiple times in the mode so that the cutting frame is enlarged or reduced, and the question picture can be accurately acquired.

The second method is that a point is selected by using an operation body on a bearing body, then a picture is taken according to the point and a question picture is obtained by a certain training model based on a preset rule, which is more intelligent than the first method, but because the recognition phenomenon is usually concentrated in the range of one point, the obtained content picture is possibly incomplete or invalid information exists, and the result of searching the question is easy to make mistakes.

The third method is a track-based method, for example, an operation body is moved on a carrier, so that a moving track forms a closed curve frame, a camera recognizes the moving track, and the smart device selects a content frame in the closed curve frame based on the moving track, or a rectangular frame is constructed on the carrier by taking a start point coordinate and an end point coordinate of content to be framed as two ends of the moving track, and then a diagonal line is formed based on the start point coordinate and the end point coordinate, so as to obtain content in the rectangular frame.

In the method of closing the curve frame, when the operation body is moved, the operation body is not displayed explicitly, and the situation that the drawing of the user is incomplete easily occurs, the intelligent terminal cannot recognize the selection frame and may need to interact with the user for many times to complete, so that the user experience is affected, and for the situation that the start point coordinate and the end point coordinate are used as diagonal lines, if the first character of the content to be framed is not retracted or the last character is at the end of the last character, it is easy to rectangle the frame and determine the framed content, as shown in fig. 1, the rectangle 14 is obtained according to the diagonal line formed by the start point coordinate 12 and the end point coordinate 13 of the movement track 11. In fact, many times, the first and last lines of the subject to be framed may not occupy the same line as the middle line, and in this case, the starting point coordinate and the ending point coordinate have no reference object, and the content in the obtained rectangular frame is likely to be incomplete, as shown in fig. 2, since the positions of the starting point and the ending point are not easy to grasp, the rectangular frame 24 obtained from the diagonal line formed by the starting point coordinate 22 and the ending point coordinate 23 of the moving track 21 cannot be framed to obtain the complete content.

Disclosure of Invention

Aiming at the defects, the embodiment of the invention discloses a frame question method and device based on a moving track, electronic equipment and a storage medium, which ensure that the frame question is accurate and the operation is more convenient.

The first aspect of the embodiment of the invention discloses a frame question method based on a moving track, which comprises the following steps:

when a frame question instruction is received, a camera is started to identify the moving track of a user on a bearing body through an operation body;

photographing the bearing body by using the moving track and a preset rule to obtain a coarse frame image;

when the moving track is a continuous track, determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image, and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image;

when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line, respectively constructing a first straight line and a second straight line which are parallel to the reference straight line based on the starting point coordinate and the end point coordinate;

and constructing a selection frame by using the first straight line, the second straight line and the edge of the rough frame image between the first straight line and the second straight line, and acquiring a picture of the character in the selection frame.

As an alternative implementation, in the first aspect of the embodiment of the present invention, the constructing a first straight line and a second straight line parallel to the reference straight line based on the start point coordinate and the end point coordinate respectively includes:

detecting whether the starting point coordinate and the end point coordinate fall on a character or not;

and if the starting point coordinate and the end point coordinate do not fall on the character, the constructed first straight line and the second straight line respectively pass through the starting point coordinate and the end point coordinate.

As an alternative implementation, in the first aspect of the embodiment of the present invention, if the start point coordinate or/and the end point coordinate falls on a character, then:

establishing a plane coordinate system of the coarse frame image by taking the base alignment line or a straight line parallel to the base alignment line as an abscissa axis;

determining the size of the longitudinal coordinate values of the starting point coordinate and the end point coordinate;

if the ordinate of the point falling on the character is a large value, adding a first preset value on the basis of the ordinate of the large value as a point through which a first straight line passes;

and if the ordinate of the point falling on the character is a smaller value, subtracting a second preset value on the basis of the ordinate of the smaller value to be used as a point through which a second straight line passes.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, a first character of a to-be-framed question is used as a start point coordinate of a movement track, and a last character of the to-be-framed question is used as an end point coordinate of the movement track; taking a line segment between the first character and the last character of the subject to be framed as a moving track;

adding a first preset value on the basis of the ordinate of the starting point coordinate to serve as a point through which a first straight line passes;

and subtracting a second preset value on the basis of the ordinate of the end point coordinate to serve as a point through which a second straight line passes.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, after the obtaining the coarse frame image, the method further includes: and correcting the coarse frame image to enable the coarse frame image to be rectangular.

As an alternative implementation manner, in the first aspect of the embodiment of the present invention, if a connection line between the start point coordinate and the end point coordinate is parallel to a reference straight line, a picture of a character through which the connection line between the start point coordinate and the end point coordinate passes is obtained.

The second aspect of the embodiments of the present invention discloses a question framing device based on a movement trajectory, which includes:

the identification unit is used for starting the camera to identify the moving track of the user on the bearing body through the operation body when the frame question instruction is received;

the photographing unit is used for photographing the bearing body by utilizing the moving track and a preset rule to obtain a coarse frame image;

the acquisition unit is used for determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image when the moving track is a continuous track;

the construction unit is used for respectively constructing a first straight line and a second straight line which are parallel to the reference straight line based on the starting point coordinate and the end point coordinate when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line;

and the frame selection unit is used for constructing a selection frame by using the first straight line, the second straight line and the edge of the thick frame image between the first straight line and the second straight line, and acquiring the picture of the characters in the selection frame.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the building unit includes:

the detection subunit is used for detecting whether the start point coordinate and the end point coordinate fall on the character;

the first judgment subunit is used for constructing a first straight line and a second straight line which respectively pass through the start point coordinate and the end point coordinate if the start point coordinate and the end point coordinate do not fall on the character;

a second judging subunit, configured to, if the start point coordinate or/and the end point coordinate falls on a character: establishing a plane coordinate system of the coarse frame image by taking the base alignment line or a straight line parallel to the base alignment line as an abscissa axis; determining the size of the longitudinal coordinate values of the starting point coordinate and the end point coordinate; if the ordinate of the point falling on the character is a large value, adding a first preset value on the basis of the ordinate of the large value as a point through which a first straight line passes; and if the ordinate of the point falling on the character is a smaller value, subtracting a second preset value on the basis of the ordinate of the smaller value to be used as a point through which a second straight line passes.

A third aspect of an embodiment of the present invention discloses an electronic device, including: a memory storing executable program code; a processor coupled with the memory; the processor calls the executable program code stored in the memory to execute the frame topic method based on the movement track disclosed by the first aspect of the embodiment of the invention.

A fourth aspect of the embodiments of the present invention discloses a computer-readable storage medium storing a computer program, where the computer program enables a computer to execute the method for framing a topic based on a movement trajectory disclosed in the first aspect of the embodiments of the present invention.

A fifth aspect of the embodiments of the present invention discloses a computer program product, which, when running on a computer, causes the computer to execute the method for framing a topic based on a movement trajectory disclosed in the first aspect of the embodiments of the present invention.

A sixth aspect of the present invention discloses an application publishing platform, where the application publishing platform is configured to publish a computer program product, and when the computer program product runs on a computer, the computer is enabled to execute a frame problem method based on a movement track disclosed in the first aspect of the present invention.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, when a frame question instruction is received, a camera is started to identify the moving track of a user on a bearing body through an operation body; photographing the bearing body by using the moving track and a preset rule to obtain a coarse frame image; when the moving track is a continuous track, determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image, and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image; when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line, respectively constructing a first straight line and a second straight line which are parallel to the reference straight line based on the starting point coordinate and the end point coordinate; and constructing a selection frame by using the first straight line, the second straight line and the edge of the rough frame image between the first straight line and the second straight line, and acquiring a picture of the character in the selection frame. Therefore, by implementing the embodiment of the invention, the selection frame can be obtained only by constructing the first straight line and the second straight line based on the starting point coordinate and the end point coordinate, so that accurate frame-based theme fetching is ensured, the operation is more convenient, and the user experience can be improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic diagram of a conventional block diagram method;

FIG. 2 is a schematic diagram of another structure of the conventional block method;

FIG. 3 is a flow chart of a block method based on a moving track according to an embodiment of the present invention;

FIG. 4 is a flow chart of another block method based on a moving track according to the embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating an example of a block method based on a moving track according to an embodiment of the present invention;

FIG. 6 is a schematic diagram illustrating an example of another block method based on a moving track according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a block device based on a moving track according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second", "third", "fourth", and the like in the description and the claims of the present invention are used for distinguishing different objects, and are not used for describing a specific order. The terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a frame question method and device based on a moving track, electronic equipment and a storage medium, wherein a selection frame can be obtained only by constructing a first straight line and a second straight line according to a starting point coordinate and an end point coordinate of the moving track, the operation is very simple and convenient, the completeness of a question can be ensured, the user experience is improved, and the following detailed description is combined with the attached drawings.

Example one

Referring to fig. 3, fig. 3 is a flowchart illustrating a method for framing a frame based on a moving track according to an embodiment of the present invention. The method described in the embodiment of the invention is suitable for intelligent devices such as learning machines, home education machines, point reading machines, tablet computers or mobile phones, and can also be suitable for desktop computers, portable computers, servers and the like. The frame question is a content part of the frame question selected by the selection frame and extracted from the image. As shown in fig. 3, the frame problem method based on the movement track includes the following steps:

110. and when a frame question instruction is received, starting the camera to identify the moving track of the user on the bearing body through the operation body.

The frame questions are used in learning, and the purpose of the frame questions can be to search for questions, store wrong questions, knowledge points and the like. The framing instruction is initiated by a user, i.e., an operator, and may be a voice instruction, or a pulse instruction formed by triggering by the user through a touch screen or a mechanical button, or formed by a specific gesture, which is not limited herein. Before the frame question instruction is received, most devices of the camera and the intelligent device are in a sleep state, so that the electric quantity can be saved, and the camera and the intelligent device are awakened through the frame question instruction. The camera can be a front camera or a rear camera carried by the intelligent equipment, and can also be an external camera separated from the intelligent equipment and in communication connection with the intelligent equipment.

The user operates on the bearing body through the operation body, the camera identifies the user intention, and the content on the bearing body can be photographed according to the user intention.

After receiving the frame question instruction, the intelligent equipment starts the camera and identifies the moving track of the user on the bearing body through the operation body. After the camera is started, a display screen on the intelligent device can display a preview image which can be collected by the camera, the image cannot be stored, the camera recognizes that the movement track of the user on the carrying body by using the operating body is realized through an affine transformation algorithm, namely, the coordinates on the carrying body are converted into the coordinates of the preview image, and the movement track recognized by the camera is the movement track on the preview image.

The carrier is exemplarily a carrier of the contents to be framed, and may be a book, an exercise book, a test paper, and the like. The operation body is used for operating on the bearing body, and further positioning of the frame selection content is achieved according to the position of the positioning operation body, and the operation body can be a finger, a touch pen, a pencil and the like, or even a small stick, and is not limited here.

120. And photographing the bearing body by utilizing the moving track and a preset rule to obtain a coarse frame image.

The thick frame image at least comprises all the subject contents covered by the moving track, and also comprises some invalid information which is irrelevant to the subject to be framed. Thus, the preset rule may specify a certain section, for example, 5 lines of characters are drawn up based on the start point coordinates, 3 lines of characters are drawn down based on the end point coordinates, and these contents are framed.

The thick frame image obtained by the camera according to the moving track and the preset rule is actually completed based on the preview image, but the process is not sensible to the user. Obviously, the moving track can be completely embodied in the thick frame image.

130. And when the moving track is a continuous track, determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image, and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image.

The moving track aimed at by the embodiment of the invention is continuous, and if the middle is interrupted, a plurality of starting point coordinates and end point coordinates appear, so that the identification cannot be realized.

Before determining the start point coordinates and the end point coordinates, the rough frame image is preferably preprocessed to ensure the accuracy of OCR recognition and the accuracy of the selection frame. The preprocessing includes but is not limited to denoising, contrast enhancement, shape correction and the like, the shape correction mainly aims at the problem of the visual angle of a camera to shoot a trapezoidal image or the curling condition of a bearing body, the shape correction can be realized by stretching the edge of a thick frame image and the like, and the finally obtained thick frame image is rectangular.

The coordinates of the start point and the coordinates of the end point of the movement trajectory are obtained by the coordinate conversion method described in step 110.

The arbitrary line includes a line where the start point coordinate is located or a line where the end point coordinate is located, and the obtaining of the reference straight line is divided into two steps: the coarse frame image is first subjected to character Recognition, and the recognized characters can be realized by mature OCR (optical character Recognition) technology, for example, and the characters include Chinese characters, letters, punctuation marks, formulas and the like. And then obtaining the position coordinates of the characters in the arbitrary line, wherein the connection line of the position coordinates of the characters can obtain a reference straight line.

When any row is selected, the row with only one character is excluded, preferably the row with 5 or more characters is excluded, and the obtained reference straight line is relatively accurate. The mode of setting the reference straight line can perform framing without adjusting the angle of the coarse frame image or the title when the character is inclined, so that the processing speed is increased.

140. And when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line, respectively constructing a first straight line and a second straight line parallel to the reference straight line through the starting point coordinate and the end point coordinate.

The embodiment of the invention is suitable for the condition that the moving track is a curve, and under the condition that the selection frame is a plurality of lines, because the moving process of the moving track is not considered by the invention, the selection frame can be obtained even if the user adopts the wavy line type movement for improving the interestingness.

Specifically, a connecting line between the start point coordinate and the end point coordinate is not necessarily a moving track, and if the connecting line is not parallel to the reference straight line, it indicates that the start point coordinate and the end point coordinate are not on the same straight line with respect to the reference straight line, and then the subject to be framed has a plurality of lines of characters, and then a first straight line and a second straight line parallel to the reference straight line are constructed based on the start point coordinate and the end point coordinate.

In the embodiment of the present invention, the first straight line and the second straight line are constructed according to the setting rule of the start point coordinate and the end point coordinate. In the first embodiment, the start point coordinate and the end point coordinate do not fall on the character, for example, after the user issues a frame question instruction, the smart device may prompt the user to place the start point coordinate in a space above a first character of the frame to be selected, and place the end point coordinate in a space below a last character of the frame to be selected.

Because the starting point coordinate and the end point coordinate do not fall on the character, as long as the first straight line and the second straight line are constructed through the starting point coordinate and the end point coordinate, the first straight line and the second straight line can not be intersected with the character in the frame to be selected, and the integrity of the frame question is ensured.

150. And constructing a selection frame by using the first straight line, the second straight line and the edge of the rough frame image between the first straight line and the second straight line, and acquiring a picture of the character in the selection frame.

The upper and lower sidelines of the selection frame are obtained, and the sidelines on the left and right sides can be directly used as the sidelines of the thick frame image.

Other marks, such as page numbers or separators, and the like, exist on two side edges of the carrier body, and can be smeared through a neural network model or a predetermined rule, for example, when isolated characters exist in a recognized selection box and when the isolated characters are page numbers, separators and other mark type characters, the characters are directly smeared by using a background template.

It should be noted that the embodiment of the present invention cannot accurately frame the multi-column typeset carrier.

By implementing the embodiment of the invention, the selection frame can be obtained based on the starting point coordinate and the end point coordinate, the user can obtain the selection frame only by placing the starting point coordinate in the blank above the first character of the selection frame and placing the end point coordinate in the blank below the last character of the selection frame, then the picture of the title is obtained according to the selection frame, and the picture of the title can be stored or searched, and the like. If the characters in the selection frame are inclined, the selection frame is a parallelogram, various operations such as rotation of a thick frame image are not needed, the operation is very simple and convenient for a user, incomplete frame questions or invalid frame information can be avoided, the frame question efficiency is improved, and the user experience is improved.

Example two

Referring to fig. 4, fig. 4 is a flowchart illustrating a method for framing a frame based on a moving track according to an embodiment of the present invention. The method described in the embodiment of the invention is suitable for intelligent devices such as learning machines, home education machines, point reading machines, tablet computers or mobile phones, and can also be suitable for desktop computers, portable computers, servers and the like. The frame question is a content part of the frame question selected by the selection frame and extracted from the image. As shown in fig. 4, the frame problem method based on the movement track includes the following steps:

210. and when a frame question instruction is received, starting the camera to identify the moving track of the user on the bearing body through the operation body.

220. And photographing the bearing body by utilizing the moving track and a preset rule to obtain a coarse frame image.

230. And when the moving track is a continuous track, determining a starting point coordinate and an end point coordinate of the moving track in the coarse frame image, and acquiring a reference straight line formed by any row of characters between the starting point coordinate and the end point coordinate in the coarse frame image.

Steps 210-230 are similar to steps 110-130 in the first embodiment, and are not described herein.

240. And when a connecting line between the starting point coordinate and the end point coordinate is not parallel to the reference straight line, respectively constructing a first straight line and a second straight line which are parallel to the reference straight line based on the starting point coordinate, the end point coordinate and a preset value.

In the embodiment of the present invention, the first straight line and the second straight line are constructed according to the setting rule of the start point coordinate and the end point coordinate. In the second embodiment, one or both of the coordinates of the start point and the coordinates of the end point fall on the character. If one of the start point coordinate or the end point coordinate does not fall on the character, the point is directly constructed into a straight line in the first embodiment, and the straight line corresponding to the remaining point is realized by the following way:

Taking the example that the start point coordinate and the end point coordinate both fall on the character, this way is also the most convenient way for the user to operate and is not easy to make mistakes. For example, after the user issues a box topic instruction, the smart device may prompt the user to place the start point coordinate on the first character of the box to be selected and to place the end point coordinate on the last character of the box to be selected.

Because the starting point coordinate and the end point coordinate are both located on the characters, the user operation is clear, and if the starting point coordinate and the end point coordinate are directly used as passing points of the first straight line and the second straight line, only part of the first line and the last line can be framed. In this case, the following operation may be adopted: adding a first preset value on the basis of the ordinate of the starting point coordinate to serve as a point through which a first straight line passes; and subtracting a second preset value on the basis of the ordinate of the end point coordinate to serve as a point through which a second straight line passes.

The first preset value and the second preset value can be the same or different and are set according to requirements. Preferably, the first straight line is located above the first line, the second straight line is located below the last line, and the first straight line and the second straight line are not intersected with irrelevant information in the thick frame image.

If the user places the start point on the last character of the last line, the end point on the first character of the first line, or any other character that places the start point on the first line or the last line, and the end point on the other characters of the last line or the first line, for example, places the start point and the end point on the first character of the first line and the last line, respectively, the same applies using the above-described method of establishing a planar coordinate system.

250. And constructing a selection frame by using the first straight line, the second straight line and the edge of the rough frame image between the first straight line and the second straight line, and acquiring a picture of the character in the selection frame.

Step 250 is similar to step 150 in the first embodiment, and is not described here again.

Fig. 5 is a schematic diagram illustrating a structure of generating a selection box by using the first character and the last character as the start point and the end point of the movement trajectory. In fig. 5, the start point coordinate 311 of the movement trajectory 310 is located on "total", and the end point coordinate 312 is located ". In the above, the connection line of the middle row of characters forms a reference straight line 320, and meanwhile, the reference straight line 320 is used as an abscissa axis to establish a coordinate system, so that a first straight line 330 is obtained according to a first preset value and a start point coordinate, a second straight line 340 is obtained according to a second preset value and an end point coordinate, a selection frame is formed by a left edge 350 and a right edge 360 of a coarse frame image between the first straight line and the second straight line, and the picture of the characters in the selection frame is selected. If the direction of the character to be framed is parallel to the upper edge or the lower edge of the image with the thick frame, the selection frame is rectangular, and if the character to be framed is inclined, as shown in fig. 6, the finally obtained selection frame is parallelogram.

By implementing the embodiment of the invention, the selection frame can be obtained based on the starting point coordinate, the end point coordinate and the preset value, the user can obtain the selection frame as long as the starting point and the terminal are respectively arranged on any characters of the first line and the last line of the selection frame, then the picture of the title is obtained according to the selection frame, and the picture of the title can be stored or searched, and the like. If the characters in the selection frame are inclined, the selection frame is a parallelogram, various operations such as rotation of a thick frame image are not needed, the operation is very simple and convenient for a user, incomplete frame questions or invalid frame information can be avoided, the frame question efficiency is improved, and the user experience is improved.

EXAMPLE III

As a special case, if the subject to be framed has only one line, the movement track is parallel to the reference straight line, and at this time, only the picture of the character through which the movement track passes is obtained.

It should be noted that, since the movement locus of the user in the question is difficult to be drawn as a straight line, the vertical coordinates of the start point coordinate and the end point coordinate in the above-mentioned plane coordinate system are not exactly equal to each other. Therefore, in the preferred embodiment of the present invention, when the included angle between the connecting line between the start point coordinate and the end point coordinate and the reference straight line is within the preset included angle range, the connecting line is considered to be parallel to the reference straight line. In addition, under certain conditions, part of characters may not be on the moving track due to shaking when a user draws a line, and therefore, if the characters passing through the moving track are used as the frame question content, the situation that the characters are incomplete may exist, so that the characters passing through the connecting line between the starting point coordinate and the end point coordinate are used as the frame question content, then the preset included angle is used as an aid, the selection range of the preset included angle is used for ensuring that the connecting line between the starting point coordinate and the end point coordinate passes through all the characters between the starting point coordinate and the end point coordinate, and finally the characters passing through the connecting line between the starting point coordinate and the end point coordinate are used as the frame question content to ensure the integrity.

Example four

Referring to fig. 7, fig. 7 is a schematic structural diagram of a frame topic device based on a moving track according to an embodiment of the present invention. As shown in fig. 7, the block device based on the movement track may include:

the identification unit 410 is used for starting the camera to identify the moving track of the user on the carrier through the operation body when receiving the frame question instruction;

the photographing unit 420 is configured to photograph the carrier by using the moving track and a preset rule to obtain a coarse frame image;

an obtaining unit 430, configured to determine a start point coordinate and an end point coordinate of the moving trajectory in the thick frame image when the moving trajectory is a continuous trajectory, and obtain a reference straight line formed by any row of characters between the start point coordinate and the end point coordinate in the thick frame image;

a constructing unit 440 configured to construct a first straight line and a second straight line parallel to the reference straight line based on the start point coordinate and the end point coordinate, respectively, when a connection line between the start point coordinate and the end point coordinate is not parallel to the reference straight line;

and the frame selecting unit 450 is configured to construct a selection frame by using the first straight line and the second straight line and the edge of the bold frame image between the first straight line and the second straight line, and acquire a picture of the character in the selection frame.

As an alternative embodiment, the building unit 440 includes:

a detecting subunit 441, configured to detect whether the start point coordinate and the end point coordinate fall on a character;

a first judging subunit 442, configured to, if the start point coordinate and the end point coordinate do not fall on the character, pass through the start point coordinate and the end point coordinate respectively by the constructed first straight line and the constructed second straight line;

a second judging subunit 443, configured to, if the start point coordinate or/and the end point coordinate falls on the character: establishing a plane coordinate system of the coarse frame image by taking the base alignment line or a straight line parallel to the base alignment line as an abscissa axis; determining the size of the longitudinal coordinate values of the starting point coordinate and the end point coordinate; if the ordinate of the point falling on the character is a large value, adding a first preset value on the basis of the ordinate of the large value as a point through which a first straight line passes; and if the ordinate of the point falling on the character is a smaller value, subtracting a second preset value on the basis of the ordinate of the smaller value to be used as a point through which a second straight line passes.

As an optional implementation manner, a first character of a to-be-framed subject is taken as a start point coordinate of a moving track, and a last character of the to-be-framed subject is taken as an end point coordinate of the moving track; taking a line segment between the first character and the last character of the subject to be framed as a moving track; adding a first preset value on the basis of the ordinate of the starting point coordinate to serve as a point through which a first straight line passes; and subtracting a second preset value on the basis of the ordinate of the end point coordinate to serve as a point through which a second straight line passes.

As an optional implementation manner, after the obtaining the coarse frame image, the method further includes: and correcting the coarse frame image to enable the coarse frame image to be rectangular.

As an alternative embodiment, if the connecting line between the start point coordinate and the end point coordinate is parallel to the reference straight line, a picture of a character through which the connecting line between the start point coordinate and the end point coordinate passes is obtained.

The frame question device based on the moving track shown in fig. 7 constructs the selection frame according to the starting point and the end point of the moving track, so that incomplete frame selection questions or invalid information obtained by frames can be avoided, the frame question efficiency is improved, and the user experience is improved.

EXAMPLE five

Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure. The electronic device can be a learning machine, a family education machine, a point reading machine, a tablet computer, a mobile phone or other intelligent devices, and can also be a desktop computer, a portable computer, a server and the like. As shown in fig. 8, the electronic device may include:

a memory 510 storing executable program code;

a processor 520 coupled to the memory 510;

the processor 520 calls the executable program code stored in the memory 510 to execute part or all of the steps of any one of the first to third embodiments of the movement trajectory-based block method.

The embodiment of the invention discloses a computer-readable storage medium which stores a computer program, wherein the computer program enables a computer to execute part or all of the steps in any one of the first to third embodiments of the invention based on the frame problem method of the movement track.

The embodiment of the invention also discloses a computer program product, wherein when the computer program product runs on a computer, the computer is enabled to execute part or all of the steps in any one of the first to third embodiments of the frame problem method based on the movement track.

The embodiment of the invention also discloses an application publishing platform, wherein the application publishing platform is used for publishing the computer program product, and when the computer program product runs on a computer, the computer is enabled to execute part or all of the steps in any one of the first embodiment to the third embodiment of the frame question method based on the movement track.

In various embodiments of the present invention, it should be understood that the sequence numbers of the processes do not mean the execution sequence necessarily in order, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present invention, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the method according to the embodiments of the present invention.

In the embodiments provided herein, it should be understood that "B corresponding to a" means that B is associated with a from which B can be determined. It should also be understood, however, that determining B from a does not mean determining B from a alone, but may also be determined from a and/or other information.

Those skilled in the art will appreciate that some or all of the steps of the methods of the embodiments may be implemented by hardware instructions of a program, which may be stored in a computer-readable storage medium, such as Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (CD-ROM), or other disk Memory, or other Memory, or may be stored in a computer-readable storage medium, A tape memory, or any other medium readable by a computer that can be used to carry or store data.

The method, the device, the electronic device and the storage medium for describing the frame problem based on the movement track disclosed by the embodiment of the invention are introduced in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A frame question method based on a moving track is characterized by comprising the following steps:

2. The framing method of claim 1, wherein constructing a first straight line and a second straight line parallel to the reference straight line based on the start point coordinates and the end point coordinates, respectively, comprises:

3. The framing method of claim 2, wherein if the start point coordinates or/and the end point coordinates fall on a character:

4. The framing method of claim 3, wherein the first character of the to-be-framed subject is taken as the start point coordinate of the movement track, and the last character of the to-be-framed subject is taken as the end point coordinate of the movement track; taking a line segment between the first character and the last character of the subject to be framed as a moving track;

5. The framing method of any one of claims 1-4, further comprising, after said obtaining the coarse frame image: and correcting the coarse frame image to enable the coarse frame image to be rectangular.

6. The framing method of any one of claims 1-4, wherein if a connecting line between the start point coordinates and the end point coordinates is parallel to a reference straight line, a picture of a character through which the connecting line between the start point coordinates and the end point coordinates passes is obtained.

7. A frame topic device based on a moving track is characterized by comprising:

8. The framing apparatus of claim 7, wherein the building unit comprises:

9. An electronic device, comprising: a memory storing executable program code; a processor coupled with the memory; the processor calls the executable program code stored in the memory for executing a movement trajectory based question method of any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute a method of framing a moving trajectory according to any one of claims 1 to 6.