CN115278108B

CN115278108B - Writing shooting method, device, learning machine and storage medium

Info

Publication number: CN115278108B
Application number: CN202210886866.1A
Authority: CN
Inventors: 王阳; 陈泽伟; 王光明; 邓泽方
Original assignee: Guangdong Genius Technology Co Ltd
Current assignee: Guangdong Genius Technology Co Ltd
Priority date: 2022-07-26
Filing date: 2022-07-26
Publication date: 2024-09-24
Anticipated expiration: 2042-07-26
Also published as: CN115278108A

Abstract

The embodiment of the application discloses a writing shooting method, a device, a learning machine and a storage medium, wherein the method is applied to the learning machine, the learning machine is provided with a camera module, the camera module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine, the vertical heights of the cameras and the desktop and the shooting range required by each camera are determined when the desktop is shot, and the distance between the cameras is determined according to the number of the cameras and the shooting range, and the method comprises the following steps: controlling all cameras to shoot a desktop to obtain a plurality of sub-images, wherein a text object is placed on the desktop; detecting a target sub-image containing a target object in the plurality of sub-images, wherein the target object is related to writing actions received by a text object; and (5) keeping the cameras corresponding to the target sub-images to shoot, and closing other cameras. By adopting the scheme, the technical problem that the definition of shooting content cannot be ensured when the wide-angle camera is used for shooting in the prior art can be solved.

Description

Writing shooting method, device, learning machine and storage medium

Technical Field

The embodiment of the application relates to the technical field of learning machines, in particular to a writing shooting method and device, a learning machine and a storage medium.

Background

Currently, artificial intelligence techniques are widely used in a variety of industries. For example, in the educational industry, artificial intelligence techniques are applied in learning machines as an auxiliary learning device for use in a user learning process. In the application process of the artificial intelligence technology, the camera is used as a key device, so that the contents such as learning handwriting and answering handwriting of a user can be shot, then the handwriting contents can be identified by utilizing the artificial intelligence technology, and further the functions of automatically counting wrong questions, searching weak knowledge points and the like are realized, so that the user can learn in a targeted manner.

It can be understood that the clearer the camera shoots, the more favorable is the accuracy of subsequent recognition and answer questions. Referring to fig. 1, when the learning machine 1 is fixed on a desktop 2 and photographs a homework book or a test paper 3 placed on the desktop 2, the camera 4 needs to be obliquely placed to photograph the desktop 2, and thus, the depth of field and the angle of view in the photographing process of the camera can affect the definition and the photographing range of photographing. When the shot homework book or test paper 3 is bigger, the whole homework book or test paper 3 can be shot by using the wide-angle camera, however, the edge definition is lower when the wide-angle camera shoots, and the follow-up identification processing is not facilitated.

In summary, how to ensure the definition of the shot content when shooting by using the camera of the learning machine becomes a technical problem to be solved urgently.

Disclosure of Invention

The application provides a writing shooting method, a writing shooting device, a learning machine and a storage medium, which are used for solving the technical problem that the definition of shooting content cannot be ensured when a wide-angle camera is used for shooting in the prior art.

In a first aspect, an embodiment of the present application provides a writing shooting method, which is applied to a learning machine, where the learning machine is provided with a camera module, the camera module includes a plurality of cameras, and a field angle of the cameras is fixed on a desktop by the learning machine and a vertical height of the cameras and the desktop and a shooting range required by each camera are determined when the desktop is shot, and a distance between the plurality of cameras is determined according to the number of the cameras and the shooting range;

the method comprises the following steps:

controlling all cameras to shoot the desktop so as to obtain a plurality of sub-images, wherein a text object is placed on the desktop, and each camera corresponds to one sub-image;

Detecting a target sub-image containing a target object in a plurality of sub-images, wherein the target object is related to writing actions received by the text object;

And keeping the cameras corresponding to the target sub-images to shoot, and closing other cameras.

In a second aspect, an embodiment of the present application further provides a writing shooting device, which is applied to a learning machine, where the learning machine is provided with a camera module, the camera module includes a plurality of cameras, and a field angle of the cameras is fixed on a desktop by the learning machine, and when the desktop is shot, a vertical height between the cameras and the desktop and a shooting range required by each camera are determined, and a distance between the plurality of cameras is determined according to the number of the cameras and the shooting range;

the device comprises:

The first shooting unit is used for controlling all cameras to shoot the desktop so as to obtain a plurality of sub-images, a text object is placed on the desktop, and each camera corresponds to one sub-image;

a first detection unit, configured to detect a target sub-image including a target object, among the plurality of sub-images, the target object being related to a writing action received by the text object;

and the second shooting unit is used for keeping the cameras corresponding to the target sub-images to shoot and closing other cameras.

In a third aspect, an embodiment of the present application further provides a learning machine, where the learning machine includes a camera module, one or more processors, and a memory, where the camera module includes a plurality of cameras, where an angle of view of the cameras is fixed on a desktop by the learning machine and a vertical height of the cameras from the desktop and a shooting range required by each camera when shooting the desktop are determined, and a distance between the plurality of cameras is determined according to the number of cameras and the shooting range;

the camera module is used for shooting according to the instruction of the processor;

the memory is used for storing one or more programs;

The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the writing photographing method as described in the first aspect.

In a fourth aspect, embodiments of the present application also provide a storage medium containing computer-executable instructions for performing the writing photographing method according to the first aspect when executed by a computer processor.

According to the writing shooting method, device, learning machine and storage medium, all cameras are controlled to shoot the desktop on which the text object is placed, the target sub-image containing the target object is identified in a plurality of sub-images obtained through shooting, then, only the camera corresponding to the target sub-image is kept to shoot, and other cameras are closed, so that the technical means of obtaining a writing picture are achieved, the technical problem that the definition of shooting content cannot be guaranteed when the wide-angle camera is used for shooting in the prior art is solved, the shooting range of each camera can be reduced through the plurality of cameras for shooting in different areas, fewer shooting details are described under the condition that pixels are unchanged, further, the shooting content of each camera is clearer, and the union area of the shooting ranges of all cameras is used as the shooting range of the learning machine camera module, so that the field of view (namely the shooting range) shot by the camera module is ensured to be large enough, and the actual requirements of users are met. Further, the angle of view of the camera is fixed on the desktop through the learning machine, and when the desktop is shot, the vertical height of the camera and the desktop and the shooting range required by each camera are determined, and the camera can be enabled to have smaller visual field by setting a reasonable angle of view for the camera, and the edge of the shooting range is closer to the center, so that the definition of the edge is improved, and the follow-up recognition analysis is facilitated. And the distance between the cameras is determined by combining the number of the cameras and the maximum shooting range, so that the reasonable arrangement of the cameras in the learning machine is ensured.

Drawings

FIG. 1 is a schematic diagram of a prior art learning machine;

fig. 2 is a schematic diagram of camera distribution according to an embodiment of the present application;

fig. 3 is a schematic diagram of camera distribution according to an embodiment of the present application;

fig. 4 is a schematic diagram of camera distribution according to an embodiment of the present application;

fig. 5 is a schematic diagram of camera distribution according to an embodiment of the present application;

FIG. 6 is a schematic view of the positions of a mirror and a camera according to an embodiment of the present application;

FIG. 7 is a photograph of an embodiment of the present application;

FIG. 8 is a schematic view of imaging a shooting range according to an embodiment of the present application;

FIG. 9 is a flowchart of a method for shooting writing according to an embodiment of the present application;

fig. 10 is a flowchart of a writing shooting method according to an embodiment of the present application;

FIG. 11 is a schematic diagram showing a relative positional relationship between a learning machine and a desktop according to an embodiment of the present application;

FIG. 12 is a plan view of triangle ABC of FIG. 11;

figure 13 is a plan view of the triangular AED of figure 11;

FIG. 14 is a diagram showing a relationship between a field angle and a shooting range according to an embodiment of the present application;

Fig. 15 is a schematic view of a camera pitch according to an embodiment of the present application;

FIG. 16 is a schematic view of another camera pitch according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of a writing shooting device according to an embodiment of the present application;

fig. 18 is a schematic structural diagram of a learning machine according to an embodiment of the present application.

Detailed Description

The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration and not of limitation. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.

An embodiment of the present application provides a writing photographing method that may be performed by a writing photographing apparatus that may be implemented in software and/or hardware and integrated in a writing photographing device. The writing photographing apparatus may be formed of two or more physical entities or may be formed of one physical entity, which is not limited in the embodiment. The writing shooting device can be a tablet computer, a notebook computer, a mobile phone, a learning machine and the like. Currently, a writing photographing apparatus is taken as an example of a learning machine. The learning machine can be used as auxiliary equipment for user learning, and can be used for making a learning plan, displaying teaching courses, recommending exercises, shooting a learning process or a question making process of a user, correcting the exercises solved by the user, and the like, or can be used as learning equipment, a learning terminal and the like. The learning machine can be fixed on the desktop in the use process, so as to shoot a user in the learning process or writing process of the desktop, wherein the fixing mode of the learning machine and the desktop is not limited currently, for example, a bracket is placed on the desktop, and the learning machine is placed on the bracket, so that the fixing of the learning machine and the desktop can be realized. Currently, the structure of the stand is fixed, i.e., the angle between the stand and the table top is fixed when the learning machine is placed on the stand, and the learning machine is known.

The learning machine is provided with at least one operating system, and at least one application program can be installed under the operating system. The application program may be an application program of the operating system, and is also installed as an application program downloaded from a third party device or a server, which is not limited at present. The learning machine is also equipped with a display screen, which may be provided with a touch function. The learning machine is also provided with a communication device through which network courses, answers to problems and the like can be obtained.

In one embodiment, the learning machine is further provided with a camera module. The camera module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine and are determined by the vertical heights of the cameras and the desktop and the shooting ranges required by the cameras when the desktop is shot, and the intervals among the cameras are determined according to the quantity of the cameras and the shooting ranges.

For example, the number of cameras is at least two, and the specific number and the relative positions of the cameras can be determined according to the actual shooting requirements, for example, fig. 2 to fig. 5 are respectively schematic diagrams of camera distribution provided in an embodiment of the present application, which respectively show the arrangement modes of the cameras when the cameras 51 in the camera module 5 are two, three, five and seven, and can also be understood as the relative positions between the cameras. It can be understood that the mounting position of the camera in the learning machine can be determined according to actual shooting requirements, for example, a plurality of cameras are mounted on one side frame of the learning machine facing the user, or a plurality of cameras are mounted on a plurality of side frames of the learning machine facing the user, and at this time, each camera can understand the front camera of the learning machine. In an embodiment, a mirror may be further provided for the camera module, where each camera may be configured with a mirror or a mirror shared by multiple cameras, where the mirror is used to reflect external light to the camera for imaging, for example, fig. 6 is a schematic diagram of positions of the mirror and the camera according to an embodiment of the present application, and referring to fig. 6, the mirror 52 is located in front of the camera 51 and presents a certain angle, so as to transmit the content 6 to be shot below into the camera for imaging. Optionally, the size of the reflector is larger than the angle of view of the camera shooting the reflector, so that the camera can not shoot the content outside the reflector when shooting is realized by the camera through the reflector.

Illustratively, the cameras are of the same type, e.g., each camera is a fixed focus camera, or each camera is a zoom camera. Further, parameters (such as a focal length, a field angle, and the like) set when each camera shoots are the same and shooting ranges are the same, wherein a shooting range refers to a real world range that can be shot when the camera shoots, and currently, taking a learning machine fixed on a desktop and shooting the desktop as an example, the shooting ranges mentioned in the embodiments refer to a desktop range that can be shot when the camera shoots the desktop. Further, the actual area corresponding to the far shooting range of the camera is larger, and the unit photosensitive area corresponding to the photosensitive area of the camera is smaller, for example, fig. 7 is a shot image provided by an embodiment of the present application, referring to fig. 7, the near lateral shooting range (the lower end of fig. 7) is less than one line, and the far lateral shooting range (the upper end of fig. 7) is more than one line when the camera shoots. In fig. 7, a mosaic process is performed on a part of the text. However, the number of pixels corresponding to the far shooting range and the near shooting range in the photograph shot by the camera is consistent, for example, fig. 8 is an imaging schematic diagram of the shooting range according to an embodiment of the present application, and in fig. 8, the image shot by the camera is taken as an example of 5×5 pixels (piexs), when the camera 51 shoots the content 6 to be shot, an image of 5×5piexs may be obtained, the far shooting range (such as the upper end in fig. 7) is represented by the upper end of the image in the horizontal direction, and the near shooting range (such as the lower end in fig. 7) is represented by the lower end of the image in the horizontal direction, and in this case, for the content of the same size in the near end and the far end, the content in the near end is represented by using more pixels, so that it is clearer. Based on this, in the embodiment, an appropriate field angle (generally smaller than the maximum field angle when a single camera shoots) is set for the camera, so that the shooting range of the camera is appropriately smaller, and further more pixels are used to present the content in the shooting range, so that the shot image is ensured to be clearer and the distortion is relatively smaller, and the processing of the subsequent image is facilitated. The angle of view may also be referred to as a field of view, and the size of the angle of view determines the field of view (currently the shooting range) of the optical instrument (currently the camera). In an embodiment, when the learning machine is fixed on the desktop, an included angle between the learning machine and the desktop is fixed, at this time, a required shooting range of the camera on the desktop and a required vertical height of the camera with the desktop in an application process of the learning machine are combined, and a suitable field angle is set for each camera, wherein a shooting range currently used is a range in which a user expects the camera to shoot, and after a suitable field angle is set for the camera in principle, the camera can meet the required shooting range. It should be noted that, when there is an intersection or close proximity between the shooting ranges corresponding to the cameras adjacent to each other, the union of the shooting ranges may be regarded as the total shooting range of the camera module.

Further, the distance between the cameras installed in the learning machine can be determined by combining the number of the cameras and the shooting range of the cameras, for example, when the number of the cameras is two, the maximum value of the distance between the cameras is the length of the edge line farthest from the cameras in the shooting range, so that the shooting ranges of the two cameras are ensured to be adjacent, and further the condition of missing shooting contents is avoided. In one embodiment, in combination with the application scenario of the learning machine, the total shooting range of the camera module should be greater than or equal to the area of the book or the test paper used when the user learns, for example, the area when the written or test paper is A3 size is the largest, and therefore, the total shooting range of the camera module should be greater than or equal to the area range corresponding to the A3 size. In one embodiment, the A3 size is 297 x 420mm, and thus the total shooting range of the shooting module is 297 x 420mm. It will be appreciated that since the total shooting range of the shooting module is known and the number of cameras is also known, a suitable shooting range may be set for the cameras in combination with the total shooting range and the number of cameras, and then the angle of view and the pitch of the cameras may be determined in combination with the shooting range of the cameras. And then the installation position of the camera in the learning machine is obtained.

Fig. 9 is a flowchart of a writing photographing method according to an embodiment of the present application, and referring to fig. 9, the writing photographing method includes:

And 110, controlling all cameras to shoot the desktop so as to obtain a plurality of sub-images, wherein a text object is placed on the desktop, and each camera corresponds to one sub-image.

At present, the learning machine is fixed on the desktop and the camera module is in a starting state, and at the moment, the learning machine can control each camera of the camera module to shoot. In one embodiment, when the learning machine shoots the desktop, the camera can shoot the desktop directly or through the reflector, and in any mode, only the desktop needs to be shot. In one embodiment, a text object is placed on the desktop, where the text object refers to a book or paper containing text content, and currently, the text object is a book or test paper. Each camera shoots the desktop, specifically, shoots the text object.

Alternatively, when the camera shoots, the camera can shoot continuously, or shoot only one image, which is not limited in the embodiment. Currently, the image shot by each camera is recorded as a sub-image, and it can be understood that the sub-image is displayed with a sub-area of the text object within the shooting range of the corresponding camera, and the sub-images obtained by the cameras can be spliced to obtain an image containing the complete text object. Currently, each acquired sub-image is a sub-image shot by each camera at the same time.

Step 120, detecting a target sub-image containing a target object in the plurality of sub-images, wherein the target object is related to the writing action received by the text object.

Illustratively, after each sub-image is acquired, each sub-image is identified to determine whether the target object is contained therein. Wherein the target object is an object associated with a writing action when the user writes on the text object, in one embodiment the target object is a pen and/or a human hand. It is understood that the user typically operates on the text object by means of a pen (which may be a writing pen, a stylus, etc.) or a human hand. When the target object is a pen, the target object can be a pen point and a pen tip, wherein the pen point refers to a part of the pen which contacts a text object, and the pen point is a part connecting the pen tip and the pen holder. Further, the manner of identifying whether the sub-image includes the target object is not limited currently, for example, a neural network model is constructed, whether the sub-image includes the target object is identified through the neural network model (already trained), or whether the target object in the sub-image is identified by using an image identification technique or an image tracking technique.

For example, if a target object is identified in a sub-image, it is determined that the user may have written in a portion of text captured by the sub-image, i.e., a writing action has been issued. In general, only one sub-image contains a target object at the same time, and at this time, the sub-image containing the target object is referred to as a target sub-image.

And 130, keeping the cameras corresponding to the target sub-images to shoot, and closing other cameras.

Illustratively, after the target sub-image is determined, a camera that captures the target sub-image (i.e., a camera corresponding to the target sub-image) is determined. And then, controlling the camera corresponding to the target sub-image to continuously shoot, namely continuously shooting. And other cameras are closed, so that hardware resources of the learning machine are saved.

Optionally, after the camera corresponding to the control target sub-image continues to shoot, the continuously acquired sub-image forms a writing picture of the user, then the learning machine can identify a track written by the user in the writing picture to determine writing content of the user, further perform a subsequent function, for example, in a answering scene, judge whether answering is accurate based on the writing content, and further, in a learning scene, determine knowledge mastering degree of the user based on the writing content and the like.

According to the writing shooting method, the desktop on which the text object is placed is shot by controlling all cameras, the target sub-image containing the target object is identified in a plurality of sub-images obtained through shooting, then, only the cameras corresponding to the target sub-image are kept for shooting, and other cameras are closed, so that the technical means of a writing picture is obtained, the technical problem that the definition of shooting content cannot be guaranteed when the wide-angle cameras are used for shooting in the prior art is solved, the shooting range of each camera can be reduced through the regional shooting of the plurality of cameras, fewer shooting details are described under the condition that pixels are unchanged, the shooting content of each camera is clearer, the union region of the shooting ranges of the cameras is used as the shooting range of the camera shooting module of the learning machine, the field of view (namely the shooting range) shot by the camera module can be ensured to be large enough, and the actual requirements of users are met. Further, the angle of view of the camera is fixed on the desktop through the learning machine, and when the desktop is shot, the vertical height of the camera and the desktop and the shooting range required by each camera are determined, and the camera can be enabled to have smaller visual field by setting a reasonable angle of view for the camera, and the edge of the shooting range is closer to the center, so that the definition of the edge is improved, and the follow-up recognition analysis is facilitated. And the distance between the cameras is determined by combining the number of the cameras and the maximum shooting range, so that the reasonable arrangement of the cameras in the learning machine is ensured.

In one embodiment, since the user has mobility in writing, that is, the writing position changes, the camera capturing the target object also changes, and at this time, the learning machine needs to instantly determine the camera capturing the target object to implement writing tracking. Based on this, in this embodiment, when the camera corresponding to the target sub-image is kept to shoot, the method further includes: and continuously detecting the target object in the target sub-image. After the camera corresponding to the target sub-image is kept to shoot and other cameras are closed, the method further comprises the following steps: and when the target sub-image is detected not to contain the target object, returning to execute the operation of controlling all cameras to shoot the desktop.

When the camera corresponding to the target sub-image shoots to acquire a writing picture, the learning machine detects the real-time target sub-image continuously shot by the camera to determine whether the target sub-image contains a target object. If the target object is included, it is indicated that the user is still writing in the shooting range corresponding to the camera, and if the target object is not included, it is indicated that the user has changed the writing position or the user has finished writing, and at this time, further judgment is required. When the judgment is further carried out, the learning machine re-controls all cameras to shoot the desktop, namely, returns to execute the operation of controlling all cameras to shoot the desktop, acquires sub-images shot by all cameras, then identifies target sub-images containing target objects in all the sub-images, continuously keeps the cameras corresponding to the target sub-images to shoot, and closes other cameras. It should be noted that, in the case where each sub-image does not include the target object, that is, the learning machine cannot recognize the target object, at this time, the learning machine confirms that the user is not writing, and continues to control each camera to perform shooting, so as to continue to detect the target object.

Optionally, a time length is preset, and a specific value of the time length can be set according to actual situations. And when the fact that the target sub-image does not contain the target object is detected, starting timing, and continuously identifying whether the target sub-image acquired in real time contains the target object or not. If the target object is detected, the cameras corresponding to the target sub-images are kept to shoot, other cameras are closed, if the time length reaches the preset time length and the target object is not detected yet, it is determined that the user does not write in the current shooting range, at the moment, the operation of controlling all cameras to shoot the desktop is performed, and therefore the influence of user writing pause on the detection result is avoided.

When the cameras corresponding to the target sub-images are kept to shoot continuously, the target objects are detected in real time, and when the target objects are not detected, all the cameras are controlled to shoot again, so that the target sub-images containing the target objects are detected again, and then the corresponding cameras are controlled to shoot, and therefore when a user changes the writing position, the learning machine can instantly master the latest writing position of the user and track shooting, and the effect of writing tracking is guaranteed.

Fig. 10 is a flowchart of a writing shooting method according to an embodiment of the present application. In this embodiment, the camera module includes two cameras, every the angle of view of camera is the same, and at this moment, the angle of view of camera passes through the learning machine is fixed on the desktop and right when the desktop is shot the camera with the required shooting range of camera and the vertical height of desktop is confirmed, the computational formula of angle of view is:

Fov＝2arctan(s/h)

h＝AF/sinα

Wherein Fov represents a view angle, s represents a half of the length of the farthest photographing end when the camera photographs, the farthest photographing end is determined according to the photographing range, AF represents the vertical height between the learning machine and the tabletop when the learning machine is fixed on the tabletop, h represents the length of a connecting line between the camera and the midpoint of the farthest photographing end when the learning machine is fixed on the tabletop and photographs the tabletop, and alpha represents an included angle between the connecting line and the tabletop.

Illustratively, the description is currently given taking an example in which the camera module includes two cameras. Fig. 11 is a schematic diagram of a relative positional relationship between a learning machine and a desktop according to an embodiment of the present application. Referring to fig. 11, when the learning machine 1 is fixed on a desk, each camera 51 is located above the side of the learning machine 1 facing the user. The included angle between the learning machine 1 and the desktop is recorded as theta, and the total shooting range of the shooting module needs to cover the area corresponding to the A3 size, namely, the shooting range in the transverse direction (parallel to the contact line between the learning machine and the desktop) needs to reach 420mm and the shooting range in the longitudinal direction needs to reach 297mm when the shooting module shoots. In an embodiment, when there are two cameras 51, the required shooting range on the desktop when each camera 51 shoots should be equal to or greater than 297×210mm, that is, the shooting range of each camera in the lateral direction should be equal to or greater than 210mm. Currently, taking a camera as an example to describe the shooting range of the camera, referring to fig. 11, in the shooting range of the camera, the edge (corresponding to the edge BC in fig. 11) farthest from the camera and parallel to the edge in contact with the learning machine and the table surface should be greater than or equal to 210mm, and the specific value thereof may be set in combination with the actual situation. The side BC is the farthest side of the shot, and the midpoint of the farthest side of the shot is the point D, and bd=dc. For ease of understanding, the triangle ABC and the triangle AED may be formed by designating the camera as point a, making a perpendicular from the camera to the bottom side of the learning machine (i.e., the side that contacts the table), and designating the intersection as point E. Fig. 12 is a plan view of the triangle ABC of fig. 11, and fig. 13 is a plan view of the triangle AED of fig. 11. The triangle ABC is an isosceles triangle, and for the triangle ABC, the angle of the angle BAC is the field angle of the camera, and the side AD may represent the length from the point a (i.e. the camera) to the side BC (i.e. the furthest side of the shot), and it is understood that the side AD is perpendicular to the side BC. Currently, the length of the side AD is denoted as h, that is, h represents the length of a line connecting the camera and the midpoint of the farthest side when the learning machine is fixed on the desktop and the desktop is photographed. at this time, tan (Fov/2) =s/h can be obtained from the trigonometric function, where s is half the length of the farthest side of the shot when the camera shoots. After the shooting range required by the camera is determined, the specific value of s can be determined, namely, s is already set, so that Fov can be obtained after h is obtained. In the triangle AED, a triangle AFD can be obtained by making a perpendicular line from the point a to the side ED and making the intersection point F, referring to fig. 13. The length of the edge AF can be understood as the vertical distance between the camera and the desktop, i.e. the vertical height between the learning machine and the desktop when the learning machine is fixed on the desktop. The length of the side AD is h, the angle AED is θ, the side AE is understood as the width of the learning machine (i.e., the length not parallel to the table surface side when fixed on the table surface), the length of the side AE is currently noted as d, the length of the side AF can be determined by d and θ, i.e., af=d×sin (1- θ), based on fig. 13, after which the length of the side FE can be derived by using a trigonometric function or the pythagorean theorem when the lengths of the side AF and the side AE are known, and since the length of the side DE is known (297 nm in the maximum photographing range), the length of the side FD can be obtained, at this time, according to the lengths of the side FD and the side AF, The length of the edge AD, i.e. the value of h, can be obtained by using the pythagorean theorem, and then the value of Fov can be obtained. Or, after the camera is fixed to the learning machine, the included angle between the camera and the connecting line of the farthest shooting edge and the desktop is known, the included angle is the angle ADE, it can be understood that the specific value of the included angle is irrelevant to the height of the camera and the desktop, at present, the angle of the angle ADE is denoted as alpha, at this moment, alpha can also represent the included angle between the connecting line AD and the desktop, and sin alpha=af/h can be obtained, and because alpha and AF are determined, h can be obtained based on the formula, and Fov can be obtained based on h and s. It will be appreciated that, referring to fig. 13, h/sinθ=d/sinα=af can also be derived using the sine theorem, and thus the relationship between h and d can be found as h=sinθ×d/sinα. it should be noted that, fig. 14 is a schematic diagram of a relationship between a field angle and a shooting range according to an embodiment of the present application, and referring to fig. 14, when the field angle of the camera is fixed, the closer the farthest side of the shot is to the camera, the smaller the shooting range is, as in fig. 14, the closer side a1 is to the camera than the side a2 is to the camera, so that the shooting range when the side a1 is the farthest side of the shot is smaller than the shooting range when the side a2 is the farthest side of the shot.

In one embodiment, the camera module includes two cameras, and the interval of two cameras is less than or equal to the camera is shot the most distal length when shooing the desktop, the shooting most distal is according to shooting scope confirms. Taking fig. 11 as an example, the length of the farthest shooting edge (i.e., edge BC) when one camera shoots is 2s, and the value of s is preset. At this time, when the distance between the two cameras is 2s, the maximum shooting ranges of the two cameras do not overlap, for example, fig. 15 is a schematic diagram of the distance between the two cameras according to an embodiment of the present application, and when the distance between the cameras is L, l=2s, a schematic diagram of a plane (i.e. a plane in which the triangle ABC is located) where the camera and the shooting furthest side of the shooting range are located is shown in fig. 15, and at this time, the shooting furthest sides of the two cameras are adjacent, i.e. the two shooting ranges are adjacent. Fig. 16 is a schematic diagram of another camera pitch provided in an embodiment of the present application, referring to fig. 16, when the pitch of the two cameras is 0, the two cameras can be understood as one camera, and at this time, a schematic plan view of the camera and the farthest shooting edge of the shooting range is shown in fig. 16, and at this time, the shooting ranges of the two cameras are completely overlapped, and based on this, the pitch of the two cameras should be greater than 0 and less than or equal to the length of the farthest shooting edge.

On this basis, referring to fig. 10, the writing photographing method includes:

Step 210, controlling all cameras to shoot the desktop so as to obtain a plurality of sub-images, wherein a text object is placed on the desktop, and each camera corresponds to one sub-image.

In one embodiment, the text object includes a title of at least one problem. The type of problem is not limited at present.

And 220, splicing the sub-images according to the relative positions of the cameras so as to obtain a text image containing the complete text object.

The learning machine determines the relative positions of the cameras in advance, where the relative positions are used to represent the arrangement of the cameras, for example, when the camera module adopts the structure shown in fig. 2, the relative positions of the two cameras are arranged horizontally, at this time, the sub-images shot by the two cameras are also arranged horizontally, based on the relationship, the two sub-images can be spliced to obtain an image, which can be considered as the image shot by the camera module, and the image contains the complete text object. Currently, the stitched image is noted as a text image.

In one embodiment, during stitching, the arrangement relation of the sub-images is determined based on the relative positions of the cameras. Then, for two adjacent sub-images, the same part in the two sub-images is searched. It can be understood that, since the shooting ranges of the adjacent cameras have partially overlapped areas, the sub-images shot by the adjacent cameras also have partially identical contents, so that the identical parts in the adjacent sub-images can be found during stitching. It will be appreciated that, because the adjacent cameras are located at different positions on the learning machine, there may be some difference in the same content in the two sub-images captured by the adjacent cameras, and at this time, the same content in at least one sub-image needs to be transformed so that the same content is aligned in the two sub-images. The method comprises the steps of aligning identical contents in two adjacent images during splicing, which is not described in detail at present, and then splicing the images based on identical contents because the identical contents between the adjacent sub-images are aligned, wherein during splicing, a weight can be set for pixels representing the identical contents in the adjacent sub-images, then the pixels representing the identical contents are fused by combining the weights, so that the splicing of the adjacent sub-images is realized, and each adjacent sub-image is spliced to obtain a text image.

And 230, performing text recognition on the text image to determine the topic content in the text object.

The text image is illustratively subjected to text recognition to obtain text content in the text object, wherein the text content of the text object is currently the topic content of the problem. The technical means used in text recognition is not limited at present, and for example, a neural network for text recognition is trained, and then a text image is input into the neural network to obtain the topic content. For another example, the text image is processed by optical character recognition (Optical Character Recognition, OCR) to obtain the topic content contained in the text image. In general, in a text image, only text (including chinese characters, english, numerals, symbols, etc.) appears in a text object, and therefore, the topic content of the text image may also be used as the topic content of the text object. Or firstly identifying the text object in the text image, and then carrying out text identification on the text object to obtain the topic content of the text object.

Step 240, searching answer content corresponding to the question content in a preset study question library.

The problem library contains a plurality of problem questions (i.e. question contents) and corresponding answer contents. The learning problem library may be stored in the learning machine or in a background server of the learning machine. Alternatively, the problem base may be classified according to discipline and grade.

Illustratively, after obtaining the topic content, one or more topic libraries are accessed to find the same topic in the topic library as the topic content. In one embodiment, one or more problems most similar to the problem content are searched in the problem library by calculating the problem similarity, and then the answer content of the searched problem is obtained and used as the answer content corresponding to the problem content. At this time, the obtained answer content can be used for judging the accuracy of the current answer of the user.

Optionally, when the target object is not detected, the cameras are required to be controlled again to shoot so as to re-detect the target object, and in the process, the topic content is not required to be identified.

Step 250, detecting a target sub-image containing a target object in the plurality of sub-images, wherein the target object is related to the writing action received by the text object.

And 260, keeping the cameras corresponding to the target sub-images to shoot, closing other cameras, and identifying the writing track corresponding to the writing action according to the target sub-images acquired in real time.

In the shooting process, after a target sub-image is obtained, the target sub-image is compared with a previous target sub-image to determine a writing track newly written in the current target sub-image, and in this way, the writing track written by the user in real time can be obtained. Meanwhile, the writing sequence of the writing track can be obtained.

And 270, determining writing content according to the writing track.

Optionally, a text library is pre-built, tracks of each text are recorded in the text library, and then the currently identified writing track is compared with the tracks of each text in the text library to determine the text represented by the currently identified writing track, so as to obtain writing content. It is understood that when a user continuously inputs a plurality of texts, whether to write a new text can be determined according to the position interval of the writing trace and the pause time of the writing trace. For example, when the user writes "we", after the writing of "me" is completed, there is a certain position interval between the first pen of "me" and the last pen of "me" and a pause time is generated, so that it can be determined that new text "s" is written based on the position interval and the pause time.

Alternatively, a neural network for text recognition is constructed, and writing tracks are input into the neural network, so that writing contents output by the neural network can be obtained.

Step 280, comparing the written content with the answer content to determine whether the written content is accurate.

The writing content is compared with the corresponding answer content, if the writing content is the same, the writing content is determined to be correct, namely the answer of the user is accurate, otherwise, the writing content is determined to be wrong, namely the answer of the user is wrong. This process may be considered as a process of modifying the answer. It can be understood that, since the writing content is determined in real time in the writing process, the real-time correction in the answering process can be realized by comparing the writing content determined in real time with the answer content.

At present, only the camera corresponding to the target sub-image is used for shooting, and the camera can only shoot the question content of one or more questions, so that the question content answered by the current user needs to be determined first, and then whether the written content is accurate or not is judged, and at the moment, the comparison of the written content and the answer content in the step can comprise: determining the topic content contained in the target sub-image according to the relative positions among the cameras; and comparing the writing content with the answer content corresponding to the question content.

For example, since the learning machine knows the relative positions of the cameras, based on the relative positions, it can determine what part of the text object corresponds to the shooting range of the camera corresponding to the target sub-image (i.e. which region of the text image corresponds to), and further determine the question content contained in the part, and then, only the answer content corresponding to the contained question content is used and compared with the written content. Optionally, after determining the question content corresponding to the current target sub-image, it may also be determined according to the writing position when the user writes, where the user is answering a problem, in general, the relative positions of the answering areas and the question content are relatively fixed, for example, the answering areas of the selected questions, the blank-filled questions and the judging questions are all located in the question content or at the rear of the question content, and the answering areas of the subjective questions and the application questions are all located at the lower positions of the question content, so that, by combining the writing positions and the relative position relation of the plurality of question contents recently connected, the problem of the current answer can be determined, and only the answer content corresponding to the problem is obtained, thereby further judging the answer accuracy.

When the problem questions are contained in the text object and the user answers, the desktop on which the text object is placed is shot by controlling all cameras, then the shot sub-images are spliced to obtain the text image containing the complete text object, then the text image is identified to obtain the question content, answer content corresponding to the question content is searched in a learning question bank, then the target sub-image containing the target object is identified in the sub-images, then the cameras corresponding to the target sub-image are kept for shooting, other cameras are closed, the writing track of the user is obtained based on the shot target sub-image, the writing content is identified based on the writing track, then the writing content and the answer content are compared to determine whether the user answers accurately. Further, the angle of view of camera passes through the learning machine to be fixed on the desktop and when shooing the desktop the camera with the vertical height of desktop and the required maximum shooting scope of each camera confirm, can combine actual shooting demand to set up reasonable angle of view for the camera for the field of view of camera is littleer, and the edge of shooting scope is closer to the center, has improved the definition at edge, is convenient for follow-up discernment and analysis, and, combines the quantity of camera and the interval between the maximum shooting scope confirm the camera, has guaranteed the reasonable arrangement of camera in the learning machine.

Fig. 17 is a schematic structural diagram of a writing shooting device according to an embodiment of the present application. The writing shooting device is applied to a learning machine, the learning machine is provided with a camera module, the camera module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine and are determined by the vertical height of the cameras and the desktop and shooting ranges required by the cameras when the desktop is shot, and the intervals among the cameras are determined according to the quantity of the cameras and the shooting ranges. Referring to fig. 17, the writing photographing apparatus includes a first photographing unit 301, a first detecting unit 302, and a second photographing unit 303.

The first shooting unit 301 is configured to control all the cameras to shoot the desktop so as to obtain a plurality of sub-images, a text object is placed on the desktop, and each camera corresponds to one sub-image; a first detection unit 302, configured to detect a target sub-image including a target object, among the plurality of sub-images, the target object being related to a writing action received by the text object; and the second shooting unit 303 is used for keeping the cameras corresponding to the target sub-images to shoot, and closing other cameras.

In one embodiment of the present application, the second photographing unit 303 includes: and the continuous detection subunit is used for continuously detecting the target object in the target sub-image when the camera corresponding to the target sub-image is kept to shoot, and the closing subunit is used for closing other cameras. The apparatus further comprises: and the return execution unit is used for keeping the cameras corresponding to the target sub-images to shoot, and returning to execute the operation of controlling all the cameras to shoot the desktop when the target sub-images are detected to not contain the target object after other cameras are closed.

In one embodiment of the present application, the second photographing unit 303 includes: the real-time identification subunit is used for keeping the cameras corresponding to the target sub-images to shoot, and identifying writing tracks corresponding to the writing actions according to the target sub-images acquired in real time when other cameras are closed; and the writing content determining subunit is used for determining writing content according to the writing track.

In one embodiment of the present application, the text object includes a title of at least one problem, and the apparatus further includes: the image splicing unit is used for controlling all the cameras to shoot the desktop so as to obtain a plurality of sub-images, and then splicing the plurality of sub-images according to the relative positions among the cameras so as to obtain a text image containing a complete text object; the topic identification unit is used for carrying out text identification on the text image so as to determine topic content in the text object; the answer searching unit is used for searching answer content corresponding to the question content in a preset study question library; and the content comparison unit is used for comparing the writing content with the answer content after determining the writing content according to the writing track so as to determine whether the writing content is accurate.

In one embodiment of the present application, the content comparing unit includes: the topic determination subunit is used for determining topic contents contained in the target sub-image according to the relative positions among the cameras; and the answer comparison subunit is used for comparing the writing content with the answer content corresponding to the question content.

In one embodiment of the present application, the camera module includes two cameras, and the angle of view of each camera is the same, and the calculation formula of the angle of view is:

Fov＝2arctan(s/h)

h＝AF/sinα

In one embodiment of the application, the camera module comprises two cameras, the distance between the two cameras is smaller than or equal to the length of the farthest shooting edge when the cameras shoot the desktop, and the farthest shooting edge is determined according to the shooting range.

The writing shooting device provided by the embodiment is contained in a learning machine, can be used for executing the writing shooting method provided by any embodiment, and has corresponding functions and beneficial effects.

Fig. 18 is a schematic structural diagram of a learning machine according to an embodiment of the present application. Specifically, as shown in fig. 18, the learning machine includes a processor 40, a memory 41, and an image pickup module 42; the number of processors 40 in the learning machine may be one or more, one processor 40 being taken as an example in fig. 18; the processor 40, memory 41 and camera module 42 in the learning machine may be connected by a bus or other means, for example in fig. 18.

The camera module 42 includes a plurality of cameras, the field angle of the cameras is fixed on the desktop by the learning machine, and when the desktop is photographed, the vertical height of the cameras and the desktop and the required photographing range of each camera are determined, and the intervals between the cameras are determined according to the number of the cameras and the photographing range. The image capturing module 42 is used for capturing images according to the instruction of the processor 40.

The memory 41 is a computer-readable storage medium that can be used to store a software program, a computer-executable program, and modules such as program instructions/modules in the writing photographing method in the embodiment of the present application (for example, the first photographing unit 301, the first detecting unit 302, and the second photographing unit 303 in the writing photographing apparatus). The processor 40 executes various functional applications of the learning machine and data processing by executing software programs, instructions, and modules stored in the memory 41, that is, implements the writing photographing method provided in any of the above embodiments.

The memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the learning machine, or the like. In addition, memory 41 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 41 may further include memory remotely located relative to processor 40, which may be connected to the learning machine via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The learning machine may further comprise input means operable to receive input numerical or character information and to generate key signal inputs relating to user settings and function control of the problem search apparatus. The learning machine may further comprise output means, which may comprise a display screen, a loudspeaker or the like. The learning machine may further comprise communication means for data communication with a background server or other device.

The learning machine comprises the writing shooting device provided by the embodiment, can be used for executing the writing shooting method provided by any embodiment, and has corresponding functions and beneficial effects.

The embodiment of the application also provides a storage medium containing computer executable instructions which are used for executing the related operations in the writing shooting method provided by any embodiment of the application when being executed by a computer processor, and the storage medium has corresponding functions and beneficial effects.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product.

Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

Note that the above is only a preferred embodiment of the present application and the technical principle applied. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, while the application has been described in connection with the above embodiments, the application is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the application, which is set forth in the following claims.

Claims

1. The writing shooting method is applied to a learning machine and is characterized in that the learning machine is provided with a camera module, the camera module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine, when the desktop is shot, the vertical heights of the cameras and the desktop and the shooting range required by each camera are determined, and the intervals among the cameras are determined according to the number of the cameras and the shooting range;

the method comprises the following steps:

2. The writing shooting method according to claim 1, wherein when the camera corresponding to the target sub-image is kept to shoot, further comprising:

Continuously detecting a target object in the target sub-image;

after the camera corresponding to the target sub-image is kept to shoot and other cameras are closed, the method further comprises the following steps:

And when the target sub-image is detected not to contain the target object, returning to execute the operation of controlling all cameras to shoot the desktop.

3. The writing shooting method according to claim 1, wherein when the camera corresponding to the target sub-image is kept to shoot and the other cameras are turned off, the writing shooting method further comprises:

According to the target sub-image obtained in real time, recognizing a writing track corresponding to the writing action;

And determining the writing content according to the writing track.

4. The method of claim 3, wherein the text object comprises a title of at least one problem,

After the controlling all the cameras to shoot the desktop so as to obtain a plurality of sub-images, the method further comprises the following steps:

splicing the sub-images according to the relative positions among the cameras to obtain a text image containing a complete text object;

performing text recognition on the text image to determine the topic content in the text object;

Searching answer content corresponding to the question content in a preset study question library;

after the writing content is determined according to the writing track, the method further comprises the following steps:

comparing the written content and the answer content to determine whether the written content is accurate.

5. The writing photographing method of claim 4, wherein the comparing the writing contents and the answer contents comprises:

determining the topic content contained in the target sub-image according to the relative positions among the cameras;

And comparing the writing content with the answer content corresponding to the question content.

6. The writing shooting method of claim 1, wherein the camera module comprises two cameras, the angle of view of each camera is the same, and the calculation formula of the angle of view is:

Fov＝2arctan(s/h)

h＝AF/sinα

7. The writing photographing method according to claim 1, wherein the camera module includes two cameras, a distance between the two cameras is smaller than or equal to a length of a farthest photographing edge when the cameras photograph the desktop, and the farthest photographing edge is determined according to the photographing range.

8. The writing shooting device is applied to a learning machine and is characterized in that the learning machine is provided with a shooting module, the shooting module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine, when the desktop is shot, the vertical heights of the cameras and the desktop and the shooting range required by each camera are determined, and the intervals among the cameras are determined according to the number of the cameras and the shooting range;

the device comprises:

9. The learning machine is characterized by comprising a camera module, one or more processors and a memory, wherein the camera module comprises a plurality of cameras, the field angles of the cameras are fixed on a desktop through the learning machine, the vertical heights of the cameras and the desktop and the shooting range required by each camera are determined when the desktop is shot, and the intervals among the cameras are determined according to the number of the cameras and the shooting range;

the memory is used for storing one or more programs;

When executed by the one or more processors, causes the one or more processors to implement the writing photography method of any of claims 1-7.

10. A storage medium containing computer executable instructions which, when executed by a computer processor, are for performing the writing photography method of any of claims 1-7.