WO2019090903A1 - 一种视频处理方法、装置、终端设备和存储介质 - Google Patents

一种视频处理方法、装置、终端设备和存储介质 Download PDF

Info

Publication number
WO2019090903A1
WO2019090903A1 PCT/CN2017/116748 CN2017116748W WO2019090903A1 WO 2019090903 A1 WO2019090903 A1 WO 2019090903A1 CN 2017116748 W CN2017116748 W CN 2017116748W WO 2019090903 A1 WO2019090903 A1 WO 2019090903A1
Authority
WO
WIPO (PCT)
Prior art keywords
writing
image
display screen
touch display
written
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/116748
Other languages
English (en)
French (fr)
Inventor
张奎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Guangzhou Shirui Electronics Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Guangzhou Shirui Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd, Guangzhou Shirui Electronics Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Publication of WO2019090903A1 publication Critical patent/WO2019090903A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/62—Control of parameters via user interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio

Definitions

  • Embodiments of the present invention relate to image processing technologies, and in particular, to a video processing method, apparatus, terminal device, and storage medium.
  • the teacher's lecture process can be recorded as a multimedia file, which can help students to easily self-study, review and preview, and can also share resources among teachers and schools, and improve the teaching power of teachers.
  • remote video teaching can be conducted to learners in different regions through the Internet.
  • the teacher when the teacher writes and writes on the writing board, the student's attention is on the content written by the teacher, but part of the content being written or written content may be blocked by the teacher's hand, as shown in Figure 1.
  • the invention provides a video processing method, device, terminal device and storage medium, so as to improve the efficiency of video teaching and the attention of students in the learning process.
  • an embodiment of the present invention provides a video processing method, where the method includes:
  • the writing trajectory is added to the writing image based on the relative position of the time axis and the writing trajectory in the touch display screen.
  • an embodiment of the present invention further provides a video processing apparatus, where the apparatus includes:
  • Obtaining a writing image module configured to acquire a writing image corresponding to an area where the touch display screen is located from video data captured by a camera facing the touch display screen;
  • a writing track module configured to collect a writing track generated on the touch display screen during the shooting of the camera, and map an acquisition time of the writing track to a time axis of the writing image
  • a writing trajectory module is added for adding the writing trajectory to the writing image based on a relative position of the time axis and the writing trajectory in the touch display screen.
  • the embodiment of the present invention further provides a terminal device, where the device includes:
  • One or more processors are One or more processors;
  • a memory for storing one or more programs
  • the one or more programs are executed by the one or more processors such that the one or more processors implement a video processing method as described above.
  • the embodiment of the present invention further provides a computer readable storage medium, where the computer program is stored, and when the program is executed by the processor, the video processing method according to any one of the above is implemented.
  • One technical solution of the present invention acquires a written image corresponding to an area where the touch display screen is located from video data captured by a camera facing the touch display screen; captures a writing track generated on the touch display screen during the shooting of the camera, and writes the writing track
  • the acquisition time is mapped to the time axis of the written image; the technical means of adding the writing trajectory to the written image based on the relative position of the time axis and the writing trajectory in the touch display screen solves the problem that the writing content is blocked by the hand during the video teaching process.
  • the problem is to improve the efficiency of video teaching and the effect of students' attention during the learning process.
  • FIG. 1 is a schematic diagram showing a state in which writing content is blocked by a hand in the prior art.
  • FIG. 2 is a flowchart of a video processing method according to Embodiment 1 of the present invention.
  • FIG. 3 is a schematic structural diagram of a video recording system according to Embodiment 1 of the present invention.
  • Embodiment 4 is a flowchart of a video processing method in Embodiment 2 of the present invention.
  • FIG. 5 is a schematic diagram showing a state in which writing content is displayed on the hand in the second embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a video processing apparatus according to Embodiment 3 of the present invention.
  • FIG. 7 is a schematic structural diagram of a terminal device according to Embodiment 4 of the present invention.
  • Embodiment 2 is a flowchart of a video processing method according to Embodiment 1 of the present invention.
  • the present embodiment is applicable to a situation in which a written content is prevented from being blocked by a hand during a recording course or a writing content is prevented from being blocked by a hand during a distance learning process.
  • the method may be implemented by a video processing device provided by an embodiment of the present invention.
  • the device may be implemented in software and/or hardware, and may be integrated in a terminal device.
  • the video processing method specifically includes the following steps:
  • a video recording of a user's teaching process is performed by a camera, as shown in FIG. 3, which is a structural schematic diagram of a video recording system.
  • the tablet computer 14 with the touch display screen 10 and the camera 11 are connected via the USB cable 13, and the camera 11 is mounted directly above the touch screen display 10, that is, the photographing direction of the camera 11 is oriented.
  • the platform computer 14 touches one side of the display screen 10.
  • the touch screen display is not limited to a tablet computer, but other electronic devices with a touch screen display, such as a smart phone with a touch screen display and a notebook computer, etc., can also be used.
  • the writing software in the tablet is turned on, and the writing mode is entered, that is, the writing pen is written on the screen of the touch screen of the tablet computer.
  • the mode of the handwriting is similar to the pen mode in the drawing board, the direction of the camera and the tablet will be adjusted so that the screen of the tablet is as close as possible to the camera, and the camera is turned on to start recording.
  • the user will be at The content to be explained is written on the screen of the touch display screen, and at the same time, the image that the user writes in the area where the touch screen is displayed is acquired through the camera.
  • the writing track generated on the touch screen is touched during the shooting of the capturing camera, and the collecting time of the writing track is mapped to the time axis of the written image.
  • the timeline starts with the camera being turned on and begins to record the content written by the user on the touch display screen, and is generated over time until the completion of recording or remote teaching of the course ends, which is used for recording
  • the writing track is composed of a myriad of writing points.
  • the touch screen will collect the set of writing points in real time. All the set of writing points are combined to form a writing trajectory, and the writing trajectory is saved in the memory of the tablet computer, and at the same time, the time at which the writing trajectory is collected is in one-to-one correspondence with the time at which the writing image is acquired.
  • a touch event of the writing software system is triggered, and the writing software in the tablet computer obtains the touch display screen through the touch event provided by the system.
  • the written trajectory is generated and the acquisition time of the written trajectory is mapped onto the time axis of the written image.
  • the written image needs to be continuously acquired, but as time goes by, the relative position of the writing track in the touch display screen is always changing during the process of the user writing the content on the touch display screen by using the writing pen. of.
  • the writing trajectory and the writing image are in one-to-one correspondence in accordance with the chronological order in which the written image is acquired and the relative position of the writing trajectory in the touch display screen, and the writing trajectory is added to the writing image.
  • the refreshing speed of the written image is relatively faster than the refreshing rate of the writing track, that is, the writing track corresponds to at least one frame of writing image, and each frame of writing image is stationary.
  • the image, and the relative position of the writing track on the touch display screen also changes.
  • the collected writing track is overwritten into the written image, that is, the set of writing points is overlaid on the corresponding pixel in the corresponding written image to
  • the set of writing points and the corresponding pixels are merged, so that the set of writing points is displayed on the display screen as a video frame of one frame, thereby preventing the user's hand from obscuring the writing content, so that the learner watching the video content It is not affected by users, but reduces its own learning efficiency.
  • the writing image corresponding to the area where the touch display screen is located is obtained from the video data captured by the camera facing the touch display screen; the writing track generated on the touch screen is captured during the shooting of the camera, and the writing is performed.
  • the acquisition time of the trajectory is mapped to the time axis of the written image; the technical means of adding the writing trajectory to the written image based on the relative position of the time axis and the writing trajectory in the touch display screen solves the problem that the writing content is blocked by the hand during the video teaching process.
  • the problem is to improve the efficiency of video teaching and the effect of students' attention during the learning process.
  • the video processing method further includes: performing audio collection based on time axis synchronization. Acquiring audio data, associating the audio data with the written image to generate video data.
  • the audio data corresponding to the acquisition time of the written image is associated and saved to generate video data of the network video course, and the related video data information is saved to the camera.
  • the video data information is saved in the memory in the tablet computer, and the captured video image is played and displayed through the tablet computer.
  • the video picture seen by the learner watching the video through the display screen is actually a still video image with a certain relationship displayed according to a certain refresh frequency, and the "frame" is usually used as a unit of the video image. Since the human eye has a "visual persistence" function, a plurality of discrete video images are displayed at a certain frequency, and a continuous video picture can be seen. Correspondingly, during the video recording by the camera, the video image acquired one frame at a time is also used.
  • the audio data is acquired by audio acquisition based on time axis synchronization, and the audio data is associated with the written image to generate video data, and the video image combined with the audio data is played in the terminal device. Display, so that learners can watch and learn the teaching content.
  • Embodiment 4 is a flowchart of a video processing method according to Embodiment 2 of the present invention.
  • the present embodiment is applicable to the case where the writing of the micro-course or the writing content in the remote teaching is blocked by the hand.
  • This embodiment is the basis of the first embodiment.
  • the steps include the following steps:
  • the camera is installed directly above the touch display screen, but during the specific recording process, the video image captured by the camera is not only the video image of the area where the touch screen is touched, but may also include Touch a video image outside the display.
  • the initial image acquired by the camera also includes The associated video image outside the specified border.
  • the initial image needs to be pre-processed to identify the initial written image in which the area where the touch display screen is located is extracted.
  • the contour detection algorithm in the Opencv open source library, the boundary of the area where the touch display screen is located in the initial image is detected, and the initial image is subjected to image recognition extraction according to the boundary to obtain an initial written image of the area where the touch display screen is located.
  • the contour detection algorithm can be divided into static image contour detection and dynamic video contour detection, wherein the first type in the dynamic video contour detection uses the optical flow method to acquire motion cues.
  • the optical flow method calculates the variation field of the brightness of an object with time and space based on the space-time gradient model.
  • the optical flow calculates the instantaneous rate of change of the pixel. The principle is that the pixel point moves from one position to the other, and the gray value remains unchanged.
  • the second type uses the frame difference method to obtain motion cues.
  • the frame difference method is also one of the common methods in moving target detection.
  • the basic principle is that the pixel difference subtracted from the two frames before and after is calculated, and the binarization processing is performed by using the fixed closed value to obtain the foreground moving target. If the target has no motion, then the pixel difference subtracted by the two frames is zero; if the target has significant motion, then the pixel difference subtracted by the two frames is greater than the closed value, and the binarization process can obtain the moving target position in the video frame.
  • the initial image acquired by the camera can be identified and extracted by the static image contour detection, so that only the written image of the area where the touch display screen is located is preserved, so that the subsequent processing of the writing track and the written image is performed.
  • the camera is mounted directly above the touch display screen, but during the specific video recording process, the pen is always in motion in the area where the touch screen is located.
  • the relative position of the trajectory of the writing pen on the touch display screen in the touch screen display also changes, that is, for the camera, the relative position of the trajectory of the writing pen on the touch screen display is not at the camera.
  • the written image captured by the camera will have a certain angle with the position of the camera, causing distortion of the captured video image.
  • the rectangular display screen may be captured, and the resulting trapezoidal image may be obtained.
  • the initial written image needs to be subjected to distortion correction to acquire a written image corresponding to the display shape of the touch display screen.
  • the initial written image is corrected by using the Opencv library function, for example, IcvCloneImage (this function is used to copy image content, and does not need to open up memory.
  • IcvCloneImage this function is used to copy image content, and does not need to open up memory.
  • 2cvGetPerspectiveTransform projected transformation by four points. In programming, you need to get the four vertices of the original image and the four vertices of the standard rectangle. And then this function performs a numerical calculation of the conversion between the two), and 34cvWarpPerspective (this function is used to perform a perspective transformation on the image. The coordinates of the four vertices of the original image are converted to the new four vertices), and the above can be used.
  • the three functions perform distortion correction on the initial written image to obtain a written image corresponding to the display shape of the touch display screen.
  • the initial written image is subjected to distortion correction using the Opencv library function, so that the obtained written image corresponds to the display shape of the touch display screen.
  • the screen shape of the touch display screen is rectangular, and the distortion of the initial written image is corrected, so that the shape of the written image obtained after the correction is also rectangular, thereby avoiding the distortion phenomenon that the camera acquires the video image.
  • S250 Scale the writing track proportionally according to a proportional relationship between the size of the written image and the size of the touch display screen.
  • the screen of the touch screen of the tablet computer is a rectangle with a ratio of 3:4, when writing the content by using the writing software, it is necessary to write in the specified frame, and the size of the written image is compared with the touch.
  • the size of the written image of the display screen is small. For example, if the size of the written border specified in the writing software is 2.7:3.6, the initial written image is corrected by the Opencv library function to make the written image.
  • the display shape also becomes a rectangle with a ratio of 2.7:3.6, and the size of the writing track is adjusted by the image scaling algorithm so that the size ratio of the writing track is also changed to 2.7:3.6.
  • the size of the touch screen of the tablet computer is 18 cm * 24 cm
  • the size of the writing frame specified by the writing software in the tablet is 16.2 cm * 21.6 cm
  • the size of the writing image is also 16.2 cm * 21.6cm
  • the writing trajectory will be scaled by the image scaling algorithm to have a size of 16.2 cm * 21.6 cm.
  • the image, as the initial written image the size of the written image obtained after the distortion correction of the initial written image is correspondingly smaller, that is, the size of the written image is smaller than the size of the touch display screen, and the written image is also required.
  • the scale relationship between the size and the size of the touch display screen is scaled by an image scaling algorithm such that the size of the written track corresponds to the size of the touch display screen, so that the combined effect of the written track and the written image is better.
  • image scaling is a process of adjusting the size of a digital image. As the size of an image increases, the visibility of the pixels that make up the image will become higher, making the image appear "soft.” Conversely, shrinking an image will enhance its smoothness and sharpness.
  • image magnification is an interpolation method, that is, a new interpolation element is used to insert a new element between pixel points based on the original image pixels.
  • the size of the writing track is adapted to the screen size of the tablet, but the size of the written image obtained by the distortion correction is not the screen size of the tablet.
  • the writing trajectory should be scaled proportionally according to the proportional relationship between the size of the written image and the size of the touch screen.
  • each frame of the written image is composed of several pixels.
  • the writing trajectory is composed of several sets of writing points. Due to the scaling of the written image and the written trajectory, the first pixel coordinates of the written image and the set of writing points of the written trajectory also change accordingly.
  • the length of time that the camera records a video image is determined by the frame rate of the camera. For example, if the frame rate of the camera is 30 frames/second, then the length of time that the camera records a video image is 1/30. Seconds; or, the frame rate of the camera is 60 frames/second, then the length of time that the camera records a video image is 1/60 second.
  • the writing content on the touch display screen does not change much during the time period of writing the image in one frame, that is, the time of writing a letter, a number, a Chinese character
  • the time of writing a letter There is at least one frame of written image inside. Therefore, in the process of writing a letter, there may be multiple frames of writing images, and similarly, one letter corresponds to a plurality of pixel coordinates, that is, one letter corresponds to a different pixel point.
  • the acquired first pixel coordinate is acquired when starting to write the P letter of the first line
  • the first line and the method of scaling the set of writing points are The writing trajectory of the second line is scaled to obtain a set of writing points of the scaled writing trajectory, and based on obtaining the time axis corresponding to the writing of the first line and the second line, and based on the writing of the first line and the second line corresponding to the first line a pixel coordinate, covering a set of writing points corresponding to the first row and the second row to corresponding pixel points in the corresponding writing image, so that the content of the writing point set is correspondingly added to the corresponding writing image, that is, when When the user writes the content on the touch display screen, the other content on the touch display screen is correspondingly presented on the user's
  • a user when a user writes content on a touch display screen (a specific writing may be a finger or a writing pen corresponding to a touch detection scheme, a writing pen in FIG. 5), it is written.
  • the above method can also be used to merge the writing track into the written image to improve the user's explanation effect and the learner's learning efficiency.
  • the technical solution of the embodiment is: reading an initial image of the area where the touch display screen is captured by the camera facing the touch display screen; performing image recognition on the initial image to extract an initial written image of the area where the touch display screen is located;
  • the initial writing image is subjected to distortion correction to obtain a writing image corresponding to the display shape of the touch display screen;
  • the writing trajectory generated on the touch screen is captured during the capturing of the camera, and the acquisition time of the writing trajectory is mapped to the time axis of the writing image;
  • the scale relationship between the size of the written image and the size of the touch screen display scales the writing trajectory proportionally; acquires the first pixel coordinates of the written image, and the set of writing points of the scaled writing trajectory; confirms the corresponding writing trajectory based on the time axis
  • the method of writing at least one frame and covering the corresponding pixel points in the corresponding written image based on the first pixel coordinates solves the problem that the written content is blocked by the user's hand during the video teaching process, and the
  • FIG. 6 is a schematic structural diagram of a video processing apparatus according to Embodiment 3 of the present invention.
  • the embodiment is applicable to the case of recording or remotely teaching a video.
  • the device may be implemented by using software and/or hardware.
  • the video processing device specifically includes: acquiring a writing image module 310 and collecting The writing track module 320 and the adding writing track module 330 are added.
  • the obtaining a writing image module 310 is configured to acquire, from the video data captured by the camera facing the touch display screen, a writing image corresponding to an area where the touch display screen is located;
  • the acquisition writing track module 320 is configured to collect a writing track generated on the touch display screen during the shooting process of the camera, and map the collecting time of the writing track to the time axis of the writing image;
  • a writing trajectory module 330 is added for adding a writing trajectory to the writing image based on the relative position of the time axis and the writing trajectory in the touch display screen.
  • the writing image corresponding to the area where the touch display screen is located is obtained from the video data captured by the camera facing the touch display screen; the writing track generated on the touch screen is captured during the shooting of the camera, and the writing is performed.
  • the acquisition time of the trajectory is mapped to the time axis of the written image; the technical means of adding the writing trajectory to the written image based on the relative position of the time axis and the writing trajectory in the touch display screen solves the problem that the writing content is blocked by the hand during the video teaching process.
  • the problem is to improve the efficiency of video teaching and the effect of students' attention during the learning process.
  • acquiring the writing image module 310 includes:
  • Reading the initial image unit 311, configured to read an initial image of the area including the touch display screen captured by the camera facing the touch display screen;
  • the writing image unit 313 is obtained for performing distortion correction on the initial writing image to acquire a writing image corresponding to the display shape of the touch display screen.
  • adding the writing track module 330 includes:
  • the scale writing track unit 331 is configured to scale the writing track proportionally according to the proportional relationship between the size of the written image and the size of the touch display screen;
  • An obtaining unit 332, configured to acquire a first pixel coordinate of the written image, and a set of writing points of the scaled writing track;
  • the overlay unit 333 is configured to confirm at least one frame of the written image corresponding to the written trajectory based on the time axis, and cover the corresponding pixel points in the corresponding written image based on the first pixel coordinates.
  • the video processing apparatus further includes: a generated video data module 340, configured to perform audio collection based on time axis synchronization to acquire audio data, and associate the audio data with the written image to generate video data.
  • a generated video data module 340 configured to perform audio collection based on time axis synchronization to acquire audio data, and associate the audio data with the written image to generate video data.
  • the video processing device can perform the video processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
  • FIG. 7 is a schematic structural diagram of a terminal device according to Embodiment 4 of the present invention.
  • FIG. 7 shows a block diagram of an exemplary computer device 412 suitable for use in implementing embodiments of the present invention.
  • the computer device 412 shown in Figure 7 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present invention.
  • computer device 412 is embodied in the form of a general purpose computing device.
  • Components of computer device 412 may include, but are not limited to, one or more processors or processing units 416, system memory 428, and a bus 418 that connects different system components, including system memory 428 and processing unit 416.
  • Bus 418 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.
  • these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA Bus, a Video Electronics Standards Association (VESA) local bus, and peripheral component interconnects ( PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI peripheral component interconnects
  • Computer device 412 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 412, including volatile and nonvolatile media, removable and non-removable media.
  • System memory 428 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 430 and/or cache memory 432.
  • Computer device 412 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • storage system 434 can be used to read and write non-removable, non-volatile magnetic media (not shown in Figure 7, commonly referred to as "hard disk drives").
  • a disk drive for reading and writing to a removable non-volatile disk such as a "floppy disk”
  • a removable non-volatile disk such as a CD-ROM, DVD-ROM
  • other optical media read and write optical drive.
  • each drive can be coupled to bus 418 via one or more data medium interfaces.
  • System memory 428 can include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.
  • Program/utility 440 having a set (at least one) of program modules 442, which may be stored, for example, in system memory 428, including, but not limited to, an operating system, one or more applications, others Program modules and program data, each of these examples or some combination may include an implementation of a network environment.
  • Program module 442 typically performs the functions and/or methods of the embodiments described herein.
  • Computer device 412 can also be in communication with one or more external devices 414 (eg, a keyboard, pointing device, display 424, etc.), and can also be in communication with one or more devices that enable a user to interact with the computer device 412, and/or Any device (eg, a network card, modem, etc.) that enables the computer device 412 to communicate with one or more other computing devices. This communication can take place via an input/output (I/O) interface 422. Moreover, computer device 412 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) via network adapter 420.
  • networks e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet
  • network adapter 420 communicates with other modules of computer device 412 via bus 418. It should be understood that although not shown in the figures, other hardware and/or software modules may be utilized in connection with computer device 412, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives. And data backup storage systems, etc.
  • the processing unit 416 performs various function applications and data processing by running a program stored in the system memory 428, for example, implementing the video processing method provided by the embodiments of the present invention:
  • the writing trajectory generated on the touch screen is captured during the shooting of the camera, and the time of collecting the writing trajectory is mapped to the time axis of the writing image;
  • a writing trajectory is added to the writing image based on the relative position of the time axis and the writing trajectory in the touch display screen.
  • acquiring a written image corresponding to an area where the touch display screen is located from the video data captured by the camera facing the touch display screen includes:
  • Distortion correction of the initial written image acquires a written image corresponding to the display shape of the touch display screen.
  • adding a writing trajectory to the writing image based on the relative position of the time axis and the writing trajectory in the touch display screen includes:
  • the writing track is scaled proportionally according to the proportional relationship between the size of the written image and the size of the touch display screen;
  • At least one frame of the written image corresponding to the written trajectory is confirmed based on the time axis, and the set of the written points is overlaid on the corresponding pixel in the corresponding written image based on the first pixel coordinates.
  • the video processing method further includes:
  • Audio acquisition is performed based on time axis synchronization to acquire audio data, and audio data is associated with the written image to generate video data.
  • the fifth embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, and when the program is executed by the processor, the video processing method provided by all the embodiments of the present invention is implemented:
  • the writing trajectory generated on the touch screen is captured during the shooting of the camera, and the time of collecting the writing trajectory is mapped to the time axis of the writing image;
  • a writing trajectory is added to the writing image based on the relative position of the time axis and the writing trajectory in the touch display screen.
  • the computer storage medium of the embodiments of the present invention may employ any combination of one or more computer readable mediums.
  • the computer readable medium can be a computer readable signal medium or a computer readable storage medium.
  • the computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.
  • a computer readable storage medium can be any tangible medium that can contain or store a program, which can be used by or in connection with an instruction execution system, apparatus or device.
  • a computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying computer readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer readable signal medium can also be any computer readable medium other than a computer readable storage medium, which can transmit, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. .
  • Program code embodied on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including an object oriented programming language - such as Java, Smalltalk, C++, and also conventional. Procedural programming language - such as the "C" language or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer via any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider) Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider Internet service provider

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本发明公开了一种视频处理方法、装置、终端设备和存储介质。该方法包括：从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像；采集所述摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将所述书写轨迹的采集时刻映射到所述书写图像的时间轴；基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像。本发明解决了在视频教学过程中，书写内容被用户的手所遮挡的问题，提高了视频教学的效率以及学生在学习过程中的注意力。

Description

一种视频处理方法、装置、终端设备和存储介质 技术领域
本发明实施例涉及图像处理技术，尤其涉及一种视频处理方法、装置、终端设备和存储介质。
背景技术
随着网络的不断发展，教学辅助手段也随之进步。进入21世纪，随着PC机、笔记本设备的不断普及，网络教育也随着网络技术的发展不断兴起。
在现有技术中，可以将老师的讲课过程录制成多媒体文件，能帮助学生方便地自学、复习和预习，还可以在教师、学校之间形成资源共享，提高师资教学力量。同时，也可以通过网络对不同地区的学习者进行远程视频教学。
而在实际教学过程中，老师在书写板上进行书写讲解时，学生注意力在老师所书写的内容上，但是正在书写的部分内容或者已书写的内容可能被老师的手遮挡住，如图1所示，老师在第一行所书写的内容为P(C)＝0.01＝1％，但由于老师在书写第二行的内容时，将第一行的“0.01＝1％”部分内容用手遮挡住，需要等到老师将手移动之后，这部分书写的内容才能被学生接收到，从而影响了学生在学习过程中的注意力。
发明内容
本发明提供一种视频处理方法、装置、终端设备和存储介质，以实现提高视频教学的效率以及学生在学习过程中的注意力。
第一方面，本发明实施例提供了一种视频处理方法，该方法包括：
从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像；
采集所述摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将所述书写轨迹的采集时刻映射到所述书写图像的时间轴；
基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像。
第二方面，本发明实施例还提供了一种视频处理装置，该装置包括：
获取书写图像模块，用于从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像；
采集书写轨迹模块，用于采集所述摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将所述书写轨迹的采集时刻映射到所述书写图像的时间轴；
添加书写轨迹模块，用于基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像。
第三方面，本发明实施例还提供了一种终端设备，所述设备包括：
一个或多个处理器；
存储器，用于存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述任一所述的视频处理方法。
第四方面，本发明实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述任一所述的视频处理方法。
本发明的一个技术方案通过从朝向触摸显示屏的摄像头拍摄到的视频数据 中获取对应于触摸显示屏所在区域的书写图像；采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像的技术手段，解决了在视频教学过程中，书写内容被手遮挡的问题，提高了视频教学的效率以及学生在学习过程中的注意力的效果。
附图说明
图1是现有技术中书写内容被手遮挡的状态示意图。
图2是本发明实施例一中的一种视频处理方法的流程图。
图3是本发明实施例一中的一种视频录制系统的结构示意图。
图4是本发明实施例二中的一种视频处理方法的流程图。
图5是本发明实施例二中书写内容在手之上显示的状态示意图。
图6是本发明实施例三中的一种视频处理装置的结构示意图。
图7是本发明实施例四中的一种终端设备的结构示意图。
具体实施方式
下面结合附图和实施例对本发明作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释本发明，而非对本发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本发明相关的部分而非全部结构。
实施例一
图2为本发明实施例一提供的视频处理方法的流程图，本实施例可适用于 在录制课程过程中避免书写内容被手遮挡或在远程教学过程中避免书写内容被手遮挡的情况，该方法可以由本发明实施例提供的视频处理装置来执行，该装置可采用软件和/或硬件的方式实现，并一般可集成在终端设备中。该视频处理方法具体包括如下步骤：
S110、从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像。
在本发明的具体实施例中，通过摄像头对用户的教学过程进行视频录制，如图3所示，为一种视频录制系统的结构示意图。在此实施例中，通过USB连接线13将带有触摸显示屏10的平板电脑14和摄像头11进行连接，并将摄像头11安装在触摸显示屏10的正上方，即摄像头11的拍摄方向是朝向平台电脑14的触摸显示屏10的一侧。其中，触摸显示屏并不仅仅限定于平板电脑，也可采用其它具有触摸显示屏的电子设备，比如，带有触摸显示屏的智能手机和笔记本电脑等。
作为示例而非限定，当用户通过USB连接线将摄像头和平板电脑进行连接后，将平板电脑中的书写软件打开，进入书写模式，即，书写笔在平板电脑的触摸显示屏的屏幕上进行书写，显示出笔迹的模式是类似于画图板中的笔模式时，将调整摄像头和平板电脑的方向，使其平板电脑的屏幕尽量处于摄像头的正下方，并打开摄像头开始录制，此时用户将在触摸显示屏的屏幕上书写所要讲解的内容，同时，将通过摄像头获取用户在触摸显示屏所在区域进行书写的图像。
S120、采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴。
其中，时间轴是随着摄像头开启，并开始对用户在触摸显示屏上书写的内容进行录制而开始的，随着时间的推移而生成直到完成课程的录制或远程教学而结束，其用于记录书写图像和书写轨迹采集过程中的时间状态以及二者之间的时间关联。书写轨迹是由无数个书写点集而组成的，当用户(可为老师，也可为对视频进行录制的其他人员)在触摸显示屏上进行书写时，触摸显示屏将实时采集书写点集，并将所有的书写点集进行组合，以形成书写轨迹，并将书写轨迹保存在平板电脑的存储器中，同时，将采集书写轨迹的时刻与获取书写图像的时刻一一对应。
在本发明的具体实施例中，当用户开始在触摸显示屏上进行内容书写时，将会触发书写软件系统的触摸事件，平板电脑中的书写软件将通过系统提供的触摸事件，获取触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴上。
S130、基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像。
在本方案中，书写图像因为需要进行连续获取，但随着时间的推移，用户利用书写笔在触摸显示屏上进行书写内容的过程中，书写轨迹在触摸显示屏中的相对位置是一直在变化的。按照获取书写图像的时间顺序和书写轨迹在触摸显示屏中的相对位置，将书写轨迹和书写图像进行一一对应，并将书写轨迹添加到书写图像中。
一般而言，在用户进行书写过程中，相对来说书写图像的刷新速度比书写轨迹的刷新速率较快，即，书写轨迹至少对应一帧书写图像，而每帧书写图像是一张张静止的图像，并且，书写轨迹在触摸显示屏上的相对位置也是变化的。 按照其在触摸显示屏上的相对位置和对应的至少一帧书写图像，将采集到的书写轨迹覆盖到书写图像中，即，将书写点集覆盖对应的书写图像中对应的像素点上，以使书写点集和对应的像素点进行合并，从而，书写点集将作为一帧帧的视频画面在显示屏上进行显示，从而避免了用户的手遮挡住书写内容，使观看视频内容的学习者不受用户的影响，而降低其自身的学习效率。
具体来说，当用户在触摸显示屏上进行内容书写时，在第一行上书写内容依次为“P(C)＝0.01＝1％”，第二行上的书写内容依次为“P(Pos C)＝0.9＝90％”，若将用户所书写的内容添加到书写图像中，当用户再次对第一行的内容进行讲解，并将手放在第一行的位置时，观看视频内容的学习者，也可看到第二行的内容，从而提高了用户讲解内容的效率，以及学习者的学习效率。
本实施例的技术方案，通过从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像；采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像的技术手段，解决了在视频教学过程中，书写内容被手遮挡的问题，提高了视频教学的效率以及学生在学习过程中的注意力的效果。
进一步地，在上述技术方案的基础上，在基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像之后，该视频处理方法，还包括：基于时间轴同步进行音频采集获取音频数据，将音频数据与书写图像关联生成视频数据。
当然，在摄像头拍摄到的视频数据中，一方面包含了在视频录制过程中，用户在触摸显示屏上进行书写的内容，另一方面，还包括在视屏录制过程中用 户对书写内容进行讲解的音频数据。
在本方案中，基于获取书写图像的时间顺序，将与书写图像的获取时刻相对应的音频数据进行关联保存，以生成网络视频课程的视频数据，并将相关视频数据信息保存到与摄像头关联的终端设备中，在本方案中，是将视频数据信息保存到平板电脑中的存储器中，并通过平板电脑将采集的视频图像进行播放显示。
当然，观看视频的学习者通过显示屏看到的视频画面，实际上是按照一定的刷新频率显示的一幅幅具有一定联系的静止的视频图像，通常将“帧”作为视频图像的单位。由于人的眼睛具有“视觉暂留”功能，因此，将多个离散的视频图像按照一定的频率显示，就可以看到连续的视频画面。相应地，在摄像头录制视频过程中，也是一帧一帧获取的视频图像。
本实施例的技术方案，进一步地，通过基于时间轴同步进行音频采集获取音频数据，将音频数据与书写图像关联生成视频数据的技术手段，将合并有音频数据的视频图像在终端设备中进行播放显示，以使学习者对教学内容进行观看学习。
实施例二
图4为本发明实施例二提供的视频处理方法的流程图，本实施例可适用于对微课的录制或者远程教学中书写内容被手遮挡的情况，本实施例是在实施例一的基础上，对从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像，以及基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像，作进一步地优化，具体包括如下步骤：
S210、读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像。
在本发明的具体实施例中，将摄像头安装在触摸显示屏的正上方，但在具体录制过程中，摄像头可拍摄得到的视频图像不仅仅为触摸显示屏所在区域的视频图像，可能也包含有触摸显示屏之外的视频图像。
同时，采用平板电脑中的书写软件进行内容书写，大部分的书写软件都会设置有一个指定的边框，使用户在指定的边框内进行内容书写，因此，摄像头获取的初始图像中，同时还包含有指定的边框外的相关视频图像。
S220、对初始图像进行图像识别提取出触摸显示屏所在区域的初始书写图像。
具体来说，为了后续将用户所书写的书写轨迹和摄像头所采集到的视频图像进行一一对应，需对初始图像进行预处理，以识别提取出触摸显示屏所在区域的初始书写图像。
示例性地，通过Opencv开源库中的轮廓检测算法，检测初始图像中触摸显示屏所在区域的边界，并根据边界对初始图像进行图像识别提取，以得到触摸显示屏所在区域的初始书写图像。具体来说，轮廓检测算法，可分为静态图像轮廓检测和动态视频轮廓检测，其中，动态视频轮廓检测中的第一类利用光流法获取运动线索。光流法基于时空梯度模型计算物体的亮度随时间和空间的变化场。光流计算像素点的瞬时变化速率，其原理是假设像素点从一个位置移动到其他位置，其灰度值保持不变，因此根据光流估计可以计算出某一像素点运动瞬时变化速率。第二类利用帧差法获取运动线索，帧差法也是运动目标检测中常见的方法之一。其基本原理是，先计算前后两帧相减的像素差，利用固定 闭值进行二值化处理，得到前景运动目标。如果目标没有运动，那么两帧相减的像素差为零；若目标有明显的运动，那么两帧相减的像素差大于闭值，二值化处理可得到视频帧中的运动目标位置。
在本方案中，可通过静态图像轮廓检测，将摄像头获取到的初始图像进行识别提取，以实现只保留触摸显示屏所在区域的书写图像，以便于后续对书写轨迹和书写图像进行合并处理。
S230、对初始书写图像进行畸变校正获取与触摸显示屏的显示形状相对应的书写图像。
具体地，将摄像头安装在触摸显示屏的正上方，但在具体视频录制过程中，书写笔在触摸显示屏所在区域中是一直处于运动状态的。同样地，书写笔在触摸显示屏上进行书写的轨迹在触摸显示屏中的相对位置也是变化的，即对于摄像头来说，书写笔在触摸显示屏上书写的轨迹的相对位置并不是在摄像头的正下方，相应地，摄像头采集到的书写图像将和摄像头的位置存在一定的角度，造成获取到的视频画面失真，比如来说，对矩形的显示屏进行拍摄，得到的可能是梯形的画面，则需要对初始书写图像进行畸变校正，以获取与触摸显示屏的显示形状相对应的书写图像。
作为示例而非限定，在本实施例的技术方案中，采用Opencv库函数，对初始书写图像进行畸变校正，比如，IcvCloneImage(该函数用来复制图像内容，且不需要开辟内存。在编程时，利用此函数将原图复制到一个新的位置，以方便后面的畸变校正)、2cvGetPerspectiveTransform(由四点计算投射变换。在编程时需要取得原图的四个顶点左边和标准矩形的四个顶点坐标，然后由此函数进行两者之间转换的数值计算)，以及34cvWarpPerspective(该函数用 来对图像进行透视变换。即将原图的四个顶点坐标转换到新的四个顶点)，可采用上述的三个函数对初始书写图像进行畸变校正，以得到触摸显示屏的显示形状相对应的书写图像。
当然，本实施例中采用Opencv库函数对初始书写图像进行畸变校正，从而获取得到的书写图像与触摸显示屏的显示形状相对应。具体地，触摸显示屏的屏幕形状为矩形的，通过对初始书写图像进行畸变校正，从而使校正后得到的书写图像的形状也为矩形的，从而避免了摄像头获取视频图像的失真现象。
S240、采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴。
S250、根据书写图像的尺寸和触摸显示屏的尺寸的比例关系对书写轨迹进行等比例的缩放。
具体来说，假如平板电脑的触摸显示屏的屏幕是比例为3∶4的矩形，但因采用书写软件对内容进行书写时，需在指定的边框内进行书写，则书写图像的尺寸相比触摸显示屏的屏幕尺寸来说，其书写图像的尺寸较小，比如，利用书写软件中指定的书写边框的尺寸为2.7∶3.6，则当通过Opencv库函数对初始书写图像进行畸变校正，使书写图像的显示形状也会变为比例为2.7∶3.6的矩形，则通过图像缩放算法对书写轨迹的尺寸进行调整，使其书写轨迹的尺寸比例也转变为2.7∶3.6。
示例性地，假如平板电脑的触摸显示屏的尺寸为18cm*24cm，则平板电脑中的书写软件指定的书写边框的尺寸为16.2cm*21.6cm，相应地，书写图像的尺寸也为16.2cm*21.6cm，为了使书写图像和书写轨迹实现更好的合并效果，将通过图像缩放算法对书写轨迹进行缩放，使其的尺寸大小也变为 16.2cm*21.6cm。
当然，当用户习惯性地在书写软件的指定边框的左侧开始进行内容书写时，比如，在第一行上书写内容依次为“P(C)＝0.01＝1％”，第二行上的书写内容依次为“P(Pos C)＝0.9＝90％”，若从初始图像中只识别提取具有P(C)＝0.01＝1％和P(Pos C)＝0.9＝90％等书写内容的图像，作为初始书写图像时，当对初始书写图像进行畸变校正后所得到的书写图像的尺寸也会相应的变小，即，书写图像的尺寸比触摸显示屏的尺寸小，也需要按照书写图像的尺寸和触摸显示屏的尺寸的比例关系，通过图像缩放算法对书写轨迹进行缩放，以使得书写轨迹的尺寸与触摸显示屏的尺寸向对应，以使得书写轨迹和书写图像的合成效果更好。
其中，图像缩放是对数字图像的大小进行调整的过程。当一个图像的大小增加之后，组成图像的像素的可见度将会变得更高，从而使得图像表现得“软”。相反地，缩小一个图像将会增强它的平滑度和清晰度。一般来说，图像放大都是采用内插值方法，即在原有图像像素的基础上在像素点之间采用合适的插值算法插入新的元素。在本方案中，因书写轨迹是在平板电脑的屏幕上进行书写的，其书写轨迹的尺寸与平板电脑的屏幕尺寸相适应，但畸变校正得到的书写图像的尺寸并不是和平板电脑的屏幕尺寸完全相同的，为了保证书写图像和书写轨迹的合成效果，需按照书写图像的尺寸和触摸显示屏的尺寸的比例关系对书写轨迹进行等比例的缩放。
S260、获取书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集。
具体来说，每一帧书写图像都是由若干个像素点组成的。同时，书写轨迹是由若干个书写点集构成的。由于对书写图像和书写轨迹进行缩放，其书写图像的第一像素坐标以及书写轨迹的书写点集也相应的发生变化。
S270、基于时间轴确认书写轨迹对应的至少一帧书写图像，基于第一像素坐标将书写点集覆盖对应的书写图像中对应的像素点。
在此需要说明的是，摄像头录制一帧视频图像的时间长度由摄像头的帧率决定，比如，摄像头的帧率为30帧/秒，那么，摄像头录制一帧视频图像的时间长度为1/30秒；或者，摄像头的帧率为60帧/秒，那么，摄像头录制一帧视频图像的时间长度为1/60秒。
而在实际的录制过程中，按照正常的书写速度而言，在一帧书写图像的时间段内，在触摸显示屏上的书写内容并没有多大变化，即，书写一个字母、数字、汉字的时间内至少含有一帧书写图像。因此，在书写一个字母的过程中，可能有多帧书写图像，同样地，一个字母对应多个像素坐标，即一个字母对应不同的像素点。
通过本技术方案，当用户在触摸显示屏上进行内容书写时，在第一行上书写内容依次为“P(C)＝0.01＝1％”，第二行上的书写内容依次为“P(Pos C)＝0.9＝90％”，则获取的第一像素坐标是在开始书写第一行的P字母时而获取的，同样地，通过上述对书写点集进行缩放的方法对该第一行和第二行的书写轨迹进行缩放，以获取缩放后的书写轨迹的书写点集，并基于获取书写第一行和第二行对应的时间轴，以及基于书写第一行和第二行对应的第一像素坐标，将第一行和第二行对应的书写点集覆盖到对应的书写图像中对应的像素点上，以使得书写点集的内容相应地添加到对应的书写图像中，即，当用户在触摸显示屏上进行内容书写时，触摸显示屏上的其它内容相应地呈现在用户的手之上，从而观看视频内容的学习者可以看到整个触摸显示屏上的内容，而不必因用户的手遮挡到书写内容，影响到用户讲解的效果，以及学习者的学习 效率。
作为示例而非限定，如图5所示，当用户在触摸显示屏上进行内容书写(具体的书写可以是手指或对应触摸检测方案的书写笔，图5中为书写笔)时，其所书写内容的书写点集利用上述的方法进行采集，并合并到书写图像中，即，将第一行用户所书写的内容：“P(C)＝0.01＝1％”等内容的书写点集均覆盖到其所对应的书写图像的像素点中，以使得用户在进行第二行内容：“P(Pos C)＝0.9＝90％”的书写时，第一行的内容“P(C)＝0.01＝1％”均在书写图像中进行显示，即书写内容在用户的手之上，从而避免了观看视频的学习者因用户的手所遮挡，而看不到在触摸显示屏上所书写的内容，进而提高了用户对内容进行讲解的效果，以及观看视频的学习者的学习效率。
同样地，由于不同用户的书写习惯，可能导致用户在书写内容的过程中，其正在书写的内容被用户自身的手所遮挡，也会造成用户在讲解内容时，观看视频的学习者的效果不佳，同样也可采用上述方法将书写轨迹合并到书写图像中，以提高用户的讲解效果和学习者的学习效率。
本实施例的技术方案，通过读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像；对初始图像进行图像识别提取出所述触摸显示屏所在区域的初始书写图像；对初始书写图像进行畸变校正获取与触摸显示屏的显示形状相对应的书写图像；采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；根据书写图像的尺寸和触摸显示屏的尺寸的比例关系对书写轨迹进行等比例的缩放；获取书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集；基于时间轴确认书写轨迹对应的至少一帧书写图像，基于第一像素坐标将书写点集覆盖对应的书 写图像中对应的像素点的技术手段，解决了在视频教学过程中，书写内容被用户的手所遮挡的问题，提高了用户讲解内容的效果，以及观看视频的学习者的学习效率。
实施例三
图6为本发明实施例三提供的一种视频处理装置的结构示意图。本实施例可适用于对视频进行录制或者远程教学的情况，该装置可以采用软件和/或硬件的方式来实现，如图6所示，该视频处理装置具体包括：获取书写图像模块310，采集书写轨迹模块320和添加书写轨迹模块330。
其中，获取书写图像模块310，用于从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像；
采集书写轨迹模块320，用于采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；
添加书写轨迹模块330，用于基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像。
本实施例的技术方案，通过从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像；采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像的技术手段，解决了在视频教学过程中，书写内容被手遮挡的问题，提高了视频教学的效率以及学生在学习过程中的注意力的效果。
进一步地，获取书写图像模块310，包括：
读取初始图像单元311，用于读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像；
提取初始书写图像单元312，用于对初始图像进行图像识别提取出触摸显示屏所在区域的初始书写图像；
获取书写图像单元313，用于对初始书写图像进行畸变校正获取与触摸显示屏的显示形状相对应的书写图像。
进一步地，添加书写轨迹模块330，包括：
缩放书写轨迹单元331，用于根据书写图像的尺寸和触摸显示屏的尺寸的比例关系对书写轨迹进行等比例的缩放；
获取单元332，用于获取书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集；
覆盖单元333，用于基于时间轴确认书写轨迹对应的至少一帧书写图像，基于第一像素坐标将书写点集覆盖对应的书写图像中对应的像素点。
进一步地，该视频处理装置，还包括：生成视频数据模块340，用于基于时间轴同步进行音频采集获取音频数据，将音频数据与书写图像关联生成视频数据。
上述视频处理装置可执行本发明任意实施例所提供的视频处理方法，具备执行方法相应的功能模块和有益效果。
实施例四
图7为本发明实施例四提供的一种终端设备的结构示意图。图7示出了适于用来实现本发明实施方式的示例性计算机设备412的框图。图7显示的计算 机设备412仅仅是一个示例，不应对本发明实施例的功能和使用范围带来任何限制。
如图7所示，计算机设备412以通用计算设备的形式表现。计算机设备412的组件可以包括但不限于：一个或者多个处理器或者处理单元416，系统存储器428，连接不同系统组件(包括系统存储器428和处理单元416)的总线418。
总线418表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(ISA)总线，微通道体系结构(MAC)总线，增强型ISA总线、视频电子标准协会(VESA)局域总线以及外围组件互连(PCI)总线。
计算机设备412典型地包括多种计算机系统可读介质。这些介质可以是任何能够被计算机设备412访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
系统存储器428可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(RAM)430和/或高速缓存存储器432。计算机设备412可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统434可以用于读写不可移动的、非易失性磁介质(图7未显示，通常称为“硬盘驱动器”)。尽管图7中未示出，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如CD-ROM，DVD-ROM或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线418相连。系统存储器428可以包括至少一个程序产品，该程序产品具有一组(例如至少 一个)程序模块，这些程序模块被配置以执行本发明各实施例的功能。
具有一组(至少一个)程序模块442的程序/实用工具440，可以存储在例如系统存储器428中，这样的程序模块442包括——但不限于——操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块442通常执行本发明所描述的实施例中的功能和/或方法。
计算机设备412也可以与一个或多个外部设备414(例如键盘、指向设备、显示器424等)通信，还可与一个或者多个使得用户能与该计算机设备412交互的设备通信，和/或与使得该计算机设备412能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口422进行。并且，计算机设备412还可以通过网络适配器420与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器420通过总线418与计算机设备412的其它模块通信。应当明白，尽管图中未示出，可以结合计算机设备412使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
处理单元416通过运行存储在系统存储器428中的程序，从而执行各种功能应用以及数据处理，例如实现本发明实施例所提供的视频处理方法：
从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像；
采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；
基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像。
进一步地，从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像，包括：
读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像；
对初始图像进行图像识别提取出触摸显示屏所在区域的初始书写图像；
对初始书写图像进行畸变校正获取与触摸显示屏的显示形状相对应的书写图像。
进一步地，基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像，包括：
根据书写图像的尺寸和触摸显示屏的尺寸的比例关系对书写轨迹进行等比例的缩放；
获取书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集；
基于时间轴确认书写轨迹对应的至少一帧书写图像，基于第一像素坐标将书写点集覆盖对应的书写图像中对应的像素点。
进一步地，视频处理方法还包括：
基于时间轴同步进行音频采集获取音频数据，将音频数据与书写图像关联生成视频数据。
实施例五
本发明实施例五还提供了一种计算机可读存储介质，其上存储有计算机程 序，该程序被处理器执行时实现如本申请所有发明实施例提供的视频处理方法：
从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于触摸显示屏所在区域的书写图像；
采集摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将书写轨迹的采集时刻映射到书写图像的时间轴；
基于时间轴和书写轨迹在触摸显示屏中的相对位置将书写轨迹添加到书写图像。
本发明实施例的计算机存储介质，可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是--但不限于--电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用 或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于无线、电线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言-诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言-诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)-连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
注意，上述仅为本发明的较佳实施例及所运用技术原理。本领域技术人员会理解，本发明不限于这里所述的特定实施例，对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本发明的保护范围。因此，虽然通过以上实施例对本发明进行了较为详细的说明，但是本发明不仅仅限于以上实施例，在不脱离本发明构思的情况下，还可以包括更多其他等效实施例，而本发明的范围由所附的权利要求范围决定。

Claims (10)

  1. 一种视频处理方法，其特征在于，包括：
    从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像；
    采集所述摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将所述书写轨迹的采集时刻映射到所述书写图像的时间轴；
    基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像。
  2. 根据权利要求1所述的方法，其特征在于，所述从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像，包括：
    读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像；
    对所述初始图像进行图像识别提取出所述触摸显示屏所在区域的初始书写图像；
    对所述初始书写图像进行畸变校正获取与所述触摸显示屏的显示形状相对应的书写图像。
  3. 根据权利要求1所述的方法，其特征在于，所述基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像，包括：
    根据所述书写图像的尺寸和所述触摸显示屏的尺寸的比例关系对所述书写轨迹进行等比例的缩放；
    获取所述书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集；
    基于所述时间轴确认所述书写轨迹对应的至少一帧书写图像，基于所述第一像素坐标将所述书写点集覆盖对应的书写图像中对应的像素点。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    基于所述时间轴同步进行音频采集获取音频数据，将所述音频数据与所述书写图像关联生成视频数据。
  5. 根据权利要求1所述的方法，其特征在于，所述摄像头安装在所述触摸显示屏的正上方。
  6. 一种视频处理装置，其特征在于，包括：
    获取书写图像模块，用于从朝向触摸显示屏的摄像头拍摄到的视频数据中获取对应于所述触摸显示屏所在区域的书写图像；
    采集书写轨迹模块，用于采集所述摄像头拍摄过程中触摸显示屏上产生的书写轨迹，并将所述书写轨迹的采集时刻映射到所述书写图像的时间轴；
    添加书写轨迹模块，用于基于所述时间轴和所述书写轨迹在所述触摸显示屏中的相对位置将所述书写轨迹添加到所述书写图像。
  7. 根据权利要求6所述的装置，其特征在于，所述获取书写图像模块，包括：
    读取初始图像单元，用于读取朝向触摸显示屏的摄像头拍摄到的包含触摸显示屏所在区域的初始图像；
    提取初始书写图像单元，用于对所述初始图像进行图像识别提取出所述触摸显示屏所在区域的初始书写图像；
    获取书写图像单元，用于对所述初始书写图像进行畸变校正获取与所述触摸显示屏的显示形状相对应的书写图像。
  8. 根据权利要求6所述的装置，其特征在于，所述添加书写轨迹模块，包括：
    缩放书写轨迹单元，用于根据所述书写图像的尺寸和所述触摸显示屏的尺寸的比例关系对所述书写轨迹进行等比例的缩放；
    获取单元，用于获取所述书写图像的第一像素坐标，以及缩放后的书写轨迹的书写点集；
    覆盖单元，用于基于所述时间轴确认所述书写轨迹对应的至少一帧书写图像，基于所述第一像素坐标将所述书写点集覆盖对应的书写图像中对应的像素点。
  9. 一种终端设备，其特征在于，所述设备包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1～6中任一所述的视频处理方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现如权利要求1～6中任一所述的视频处理方法。
PCT/CN2017/116748 2017-11-10 2017-12-17 一种视频处理方法、装置、终端设备和存储介质 Ceased WO2019090903A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711103557.8 2017-11-10
CN201711103557.8A CN107909022B (zh) 2017-11-10 2017-11-10 一种视频处理方法、装置、终端设备和存储介质

Publications (1)

Publication Number Publication Date
WO2019090903A1 true WO2019090903A1 (zh) 2019-05-16

Family

ID=61844613

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/116748 Ceased WO2019090903A1 (zh) 2017-11-10 2017-12-17 一种视频处理方法、装置、终端设备和存储介质

Country Status (2)

Country Link
CN (1) CN107909022B (zh)
WO (1) WO2019090903A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112580574A (zh) * 2020-12-28 2021-03-30 北京翰立教育科技有限公司 一种基于识别手写文字的智能学习方法及装置
CN114202601A (zh) * 2021-11-17 2022-03-18 北京奕斯伟计算技术有限公司 板书信息处理方法、装置、电子设备及存储介质
CN114527922A (zh) * 2022-01-13 2022-05-24 珠海视熙科技有限公司 一种基于屏幕识别实现触控的方法及屏幕控制设备
CN115705133A (zh) * 2021-08-04 2023-02-17 青岛海尔科技有限公司 用于自动适配显示屏幕的方法及装置、电子设备、介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10873709B2 (en) * 2018-07-10 2020-12-22 Fuji Xerox Co., Ltd. Systems and methods for automatic detection and insetting of digital streams into a 360-degree video
JP2020098420A (ja) * 2018-12-17 2020-06-25 ソニー株式会社 画像処理装置、画像処理方法及びプログラム
CN109905733A (zh) * 2019-01-21 2019-06-18 侯力宇 一种新型视频购物系统
CN113014846B (zh) * 2019-12-19 2022-07-22 华为技术有限公司 一种视频采集控制方法、电子设备、计算机可读存储介质
CN115516505A (zh) * 2020-05-15 2022-12-23 索尼集团公司 图像处理设备、图像处理方法和程序
CN112770160A (zh) * 2020-12-24 2021-05-07 沈阳麟龙科技股份有限公司 一种股票分析视频创作系统及方法
CN113554904B (zh) * 2021-07-12 2023-04-07 江苏欧帝电子科技有限公司 一种多模式协同教育智能处理方法和系统
CN113971807B (zh) * 2021-10-12 2024-12-13 科大讯飞股份有限公司 一种笔迹信息的获取方法和设备
CN114120337B (zh) * 2021-10-15 2024-04-09 华南理工大学 实时的纸张书写笔迹恢复方法、系统、设备和存储介质
CN116844166B (zh) * 2023-08-24 2023-11-24 青岛罗博数码科技有限公司 一种基于学习行为的视频定位装置及方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101105895A (zh) * 2007-08-10 2008-01-16 上海迈辉信息技术有限公司 音视频多流合并教学训练系统及实现方法
CN102903265A (zh) * 2012-09-19 2013-01-30 河南智游网络技术有限公司 一种课堂教学自动记录的方法
CN103177611A (zh) * 2011-12-23 2013-06-26 李云峰 一种在电纸书上实现多媒体课件的方法
CN105120195A (zh) * 2015-09-18 2015-12-02 谷鸿林 内容录制、再现系统和方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100693343B1 (ko) * 2005-12-29 2007-03-12 이코리아시스템 (주) 사이버 전자칠판
CN101436353A (zh) * 2007-11-16 2009-05-20 北京新学堂网络科技有限公司 一种模拟真实课堂授课效果的远程授课视频制作方法
CN102929387B (zh) * 2012-09-25 2015-06-10 长安大学 一种基于普通纸笔的人机交互方法及人机交互系统
CN103945160A (zh) * 2014-02-27 2014-07-23 广州视睿电子科技有限公司 Android系统中实现远程视频对话方法和装置
CN105451065A (zh) * 2014-08-26 2016-03-30 天脉聚源（北京）教育科技有限公司 多种媒体的同步录制方法
CN104575120B (zh) * 2015-01-09 2017-02-22 代四广 一种用于辅助教学的展示系统
CN104575137B (zh) * 2015-01-19 2018-01-12 广州贝远信息技术有限公司 分体式情景互动多媒体智能终端
CN105635783A (zh) * 2015-12-31 2016-06-01 田雪松 多媒体文件制作方法
CN105513438A (zh) * 2016-01-21 2016-04-20 郑尚勤 教学示范仪
CN106970681A (zh) * 2017-02-21 2017-07-21 广州视源电子科技股份有限公司 书写显示方法及其系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101105895A (zh) * 2007-08-10 2008-01-16 上海迈辉信息技术有限公司 音视频多流合并教学训练系统及实现方法
CN103177611A (zh) * 2011-12-23 2013-06-26 李云峰 一种在电纸书上实现多媒体课件的方法
CN102903265A (zh) * 2012-09-19 2013-01-30 河南智游网络技术有限公司 一种课堂教学自动记录的方法
CN105120195A (zh) * 2015-09-18 2015-12-02 谷鸿林 内容录制、再现系统和方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112580574A (zh) * 2020-12-28 2021-03-30 北京翰立教育科技有限公司 一种基于识别手写文字的智能学习方法及装置
CN115705133A (zh) * 2021-08-04 2023-02-17 青岛海尔科技有限公司 用于自动适配显示屏幕的方法及装置、电子设备、介质
CN114202601A (zh) * 2021-11-17 2022-03-18 北京奕斯伟计算技术有限公司 板书信息处理方法、装置、电子设备及存储介质
CN114527922A (zh) * 2022-01-13 2022-05-24 珠海视熙科技有限公司 一种基于屏幕识别实现触控的方法及屏幕控制设备

Also Published As

Publication number Publication date
CN107909022B (zh) 2020-06-16
CN107909022A (zh) 2018-04-13

Similar Documents

Publication Publication Date Title
CN107909022B (zh) 一种视频处理方法、装置、终端设备和存储介质
US12008167B2 (en) Action recognition method and device for target object, and electronic apparatus
CN112261477B (zh) 视频处理方法及装置、训练方法和存储介质
CN113132800B (zh) 视频处理方法、装置、视频播放器、电子设备及可读介质
US11758217B2 (en) Integrating overlaid digital content into displayed data via graphics processing circuitry
WO2021213067A1 (zh) 物品显示方法、装置、设备及存储介质
KR101647969B1 (ko) 사용자 시선을 검출하기 위한 사용자 시선 검출 장치 및 그 방법과, 그 방법을 실행하기 위한 컴퓨터 프로그램
US10990226B2 (en) Inputting information using a virtual canvas
CN113934297B (zh) 一种基于增强现实的交互方法、装置、电子设备及介质
US12019669B2 (en) Method, apparatus, device, readable storage medium and product for media content processing
CN107835366A (zh) 多媒体播放方法、装置、存储介质及电子设备
US20250292444A1 (en) Video synthesis method, apparatus, device, medium and product
CN107479841A (zh) 双板展示系统的控制方法、装置、设备和存储介质
US10812769B2 (en) Visualizing focus objects from video data on electronic maps
CN111881904A (zh) 板书记录方法和系统
CN116744065A (zh) 视频播放方法、装置
CN116434253A (zh) 图像处理方法、装置、设备、存储介质及产品
CN111144255A (zh) 一种教师的非语言行为的分析方法及装置
WO2024007819A1 (zh) 黑板板书获取方法、装置、设备、系统及存储介质
CN109857244A (zh) 一种手势识别方法、装置、终端设备、存储介质及vr眼镜
CN114449323A (zh) 视频生成方法和电子设备
CN115695714B (zh) 图像显示方法、装置、电子设备及介质
US12541859B2 (en) Image processing apparatus, image processing method, and non-transitory computer-readable storage medium
CN108805951B (zh) 一种投影图像处理方法、装置、终端和存储介质
CN116311520B (zh) 一种基于单目摄像头的空中手写人机交互技术

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17931208

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14/10/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17931208

Country of ref document: EP

Kind code of ref document: A1