CN116778121A - Virtual screen writing control synthesis system and method - Google Patents
Virtual screen writing control synthesis system and method Download PDFInfo
- Publication number
- CN116778121A CN116778121A CN202310788432.2A CN202310788432A CN116778121A CN 116778121 A CN116778121 A CN 116778121A CN 202310788432 A CN202310788432 A CN 202310788432A CN 116778121 A CN116778121 A CN 116778121A
- Authority
- CN
- China
- Prior art keywords
- writing control
- virtual
- writing
- control action
- transparent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000015572 biosynthetic process Effects 0.000 title claims abstract description 50
- 238000003786 synthesis reaction Methods 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims description 48
- 230000009471 action Effects 0.000 claims abstract description 116
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000001308 synthesis method Methods 0.000 claims abstract description 11
- 230000008569 process Effects 0.000 claims description 33
- 238000000605 extraction Methods 0.000 claims description 16
- 230000006978 adaptation Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 2
- 230000002093 peripheral effect Effects 0.000 claims description 2
- 230000003044 adaptive effect Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 21
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000001276 controlling effect Effects 0.000 description 4
- 239000004744 fabric Substances 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000008921 facial expression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Graphics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a virtual screen writing control synthesis system and a virtual screen writing control synthesis method, and belongs to the field of online electronic demonstration. The system mainly comprises a transparent virtual writing control board, a display board and a control board, wherein the transparent virtual writing control board is used for shooting a demonstrator through the transparent virtual writing control board and acquiring writing control action track data of the presenter on the surface of the presenter; the image pickup device is used for shooting the writing control action of the demonstrator to obtain writing control action image data; the demonstration device is used for providing demonstration pictures; the acquisition device is used for acquiring the writing control action track data, the writing control action image data and the demonstration pictures; the virtual picture synthesis device is used for processing and matching and synthesizing the writing control action track data, the writing control action image data and the demonstration pictures to obtain virtual demonstration pictures; and a display device for displaying the virtual presentation to provide the presenter with a reference prompt. The invention can be beneficial to improving the demonstration effect of online demonstration and improving the user experience.
Description
Technical Field
The invention relates to the technical field of online electronic demonstration, in particular to a virtual screen writing control synthesis system and method.
Background
At present, in the application of remote electronic demonstration such as online live broadcasting courses and the like, when a presenter writes or controls on a demonstration courseware or other drawing board software by using a screen-writable computer or an electronic writing board and the like in the prior art, the presenter can only see writing writings or cursors, but cannot see the scene of writing and controlling on a blackboard like a teacher in a classroom, and cannot see the facial expression and limb language of the teacher. The electronic large screen capable of being written by the screen is also used, and a camera is placed in front of the electronic large screen to shoot the writing process of a teacher on the large screen in real view.
Disclosure of Invention
The embodiment of the invention provides a virtual screen writing control synthesis system and a virtual screen writing control synthesis method, which utilize a transparent virtual writing control board, camera equipment, acquisition equipment, presentation equipment and virtual synthesis equipment to acquire writing or control track data and acquire writing or control action picture data and a presentation picture at the same time, and synthesize the writing or control action picture data and the presentation picture, so that a presenter can write and control correspondingly to a presenter on the premise of ensuring the clarity of the presentation picture, the presentation effect is improved, and the presenter can see a virtual picture which is written or controlled on a screen like the presenter. Its application has AR effects and meta-cosmic scenes.
In a first aspect, an embodiment of the present invention provides a virtual screen writing control synthesis system and method, including: the transparent virtual writing control board comprises a transparent board and a track extraction device, wherein the transparent board is used for a demonstrator to perform writing control actions on the surface of the transparent board and shoot through the transparent board; the track extraction device is a non-shielding device and is used for acquiring the writing control action track data of the demonstrator on the surface of the transparent plate; the imaging device is used for shooting the writing control action of the demonstrator through the transparent virtual writing control panel to obtain writing control action image data corresponding to the writing control action track data; the demonstration device is used for providing demonstration pictures required to be demonstrated for the demonstrator; the acquisition equipment is used for correspondingly acquiring the writing control action track data, the corresponding writing control action image data and the demonstration pictures from the transparent virtual writing control panel, the camera equipment and the demonstration equipment respectively; the virtual picture synthesis device is used for processing and matching and synthesizing the writing control action track data, the corresponding writing control action image data and the demonstration pictures to obtain virtual demonstration pictures for being demonstrated to audiences; and the display equipment is used for displaying the virtual display picture, and further providing a reference prompt for the writing control action of the demonstrator on the surface of the transparent virtual writing control panel by utilizing the virtual display picture.
In a second aspect, an embodiment of the present invention provides a virtual screen writing manipulation synthesis method, including:
acquiring writing control action track data of a demonstrator in the writing control action process of the surface of a transparent virtual writing control board by utilizing the transparent virtual writing control board, wherein the transparent virtual writing control board comprises a transparent board and a non-shielding track extraction device; shooting the writing control action of the demonstrator through the transparent virtual writing control panel by using an image pickup device to obtain writing control action image data corresponding to the writing control action track data; providing a demonstration picture required to be demonstrated for the presenter by using demonstration equipment; collecting the writing control action track data, the corresponding writing control action image data and the demonstration pictures from the transparent virtual writing control board, the camera shooting equipment and the demonstration equipment by using a collecting device; the virtual picture synthesizing equipment is utilized to process, match and synthesize the acquired writing control action track data, corresponding writing control action image data and the demonstration pictures acquired by the acquisition equipment to obtain virtual demonstration pictures; and displaying the virtual display picture by using a display device, and further providing a reference prompt for the writing control action of the demonstrator on the surface of the transparent virtual writing control panel by using the virtual display picture.
According to the virtual screen writing control synthesis system and method provided by the embodiment of the invention, in the online demonstration process, the transparent virtual writing control board can be utilized to directly obtain clear demonstration courseware pictures, writing writings and control tracks of a demonstrator, and meanwhile, the front demonstration action of the presenter is obtained, so that the presenter can face the camera equipment, namely the observer, and the writing writings and control tracks can be completely matched with the writing action and control action of the presenter, thereby enabling the observer to have the personally-on demonstration experience, improving the demonstration effect, improving the user experience, and simultaneously displaying the facial expression and limb language of the presenter. But also has AR effects and meta-cosmic scenes.
Drawings
FIG. 1 is a flow diagram of one embodiment of a virtual screen writing manipulation synthesis system of the present invention;
FIG. 2 is a flow diagram of one embodiment of a virtual screen writing manipulation synthesis system of the present invention;
FIG. 3 is a flow diagram of one embodiment of a virtual screen writing manipulation synthesis system of the present invention;
FIG. 4 is a flow diagram of one embodiment of a virtual screen writing manipulation synthesis system of the present invention;
FIG. 5 is a flow diagram of one embodiment of a virtual screen writing manipulation synthesis system of the present invention;
FIG. 6 is a schematic diagram of a transparent virtual tablet and its connections in one embodiment of a virtual screen writing manipulation composition system of the present invention;
FIG. 7 is a flow chart of one embodiment of a virtual screen writing manipulation synthesis method of the present invention;
reference numerals:
1-a transparent virtual writing control board; 2-video camera; 3-acquisition equipment; 4-virtual picture synthesizing device; a 5-display; 6-green curtain; 7-a prompter; 8-a transparent plate; 9-an infrared touch frame; 10-a capacitive touch film; 11-presentation device.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
At present, in the application of remote electronic demonstration such as online live broadcasting courses and the like, when a presenter uses a screen-writable computer or an electronic writing board and the like in the prior art to write or control in the demonstration, a viewer can only see writing writings or cursors, but cannot see scenes of writing and controlling on a blackboard like a teacher in a classroom, and cannot see the language of limbs of the presenter. If the electronic large screen capable of being written is used, a camera is placed in front of the electronic large screen to shoot the writing process of the demonstrator on the large screen, the demonstrator is not only required to be away from the observer during writing, but also the shot demonstration picture is unclear. When in remote interaction, only the voice of a remote person is usually needed, the person can not be seen, and even the person can not write with the screen.
In order to realize the splicing of writing handwriting and control and the writing gesture and control action of a demonstrator in online demonstration, the presenter is right opposite to the screen to write and control, and the presenter looks like writing and control on the screen, the invention provides a virtual screen writing and control synthesis system and a virtual screen writing and control synthesis method for realizing the scene.
In one embodiment of the present invention, the virtual screen writing manipulation synthesis system of the present invention comprises: the transparent virtual writing control board comprises a transparent board and a track extraction device, wherein the transparent board is used for a demonstrator to perform writing control actions on the surface of the transparent board and shoot through the transparent board; the track extraction device is a non-shielding device and is used for acquiring the writing control action track data of a demonstrator on the surface of the transparent plate; the camera equipment is used for shooting the writing control action of the demonstrator through the transparent virtual writing control panel to obtain writing control action image data corresponding to the writing control action track data; the acquisition equipment is used for acquiring the writing control action track data, the corresponding writing control action image data and the demonstration picture from the transparent virtual writing control panel, the camera equipment and the demonstration equipment respectively; the virtual picture synthesis device is used for carrying out matching synthesis on the writing control action track data, the corresponding writing control action image data and the demonstration pictures to obtain virtual demonstration pictures; and the display equipment is used for displaying the control picture and the demonstration picture, displaying the virtual demonstration picture, and further providing a reference prompt for the writing control action of the demonstrator on the surface of the transparent virtual writing control board by utilizing the virtual demonstration picture.
The virtual screen writing control synthesis system can directly acquire clear demonstration courseware pictures, writing writings and control tracks of a demonstrator by using the transparent virtual writing control board in the online demonstration process, and simultaneously acquires front demonstration actions of the demonstrator, so that the writing writings and the control tracks can be completely matched with the writing actions and the control actions of the presenter, the presenter can always face the spectator, thereby enabling the spectator to have full-scene spectator experience, improving the demonstration effect, improving the user experience, and enabling the classroom learning atmosphere to be easier to mobilize and improving the teaching quality especially in the scene of online teaching. In more application scenes, the method also has AR effect and meta-universe effect.
Specifically, the acquisition device may be a combination device including a computer, software, an acquisition card, a camera device, and the like, which processes and acquires necessary information.
Specifically, the writing manipulation actions include the action of the presenter and writing and manipulation actions thereof, and the corresponding writing manipulation track includes the writing track and manipulation track information. The writing track comprises information such as handwriting, pictures and the like written by a demonstrator at any time in the demonstration process. The manipulation trajectory includes information of the presenter opening or using software, such as various manipulation process information such as the presenter clicking on presentation interface icons, clicking on various menu buttons, inputting, selecting, setting options, adjusting parameters, and editing. The writing manipulation action may be acquired by the image capturing apparatus, and the writing manipulation trajectory may be acquired by the transparent virtual writing manipulation panel.
In an alternative embodiment of the present invention, the transparent virtual writing control board includes a plurality of track extraction devices, where the plurality of track extraction devices include an infrared touch frame disposed along a peripheral edge of the transparent board, and/or a transparent capacitive touch film adhered to the transparent board.
In particular, in a virtually synthesized video frame, multiple writing control interfaces may be simultaneously presented or nested, i.e., multiple layers of multiple control windows, and the size of the frame windows may be different, requiring one or more additional writing or control information extraction devices for manipulating one or more additional layers of windows.
In an alternative embodiment of the present invention, a transparent virtual writing control board with multiple touch windows including an infrared touch frame and a capacitive touch film may be selected, so as to implement simultaneous writing control in multiple windows. Fig. 6 shows a case where the track extraction device includes an infrared touch frame and two capacitive touch films, where the connection lines only indicate the correspondence of the writing control information, and the internal window indicates the touch sensing area.
In an optional embodiment of the present invention, the above multi-window writing operation may be implemented by using a transparent virtual writing operation board only including a capacitive touch film, and specifically, the parameter setting of the capacitive touch film may also enable the effect that multiple window frames are respectively written and operated in the final virtual synthesized frame to be displayed.
In an optional embodiment of the present invention, the above multi-window writing operation may be implemented by using a transparent virtual writing operation board only including an infrared touch frame, and specifically, the parameter setting of the infrared touch frame may also enable the final virtual synthesized picture to display the effect that the multiple window pictures are respectively written and operated.
In an actual application scene, the shape, size and performance of the transparent writing control board and the track extraction device thereof can be configured according to parameters of computer equipment, parameters of camera equipment, use requirements and the like, such as the screen ratio of a computer picture, the resolution of the computer screen, parameters of focal length, visual field, geometric picture restoration degree, picture resolution and the like of a camera, vertical screen or horizontal screen display, the resolution of touch equipment, the number of touch points and the like.
In an optional embodiment of the present invention, the image capturing apparatus further includes an image matting device for matting the presenter. Specifically, the image matting device can comprise matting cloth, the matting cloth is arranged behind the demonstrator, and the setting of the matting device can be beneficial to matting when the writing control action of the demonstrator is acquired in the later stage, so that a limb image only comprising the demonstrator is obtained. Optionally, the image matting cloth is a green curtain. Optionally, the intelligent image matting device can also be used without matting cloth.
In an alternative embodiment of the present invention, the image capturing apparatus includes a camera, as shown in fig. 1, disposed directly in front of the presenter, and a transparent virtual writing control board is disposed between the camera and the presenter. Because the virtual writing control board is transparent, the virtual writing control board is arranged between the camera and the demonstrator, and can normally shoot the action of the demonstrator, so that the writing action and the control action of the demonstrator can be matched and synthesized with corresponding writing track and control track information.
Specifically, the camera can select a mirror image camera to shoot the demonstration action of the demonstrator, so that the matching and synthesis of the later track and the action are facilitated.
In an alternative embodiment of the present invention, the image capturing apparatus may include a plurality of cameras, and the positions of the cameras may be on the upper, lower, left, right, and lateral sides of the monitor in front of the camera, so as to facilitate subsequent composition of pictures of the presenter photographed in different directions, and obtain a better virtual composite picture effect.
In an alternative embodiment of the present invention, the image capturing apparatus may include various video or image capturing apparatuses such as various video cameras, conference cameras, monitoring cameras, computer cameras, network cameras, and mobile phone cameras. In an actual use scene, a proper image pickup device can be configured according to requirements on focal length, field of view, geometric picture restoration degree, picture resolution and the like of a camera.
In an alternative embodiment of the present invention, the presentation device comprises one or more computer devices.
Specifically, the one or more computer devices included in the presentation device may provide a writing control screen that needs to be presented. The screen on the display device is manipulated through the transparent virtual writing manipulation panel, including various operations of inputting, clicking, opening and manipulating software, selecting, setting options and editing, and writing on the presentation screen.
In particular, in the case of multiple computer devices, different computer devices may be used to provide different pictures that need to be demonstrated, for example, a picture that needs to be demonstrated when giving lessons, a page of questions posed by students, and so on.
In an alternative embodiment of the present invention, the virtual picture synthesizing apparatus may include one or more synthesizing function computers configured with related software.
In an alternative embodiment of the present invention, the above-mentioned method for processing and matching the writing manipulation action track data, the corresponding writing manipulation action image data, and the presentation screen may include various methods. The writing track data on the transparent virtual writing control board is directly displayed as writing tracks on a demonstration picture; the writing track data on the transparent virtual writing control board can be independently extracted and then combined to the demonstration picture to display writing tracks.
In particular, in some specific usage scenarios, a virtual picture composition device requires a composition function computer. When the synthesis function computer equipment only comprises one synthesis function computer, the transparent virtual writing control board, the camera equipment, the acquisition equipment and the like are only connected to the synthesis function computer, and at the moment, the synthesis function computer integrates various functions of the demonstration equipment, the monitoring equipment, the display equipment, the virtual picture synthesis equipment and the like, so that the system cost can be integrally saved.
In an alternative embodiment of the present invention, the virtual picture synthesizing apparatus includes a plurality of synthesizing function computers. In particular, if the virtual picture composing apparatus includes a plurality of composing-function computers, the virtual picture composing apparatus may be used for processing different data information, for example, one composing-function computer is used for processing writing track and/or manipulating action track data information, and another computer is used for processing image data photographed by the camera and composing data. Therefore, the processing efficiency of the computer equipment with the synthesis function can be improved, and the phenomena of blocking, mismatching and the like are avoided.
Specifically, the plurality of synthesis function computers may be connected together, and one of the synthesis function computers is set as a master synthesis function computer.
Specifically, the virtual picture synthesizing device may include a synthesizing function computer on which required specialized software, such as matting software, specialized track action synthesizing software, and the like, may be installed in advance.
In an alternative embodiment of the invention, the display device comprises one or more monitors, wherein one monitor is arranged directly in front of the presenter and the camera is located between the directly in front monitor and the transparent virtual writing dashboard. The case where the display device includes a monitor is shown in fig. 1.
Specifically, the monitor is used as display equipment, the above pictures are virtual display pictures obtained by matching and synthesizing writing operation motion track data and corresponding writing operation control motion image data by using professional synthesis software, and the virtual display pictures are finally required pictures and can be recorded or promoted. The pictures comprise demonstration pictures, handwriting, computer or software pictures, figures, dynamic postures of writing or controlling, pictures, videos and the like, and a part of pictures are pictures on a computer and can change in real time. When the screen on the monitor is used as a reference screen for the presenter to perform various writing and operation, the presenter can see the monitor screen through the transparent virtual writing control board, and can decide what to write on the transparent virtual writing control board, where to write, what to operate, what to click on and what to open as if the presenter were operating on a computer at ordinary times, but the actual operation is performed on the transparent virtual writing control board.
When a monitor is used as a presenter reference, it is inevitable that the presenter is not natural to look at a place. Therefore, a plurality of monitors can be arranged, so that the presenter can change the visual angle of the eyes, and the natural effect that the presenter looks at the virtual handwriting is achieved.
Since the monitor is opaque, it is necessary to place a camera between the front of the monitor and the transparent virtual writing control panel, avoiding affecting the shooting of the presenter's presentation action.
In an alternative embodiment of the present invention, the display device includes one or more monitors, wherein one monitor is sleeved on the camera, and the display device includes a sleeved monitor scene as shown in fig. 2 and 3.
Since the monitor is opaque, a splice screen or a hole digging screen monitor with a through hole can be selected to be sleeved with the camera.
Alternatively, the monitor and the camera with the through hole splicing screen or the hole digging screen can be integrated into a whole, as shown in fig. 2, so as to facilitate the installation and operation of the system.
Optionally, the monitor, the camera and the transparent virtual writing control panel with the through hole may be integrated to form an integrated body, as shown in fig. 3, so as to be more convenient in use.
Optionally, the monitor may also select a tiled screen mode, or may integrate the camera with it.
Alternatively, the monitor may select the projection screen mode, or the camera may be integrated therewith.
In an alternative embodiment of the invention, the display device comprises one or more prompter architecture devices, wherein one prompter architecture device is disposed directly in front of the presenter, between the camera and the presenter, and the display device comprises a scene of one prompter architecture device as shown in fig. 4 and 5.
Specifically, the aforementioned prompter or prompter frame apparatus includes a display, a beam splitter, a light shield, and a bracket.
Optionally, the placement positions of the display and the spectroscope in the above-mentioned prompter framework can be set according to the needs, and the display can be at the upper, lower, left and right positions of the spectroscope, and the display can be correspondingly displayed by a horizontal screen or a vertical screen.
Alternatively, the camera and the device of the prompter frame may be integrated into an integrated unit, as shown in fig. 4, to facilitate installation, operation, and use of the system.
Optionally, the device, the camera and the transparent virtual writing control board of the prompter framework may be integrated to form an integrated body, as shown in fig. 5, so as to be more convenient for installation and operation during use.
Specifically, when the device of the prompter framework is used as the display device, the spectroscope of the prompter can be shielded by using the shielding devices such as the shielding cover, so that the display picture can be clearly and clearly displayed.
In one embodiment of the present invention, the functional modules of a virtual screen writing manipulation synthesis system of the present invention may be directly in hardware, in software modules executed by a processor, or in a combination of both.
In one embodiment of the present invention, two or more remote virtual screen writing manipulation synthesis systems may be connected through a network, and remote presentation frames, monitoring frames, image capturing devices, virtual synthesis frames, etc. may be acquired from each other through the network and then superimposed into a new virtual synthesis frame, in which two or more demonstrators may write and/or manipulate.
Optionally, the transparent virtual writing control board, the image capturing device, the demonstration image and the virtual image synthesizing device need to perform parameter matching, including a focal length, a field of view and a distance from the transparent virtual writing control board of the image capturing device, a size, an induction and/or touch area of the transparent virtual writing control board, and positions and/or proportions of each image in the virtual synthesizing image.
Fig. 7 shows a specific embodiment of a virtual screen writing manipulation synthesis method of the present invention.
In the specific embodiment of the virtual screen writing manipulation synthesis method shown in fig. 7, the method includes:
the method comprises the following steps of S701, acquiring writing control action track data of a demonstrator in the writing control action process of the surface of a transparent virtual writing control board by using the transparent virtual writing control board, wherein the transparent virtual writing control board comprises a transparent board and a non-shielding track extraction device;
the process S702 of shooting the writing operation of the presenter through the transparent virtual writing operation panel by using the image capturing device to obtain writing control operation image data corresponding to the writing operation track data;
a process S703 of providing a demonstration screen to be demonstrated for a presenter by using a demonstration device;
a process S704 of collecting writing operation track data, corresponding writing operation image data and a demonstration image from the transparent virtual writing operation board, the camera device and the demonstration device by using the collecting device;
step S705, processing, matching and synthesizing the writing operation track data acquired by the acquisition device, the corresponding writing operation image data and the demonstration pictures by using the virtual picture synthesizing device to obtain virtual demonstration pictures; and
and S706, displaying the virtual display picture by using the display equipment, and further providing a reference prompt for the writing operation performed by the presenter on the surface of the transparent virtual writing operation panel by using the virtual display picture.
The method can directly acquire clear demonstration courseware pictures, writing writings and control tracks of a demonstrator by using the transparent virtual writing control board, the acquisition equipment and the like in the online demonstration process, and simultaneously acquire front demonstration actions and images of the presenter, so that the presenter can face the camera equipment, namely the observer, and the writing writings and control information can be completely and seamlessly matched and synthesized with the writing actions and control actions of the presenter, thereby enabling the observer to have full-scene observation experience, improving the demonstration effect, improving the user experience, and particularly enabling learning atmosphere to be easier to mobilize and improving the teaching quality in the scene of online teaching. In more application scenes, the method has AR effect and meta-universe effect.
The transparent virtual writing control board that process S701 represents is utilized to acquire the action track data that writes in the action process is controlled to the demonstrator in transparent virtual writing control board surface, can utilize transparent virtual control board to obtain clear track data information that writes the action, can do benefit to later stage acquisition and synthesis, guarantees the definition of demonstration, and then guarantees the demonstration effect. The transparent virtual writing control board not only has a single-window writing control function, but also has a multi-window writing control function, and can perform writing control on a plurality of demonstration pictures, and simultaneously or respectively perform writing control. The writing control information is transmitted to the computer equipment, the acquisition equipment and the virtual picture synthesis equipment, and is synthesized into the virtual picture through the equipment and the software, wherein parameters of the writing control information are set and adjusted according to the needs.
The process S702 represents a process of capturing a writing operation of a presenter through a transparent virtual writing operation panel by using an image capturing device to obtain writing control operation image data corresponding to writing operation track data, and the process can capture the writing operation of the presenter through the transparent virtual writing operation panel, so that the presenter can keep the front of the presenter facing the lens, and writing or operation can be captured in time. The quality of the image information captured by the image capturing device is affected by the performance of the transparent virtual writing control panel, including optical performance, touch area size, geometric dimensions, and the like. In order to obtain high-quality image information, parameters such as a focal length, a field of view, a distance from a writing control board and the like of the image pickup apparatus need to be matched with parameters such as performance and shape of the writing control board. The method is convenient for synthesizing high-quality virtual pictures in the later period, is beneficial to obtaining virtual pictures written or controlled on the display screen by a demonstrator, and improves the demonstration effect.
The demonstration device denoted by the process S703 provides the demonstration picture to be demonstrated for the presenter, which can facilitate the subsequent synthesis of the demonstration picture and the writing manipulation motion trajectory to obtain the virtual demonstration picture.
The process S704 represents a process of collecting the writing manipulation action trajectory data, the corresponding writing manipulation action image data, and the presentation screen from the transparent virtual writing manipulation panel, the image capturing device, and the presentation device, respectively, using the collecting device. The method can be favorable for the professional synthesis software on the computer equipment to carry out matching synthesis of the writing operation track data and the corresponding writing operation image data. Most of the synthesized virtual pictures originate from the presentation device. The change of the picture caused by the writing operation is completed on the demonstration device.
The process S705 represents a process of processing, matching and synthesizing the writing operation track data acquired by the acquisition device, the corresponding writing operation image data, and the presentation image to obtain a virtual presentation image by using the virtual image synthesizing device, so that the virtual presentation image of the integrating courseware, software, writing and/or operation and writing control actions, which can be seen by a viewer, can be obtained, thereby enabling the viewer to have a full-scene viewing experience, improving the presentation effect, and improving the user experience. The virtual composite picture may have AR effects and meta-cosmic scenes.
The process S706 represents a process of providing a reference prompt for the writing operation performed by the presenter on the surface of the transparent virtual writing operation panel by using the display device to display the virtual display screen, so that the presenter can accurately operate, the natural smoothness of the virtual display screen is ensured, and the demonstration effect is ensured.
In an alternative embodiment of the present invention, the process S705 of processing, matching and synthesizing the acquired writing operation motion trajectory data, the corresponding writing operation motion image data, and the presentation frame to obtain the virtual presentation frame by using the virtual frame synthesizing device includes:
performing adaptation adjustment on corresponding image positions and/or image proportions of corresponding writing control action image data according to the writing control action track data; according to the writing control action image data, the track position and/or track proportion corresponding to the writing control action track data are adapted and adjusted; the track position and/or track proportion corresponding to the writing control action track data and the image position and/or image proportion corresponding to the corresponding writing control action image data are mutually adapted and adjusted; and/or the positions and/or proportions of various elements in the demonstration picture, the track positions and/or track proportions corresponding to the writing control action track data, and the image positions and/or image proportions corresponding to the corresponding writing control action image data are mutually adapted and adjusted.
And in the multi-picture process, the positions and/or proportions of the elements in the written interface or the controlled interface, the positions and/or proportions corresponding to the writing or controlling action data information and the image positions and/or proportions corresponding to the writing control action image data information are mutually matched and adjusted. And the position and/or the proportion of other pictures can be adjusted by taking any one picture as a reference object. And the synthesized virtual pictures achieve the effect of seamless splicing.
Specifically, the video camera extracts the writing action image information data and/or the control action image data of the demonstrator, the writing control action data information extracted by the transparent electronic virtual writing control board is obtained by two channels and is respectively and independently presented, and the writing action data information and the control action data information are required to be combined into a virtual screen writing picture or a virtual control picture together with a written interface or a controlled interface to be subjected to adaptation adjustment. The synthesis process requires not only a high match of parameters and indices for each device, but also further fine tuning. The data obtained by other channels can be adjusted according to the data obtained by one channel, and the data obtained by a plurality of channels can be mutually adapted and debugged.
The embodiment of the invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the virtual screen writing control synthesis method provided by any embodiment when executing the program.
The embodiment of the invention also provides a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the virtual screen writing control synthesis method provided in any of the above embodiments.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A virtual screen writing manipulation synthesis system, comprising:
the transparent virtual writing control board comprises a transparent board and a track extraction device, wherein the transparent board is used for a demonstrator to perform writing control actions on the surface of the transparent board and shoot through the transparent board; the track extraction device is a non-shielding device and is used for acquiring the writing control action track data of the demonstrator on the surface of the transparent plate;
the imaging device is used for shooting the writing control action of the demonstrator through the transparent virtual writing control panel to obtain writing control action image data corresponding to the writing control action track data;
the demonstration device is used for providing demonstration pictures required to be demonstrated for the demonstrator;
the acquisition equipment is used for correspondingly acquiring the writing control action track data, the corresponding writing control action image data and the demonstration pictures from the transparent virtual writing control panel, the camera equipment and the demonstration equipment respectively;
the virtual picture synthesis device is used for processing and matching and synthesizing the writing control action track data, the corresponding writing control action image data and the demonstration pictures to obtain virtual demonstration pictures for being demonstrated to audiences; and
and the display equipment is used for displaying the virtual display picture, and further providing a reference prompt for the writing control action of the demonstrator on the surface of the transparent virtual writing control panel by utilizing the virtual display picture.
2. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the transparent virtual writing control board comprises a plurality of track extraction devices, wherein the track extraction devices comprise infrared touch frames arranged along the peripheral edges of the transparent board and/or transparent capacitance touch films adhered to the surface of the transparent board.
3. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the camera device comprises one or more cameras, wherein one camera is arranged in front of the presenter, and the transparent virtual writing control board is positioned between the camera and the presenter.
4. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the presentation device includes a plurality of computer devices.
5. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the display device includes one or more monitors, one of which is disposed directly in front of the presenter, the camera being located between the monitor directly in front and the transparent virtual writing manipulation panel.
6. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the display device comprises one or more monitors, wherein one monitor is sleeved on the camera.
7. The virtual screen writing manipulation synthesis system of claim 1, wherein,
the display device includes one or more prompter architecture devices, one of which is disposed directly in front of the presenter, between the camera and the transparent virtual writing control panel.
8. A virtual screen writing manipulation synthesis system according to claim 1, wherein the image capturing apparatus further comprises an image matting device for matting the presenter.
9. A virtual screen writing manipulation synthesis method, characterized by comprising:
acquiring writing control action track data of a demonstrator in the writing control action process of the surface of a transparent virtual writing control board by utilizing the transparent virtual writing control board, wherein the transparent virtual writing control board comprises a transparent board and a non-shielding track extraction device;
shooting the writing control action of the demonstrator through the transparent virtual writing control panel by using an image pickup device to obtain writing control action image data corresponding to the writing control action track data;
providing a demonstration picture required to be demonstrated for the presenter by using demonstration equipment;
collecting the writing control action track data, the corresponding writing control action image data and the demonstration pictures from the transparent virtual writing control board, the camera shooting equipment and the demonstration equipment by using a collecting device;
the virtual picture synthesizing equipment is utilized to process, match and synthesize the acquired writing control action track data, corresponding writing control action image data and the demonstration pictures acquired by the acquisition equipment to obtain virtual demonstration pictures; and
and displaying the virtual display picture by using a display device, and further providing a reference prompt for the writing control action of the demonstrator on the surface of the transparent virtual writing control panel by using the virtual display picture.
10. The virtual screen writing manipulation synthesis method according to claim 9, wherein the process of processing, matching and synthesizing the acquired writing manipulation action track data, the corresponding writing manipulation action image data, and the presentation picture to obtain the virtual presentation picture by the acquisition device comprises:
performing adaptation adjustment on corresponding image positions and/or image proportions of corresponding writing control action image data according to the writing control action track data;
performing adaptive adjustment on track positions and/or track proportions corresponding to the writing control action track data according to the writing control action image data;
the track position and/or track proportion corresponding to the writing control action track data and the image position and/or image proportion corresponding to the corresponding writing control action image data are mutually adapted and adjusted; and/or the positions and/or proportions of various elements in the demonstration picture, the track positions and/or track proportions corresponding to the writing control action track data and the image positions and/or image proportions corresponding to the corresponding writing control action image data are mutually adapted and adjusted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310788432.2A CN116778121A (en) | 2023-06-29 | 2023-06-29 | Virtual screen writing control synthesis system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310788432.2A CN116778121A (en) | 2023-06-29 | 2023-06-29 | Virtual screen writing control synthesis system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116778121A true CN116778121A (en) | 2023-09-19 |
Family
ID=87985808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310788432.2A Pending CN116778121A (en) | 2023-06-29 | 2023-06-29 | Virtual screen writing control synthesis system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116778121A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117596351A (en) * | 2024-01-16 | 2024-02-23 | 江苏欧帝电子科技有限公司 | Interactive system, method and terminal for infrared writing as recorded broadcast picture |
| CN117596351B (en) * | 2024-01-16 | 2024-06-07 | 江苏欧帝电子科技有限公司 | Interactive system, method and terminal for infrared writing as recorded broadcast picture |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108604125A (en) * | 2016-03-29 | 2018-09-28 | 谷歌有限责任公司 | For the system and method based on tracking generation virtual tag is stared |
| CN111405219A (en) * | 2020-04-30 | 2020-07-10 | 李东旭 | Video shooting and recording device and method |
| CN111698390A (en) * | 2020-06-23 | 2020-09-22 | 网易（杭州）网络有限公司 | Virtual camera control method and device, and virtual studio implementation method and system |
| CN113010129A (en) * | 2021-03-08 | 2021-06-22 | 辽宁向日葵教育科技有限公司 | Virtual studio full-flow multi-terminal blackboard writing extraction method and device |
| CN113436343A (en) * | 2021-06-21 | 2021-09-24 | 广州博冠信息科技有限公司 | Picture generation method and device for virtual studio, medium and electronic equipment |
-
2023
- 2023-06-29 CN CN202310788432.2A patent/CN116778121A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108604125A (en) * | 2016-03-29 | 2018-09-28 | 谷歌有限责任公司 | For the system and method based on tracking generation virtual tag is stared |
| CN111405219A (en) * | 2020-04-30 | 2020-07-10 | 李东旭 | Video shooting and recording device and method |
| CN111698390A (en) * | 2020-06-23 | 2020-09-22 | 网易（杭州）网络有限公司 | Virtual camera control method and device, and virtual studio implementation method and system |
| CN113010129A (en) * | 2021-03-08 | 2021-06-22 | 辽宁向日葵教育科技有限公司 | Virtual studio full-flow multi-terminal blackboard writing extraction method and device |
| CN113436343A (en) * | 2021-06-21 | 2021-09-24 | 广州博冠信息科技有限公司 | Picture generation method and device for virtual studio, medium and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117596351A (en) * | 2024-01-16 | 2024-02-23 | 江苏欧帝电子科技有限公司 | Interactive system, method and terminal for infrared writing as recorded broadcast picture |
| CN117596351B (en) * | 2024-01-16 | 2024-06-07 | 江苏欧帝电子科技有限公司 | Interactive system, method and terminal for infrared writing as recorded broadcast picture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11335379B2 (en) | Video processing method, device and electronic equipment | |
| CN107909022B (en) | Video processing method and device, terminal equipment and storage medium | |
| WO2022022028A1 (en) | Virtual object control method and apparatus, and device and computer-readable storage medium | |
| US8996974B2 (en) | Enhancing video presentation systems | |
| CN111654715B (en) | Live video processing method and device, electronic equipment and storage medium | |
| JP2006162692A (en) | Automatic lecture content creating system | |
| US20220044580A1 (en) | System for recording and producing multimedia presentations | |
| JP2015119277A (en) | Display apparatus, display method, and display program | |
| CN107682638A (en) | Generation, the method and device of display panoramic picture | |
| CN110636365A (en) | Video character adding method and device | |
| WO2019124111A1 (en) | Control device and control method | |
| JP2019512177A (en) | Device and related method | |
| CN101405763B (en) | Method and system for acquiring multiple views of real-time video output object | |
| CN116320364B (en) | Virtual reality shooting method and display method based on multi-layer display | |
| KR101554574B1 (en) | System and Method for Recording Lecture Video Using Transparent Board | |
| WO2023236656A1 (en) | Method and apparatus for rendering interactive picture, and device, storage medium and program product | |
| KR20120074977A (en) | Educational materials and methods using augmented reality the performance of voice recognition and command | |
| US20230113359A1 (en) | Full color spectrum blending and digital color filtering for transparent display screens | |
| CN116778121A (en) | Virtual screen writing control synthesis system and method | |
| CN114615556B (en) | Virtual live broadcast enhanced interaction method and device, electronic equipment and storage medium | |
| US20220360755A1 (en) | Interactive display with integrated camera for capturing audio and visual information | |
| CN115580691A (en) | Image rendering and synthesizing system for virtual film production | |
| CN112887653B (en) | Information processing method and information processing device | |
| JP2021009351A (en) | Content control system, content control method, and content control program | |
| CN115052114B (en) | Electronic semitransparent green curtain image matting and lesson recording system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |