CN113010129A - Virtual studio full-flow multi-terminal blackboard writing extraction method and device - Google Patents
Virtual studio full-flow multi-terminal blackboard writing extraction method and device Download PDFInfo
- Publication number
- CN113010129A CN113010129A CN202110252053.2A CN202110252053A CN113010129A CN 113010129 A CN113010129 A CN 113010129A CN 202110252053 A CN202110252053 A CN 202110252053A CN 113010129 A CN113010129 A CN 113010129A
- Authority
- CN
- China
- Prior art keywords
- portrait
- blackboard
- image
- writing
- virtual studio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 35
- 230000004044 response Effects 0.000 claims abstract description 50
- 230000005540 biological transmission Effects 0.000 claims abstract description 34
- 238000001914 filtration Methods 0.000 claims abstract description 25
- 230000007246 mechanism Effects 0.000 claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 14
- 230000011218 segmentation Effects 0.000 claims description 32
- 238000005070 sampling Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000011800 void material Substances 0.000 description 2
- 101000822695 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C1 Proteins 0.000 description 1
- 101000655262 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C2 Proteins 0.000 description 1
- 101000655256 Paraclostridium bifermentans Small, acid-soluble spore protein alpha Proteins 0.000 description 1
- 101000655264 Paraclostridium bifermentans Small, acid-soluble spore protein beta Proteins 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004573 interface analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/186—Templates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30176—Document
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Abstract
The invention relates to the field of artificial intelligence, and discloses a full-flow multi-terminal blackboard writing extraction method for a virtual studio, which comprises the following steps: filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol; constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture; carrying out portrait cutout on the blackboard writing image by utilizing a pre-trained portrait cutout model to generate a portrait image; and generating a blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image. In addition, the invention provides a virtual studio full-flow multi-terminal blackboard-writing extraction device, electronic equipment and a computer-readable storage medium. The method and the system can realize dynamic display of the blackboard writing interface of the virtual studio and improve the fluency of the blackboard writing interface.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a full-flow multi-terminal blackboard writing extraction method and device for a virtual studio, electronic equipment and a computer-readable storage medium.
Background
The blackboard writing interface can be understood as a visual picture displayed on an intelligent interactive software platform, and is applied to the industrial fields of media, monitoring, education propagation and the like, such as playing of a virtual studio, and using an intelligent interface to help a user to intuitively and clearly know data to be displayed, most of the blackboard writing interfaces are generated by the traditional intelligent interactive software platform at present, but the blackboard writing interface generated by the traditional intelligent interactive software platform cannot support data and portrait separation, so that the generated blackboard writing interface can not be displayed dynamically while displaying is fuzzy, the experience of the user is influenced, meanwhile, the traditional intelligent interactive software platform can only write the data to be displayed on a fixed software interface, the blackboard writing picture switching process can be realized, the switching among a plurality of display interfaces can only be realized through the minimization and maximization functions of software, and the phenomenon of blocking during the picture switching process is easily caused, thereby also affecting the user experience.
Disclosure of Invention
The invention provides a full-flow multi-terminal blackboard writing extraction method and device for a virtual studio, electronic equipment and a computer readable storage medium, and mainly aims to realize dynamic display of a blackboard writing interface of the virtual studio and improve fluency of the blackboard writing interface so as to improve user experience.
In order to achieve the above object, the present invention provides a method for extracting a full-flow multi-terminal blackboard writing in a virtual studio, comprising:
filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol;
constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture;
carrying out portrait cutout on the blackboard writing image by utilizing a pre-trained portrait cutout model to generate a portrait image;
and generating a blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
Optionally, the transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol includes:
acquiring a program running file packet of the filtered blackboard-writing text data, and identifying interface parameters of the program running file packet;
acquiring port parameters of the virtual studio;
configuring the program operation file packet and a data transmission channel of the virtual studio by using the data transmission protocol according to the interface parameter and the port parameter;
and transmitting the filtered blackboard writing text data to the virtual studio based on the data transmission channel.
Optionally, the creating the response event button of the text screen includes:
acquiring a compiling environment of the text picture;
creating a response button script of the text screen in the compiling environment;
and loading an event trigger button of the corresponding text picture based on the response event script, and generating a response event button.
Optionally, the portrait matting is performed on the blackboard-writing image by using the pre-trained portrait matting model, and a portrait image is generated, including:
utilizing a portrait segmentation module in the portrait cutout model to perform portrait segmentation processing on the blackboard-writing image to obtain a portrait segmentation image;
and carrying out cutout processing on the portrait sampling image by utilizing a portrait regression module in the portrait cutout model to obtain a portrait image.
Optionally, the matting the image sample by using an image regression module in the image matting model to obtain an image includes:
utilizing the portrait regression module to perform up-sampling on the portrait segmentation image to obtain a portrait sampling image;
splicing the portrait sampling image and the blackboard-writing image to obtain a spliced image of the characteristic sampling image and the original image;
performing fusion operation on the spliced image to obtain a fused image of the spliced image;
and outputting the fusion image by using an activation function in the portrait regression module to obtain a portrait image.
Optionally, the generating a blackboard-writing interface of the virtual studio according to the text screen, the response event button, and the portrait image includes:
constructing an interface display layer in the virtual studio, wherein the blackboard writing interface display layer comprises a blackboard card layer, an operation picture layer and a portrait layer;
loading the text picture, the response event button and the portrait image into the board card layer, the operation picture layer and the portrait layer respectively to generate a board card picture, a response picture and a portrait picture;
and fusing the board card picture, the response picture and the portrait picture to generate a board writing interface of the virtual studio.
In order to solve the above problem, the present invention further provides a virtual studio full-flow multi-terminal blackboard writing extraction device, including:
the transmission module is used for filtering the blackboard writing text data based on a preset data filtering mechanism and transmitting the filtered blackboard writing text data to the virtual studio by using a data transmission protocol;
the creating module is used for creating a text picture for the blackboard writing text data in the virtual studio and creating a response event button of the text picture;
the cutout module is used for carrying out cutout on the blackboard writing image by utilizing the pre-trained cutout model to generate a portrait image;
and the generating module is used for generating a blackboard writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to implement the virtual studio full-flow multi-terminal blackboard writing extraction method.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, where at least one computer program is stored, where the at least one computer program is executed by a processor in an electronic device to implement the virtual studio full-flow multi-terminal blackboard-writing extraction method described above.
The embodiment of the invention firstly filters the blackboard writing text data based on a preset data filtering mechanism, and transmits the filtered blackboard writing text data to a virtual studio by using a data transmission protocol, so that illegal data of the blackboard writing text data can be identified, and the subsequent data transmission speed is improved; secondly, a text picture is constructed for the blackboard-writing text data in the virtual studio, a response event button of the text picture is created, so that the subsequent blackboard-writing interface can smoothly and quickly switch pictures, the switching smoothness is improved, a pre-trained portrait matting model is utilized to carry out portrait matting on a blackboard-writing image, a portrait image is generated, the image and data separation is supported, and the dynamic display of the blackboard-writing interface is realized; further, the embodiment of the invention generates the blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image. Therefore, the virtual studio full-flow multi-terminal blackboard writing extraction method, the device, the electronic equipment and the storage medium provided by the invention can realize dynamic display of a blackboard writing interface of the virtual studio and improve fluency of the blackboard writing interface so as to improve user experience.
Drawings
Fig. 1 is a schematic flow chart of a full-flow multi-terminal blackboard writing extraction method for a virtual studio according to an embodiment of the present invention;
fig. 2 is a detailed flowchart of a step of the virtual studio full-flow multi-terminal blackboard-writing extracting method provided in fig. 1 according to a first embodiment of the present invention;
fig. 3 is a detailed flowchart of another step of the virtual studio full-flow multi-terminal blackboard-writing extracting method provided in fig. 1 according to the first embodiment of the present invention;
fig. 4 is a schematic block diagram of a full-flow multi-terminal blackboard-writing extracting apparatus for a virtual studio according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an internal structure of an electronic device for implementing a virtual studio full-flow multi-terminal blackboard writing extraction method according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a virtual studio full-flow multi-terminal blackboard writing extraction method. The execution main body of the virtual studio full-flow multi-terminal blackboard writing extraction method includes but is not limited to at least one of electronic devices such as a server and the like which can be configured to execute the method provided by the embodiment of the application. In other words, the virtual studio full-flow multi-terminal blackboard writing extraction method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Fig. 1 is a schematic flow chart of a full-flow multi-terminal blackboard-writing extracting method for a virtual studio according to an embodiment of the present invention. In the embodiment of the present invention, the virtual studio full-flow multi-terminal blackboard writing extraction method includes:
and S1, filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the standard blackboard-writing data to the virtual studio by using a data transmission protocol.
In this embodiment of the present invention, the blackboard writing text data refers to data that needs to be displayed in a virtual studio, and is obtained based on different service scenarios, for example, in a virtual studio playing scenario, the blackboard writing text data may include: a presentation object, a presentation screen, presentation data, and the like. Further, in the embodiment of the present invention, the blackboard-writing text data may be edited by a vsto (Visual Studio Tools for Office) tool to meet the requirements of users in different applications, where the vsto tool is a set of Visual Studio toolkit for creating custom Office applications, so that users may extend Office applications (such as Word, Excel, InfoPath, and Outlook) with Visual Basic or Visual C #.
It should be understood that some sensitive or illegal data may exist in the acquired blackboard writing text data, and in order to ensure the validity of the blackboard writing text data, the embodiment of the present invention filters the blackboard writing text data based on a preset data filtering mechanism, wherein the data filtering mechanism filters a data filtering script developed through a Java language.
In detail, the filtering the blackboard text data based on a preset data filtering mechanism includes: and acquiring the attribute parameters of the blackboard-writing text data through traversal operation, judging whether the attribute parameters are matched with the preset data filtering mechanism, if not, deleting the corresponding blackboard-writing text data, and if so, retaining the corresponding blackboard-writing text data.
In an alternative embodiment, the attribute parameters include: string, field, and Id.
Further, the invention transmits the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol to determine a text content module of a subsequent blackboard-writing interface, wherein the virtual studio can be understood as a client device, such as a display screen, a PC and the like, displaying the blackboard-writing interface, the data transmission protocol refers to a communication protocol between the blackboard-writing text data and the virtual studio, and in an optional embodiment, the data transmission protocol can be constructed by socket sockets.
In detail, referring to fig. 2, the transmitting the filtered blackboard writing text data to a virtual studio by using a data transmission protocol includes:
s20, acquiring a program running file package of the filtered blackboard-writing text data, and identifying interface parameters of the program running file package;
s21, acquiring the port parameters of the virtual studio;
s22, configuring the program operation file packet and the data transmission channel of the virtual studio by using the data transmission protocol according to the interface parameters and the port parameters;
and S23, transmitting the filtered blackboard writing text data to the virtual studio based on the data transmission channel.
In an optional embodiment, the program execution file package is a jar package, and is used for executing the board text data in the corresponding virtual studio.
In an alternative embodiment, the interface parameters of the program run package are identified by the get () command.
In an optional embodiment, the port parameters include: port number and url address and api interface.
In an alternative embodiment, the program operation file packet and the data transmission channel are compiled by using Java language.
S2, constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture.
It should be understood that some screen switching operations may be involved in an interface in a virtual studio, and in order to enable a subsequently generated blackboard writing interface to smoothly and quickly switch screens, in the embodiment of the present invention, a text screen is constructed for blackboard writing text data in the virtual studio, and a response event button of the text screen is created, so as to ensure that the subsequently generated blackboard writing interface can be quickly switched in the virtual studio, thereby improving user experience.
Further, in the embodiment of the present invention, a pre-created text template engine is used to perform page interface analysis on the blackboard-writing text data in the virtual studio to generate a corresponding text picture. Wherein the text template engine comprises a server side template engine (Ejs template), and the page interface parsing refers to a text format parsing of the blackboard-writing text data, and the text format comprises josn format text and xml format text.
Further, the response event button refers to a button for triggering switching of a text screen, and in detail, the response event button for creating the text screen includes: and acquiring a compiling environment of the text picture, creating a response button script of the text picture in the compiling environment, loading an event trigger button corresponding to the text picture based on the response event script, and generating a response event button.
In an alternative embodiment, the response button script may be compiled through a JavaScript language, and the loading of the event trigger button is implemented through a post () command.
And S3, performing portrait matting on the blackboard-writing image by using the pre-trained portrait matting model to generate a portrait image.
In an embodiment of the present invention, the blackboard-writing image includes: in order to report the state of an object through a plate writing interface, a background picture and a user picture more visually and stereoscopically, the embodiment of the invention utilizes a pre-trained portrait matting model to perform portrait matting on a plate writing image, wherein the portrait matting model comprises: the portrait segmentation module is used for carrying out portrait segmentation on the board book image, and the portrait regression module is used for carrying out portrait cutout on the board book image after segmentation so as to generate a more accurate portrait image. Wherein the portrait segmentation module comprises a convolution layer (conv), a batch normalization layer (bn), an activation function (relu), and a void space convolution pooling pyramid (ASSP)
In detail, referring to fig. 4, the generating of portrait images by portrait matting blackboard writing images with pre-trained portrait matting models includes:
s30, performing portrait segmentation processing on the blackboard-writing image by utilizing a portrait segmentation module in the portrait cutout model to obtain a portrait segmentation image;
and S31, carrying out matting processing on the portrait sampling image by using a portrait regression module in the portrait matting model to obtain a portrait image.
Wherein the S30 includes: performing feature extraction on the blackboard writing image by using the portrait segmentation module to obtain a feature map; and performing convolution operation on the feature map and outputting to obtain the portrait segmentation map.
The S31 includes: utilizing the portrait regression module to perform up-sampling on the portrait segmentation image to obtain a portrait sampling image, and splicing the portrait sampling image and the blackboard-writing image to obtain a spliced image of the characteristic sampling image and the original image; performing fusion operation on the spliced image to obtain a fused image of the spliced image; and outputting the fusion image by using an activation function in the portrait regression module to obtain the portrait image.
Further, the extracting features of the blackboard-writing image by using the portrait segmentation module to obtain a feature map includes: performing convolution feature extraction on the blackboard-writing image by using a convolution layer of the portrait segmentation module to obtain an initial feature map, standardizing the initial feature map by using a batch normalization layer of the portrait segmentation module to obtain a standard feature map, outputting the standard feature map by using an activation function of the portrait segmentation module, and performing multi-scale feature fusion on the standard feature map by using a void space convolution pooling pyramid of the portrait segmentation module to obtain the feature map.
Further, the upsampling is to sample the portrait segmentation map to a specified resolution size, for example, after a blackboard-writing image (416, 3) is subjected to a series of convolution pooling operations, a portrait segmentation map (13,13,16) is obtained, in order to compare the specific portrait segmentation map with the corresponding blackboard-writing image, the portrait segmentation map needs to be changed into (416, 3) size, and the process is called upsampling. Further, in order to better understand the characteristic semantic information of the portrait sampling image, the embodiment of the application splices the portrait sampling image and the blackboard-writing image to obtain a spliced image.
In an alternative embodiment, the upsampling may be implemented by a currently known linear interpolation algorithm, and the stitching may be implemented by a currently known stitching algorithm, such as surf (speeded Up Robust features) algorithm.
In an alternative embodiment, the fusing operation comprises: convolution, batch normalization and activation operations. The convolution, batch normalization and activation operations may refer to feature extraction of the blackboard-writing image, and are not described herein again.
And S4, generating a blackboard writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
In an embodiment of the present invention, the generating a blackboard writing interface of the virtual studio according to the text screen, the response event button, and the portrait image includes: constructing an interface display layer in the virtual studio, wherein the blackboard writing interface display layer comprises: the board card layer, the operation picture layer and the portrait layer; loading the text picture, the response event button and the portrait image into the board card layer, the operation picture layer and the portrait layer respectively to generate a board card picture, a response picture and a portrait picture; and fusing the board card picture, the response picture and the portrait picture to generate a board writing interface of the virtual studio.
The board card layer is used for displaying text pictures, the operation picture layer is used for switching the text pictures, and the portrait layer is used for displaying portrait images.
In an alternative embodiment, the interface display layer may be constructed by a 3d virtual technology (such as Unity), and the loading of the text screen, the response event button, and the portrait image may be implemented by a load () function.
The embodiment of the invention firstly filters the blackboard writing text data based on a preset data filtering mechanism, and transmits the filtered blackboard writing text data to a virtual studio by using a data transmission protocol, so that illegal data of the blackboard writing text data can be identified, and the subsequent data transmission speed is improved; secondly, a text picture is constructed for the blackboard-writing text data in the virtual studio, a response event button of the text picture is created, so that the subsequent blackboard-writing interface can smoothly and quickly switch pictures, the switching smoothness is improved, a pre-trained portrait matting model is utilized to carry out portrait matting on a blackboard-writing image, a portrait image is generated, the image and data separation is supported, and the dynamic display of the blackboard-writing interface is realized; further, the embodiment of the invention generates the blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image. Therefore, the virtual studio full-flow multi-terminal blackboard writing extraction method provided by the invention can realize dynamic display of the blackboard writing interface of the virtual studio and improve the fluency of the blackboard writing interface so as to improve the user experience.
Fig. 4 is a functional block diagram of the virtual studio full-flow multi-terminal blackboard-writing extracting apparatus according to the present invention.
The virtual studio full-flow multi-terminal blackboard writing extraction device 100 can be installed in electronic equipment. According to the realized functions, the virtual studio full-flow multi-terminal blackboard-writing extraction device can comprise a transmission module 101, a creation module 102, a matting module 103 and a generation module 105. A module according to the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the transmission module 101 is configured to filter the blackboard writing text data based on a preset data filtering mechanism, and transmit the filtered blackboard writing text data to a virtual studio by using a data transmission protocol;
the creating module 102 is configured to construct a text picture for the blackboard writing text data in the virtual studio, and create a response event button of the text picture;
the matting module 103 is configured to perform portrait matting on a blackboard writing image by using a pre-trained portrait matting model to generate a portrait image;
the generating module 104 is configured to generate a blackboard writing interface of the virtual studio according to the text picture, the response event button, and the portrait image.
In detail, in the embodiment of the present invention, when the modules in the virtual studio full-flow multi-terminal blackboard-writing extraction apparatus 100 are used, the same technical means as the virtual studio full-flow multi-terminal blackboard-writing extraction method described in fig. 1 to fig. 3 are adopted, and the same technical effects can be produced, which is not described herein again.
Fig. 5 is a schematic structural diagram of an electronic device for implementing the virtual studio full-flow multi-terminal blackboard writing extraction method according to the present invention.
The electronic device 1 may include a processor 10, a memory 11 and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 10, such as a virtual studio full-flow multi-terminal blackboard-writing extraction program 12.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used to store not only application software installed in the electronic device 1 and various types of data, such as codes of the virtual studio full-flow multi-terminal blackboard-writing extraction program 12, but also temporarily store data that has been output or is to be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the whole electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules stored in the memory 11 (for example, executing the virtual studio full-flow multi-terminal board book extraction program 12 and the like) and calling data stored in the memory 11.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 5 only shows an electronic device with components, and it will be understood by a person skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The virtual studio full-flow multi-terminal blackboard-writing extraction program 12 stored in the memory 11 of the electronic device 1 is a combination of a plurality of programs, and when running in the processor 10, can realize:
filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol;
constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture;
carrying out portrait cutout on the blackboard writing image by utilizing a pre-trained portrait cutout model to generate a portrait image;
and generating a blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the foregoing program, which is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a non-volatile computer-readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol;
constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture;
carrying out portrait cutout on the blackboard writing image by utilizing a pre-trained portrait cutout model to generate a portrait image;
and generating a blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A virtual studio full-flow multi-terminal blackboard writing extraction method is characterized by comprising the following steps:
filtering the blackboard-writing text data based on a preset data filtering mechanism, and transmitting the filtered blackboard-writing text data to a virtual studio by using a data transmission protocol;
constructing a text picture for the blackboard writing text data in the virtual studio, and creating a response event button of the text picture;
carrying out portrait cutout on the blackboard writing image by utilizing a pre-trained portrait cutout model to generate a portrait image;
and generating a blackboard-writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
2. The virtual studio full-flow multi-terminal blackboard writing extraction method of claim 1, wherein the transmitting the filtered blackboard writing text data to the virtual studio by using a data transmission protocol comprises:
acquiring a program running file packet of the filtered blackboard-writing text data, and identifying interface parameters of the program running file packet;
acquiring port parameters of the virtual studio;
configuring the program operation file packet and a data transmission channel of the virtual studio by using the data transmission protocol according to the interface parameter and the port parameter;
and transmitting the filtered blackboard writing text data to the virtual studio based on the data transmission channel.
3. The virtual studio full-flow multi-terminal blackboard writing extraction method of claim 1, wherein the creating the response event button of the text screen includes:
acquiring a compiling environment of the text picture;
creating a response button script of the text screen in the compiling environment;
and loading an event trigger button of the corresponding text picture based on the response event script, and generating a response event button.
4. The virtual studio full-flow multi-terminal blackboard writing extraction method of claim 1, wherein the portrait matting the blackboard writing image by using the pre-trained portrait matting model to generate the portrait image comprises:
utilizing a portrait segmentation module in the portrait cutout model to perform portrait segmentation processing on the blackboard-writing image to obtain a portrait segmentation image;
and carrying out cutout processing on the portrait sampling image by utilizing a portrait regression module in the portrait cutout model to obtain a portrait image.
5. The method for extracting a full-flow multi-terminal blackboard writing in a virtual studio according to claim 4, wherein the step of performing portrait segmentation processing on the blackboard writing image by using a portrait segmentation module in the portrait cutout model to obtain a portrait segmentation map comprises the steps of:
performing feature extraction on the blackboard writing image by using the portrait segmentation module to obtain a feature map;
and performing convolution operation on the feature map and outputting to obtain the portrait segmentation map.
6. The virtual studio full-flow multi-terminal blackboard writing extraction method of claim 4, wherein the matting processing of the portrait sampling drawing by the portrait regression module in the portrait matting model to obtain a portrait image comprises:
utilizing the portrait regression module to perform up-sampling on the portrait segmentation image to obtain a portrait sampling image;
splicing the portrait sampling image and the blackboard-writing image to obtain a spliced image of the characteristic sampling image and the original image;
performing fusion operation on the spliced image to obtain a fused image of the spliced image;
and outputting the fusion image by using an activation function in the portrait regression module to obtain a portrait image.
7. The virtual studio full-flow multi-terminal blackboard writing extraction method of any one of claims 1 to 6, wherein the generating of the blackboard writing interface of the virtual studio according to the text picture, the response event button and the portrait image comprises:
constructing an interface display layer in the virtual studio, wherein the blackboard writing interface display layer comprises a blackboard card layer, an operation picture layer and a portrait layer;
loading the text picture, the response event button and the portrait image into the board card layer, the operation picture layer and the portrait layer respectively to generate a board card picture, a response picture and a portrait picture;
and fusing the board card picture, the response picture and the portrait picture to generate a board writing interface of the virtual studio.
8. The utility model provides a virtual studio total flow multi-terminal blackboard writing extraction element which characterized in that, the device includes:
the transmission module is used for filtering the blackboard writing text data based on a preset data filtering mechanism and transmitting the filtered blackboard writing text data to the virtual studio by using a data transmission protocol;
the creating module is used for creating a text picture for the blackboard writing text data in the virtual studio and creating a response event button of the text picture;
the cutout module is used for carrying out cutout on the blackboard writing image by utilizing the pre-trained cutout model to generate a portrait image;
and the generating module is used for generating a blackboard writing interface of the virtual studio according to the text picture, the response event button and the portrait image.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the virtual studio full flow multi-terminal blackboard writing extraction method of any one of claims 1 to 7.
10. A computer-readable storage medium, storing a computer program, wherein the computer program, when executed by a processor, implements the virtual studio full-flow multi-terminal blackboard writing extraction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110252053.2A CN113010129A (en) | 2021-03-08 | 2021-03-08 | Virtual studio full-flow multi-terminal blackboard writing extraction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110252053.2A CN113010129A (en) | 2021-03-08 | 2021-03-08 | Virtual studio full-flow multi-terminal blackboard writing extraction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113010129A true CN113010129A (en) | 2021-06-22 |
Family
ID=76408760
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110252053.2A Pending CN113010129A (en) | 2021-03-08 | 2021-03-08 | Virtual studio full-flow multi-terminal blackboard writing extraction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113010129A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116778121A (en) * | 2023-06-29 | 2023-09-19 | 南京云视全映科技有限公司 | Virtual screen writing control synthesis system and method |
-
2021
- 2021-03-08 CN CN202110252053.2A patent/CN113010129A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116778121A (en) * | 2023-06-29 | 2023-09-19 | 南京云视全映科技有限公司 | Virtual screen writing control synthesis system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108292231B (en) | Method and system for generating applications from data | |
| EP3433732B1 (en) | Converting visual diagrams into code | |
| CN110990053A (en) | Method for creating and using machine learning scheme template and device | |
| CN112396677B (en) | Animation production method, electronic device, and storage medium | |
| CN114020256A (en) | Front-end page generation method, device and equipment and readable storage medium | |
| CN113010129A (en) | Virtual studio full-flow multi-terminal blackboard writing extraction method and device | |
| CN113360139A (en) | Integration method and device of front-end frame, electronic equipment and storage medium | |
| CN114398282A (en) | Test script generation method, device, equipment and storage medium | |
| CN114911479A (en) | Interface generation method, device, equipment and storage medium based on configuration | |
| CN115409041A (en) | Unstructured data extraction method, device, equipment and storage medium | |
| CN115016754A (en) | Method and device for synchronously displaying pages among devices, electronic device and medium | |
| CN112233194B (en) | Medical picture optimization method, device, equipment and computer readable storage medium | |
| CN114004591A (en) | Automatic user-defined form auditing method and device, electronic equipment and storage medium | |
| CN114443022A (en) | Method for generating page building block and electronic equipment | |
| CN114721952A (en) | Method, device, equipment and storage medium for synchronously deploying multiple sets of test environments | |
| CN106446611A (en) | Security-level information generation and reading methods and apparatuses | |
| CN111667411A (en) | Image transmission method and device, electronic equipment and storage medium | |
| CN114363718B (en) | Classroom control method, device, equipment and medium for remote multimedia | |
| CN113657076B (en) | Page operation record table generation method and device, electronic equipment and storage medium | |
| CN112214981B (en) | Presentation file generation method and device, electronic equipment and storage medium | |
| CN113672565B (en) | File marking method and device, electronic equipment and storage medium | |
| CN115129415A (en) | Cross-domain dragging method, device, equipment and storage medium based on webpage interface | |
| CN114995940A (en) | Bullet frame configuration and display method, device, equipment and storage medium | |
| CN113656431A (en) | Graph display method based on big data | |
| CN115277469A (en) | Weak network visualization control method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: No. 868-3, Shangshengou Village, Hunnan District, Shenyang City, Liaoning Province, 110000 (101, 201) Applicant after: Liaoning Sunflower Digital Technology Co.,Ltd. Address before: Room c418, 860-1, shangshengou village, Dongling District, Shenyang City, Liaoning Province, 110000 Applicant before: LIAONING SUNFLOWER EDUCATION TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| CB02 | Change of applicant information |