CN112804459A - Image display method and device based on virtual camera, storage medium and electronic equipment - Google Patents
Image display method and device based on virtual camera, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112804459A CN112804459A CN202110038745.7A CN202110038745A CN112804459A CN 112804459 A CN112804459 A CN 112804459A CN 202110038745 A CN202110038745 A CN 202110038745A CN 112804459 A CN112804459 A CN 112804459A
- Authority
- CN
- China
- Prior art keywords
- image data
- camera
- virtual camera
- management module
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000004891 communication Methods 0.000 claims abstract description 6
- 238000005516 engineering process Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 abstract description 23
- 238000009877 rendering Methods 0.000 description 29
- 230000000694 effects Effects 0.000 description 10
- 230000007704 transition Effects 0.000 description 6
- 206010044565 Tremor Diseases 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000003139 buffering effect Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 239000011230 binding agent Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
- H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Abstract
The invention discloses an image display method based on a virtual camera, which comprises the following steps: after the virtual camera acquires the preview interface calling request, establishing communication connection with a camera management module, and initializing a cache queue capable of transmitting data across processes; the camera management module acquires a plurality of same or different image data sources, target image data are obtained from the image data sources, and the target image data are output to at least one virtual camera through the cache queue; the virtual camera sends the target image data to at least one data demand main body for image preview, wherein the data demand main body comprises other windows and/or third-party applications of a single application; the virtual camera is constructed from the camera driving layer, the expansibility is strong, the requirement of simultaneously previewing the data of the camera on a plurality of applications or platforms can be met, and the camera management module is an upper application and has great expansibility and space for processing images.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image display method and device based on a virtual camera, a storage medium and electronic equipment.
Background
The process of generally using a camera on a mobile terminal is as follows:
and calling a camera open interface, and establishing connection with the bottom layer camera drive. Examples of bottom layer cameras include front-facing cameras, rear-facing cameras, USB cameras, etc., also known as real cameras.
Setting preview parameters of a real camera, and binding a preview image to a display window of the Android device. The preview parameters include the width, height, frame rate and the like of the preview image, and the display window is called Surface.
And calling a start preview interface of the camera to inform a bottom layer camera to drive to start outputting the video image stream, namely starting previewing, and displaying the image of the camera on a screen of the mobile terminal. And closing the camera after the preview is finished, and releasing the bottom layer driving object.
In general, when an application uses a camera, only one camera instance can be obtained at a time to preview a data stream (and image data content) of one camera, if the camera needs to be switched, the previous camera needs to be closed and the next camera needs to be opened, and multiple applications cannot use the camera at the same time, and only one application can use the camera service at the same time.
Chinese patent publication No. CN 108845861 a discloses a method and an apparatus for processing a virtual camera, which creates a virtual camera by hooking some interfaces on the upper layer of the camera through technologies such as reflection, etc., thereby implementing preview of image data, extending the source of image data, and making the source of image data of electronic equipment no longer depend on hardware camera. However, the source code level is not modified, and the way of hooking the upper interface of the camera by the reflection technology has great limitation, and the virtual camera can only be used in a single application, and cannot realize cross-process, that is, the virtual camera cannot be used by third-party applications such as jittering and the like. In addition, the hitchable interface is limited, and the situation of surface preview cannot be handled.
Disclosure of Invention
In view of the foregoing defects in the prior art, an object of the present invention is to provide an image display method, an image display apparatus, a storage medium, and an electronic device based on a virtual camera, which are capable of outputting multiple image data to different third-party applications simultaneously across processes. The method is realized by the following technical scheme:
in scheme 1, an image display method based on a virtual camera includes:
after the virtual camera acquires the preview interface calling request, establishing communication connection with a camera management module, and initializing a cache queue capable of transmitting data across processes;
the camera management module acquires a plurality of same or different image data sources, target image data are obtained from the image data sources, and the target image data are output to at least one virtual camera through the cache queue;
the virtual camera sends the target image data to at least one data demand main body for image preview, wherein the data demand main body comprises other windows and/or third-party applications of a single application;
the virtual camera is located on the camera driving level, and the camera management module is located on the system application level.
In the scheme 2, on the basis of the scheme 1, the virtual camera and the image data source are bound according to a preset corresponding relationship, and the camera management module acquires a plurality of identical or different image data sources, including: and the camera management module determines and acquires the image data source bound with the virtual camera according to the preset corresponding relation. Such as: the multiple same or different image data sources may be video and LOGO pictures, available for post-editing processing.
In scheme 3, on the basis of scheme 1, the virtual camera sends the target image data to at least one data demand subject for image preview, including: a single virtual camera sends a plurality of same or different target image data to a data demand subject for image preview; or a single virtual camera sends a plurality of same or different target image data to a plurality of data demand subjects for image preview; or a plurality of virtual cameras send a plurality of same or different target image data to a data demand subject for image preview; or the plurality of virtual cameras send a plurality of same or different target image data to a plurality of data demand subjects for image preview.
Wherein, a single virtual camera sends a plurality of same or different target image data to a data demand subject for image preview, such as: the image data source can be a video source and a LOGO picture, is edited into a plurality of identical or different videos with LOGO, and is respectively sent to a third-party application or a plurality of display windows of a single application for image preview;
wherein: a single virtual camera sends a plurality of same or different target image data to a plurality of data demand subjects for image preview, such as: respectively sending the same or different videos with LOGO to a plurality of third-party applications such as trembler, fast hand and the like for image preview;
wherein: the multiple virtual cameras send multiple same or different target image data to one data demand subject for image preview, such as: simultaneously sending the same or different videos with LOGO to a third-party application such as a trembler application, a fast hand application and the like for image preview;
the multiple virtual cameras send multiple same or different target image data to multiple data demand subjects for image preview, for example: and simultaneously sending the same or different videos with LOGO to a plurality of third-party applications such as tremble, fast hands and the like for image preview.
And in the scheme 4, on the basis of the scheme 1, the image data source is from a real camera, a virtual camera, a local storage file, a website or live webcasting, so that the providing mode of the image data source is widened.
In the scheme 5, on the basis of the scheme 1, the buffer queue capable of transmitting data across processes is implemented by an ion technology.
And according to the scheme 6, on the basis of the scheme 1, the system application layer is applied to one or more of an Android system, an iOS system, a YunOS system, a Windows Phone system and a Hongmon system. The application level of the system is particularly the system used by mobile terminals such as mobile phones and tablet computers, and the system is different from a PC (personal computer) and cannot be used across frames.
In the scheme 7, on the basis of any one of the schemes 1 to 6, after the camera management module acquires the image data source, the method further includes: and editing the image data source according to a preset editing instruction or a specified editing instruction generated based on user operation to obtain target image data. The editing process can be understood as beautifying the video content, wherein the beautifying includes presetting the beautifying degree, and the beautifying degree can also be set by the user.
Scheme 8, this application still provides an image display device based on virtual camera, includes:
virtual camera: after a preview interface calling request is obtained, establishing communication connection with a camera management module, and initializing a cache queue capable of transmitting data in a cross-process manner;
an image data acquisition module: for acquiring a plurality of identical or different image data sources;
the camera management module: target image data obtained by an image data source is output to at least one virtual camera through the buffer queue,
the virtual camera sends the target image data to at least one data demand main body for image preview, wherein the data demand main body comprises other windows and/or third-party applications of a single application;
the virtual camera is located on the camera driving level, and the camera management module is located on the system application level.
In solution 9, the present application further provides a readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the image displaying method according to any one of solutions 1 to 7.
In scheme 10, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image display method according to any one of schemes 1 to 7 when executing the program. The electronic device may be a mobile phone, a direct broadcast, a tablet computer, a Personal Digital Assistant (PDA), or other terminal devices.
In the scheme 11, on the basis of the scheme 2, the preset corresponding relationship between the virtual camera and the image data source can be edited and adjusted on a system application level. The image data source can be appointed to be output according to the user requirement or the target image data in the process of editing and adjusting can be appointed. If the corresponding relation is preset between the virtual camera and one video data, if a user needs to obtain target video data with application LOGO added to the original video data, the virtual camera can be adjusted to be corresponding to the target video data with the LOGO added.
In the scheme 12, on the basis of the scheme 1, the image data source includes video files, pictures, AR images, or customized contents, and the customized contents include characters, graphics, or animations drawn by using a system image tool, so that the format type range of the image data source is expanded.
In the scheme 13, on the basis of the scheme 1, the camera management module may obtain the target image data from the image data source through an editing process.
Compared with the prior art, the invention also has the following beneficial effects:
1) the interface of the upper layer of the system is not modified, and the intrusion to the system is small;
2) the virtual cameras are constructed from the camera driving layer, the expansibility is strong, namely a plurality of virtual cameras can be provided, and the requirement that a plurality of applications or platforms preview the data of the cameras at the same time can be met;
3) the camera management module is an upper application, and has great expansibility and space for processing images, and the following expansion examples are as follows:
the camera management module can output the local video and the network video to the virtual camera as video sources; the camera management module can output the local image or the network image to the virtual camera as a film source; the camera management module can output self-defined contents such as self-drawn images, subtitles, UI and other resources to the virtual camera; the camera management module can combine the video sources, film sources or custom contents, add special effects such as beauty, green screen and the like, and output the special effects to the virtual camera.
The camera management module can simultaneously output the edited target image data to a plurality of different third-party platforms; different edits can be carried out on the same video file on the interactive editing page, and a plurality of different video contents are respectively output to a plurality of different third-party platforms at the same time, so that the content customization requirements of the application can be met.
Drawings
FIG. 1 is a flowchart of an image display method based on a virtual camera according to an embodiment of the present invention;
FIG. 2 is a block diagram of an image display apparatus based on a virtual camera according to an embodiment of the present invention;
fig. 3 is a block diagram of an electronic device of an image display apparatus based on a virtual camera according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention are further described below by using specific embodiments or examples, and it should be noted that the specific embodiments or examples described below are not intended to limit the claims of the present invention.
An image display processing device based on a virtual camera, as shown in fig. 2, includes:
the system comprises an image data acquisition module 1, a virtual camera 3, a camera management module 2 and a data demand main body 4, wherein the virtual camera 3 is located at a camera driving level, and the camera management module 2 is located at a system application level.
The system application layer of this embodiment takes an Android system as an example. The preview-to-display window of the present embodiment can be a surface preview. In this embodiment, the camera management module 2 may invoke a GPU, a memory, and the like to perform an editing process.
The aforementioned device, operating by the following steps, as shown in fig. 1, comprises:
s1: after acquiring a preview interface calling request, the virtual camera 3 establishes communication connection with the camera management module 2 and initializes a cache queue capable of transmitting data across processes;
s2: the image data acquisition module 1 acquires a plurality of same or different image data sources; the camera management module 2 outputs the target image data obtained from the image data source to at least one virtual camera 3 through the cache queue;
s3: the virtual camera 3 sends the target image data to at least one data demand main body 4 for image preview, and the data demand main body 4 comprises other windows of a single application (such as own video content and other people video content in a multi-people PK video picture of live broadcast software are displayed simultaneously) and/or third-party applications (including different live broadcast software, a shopping live broadcast platform, a video social platform and the like).
In the embodiment, the method can be realized by the following technical scheme:
in this embodiment, the virtual camera 3 and the image data source are bound according to a preset corresponding relationship, and the camera management module 2 obtains a plurality of identical or different image data sources, including: and the camera management module 2 determines and acquires the image data source bound with the virtual camera 3 according to the preset corresponding relation. Such as: the multiple same or different image data sources may be video and LOGO pictures, available for post-editing processing.
In this embodiment, the preset corresponding relationship between the virtual camera 3 and the image data source may be edited and adjusted in the system application layer. The image data source can be appointed to be output according to the user requirement or the target image data in the process of editing and adjusting can be appointed. If the corresponding relationship is preset between the virtual camera 3 and one video data, if a user needs to obtain target video data with the application LOGO added to the original video data, the virtual camera 3 can be adjusted to correspond to the target video data with the LOGO added.
In this embodiment, the sending, by the virtual camera 3, the target image data to at least one data demand subject 4 for image preview includes: a single virtual camera 3 sends a plurality of same or different target image data to one data demand subject 4 for image preview, such as: the multiple same or different image data sources can be video sources and LOGO pictures, are edited into a video with LOGO and are respectively sent to a third-party application or multiple display windows of a single application for use;
or the single virtual camera 3 sends a plurality of same or different target image data to a plurality of data demand bodies 4 for image preview, such as: respectively sending the video with LOGO to a plurality of third-party applications such as tremble or fast hands for use;
or a plurality of virtual cameras 3 send a plurality of same or different target image data to one data demand subject 4 for image preview, such as: simultaneously sending the same or different videos with LOGO to a third-party application such as a trembler application, a fast hand application and the like for image preview;
or the plurality of virtual cameras 3 send a plurality of same or different target image data to the plurality of data demand subjects 4 for image preview, for example: and simultaneously sending the same or different videos with LOGO to a plurality of third-party applications such as tremble, fast hands and the like for image preview.
In this embodiment, the image data source may select a video stream, and the content includes: the video file can be a local sd or an internally stored video file, can also be a video file on the network, and can also be a video stream, a picture, a network picture of network live broadcast or a local album picture, gif, AR image and the like; and can also be custom content such as text, graphics or animation drawn using system graphics tools.
The content of the video stream can be acquired through a real camera, such as a common camera, a USB camera, an HDMI camera and the like. The acquisition processes of the video stream contents are different, and the common camera acquires the video stream contents through a system interface; the USB camera acquires the video stream content through the expanded system interface; the network video or the local video is unpacked through a system Media interface, and the video is decoded and the data is output to a surface; the picture is read and written into the memory through the file, and then the picture format is decoded and uploaded to the GPU; the AR image is provided by the AR core provided by Google; the self-defined content is drawn through a system image interface, and mainly comprises a view technology, a canvas drawing technology and the like.
The buffer queue capable of transmitting data across processes is implemented by an ion technology.
In this embodiment, after the camera management module 2 acquires an image data source, the method further includes: and editing the image data source according to a preset editing instruction or a specified editing instruction generated based on user operation to obtain target image data. The editing process can be understood as beautifying the video content, wherein the beautifying includes presetting the beautifying degree, and the beautifying degree can also be set by the user.
The present embodiment also provides a readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the image presentation method according to any one of the preceding claims.
The present embodiment further provides an electronic device, as shown in fig. 3, including a memory 101, a processor 102, and a computer program stored on the memory 101 and operable on the processor 102, wherein the processor 102 implements the steps of the image displaying method according to any one of the foregoing schemes when executing the program. The electronic device may be a mobile phone, a direct broadcast, a tablet computer, a Personal Digital Assistant (PDA), or other terminal devices.
In addition, the camera management module 2 of this embodiment may obtain the target image data from the image data source through an editing process.
The editing process comprises the following steps:
the camera management module 2 edits the image data source to obtain at least one target image data, and distributes the target image data to at least one virtual camera 3 according to a preset corresponding relation;
in this embodiment, the following technical solutions can be specifically adopted:
in this embodiment, the editing the image data source to obtain at least one target image data includes: performing multi-node output on a single editing processing process of the image data source to obtain at least one target image data; or, a plurality of editing processes are executed on the image data source to obtain at least one target image data.
Wherein, the multi-node output is performed on the single editing processing process of the image data source to obtain at least one target image data, and the method includes:
determining a node path of each node;
forming a directed acyclic graph of a single editing processing process according to the node path of each node;
and editing the image data source according to the directed acyclic graph to obtain at least one target image data.
Specifically, in this scheme, the editing process may be formed by combining a plurality of nodes in a directed acyclic graph, where each node is a pass, the node processing result is stored and output by binding fbo, the node processing result stored in fbo is output to an output destination or output to one or more next nodes as input, when the node result stored in fbo is output to the output destination, the output destination may be used as a new node and input to the directed acyclic graph as input of the next node or nodes, a plurality of node processing results are obtained by processing a plurality of nodes of different paths in the directed acyclic graph, and the plurality of node processing results are stored in a plurality of fbo and are respectively output to the plurality of output destinations correspondingly.
In this embodiment, after acquiring the image data source, the method further includes:
if the fact that the interactive interface is instructed to be edited is detected, an editing instruction is generated through an Open GL interface technology;
the editing the image data source to obtain at least one target image data includes:
and editing the image data source according to the editing instruction to obtain at least one target image data.
In this embodiment, the editing process includes video content assembly, superimposed content assembly, video animation, video special effect, transition, and rendering, which are performed in sequence.
The video content is assembled to arrange one or more video pictures to an editing position, wherein the arrangement of the video pictures can be front-back superposition, left-right arrangement, up-down arrangement and the like, and the arrangement of the video pictures can be supported at will;
the overlapped content is static resources such as pictures and characters, and can also be a gif dynamic image, elements with a certain shielding relation exist on the upper layer of the video, and the elements comprise watermarks, portrait stickers and the like;
the video animation is to change the picture in time, such as moving, zooming, rotating, cutting, and the like;
the video special effect is to beautify and enhance the video picture, such as using some video filters and the like;
the transition is a transition special effect when one video picture is switched to another video scene;
and the rendering is to generate rendering instructions of the GPU through the rendering parameters and resources generated by the processes through an open GL interface, and transmit the rendering instructions to the GPU to drive the GPU to perform image synthesis operation of the processes.
The above process is a rendering process, which implements a simple rendering engine.
Specifically, in this embodiment, the output target includes one or more of a system surface component, GPU texture buffering, or a custom image queue implementation, and in this embodiment, the system surface component is selected to preview the target image data.
In this embodiment, the editing process performed in the camera management module 2 is implemented by a rendering engine. The video stream is specifically subjected to composition processing using Open GL.
The rendering engine realizes a simple rendering process through video content assembly, superposed content assembly, video animation, video special effects, transition and rendering.
The video content is assembled to arrange one or more video pictures to an editing position, wherein the arrangement of the video pictures to the editing position can be front-back superposition, left-right arrangement, up-down arrangement and the like, and the random arrangement of a plurality of video pictures is supported;
the superposed contents assembled by the superposed contents can be static resources such as pictures and characters, and can also be gif dynamic images, elements with certain shielding relation exist on the upper layer of the video, the elements can not fully cover the screen, and the elements are generally applied to watermarking, portrait stickers and the like;
video animation is the temporal change of a picture, such as moving, zooming, rotating, cutting, and the like;
the video special effect is to beautify and enhance the video picture, such as using some video filters and the like;
transition is a transition special effect made when one video picture is switched to another video scene;
and the rendering is to generate rendering instructions of the GPU by the rendering parameters and resources generated by the processes through an interface of open GL, and transmit the rendering instructions to the GPU to drive the GPU to perform image synthesis operation of the processes.
The video rendering processing can output multiple targets, and the output targets are defined as the following:
the display window of the system can be directly displayed on a screen;
GPU texture buffering can be used for secondary processing;
and the image data queue can be used for cross-process transmission of target image data.
The rendering simple expression can be composed of a plurality of passes, namely a process, one pass is one-time drawing and comprises vertex processing and fragment source processing, the pass is packaged into a program object in a program, and one rendering effect is basically completed by combining the plurality of passes.
The GPU related technology described in this embodiment includes the following three types:
fbo buffering technique: fbo rendering buffer provided by the GPU, which can be used to save the intermediate process of rendering, including texture image information, depth information, etc.;
pbo technique: the API interface provided by the GPU for efficiently transmitting data to the GPU or transmitting data from the GPU to the memory can be used for processing video stream transmission to the GPU;
image read technology: the Android system provides an interface for efficiently copying and converting image data from the GPU, and the interface can be used for outputting the image data.
In this embodiment, the camera management module 2 acquires a video stream, uploads the video stream to the GPU through the surface technology, and provides an interactive interface to enable a user to edit an image data source of the user, an instruction that can be executed by the GPU is generated by a rendering engine in the editing process to render and synthesize an image, then the GPU outputs the synthesized content (i.e., target image data) to the memory, and the camera management module 2 distributes the output content to the virtual camera 3, thereby completing customized preview of the third-party application content.
In the embodiment, because the GPU has extremely high parallel computing capability, the method has high efficiency on the computation-intensive task of the video image; the surface is used as a cache component for system optimization, and can upload data to GPU storage by using dma technology without consuming CPU time, so that the performance of the machine is greatly saved.
In the present embodiment, the arrangement of video images and the like, the selection of images, the addition of special effects, and the like in the rendering engine are edited and completed through the interaction applied by the camera management module 2. And then, the rendering resources and parameters generated by the user through editing are used for generating rendering instructions of the GPU through the rendering engine to complete the rendering process. The rendering engines are sequentially executed in the GPU, and most of editing requirements of the video stream are met.
Specifically, the rendering engine (i.e., the editing process) in the camera management module 2 may support outputting a plurality of different contents in design, the rendering process is packaged into one flow, the flow may specify an output target, a plurality of targets may be added, for scenes in which a plurality of users need different contents, a plurality of flows may be created to edit different contents and output the different contents to different targets, and one flow may also output the process contents in the execution process to different targets. The design can meet the requirement that different applications customize different contents. The output target described above refers to a component that can store and output image data. In this embodiment, the component for storing and outputting image data is a system surface component.
In this embodiment, the camera management module 2 transfers the data of the GPU to the memory through the surface technology. The method includes the steps that data are provided for the virtual camera 3 and must be copied from a memory, data in the GPU need to be copied to the memory, the GPU can only output RGBA data, the virtual camera 3 only receives YUV data formats, and therefore a data conversion process exists, the process is a calculation-intensive task.
In this embodiment, specifically, the user output target (specifically, a cache address of the surface component in this embodiment) stored by the virtual camera 3 is obtained by communicating with the virtual camera 3 through the binder interface, and the user output target is returned to the application layer in a file handle manner, in the process, a memory transfers the fd handle to another process through the binder, and in this manner, memory copy is not performed, so that the two processes can share one memory, which greatly improves efficiency.
The output content can be copied to the cache address of the surface component of the user through the processes, and then the virtual camera 3 informs the third-party application of the data so as to finish the preview.
The process for meeting the requirement of customizing different contents by different applications and distributing different contents to different applications comprises the following specific steps: obtaining a video stream subjected to synthesis processing in a GPU; the memory obtains the video stream data output from the GPU after processing to generate different contents, and then the contents are processed through the following steps: and the synthesized video stream data is distributed to the virtual camera 3 from the memory, so that different contents are distributed to different applications for display.
It should be noted that the english abbreviation techniques described in the examples and the specification of the present application are all commonly used in the art, and need not be described in detail. Furthermore, the technical features of the embodiments or examples described in the specification may be different in name, and all equivalent changes or simple changes made according to the technical features and principles described in the patent idea of the invention are included in the protection scope of the patent of the invention.
Claims (10)
1. An image display method based on a virtual camera is characterized by comprising the following steps:
after the virtual camera acquires the preview interface calling request, establishing communication connection with a camera management module, and initializing a cache queue capable of transmitting data across processes;
the camera management module acquires a plurality of same or different image data sources, target image data are obtained from the image data sources, and the target image data are output to at least one virtual camera through the cache queue;
the virtual camera sends the target image data to at least one data demand main body for image preview, wherein the data demand main body comprises other windows and/or third-party applications of a single application;
the virtual camera is located on the camera driving level, and the camera management module is located on the system application level.
2. The image display method of claim 1, wherein the virtual camera is bound to the image data source according to a preset correspondence, and the acquiring, by the camera management module, a plurality of identical or different image data sources comprises: and the camera management module determines and acquires the image data source bound with the virtual camera according to the preset corresponding relation.
3. The image display method of claim 1, wherein the virtual camera sends the target image data to at least one data demand subject for image preview, comprising: a single virtual camera sends a plurality of same or different target image data to a data demand subject for image preview; or a single virtual camera sends a plurality of same or different target image data to a plurality of data demand subjects for image preview; or a plurality of virtual cameras send a plurality of same or different target image data to a data demand subject for image preview; or the plurality of virtual cameras send a plurality of same or different target image data to a plurality of data demand subjects for image preview.
4. The image presentation method of claim 1, wherein the image data source is derived from a real camera, a virtual camera, a locally stored file, a website, or a live webcast.
5. The image presentation method of claim 1, wherein the buffer queue capable of transmitting data across processes is implemented by an ion technology.
6. The image display method according to claim 1, wherein the system application layer is applied to one or more of an Android system, an iOS system, a YunOS system, a Windows Phone system, and a dammon system.
7. The image display method according to any one of claims 1 to 6, wherein after the camera management module obtains an image data source, the method further comprises: and editing the image data source according to a preset editing instruction or a specified editing instruction generated based on user operation to obtain target image data.
8. An image display device based on a virtual camera is characterized by comprising:
virtual camera: after a preview interface calling request is obtained, establishing communication connection with a camera management module, and initializing a cache queue capable of transmitting data in a cross-process manner;
an image data acquisition module: for acquiring a plurality of identical or different image data sources;
the camera management module: outputting the target image data obtained by an image data source to at least one virtual camera through the cache queue;
the virtual camera sends the target image data to at least one data demand main body for image preview, wherein the data demand main body comprises other windows and/or third-party applications of a single application;
the virtual camera is located on the camera driving level, and the camera management module is located on the system application level.
9. A readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the image presentation method according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the image presentation method according to any one of claims 1 to 7 are implemented when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110038745.7A CN112804459A (en) | 2021-01-12 | 2021-01-12 | Image display method and device based on virtual camera, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110038745.7A CN112804459A (en) | 2021-01-12 | 2021-01-12 | Image display method and device based on virtual camera, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112804459A true CN112804459A (en) | 2021-05-14 |
Family
ID=75810224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110038745.7A Pending CN112804459A (en) | 2021-01-12 | 2021-01-12 | Image display method and device based on virtual camera, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112804459A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113596321A (en) * | 2021-06-16 | 2021-11-02 | 荣耀终端有限公司 | Transition dynamic effect generation method, apparatus, storage medium, and program product |
| CN113873158A (en) * | 2021-09-30 | 2021-12-31 | 联想(北京)有限公司 | Electronic equipment and control method thereof |
| CN113890998A (en) * | 2021-10-26 | 2022-01-04 | Oppo广东移动通信有限公司 | Image data processing method and device, electronic equipment and storage medium |
| CN113992854A (en) * | 2021-10-29 | 2022-01-28 | Oppo广东移动通信有限公司 | Image preview method and device, electronic equipment and computer readable storage medium |
| CN114257831A (en) * | 2021-12-21 | 2022-03-29 | 广州播丫科技有限公司 | Access management system and method for multi-channel virtual video source |
| CN114363654A (en) * | 2022-01-12 | 2022-04-15 | 北京字节跳动网络技术有限公司 | Video plug-flow method, device, terminal equipment and storage medium |
| CN114449171A (en) * | 2022-01-28 | 2022-05-06 | 青岛海信移动通信技术股份有限公司 | Method for controlling camera, terminal device, storage medium and program product |
| CN114500877A (en) * | 2021-12-30 | 2022-05-13 | 惠州华阳通用智慧车载系统开发有限公司 | Camera sharing method, system and storage medium |
| CN115119003A (en) * | 2022-05-16 | 2022-09-27 | 澳克多普有限公司 | Application method and system of virtual camera in live broadcast room |
| CN115185711A (en) * | 2022-09-06 | 2022-10-14 | 湖北芯擎科技有限公司 | Data interaction method and device based on virtual camera, electronic equipment and medium |
| CN116708867A (en) * | 2023-08-07 | 2023-09-05 | 深圳市亿晟科技有限公司 | Live broadcast data processing method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100231754A1 (en) * | 2009-03-11 | 2010-09-16 | Wang Shaolan | Virtual camera for sharing a physical camera |
| KR101697713B1 (en) * | 2016-06-28 | 2017-01-19 | (주)인트라테크 | Method and apparatus for generating intelligence panorama VR(virtual reality) contents |
| CN107659768A (en) * | 2017-08-08 | 2018-02-02 | 珠海全志科技股份有限公司 | A kind of system and method based on the more Application share cameras of Android |
| CN109168021A (en) * | 2018-10-25 | 2019-01-08 | 京信通信系统（中国）有限公司 | A kind of method and device of plug-flow |
| CN112118459A (en) * | 2020-08-28 | 2020-12-22 | 上海商米科技集团股份有限公司 | Method for realizing common live broadcast of multiple apps by sharing camera data and readable medium |
-
2021
- 2021-01-12 CN CN202110038745.7A patent/CN112804459A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100231754A1 (en) * | 2009-03-11 | 2010-09-16 | Wang Shaolan | Virtual camera for sharing a physical camera |
| KR101697713B1 (en) * | 2016-06-28 | 2017-01-19 | (주)인트라테크 | Method and apparatus for generating intelligence panorama VR(virtual reality) contents |
| CN107659768A (en) * | 2017-08-08 | 2018-02-02 | 珠海全志科技股份有限公司 | A kind of system and method based on the more Application share cameras of Android |
| CN109168021A (en) * | 2018-10-25 | 2019-01-08 | 京信通信系统（中国）有限公司 | A kind of method and device of plug-flow |
| CN112118459A (en) * | 2020-08-28 | 2020-12-22 | 上海商米科技集团股份有限公司 | Method for realizing common live broadcast of multiple apps by sharing camera data and readable medium |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113596321A (en) * | 2021-06-16 | 2021-11-02 | 荣耀终端有限公司 | Transition dynamic effect generation method, apparatus, storage medium, and program product |
| CN113596321B (en) * | 2021-06-16 | 2023-05-09 | 荣耀终端有限公司 | Method, device and storage medium for generating transition dynamic effect |
| CN113873158A (en) * | 2021-09-30 | 2021-12-31 | 联想(北京)有限公司 | Electronic equipment and control method thereof |
| CN113890998A (en) * | 2021-10-26 | 2022-01-04 | Oppo广东移动通信有限公司 | Image data processing method and device, electronic equipment and storage medium |
| CN113992854A (en) * | 2021-10-29 | 2022-01-28 | Oppo广东移动通信有限公司 | Image preview method and device, electronic equipment and computer readable storage medium |
| CN114257831A (en) * | 2021-12-21 | 2022-03-29 | 广州播丫科技有限公司 | Access management system and method for multi-channel virtual video source |
| CN114257831B (en) * | 2021-12-21 | 2023-11-24 | 广州播丫科技有限公司 | Access management system and method for multipath virtual video source |
| CN114500877A (en) * | 2021-12-30 | 2022-05-13 | 惠州华阳通用智慧车载系统开发有限公司 | Camera sharing method, system and storage medium |
| CN114363654B (en) * | 2022-01-12 | 2023-12-19 | 北京字节跳动网络技术有限公司 | Video push method, device, terminal equipment and storage medium |
| WO2023134509A1 (en) * | 2022-01-12 | 2023-07-20 | 北京字节跳动网络技术有限公司 | Video stream pushing method and apparatus, and terminal device and storage medium |
| CN114363654A (en) * | 2022-01-12 | 2022-04-15 | 北京字节跳动网络技术有限公司 | Video plug-flow method, device, terminal equipment and storage medium |
| CN114449171B (en) * | 2022-01-28 | 2023-10-27 | 青岛海信移动通信技术有限公司 | Method for controlling camera, terminal device, storage medium and program product |
| CN114449171A (en) * | 2022-01-28 | 2022-05-06 | 青岛海信移动通信技术股份有限公司 | Method for controlling camera, terminal device, storage medium and program product |
| CN115119003A (en) * | 2022-05-16 | 2022-09-27 | 澳克多普有限公司 | Application method and system of virtual camera in live broadcast room |
| CN115185711B (en) * | 2022-09-06 | 2023-10-10 | 湖北芯擎科技有限公司 | Data interaction method and device based on virtual camera, electronic equipment and medium |
| CN115185711A (en) * | 2022-09-06 | 2022-10-14 | 湖北芯擎科技有限公司 | Data interaction method and device based on virtual camera, electronic equipment and medium |
| CN116708867B (en) * | 2023-08-07 | 2023-11-10 | 深圳市亿晟科技有限公司 | Live broadcast data processing method, device, equipment and storage medium |
| CN116708867A (en) * | 2023-08-07 | 2023-09-05 | 深圳市亿晟科技有限公司 | Live broadcast data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112804459A (en) | Image display method and device based on virtual camera, storage medium and electronic equipment | |
| CN106611435B (en) | Animation processing method and device | |
| CN107770626B (en) | Video material processing method, video synthesizing device and storage medium | |
| US6763175B1 (en) | Flexible video editing architecture with software video effect filter components | |
| US10021302B2 (en) | Video recording method and device | |
| JP6120169B2 (en) | Image editing device | |
| US20220007083A1 (en) | Method and stream-pushing client for processing live stream in webrtc | |
| US8306399B1 (en) | Real-time video editing architecture | |
| JP2018117385A (en) | Displaying simulated media content item enhancements on mobile devices | |
| US6763176B1 (en) | Method and apparatus for real-time video editing using a graphics processor | |
| KR100889367B1 (en) | System and Method for Realizing Vertual Studio via Network | |
| KR20180036153A (en) | Video editing system and method | |
| CN108986009A (en) | Generation method, device and the electronic equipment of picture | |
| CN112804460A (en) | Image processing method and device based on virtual camera, storage medium and electronic equipment | |
| KR101663085B1 (en) | System for producing video card in a portable smart device using alpha channel image composition based on video and method therefor | |
| CN103312981A (en) | Synthetic multi-picture taking method and shooting device | |
| CN112153472A (en) | Method and device for generating special picture effect, storage medium and electronic equipment | |
| RU105102U1 (en) | AUTOMATED SYSTEM FOR CREATING, PROCESSING AND INSTALLING VIDEOS | |
| JP4321751B2 (en) | Drawing processing apparatus, drawing processing method, drawing processing program, and electronic conference system including the same | |
| KR20100062822A (en) | System for cooperative digital image production | |
| KR20180054537A (en) | Video editing system and method | |
| CN111367598B (en) | Method and device for processing action instruction, electronic equipment and computer readable storage medium | |
| KR20180117826A (en) | Method and apparatus for production of webtoon movies | |
| TWI802204B (en) | Methods and systems for derived immersive tracks | |
| JP6108723B2 (en) | Information processing apparatus, information processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210514 |