CN115484486A - Video presentation method, electronic device, computer storage medium, and program product - Google Patents

Video presentation method, electronic device, computer storage medium, and program product Download PDF

Info

Publication number
CN115484486A
CN115484486A CN202110837120.7A CN202110837120A CN115484486A CN 115484486 A CN115484486 A CN 115484486A CN 202110837120 A CN202110837120 A CN 202110837120A CN 115484486 A CN115484486 A CN 115484486A
Authority
CN
China
Prior art keywords
video
frame
client device
shooting
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110837120.7A
Other languages
Chinese (zh)
Inventor
郑建明
王金波
侯哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to PCT/CN2022/084913 priority Critical patent/WO2022252797A1/en
Publication of CN115484486A publication Critical patent/CN115484486A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/40Scaling the whole image or part thereof
    • G06T3/4038Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/32Indexing scheme for image data processing or generation, in general involving image mosaicing

Abstract

Embodiments of the present disclosure provide a video presentation method, an electronic device, a computer storage medium, and a program product. The method comprises the following steps: the client equipment receives a composite video from the central equipment, wherein the ith frame of the composite video is obtained based on the ith frame of the video shot by a plurality of shooting equipment in the distributed shooting system at the same moment, and i is any positive integer; the client device determines a video to be presented based on the composite video, the video to be presented being associated with at least one of the plurality of photographing devices; and the client device presents the video to be presented. In this way, the client device may determine the video to be presented for presentation based on the composite video from the central device, such that the presentation at the client device is more flexible and diverse, thereby enhancing user experience.

Description

Video presentation method, electronic device, computer storage medium, and program product
Technical Field
Embodiments of the present disclosure relate to the field of multimedia processing, and more particularly, to a video presentation method, an electronic device, a computer storage medium, and a program product.
Background
The application scenes of collaborative interconnection among a plurality of different devices are increasing, for example, in a scene where a plurality of shooting devices are collaborative, multi-channel on-screen display, multi-channel image real-time splicing and the like can be realized, so that different scene experiences with a single device are brought to a user. Such a scene in which a plurality of photographing apparatuses constitute a photographing array may also be referred to as a distributed photographing system.
Images shot by the distributed shooting system can be presented on a client, but the current presentation mode is single, so that the user experience is poor.
Disclosure of Invention
Embodiments of the present disclosure provide a solution for presenting videos to be presented based on composite videos from a central device on a client device.
In a first aspect, a method of video presentation is provided. The method comprises the following steps: the client equipment receives a composite video from the central equipment, wherein the ith frame of the composite video is obtained based on the ith frame of the video shot by a plurality of shooting equipment in the distributed shooting system at the same moment, and i is any positive integer; the client device determines a video to be presented based on the composite video, the video to be presented being associated with at least one of the plurality of photographing devices; and the client device presents the video to be presented.
In this way, the client device may determine the video to be presented for presentation based on the composite video from the central device, such that the video to be presented is no longer passively received but is determined by the client device, such that the presentation at the client device is more flexible and varied, thereby enhancing user experience.
In some embodiments of the first aspect, prior to the client device receiving the composite video from the hub device, further comprising: the center device is connected with the plurality of shooting devices respectively. In some embodiments, the center device establishes a wireless connection with each of the plurality of photographing devices, respectively, and the center device and the plurality of photographing devices are in the same local area network environment.
In some embodiments of the first aspect, the client device determining a video to present based on the composite video comprises determining each frame of the video to present by: the client device determines an ith frame of a video shot by a target shooting device from the ith frame of the composite video, wherein the target shooting device is a shooting device located at a target position in the plurality of shooting devices; and the client equipment determines the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
In this way, the client device can determine and present the video shot by the target shooting device based on the composite video, which can simplify user operations.
In some embodiments of the first aspect, the client device determining the video to present based on the composite video comprises: the client equipment receives a user input instruction, and the user input instruction indicates target shooting equipment; the client device determines each frame of the video to be presented by: determining an ith frame of a video shot by a target shooting device from the ith frame of the composite video; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
Therefore, the client device can determine and present the video shot by the target shooting device corresponding to the user input instruction based on the composite video based on the user input instruction, so that the user can view interested videos as required, the presentation at the shooting device can be more diversified, and the user experience is improved.
In some embodiments of the first aspect, further comprising: the method comprises the steps that client equipment receives a look-around viewing operation of a user for a current frame of a video to be presented; and in response to the look-around viewing operation, the client device presents a sequence of look-around images corresponding to a current frame of the video to be presented.
Therefore, the client device can present the all-round-view image sequence based on the all-round-view viewing operation of the user, so that the user can view the all-round-view effect more intuitively, and the diversified presentation mode can improve the user experience.
In some embodiments of the first aspect, the client device presenting the sequence of around-the-eye images comprises: the client equipment responds to the look-around viewing operation and determines a frame corresponding to a current frame of the video to be presented from the composite video; the client device splits a frame of the determined composite video, which corresponds to a current frame of the video to be presented, into a plurality of images corresponding to the plurality of shooting devices respectively; obtaining a sequence of surround view images based on a plurality of images; and the client device presents the sequence of surround view images.
In this way, the panoramic image sequence is obtained by a plurality of images shot by a plurality of shooting devices, and each shooting device in the distributed shooting system can be fully utilized to realize the maximization of resource utilization.
In some embodiments of the first aspect, the number of the plurality of images is equal to the number of the plurality of photographing devices.
In some embodiments of the first aspect, the client device obtaining the sequence of all-around images based on the plurality of images comprises: the client device arranges the plurality of images in order of positions of the plurality of photographing devices to obtain a panoramic image sequence.
Therefore, the all-around view image sequence is obtained according to the positions of the plurality of shooting devices, the presenting effect of the all-around view image sequence can be ensured, and errors are avoided.
In some embodiments of the first aspect, the client device obtaining the sequence of around looking images based on the plurality of images comprises: the client device arranges the plurality of images according to the position sequence of the plurality of shooting devices; and the client device inserts an intermediate frame between each two adjacent images of the plurality of images through the frame insertion operation to obtain the all-around image sequence.
Therefore, by inserting frames between adjacent images, the continuity of looking around can be determined, image jumping is avoided, the continuity of the effect viewed by a user is ensured, and the user experience is improved.
In some embodiments of the first aspect, the ith frame of the composite video is obtained by stitching ith frames respectively captured by a plurality of capturing devices at the same time.
In a second aspect, a method of video presentation is provided. The method comprises the following steps: the method comprises the steps that a central device receives videos shot by a plurality of shooting devices in a distributed shooting system; the central equipment obtains a composite video based on videos shot by the plurality of shooting equipment respectively, wherein the ith frame of the composite video is obtained based on the ith frame of the videos shot by the plurality of shooting equipment respectively at the same moment, and i is any positive integer; and the central device sends the composite video to the client device.
In some embodiments of the second aspect, the central device obtaining the composite video includes determining each frame of the composite video by: the central equipment splices the ith frames of the videos shot by the shooting equipment at the same moment respectively to obtain the ith frame of the composite video.
In some embodiments of the second aspect, further comprising: the central device presents video captured by a particular capture device in the distributed capture system.
In a third aspect, an apparatus for video presentation is provided. The device includes: a receiving module configured to receive a composite video from a center device, an ith frame of the composite video being obtained based on the ith frame of a video respectively captured by a plurality of capturing devices at the same time in a distributed capturing system, i being any positive integer; a determination module configured to determine a video to present based on the composite video, the video to present being associated with at least one of the plurality of capture devices; and a presentation module configured to present the video to be presented.
In some embodiments of the third aspect, the determination module is configured to determine each frame of the video to be presented by: determining an ith frame of a video shot by a target shooting device from the ith frame of the composite video, wherein the target shooting device is a shooting device located at a target position in the plurality of shooting devices; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
In some embodiments of the third aspect, the receiving module is further configured to receive a user input instruction, the user input instruction being indicative of the target capturing device. The determination module is configured to determine each frame of the video to be presented by: determining an ith frame of a video shot by a target shooting device from the ith frame of the composite video; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
In some embodiments of the third aspect, the receiving module is further configured to receive a look-around viewing operation by a user for a current frame of the video to be presented; and the presentation module is further configured to present a sequence of look-around images corresponding to a current frame of the video to be presented in response to the look-around viewing operation.
In some embodiments of the third aspect, the determination module is configured to: responding to the look-around viewing operation, and determining a frame corresponding to a current frame of the video to be presented from the composite video; splitting a frame corresponding to a current frame of a video to be presented in the determined composite video into a plurality of images respectively corresponding to a plurality of shooting devices; and obtaining a sequence of surround view images based on the plurality of images.
In some embodiments of the third aspect, the number of the plurality of images is equal to the number of the plurality of photographing devices.
In some embodiments of the third aspect, the determination module is configured to: the plurality of images are arranged in order of positions of the plurality of photographing apparatuses to obtain a panoramic image sequence.
In some embodiments of the third aspect, the determination module is configured to: arranging a plurality of images in the order of positions of a plurality of photographing apparatuses; and inserting an intermediate frame between every two adjacent images of the plurality of images through an interpolation operation to obtain a panoramic image sequence.
In some embodiments of the third aspect, the ith frame of the composite video is obtained by stitching ith frames respectively captured by a plurality of capturing devices at the same time.
In a fourth aspect, an apparatus for video presentation is provided. The device includes: a receiving module configured to receive videos captured by a plurality of capturing devices in a distributed capturing system; a determination module configured to obtain a composite video based on videos captured by the plurality of capturing devices, respectively, an ith frame of the composite video being obtained based on the ith frame of the videos captured by the plurality of capturing devices at the same time, i being any positive integer; and a sending module configured to send the composite video to the client device.
In some embodiments of the fourth aspect, the determination module is configured to determine each frame of the composite video by: splicing the ith frames of the videos respectively shot by the shooting devices at the same moment to obtain the ith frame of the composite video.
In some embodiments of the fourth aspect, further comprising a presentation module configured to present video captured by a particular capture device in the distributed capture system.
In a fifth aspect, an electronic device is provided. The electronic device includes a transceiver, a processor, and a memory having instructions stored thereon that, when executed by the processor, cause the electronic device to: receiving a composite video from a center device via a transceiver, an ith frame of the composite video being obtained based on the ith frame of a video respectively captured by a plurality of capturing devices at the same time in a distributed capturing system, i being any positive integer; determining a video to be presented based on the composite video, the video to be presented being associated with at least one of the plurality of capture devices; and presenting the video to be presented.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to enable determination of each frame of the video to be presented by: determining an ith frame of a video photographed by a target photographing apparatus from among the ith frames of the composite video, the target photographing apparatus being a photographing apparatus located at a target position among the plurality of photographing apparatuses; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to: receiving, via a transceiver, a user input instruction, the user input instruction indicating a target capture device; and determining each frame of the video to be presented by: determining an ith frame of a video shot by a target shooting device from the ith frame of the composite video; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to: receiving a look-around viewing operation of a user for a current frame of a video to be presented; and presenting a look-around image sequence corresponding to the current frame of the video to be presented in response to the look-around viewing operation.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to: responding to the look-around viewing operation, and determining a frame corresponding to a current frame of the video to be presented from the composite video; splitting a frame corresponding to a current frame of a video to be presented in the determined composite video into a plurality of images respectively corresponding to a plurality of shooting devices; obtaining a sequence of surround view images based on a plurality of images; and presenting the sequence of around looking images.
In some embodiments of the fifth aspect, the number of the plurality of images is equal to the number of the plurality of photographing apparatuses.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to: the plurality of images are arranged in order of positions of the plurality of photographing apparatuses to obtain a panoramic image sequence.
In some embodiments of the fifth aspect, execution of the instructions by the processor causes the electronic device to: arranging a plurality of images in the order of positions of a plurality of photographing apparatuses; and inserting an intermediate frame between every two adjacent images of the plurality of images through an interpolation operation to obtain a panoramic image sequence.
In some embodiments of the fifth aspect, the ith frame of the composite video is obtained by stitching ith frames of videos respectively captured by a plurality of capturing devices at the same time.
In some embodiments of the fifth aspect, the electronic device includes a display screen for presenting the video to be presented or the sequence of around looking images.
In a sixth aspect, an electronic device is provided. The electronic device includes a transceiver, a processor, and a memory having stored thereon instructions for execution by the processor, which when executed by the processor, cause the electronic device to implement: receiving videos shot by a plurality of shooting devices in a distributed shooting system respectively through a transceiver; obtaining a composite video based on videos shot by a plurality of shooting devices respectively, wherein the ith frame of the composite video is obtained based on the ith frame of the videos shot by the plurality of shooting devices respectively at the same moment, and i is any positive integer; and transmitting the composite video to the client device via the transceiver.
In some embodiments of the sixth aspect, execution of the instructions by the processor causes the electronic device to determine each frame of the composite video by: splicing the ith frames of the videos respectively shot by the shooting devices at the same moment to obtain the ith frame of the composite video.
In some embodiments of the sixth aspect, execution of the instructions by the processor causes the electronic device to: video captured by a particular capture device in a distributed capture system is presented.
In some embodiments of the sixth aspect, the electronic device comprises a camera.
In a seventh aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the operations of the method according to the first or second aspect or any implementation thereof.
In an eighth aspect, a chip or chip system is provided. The chip or chip system comprises processing circuitry configured to perform operations of the method according to the first or second aspect described above or any implementation thereof.
In a ninth aspect, a computer program or computer program product is provided. The computer program or computer program product is tangibly stored on a computer-readable medium and includes computer-executable instructions that, when run on a computer, cause the computer to perform operations of the method according to the first or second aspect described above or any implementation thereof.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:
FIG. 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;
FIG. 2 shows one schematic diagram of an example scenario in which embodiments of the present disclosure may be implemented;
FIG. 3 shows a schematic interaction diagram of a video presentation process, in accordance with some embodiments of the present disclosure;
FIG. 4 shows a schematic flow diagram of a process of determining a composite video in accordance with some embodiments of the present disclosure;
FIG. 5 illustrates a schematic diagram of a determination of composite video according to some embodiments of the present disclosure;
FIG. 6 shows a schematic flow chart diagram of a process of presenting a sequence of surround view images, in accordance with some embodiments of the present disclosure;
fig. 7 illustrates a schematic diagram of an interpolation frame, according to some embodiments of the present disclosure;
FIG. 8 shows a schematic flow chart diagram of a video presentation process in accordance with some embodiments of the present disclosure;
FIG. 9 shows a schematic flow diagram of a video presentation process in accordance with some embodiments of the present disclosure;
fig. 10 shows a schematic block diagram of an apparatus for video presentation in accordance with some embodiments of the present disclosure;
fig. 11 illustrates another schematic block diagram of an apparatus for video presentation in accordance with some embodiments of the present disclosure; and
FIG. 12 is a schematic block diagram of an example device that may be used to implement embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
In describing embodiments of the present disclosure, the terms "include" and its derivatives should be interpreted as being inclusive, i.e., "including but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first," "second," and the like may refer to different or the same objects. Other explicit and implicit definitions are also possible below.
The distributed shooting system may include a shooting array composed of at least two shooting devices, the shooting devices may shoot images or videos, the shooting devices may be referred to as image capturing devices, and the like, and the distributed shooting system may also be referred to as a distributed camera system or a distributed shooting array or a distributed image capturing system, and the like. It is to be understood that the "image" in the embodiment of the present disclosure may be an image captured by a capturing device or may be a frame of a video captured by an image capturing device. In addition, video may also be referred to as an image stream, a frame stream, a video stream, a media stream, and so on, and the disclosure is not limited thereto.
At least two shooting devices in the distributed shooting system can shoot simultaneously to acquire more visual information, and then multi-path images can be displayed on the same screen, spliced in real time and the like through mutual cooperation. The distributed photographing system may be implemented in various different scenarios. For example, a distributed camera system may be implemented as a look-around camera array, where multiple cameras may be arranged at certain angles and intervals around a target object, and each camera is responsible for taking a picture of the target object in a certain field of view, so that when images taken by the respective cameras are played in sequence, the target object is viewed from different angles along an arc in one direction as if the human eye were centered on the target object.
The distributed shooting system is also applied to live webcasting. In a live network scene, multiple shooting devices of the distributed shooting system can respectively shoot, and the anchor can select videos to be presented so that customers can view the videos through the client devices. In this process, the content viewed by the patrons is determined by the anchor, i.e., the client devices are only passive recipients of the video, resulting in all of the patrons viewing the live content from the respective different clients to be identical. Even if the anchor does not switch the shooting devices, the video shot by other shooting devices in the distributed shooting system is wasted, and the videos shot by a plurality of shooting devices cannot be fully utilized. Thus, the experience and feeling of watching the live broadcast are seriously influenced.
In view of this, the present disclosure provides a video presentation scheme, where a client device may receive a composite video obtained from videos shot by multiple shooting devices, and perform local presentation based on the composite video, so that different requirements of different users for presentation can be met, and user experience is improved.
FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented. In the example environment 100, a central device 110, a distributed camera system 120, a client device 130-1, a client device 130-2, \ 8230, and a client device 130-N are shown. Distributed camera system 120 includes multiple cameras, such as camera 122-1, camera 122-2, and camera 122-3 in FIG. 1.
For convenience of description below, the photographing device 122-1, the photographing device 122-2, and the photographing device 122-3 are collectively referred to as the photographing device 122 in the embodiments of the present disclosure, and the client device 130-1, the client device 130-2, \ 8230, and the client device 130-N are collectively referred to as the client device 130 in the embodiments of the present disclosure. And it is understood that although it is illustrated in fig. 1 that the distributed photographing system 120 includes three photographing apparatuses, in an actual application, the number of photographing apparatuses included in the distributed photographing system 120 may be set according to a scene or the like. The photographing device 122 may be a stand-alone device or may be a peripheral of other electronic devices, for example, the photographing device 122 may be implemented as an electronic device having an image capturing function, and the like. The capture device 122 may include a camera, a camcorder, a snap-shot device, a cell phone, a tablet, a wearable device, etc., to which the present disclosure is not limited.
As shown in fig. 1, the center device 110 may interact with the client devices 130, and in a scenario such as webcast, the center device 110 may be an electronic device that interacts with a host and the client devices 130 may be electronic devices that interact with a user. That is, the center device 110 has a communication connection with the client device 130.
In some embodiments, the hub device 110 may interact with the client devices 130 via a server (e.g., a streaming media server). In some embodiments, the central device 110 and the client devices 130 may be implemented as electronic devices such as smartphones, tablets, wearable devices, and the like. Embodiments of the present disclosure do not limit the number (N) of client devices, for example in a live webcast scenario, the number of client devices may be on the order of hundreds, tens of thousands, or even greater.
As shown in FIG. 1, central facility 110 may interact with distributed camera system 120, for example, images or videos taken by camera 122-1, camera 122-2, and camera 122-3, respectively, may be transmitted to central facility 110.
The center device 110 and the shooting device 122 may have a communication connection therebetween, and the embodiment of the disclosure does not limit the connection manner, for example, the connection is performed in a wired or wireless manner. Wired means may include, but are not limited to, fiber optic connections, universal Serial Bus (USB) connections, etc., and wireless means may include, but are not limited to, mobile communication technologies (including, but not limited to, 2g,3g,4g,5g,6g, etc., wi-Fi, bluetooth (Bluetooth), point-to-Point (P2P), etc.).
Taking a Wi-Fi connection as an example, the center device 110 and the photographing device 122 may be in the same local area network environment, and the center device 110 may discover the photographing device 122 located in the same local area network environment through its distributed photographing system control module (or a connection discovery module or other modules, etc.), and establish a Wi-Fi connection with the photographing device 122, for example, the center device 110 and the photographing device 122 may be connected to the same router. It should be noted that the communication connection between the center device 110 and the different photographing devices 122 may be the same or different. For example, the center apparatus 110 and the photographing apparatus 122-1 may be connected in a different manner from the center apparatus 110 and the photographing apparatus 122-2.
Although the central device 110 is shown in fig. 1 as a device separate from the distributed camera system 120, in some embodiments, the central device 110 may be implemented as part of the distributed camera system 120, e.g., the central device 110 may be an electronic device to which the camera device 122-2 corresponds.
The embodiment of the present disclosure does not limit the arrangement manner of each shooting device in the distributed shooting system 120.
In some embodiments, the photographing apparatus 122-1, the photographing apparatus 122-2, and the photographing apparatus 122-3 may be arranged side by side so that photographing directions of the target objects are parallel or substantially uniform when photographing the target objects.
In some embodiments, the photographing apparatus 122-1, the photographing apparatus 122-2, and the photographing apparatus 122-3 may be arranged around the target object so as to make an angle with respect to a photographing direction of the target object when photographing the target object. In connection with fig. 2, assume that the distributed photographing system 120 includes 7 photographing devices, respectively photographing devices 122-1 to 122-7, and an electronic device corresponding to the photographing device 122-4 is the center device 110.
In the scene 200 shown in fig. 2, 7 photographing apparatuses may be mounted on the fixed stand 201, and each of the 7 photographing apparatuses may perform image photographing on the target object 202. In a scenario such as a webcast, the target object 202 may be an item to be displayed by the anchor.
In fig. 2, the fixing bracket 201 is implemented as a ring bracket, and a plurality of fixing buckles are provided on the fixing bracket 201 for fixing a plurality of photographing apparatuses 122. And after the plurality of photographing devices 122 are respectively mounted on the fixed support 201, the position and angle of each photographing device 122 with respect to the center of the fixed support 201 (e.g., the center of a circle on which the ring is located) may be fixed. It is to be understood that although the plurality of photographing devices 122 in fig. 2 may photograph toward the center of the fixing bracket 201, that is, the target object 202 is located near the center of the fixing bracket 201. However, the embodiment of the present disclosure is not limited thereto, and for example, the plurality of photographing apparatuses 122 may photograph toward the outside of the fixed stand 201, which can enlarge the field of view to perform a panoramic live broadcast.
In some embodiments, each of the 7 capturing devices in fig. 2 may be implemented as a camera on a smart terminal, for example, in case the smart terminal is a mobile phone, 7 mobile phones may be installed at corresponding positions of the fixing bracket 201. In one embodiment, 7 photographing apparatuses are respectively mounted on the fixing brackets 201, so that photographing angles at which the 7 photographing apparatuses photograph the target object 202 are fixed, i.e., each photographing apparatus is not movable and rotatable. In some embodiments, the angle between the centerlines of each two adjacent cameras may be fixed, for example, the angle may be set to 20 ° or other value.
In the embodiment of the present disclosure, the shooting areas of two adjacent shooting devices in the distributed shooting system 120 may have partial overlap. For example, the photographing device 122-1 photographs the target object 202 to obtain a first image, the photographing device 122-2 photographs the target object 202 to obtain a second image, and the first area in the first image and the second area in the second image are for the same photographing area of the target object 202. In one example, the first area occupies 1/4 or more of the first image and the second area occupies 1/4 or more of the second image.
The embodiment of the disclosure can be applied to the scene of network live broadcast. For live webcasts, the anchor may prepare and fix the various cameras 122 in advance to form a distributed camera system. A plurality of photographing apparatuses 122 are mounted on the fixing bracket 201, for example, in a manner as shown in fig. 2. The anchor may also select the center device 110, for example, setting the electronic device corresponding to the photographing device 122-4 as the center device 110. The anchor may then create a live room through the hub device 110, for example by connecting to a server of a live platform through the hub device 110 to create a live room. The central device 110 may request a server of the live platform to obtain a stream pushing address, such as a Uniform Resource Locator (URL). The push streaming may be a process of the central device 110 pushing the audio and video stream to a server of the live platform, and the push streaming address is an address corresponding to the push streaming process, and a format and the like of the push streaming address depend on a protocol and the like used. Similarly, the client device 130 may obtain the corresponding audio/video stream from the server of the live platform through a pull stream address corresponding to the push stream address, where the pull stream may be a process in which the client device 130 pulls the audio/video stream on the server of the live platform to the local, and the pull stream address is an address corresponding to the pull stream process, and a format of the pull stream address depends on a protocol used, and the like. It will be appreciated that the number of client devices 130 connected to the central device 110 may vary with the customer's entry or exit from the live room.
It will be appreciated that the client device 130 may be caused to establish a communication connection with the central device 110 by a user entering a live room operation. In some embodiments, the hub device 110 may send the live room information to the client device 130. In some embodiments, the client device 130 may send an information request to the live platform server to obtain live room information from the central device 110.
Illustratively, the live room information may include system information of the distributed shooting system. In some embodiments, the system information of the distributed photographing system may include the number of photographing devices included in the distributed photographing system. For example, in the scenario shown in fig. 2, the number is 7. In some embodiments, the system information of the distributed photographing system may include sizes, such as width and height, of images photographed by the respective photographing apparatuses. For example, if the images captured by the respective photographing apparatuses are equal in size, w and h may be included to respectively represent the width and height of the image captured by a single photographing apparatus. In some embodiments, the system information of the distributed camera system may include an identification of the camera device associated with the central device 110. For example, in the scenario shown in fig. 2, the center device 110 is an electronic device corresponding to the camera device 122-4, and the identification of the camera device associated with the center device 110 may be 4. It is understood that the system information of the distributed photographing system may further include other information, etc., such as the resolution of the photographing apparatus, etc., which are not listed one by one here.
Illustratively, the live room information may also include a pull address for the client device 130, through which the client device 130 may retrieve the video. It will be appreciated that live room information may also include other information, etc., such as live room address, live play time, etc., which are not listed here.
Embodiments of the present disclosure will be described in more detail below with reference to fig. 3 to 7.
Fig. 3 shows a schematic interaction diagram of a video presentation process 300 according to some embodiments of the present disclosure. The process 300 shown in fig. 3 involves the hub device 110 and the client device 130.
In process 300, the center device 110 determines 310 a composite video based on a plurality of videos respectively captured by a plurality of capture devices 122 of the distributed capture system 120.
Exemplarily, the process of determining the composite video by the center device 110 may be as shown in fig. 4, and fig. 4 shows a schematic flow chart of a process 400 of determining the composite video by the center device 110.
In process 400, capture device 122 performs video capture 410. Taking 3 cameras as an example, it can be assumed that camera 122-1 captures video 1, camera 122-2 captures video 2, and camera 122-3 captures video 3.
Alternatively or additionally, the central device 110 and the capture device 122 may be time synchronized 402. The embodiments of the present disclosure do not limit the specific manner of time synchronization. In some embodiments, through time synchronization, time synchronization information between the local clock of the center apparatus 110 and the local clock of the photographing apparatus 122 may be determined. In some embodiments, after time synchronization, different capture devices 122 may take images at the same time. For example, the i-th frame photographed by the photographing device 122-1 and the i-th frame photographed by the photographing device 122-2 are simultaneously acquired.
In process 400, the capture device 122 sends 420 the captured video to the hub device 110. Taking 3 shooting devices as an example, the center device 110 can acquire video 1, video 2, and video 3.
Alternatively or additionally, the hub device 110 may pre-process 422 the video from the capture device 122. In some embodiments, the pre-processing may be for a portion of the video or the entire video. In some embodiments, the pre-processing may include, but is not limited to: beautifying, watermarking, mosaic printing and the like. As an example, watermarking may include adding all or part of the following information over part or all of the frames of the video: the name of the anchor, information of the target object, the identification of the photographing device 122, and the like.
In process 400, the hub device 110 obtains 430 a composite video. Taking 3 shooting devices as an example, the center device 110 may synthesize video 1, video 2, and video 3 to obtain a composite video.
In an embodiment of the present disclosure, the ith frame of the composite video may be obtained based on the ith frame captured by the capturing device 122. In some embodiments, the ith frame captured by each capture device 122 may be combined to obtain the ith frame of the composite video. i is any positive integer, so that the composite video can be obtained by obtaining each frame of the composite video.
In some embodiments, the ith frame captured by each of the capture devices 122 may be stitched in the order of the capture devices 122 to obtain the ith frame of the composite video. For example, assume that the ith frame captured by each of the capturing devices 122 has the same size, e.g., w width and h height. The width of the ith frame of the composite video may be equal to the sum of the widths of the ith frames captured by the respective capture devices 122, and the height of the ith frame of the composite video is higher than h. In some embodiments, the order of the capture device 122 may be: the order of photographing devices arranged in the clockwise direction with the target object 202 as a reference. With reference to fig. 2, taking the target object 202 as a central point, the clockwise sequence of the photographing apparatus 122 is: capture device 122-1, capture device 122-2, capture device 122-3, capture device 122-4, capture device 122-5, capture device 122-6, and capture device 122-7.
For simplicity of description, a process of obtaining a composite video is described taking 3 photographing devices as an example. Specifically, frames in the video 1 captured by the capture device 122-1 are denoted as f11, f12, \8230;, and f1n, frames in the video 2 captured by the capture device 122-2 are denoted as f21, f22, \8230;, and f2n, and frames in the video 3 captured by the capture device 122-3 are denoted as f31, f32, \8230;, and f3n. Then the 1 st frame of the composite video may be f11, f21, and f31 sequentially stitched, \ 8230, and the nth frame of the composite video may be f1n, f2n, and f3n sequentially stitched, as shown in fig. 5.
In other words, the center device 110 may take first frames from the plurality of videos respectively in the order of the photographing device 122 from the corresponding videos, and after finishing taking the first frames of all the videos, splice the plurality of first frames in the order of the photographing device 122 as the first frame of the composite video. Then, the center device 110 may take the second frames from the plurality of videos respectively in the order of the shooting device 122, and after finishing taking the second frames of all the videos, splice the plurality of second frames in the order of the shooting device 122 to obtain the second frame of the composite video. And circulating the steps to obtain the composite video.
It is understood that the above embodiments are only illustrative of the manner in which the composite video is obtained, and the embodiments of the present disclosure are not limited thereto. For example, the composition may be in a reverse order of the photographing apparatus 122, or may be in an order according to other predetermined rules. For example, the splicing may be performed in the width direction or in the height direction, or may be performed in another manner.
In process 300, the hub device 110 sends 320 the composite video to the client device 130.
In some embodiments, the client device 130 may retrieve the composite video via a streaming server by pulling a streaming address.
In some embodiments, the hub device 110 may encode, compress, encapsulate, etc. the composite video before transmitting to the client devices 130. Illustratively, video compression techniques such as h.264 may be employed for the encoding compression. Illustratively, the video may be packaged into a streaming media format such as FLV or TS. Therefore, the requirements for network bandwidth and the like can be reduced, the transmission rate is improved, and the real-time performance is ensured.
Accordingly, client device 130 may determine the composite video by decapsulating, decoding decompression, and the like. In some embodiments, the client device 130 may obtain the encapsulated FLV or TS formatted video data by pulling from the live platform server, and may then obtain the encoded compressed video data by parsing, etc. Further, the client device 130 may operate by decoding to restore the composite video.
In process 300, the client device 130 determines 330 a video to present, wherein the video to present is associated with at least one capture device 122. Further, the client device 130 renders 340 the video to be rendered.
In the embodiment of the present disclosure, presenting a video may refer to displaying frames of the video frame by frame, or may be understood as playing the video.
In some embodiments, the video to be presented may be a composite video. Since one frame of the composite video includes images captured by the respective capturing devices in the distributed capturing system, images taken by the plurality of capturing devices about the target object 202 can be presented at the client device 130 at the same time. In this way, the user at the client device 130 can see images of the target object 202 from various angles at the same time, which can facilitate subsequent selection by the user, such as viewing a video for which capture device.
In some embodiments, the video to be presented may be a video captured by a particular capture device in a distributed capture system. Specifically, a portion photographed by a specific photographing apparatus may be separated from each frame of the composite video, thereby determining a video to be presented. In the embodiment of the present disclosure, the specific photographing apparatus may be any one of: a photographing device located at a target position, a photographing device designated by a user of the client device 130, and the like.
In some embodiments, the target location may be the center device 110, and the video to be presented may be a video captured by a capturing device corresponding to the center device 110. Illustratively, the photographing apparatus corresponding to the center apparatus 110 may be a photographing apparatus included in the center apparatus 110.
For simplicity, the photographing apparatus corresponding to the center apparatus 110 may be referred to as a center photographing apparatus. In some embodiments, the client device 130 may separate the video to be rendered corresponding to the central capture device from the composite video based on the number of multiple capture devices in the distributed capture system (assumed to be M) and the identification of the central capture device (assumed to be p).
For example, if the composite video is determined in a manner similar to stitching as shown in FIG. 5. Then, for any frame of the composite video (i-th frame is assumed), a portion having a width of [ (p-1) × w, p × w ] may be cut out from the i-th frame of the composite video as the i-th frame of the video to be presented. For another example, the ith frame of the composite video may be split into M images, that is, the images captured by the M capturing devices are recovered, and then the image captured by the central capturing device is determined from the M images.
Since the hub device 110 is typically an electronic device operated by the anchor, in this way, the rendered video at the client device 130 can be made consistent with the content on the hub device 110 that the anchor interacts with. Particularly when the anchor makes a voice introduction to the target object 202, the user can be enabled to view the specific details introduced by the anchor in time.
In some embodiments, the target location may be an intermediate location of the plurality of capture devices, and the video to be presented may be a video captured by a capture device located at the intermediate location in the distributed capture system. Assume that the number of a plurality of photographing apparatuses in the distributed photographing system is M. Then if M is an odd number, the number of the photographing apparatus located at the middle position is (M + 1)/2. If M is even, the number of the photographing apparatus located at the middle position may be M/2 or M/2+1.
In this way, the image of the front side of the target object 202 can be viewed by the user at the client device 130, enabling the user to view more details of the target object 202.
In some embodiments, the video to be presented may be a video captured by a user-specified capture device. Specifically, the client device 130 may receive an input instruction of the user, which may indicate which photographing device of the plurality of photographing devices.
For example, the user may input a photographing apparatus number, such as "2", so that the client apparatus 130 can acquire the input instruction. For example, during the process that the client device 130 is playing a video shot by a certain shooting device (assuming the number n 1), the user can determine the shooting device number by sliding left or right. For example, a left slide indicates that the photographing apparatus number is reduced by one, i.e., the photographing apparatus number is designated as n1-1. For example, a right slide indicates that the photographing apparatus number is increased by one, i.e., the photographing apparatus number is designated as n1+1.
Thus, in the embodiment of the present disclosure, the user at the client device 130 can determine which shooting device video is to be presented by inputting an instruction, so that switching of videos presented on the client device 130 is realized, the autonomy of the user is improved, and the requirements of the customer can be met better. In this way, the video presented on the client device 130 need not be identical to the video on the anchor electronic device and is no longer a passive video recipient, but rather, the video of interest can be selected autonomously by the user without affecting the anchor electronic device and other client devices 130.
In addition, the video presented on the center device 110 is not limited by the embodiment of the present disclosure, and may be, for example, a video shot by the center shooting device.
Alternatively or additionally, the client device 130 may receive 350 a look-around viewing operation by the user. Further, the client device 130 may present 360 a sequence of panoptic images.
In some embodiments, the user may click on a particular area on the interface of the client device 130 to perform a look-around viewing operation, such as clicking on a "look-around" button on the particular area. In some embodiments, the user may manipulate a particular gesture on the interface of the client device 130 to perform a look-around viewing operation, for example, the particular gesture is a circled or semi-arc.
Fig. 6 illustrates a schematic flow diagram of a process 600 of presenting a sequence of surround view images, in accordance with some embodiments of the present disclosure.
At block 610, the client device 130 determines a current frame of the composite video that corresponds to a current frame of the video to be presented in response to the look-around viewing operation.
Specifically, assuming that the current frame of the video to be presented is the tth frame, the tth frame of the composite video may be acquired, and it is understood that the tth frame of the composite video includes images captured by each of the capturing devices in the distributed capturing system.
At block 620, the client device 130 splits the current frame of the composite video into a plurality of images.
Specifically, the splitting may be performed based on the number of the plurality of photographing devices, that is, the number of the plurality of images is equal to the number of the plurality of photographing devices. In some embodiments, the plurality of images are images respectively captured by a plurality of capturing devices.
At block 630, the client device 130 obtains a sequence of panoramic images based on the plurality of images.
In some embodiments, the plurality of images may be ordered according to the position of the capture device to obtain a sequence of all-round images. That is, a plurality of images may be sequentially arranged in order of positions of a plurality of photographing apparatuses to obtain a panoramic image sequence. For example, as shown in fig. 2, the image captured by the capturing device 122-i is located at the ith position of the all-round image sequence, and i is any value from 1 to 7.
In some embodiments, the plurality of images may be sequentially arranged in the positional order of the plurality of photographing devices with at least one frame interposed between each two adjacent images to form a panoramic image sequence. It is understood that, in this embodiment, the number of images in the image sequence is viewed around by the number of the plurality of photographing apparatuses.
Specifically, at least one frame may be inserted between two adjacent images by the frame insertion operation. The inserted at least one frame may be referred to as a virtual frame or an intermediate frame, and embodiments of the present disclosure do not limit the manner in which the frame is inserted. The interpolated frame may also be called a complement frame or an animation complement frame, and a virtual frame may be obtained by an algorithm such as local interpolation. In this way, by the frame interpolation processing, a virtual frame is interpolated between two adjacent images, and continuity of image change between the two adjacent images can be ensured.
For example, assume that the current frame is the t-th frame, the number of the plurality of photographing devices is m, and the plurality of images are images respectively photographed by the plurality of photographing devices, denoted as f1t, f2t, f3t, \8230;, fmt. As shown in fig. 7, 4 virtual frames may be inserted between each two images, resulting in a panoramic image sequence 700. Thus, the sequence of surround-view images obtained after frame interpolation includes the number of images: m +4 × (m-1).
In this way, the number of images in the sequence of surround view images can be expanded, making it more fluent when the sequence of surround view images is presented later.
It should be noted that the number of virtual frames inserted in the frame insertion process is not limited in the embodiment of the present disclosure. In some embodiments, the number of dummy frames inserted between each two adjacent frames may be a predetermined value, for example, the predetermined value is 4 in fig. 7, it is understood that the predetermined value may be other values. The preset value may be preset according to an angle between two adjacent photographing apparatuses, the number of photographing apparatuses, and the like. In some embodiments, the number of virtual frames inserted between two different adjacent frames may be equal or unequal, for example, the virtual frames inserted between f1t and f2t have a first number, while the virtual frames inserted between f2t and f3t have a second number, and the first number may be equal to or not equal to the second number.
At block 640, the client device 130 sequentially presents each image in the sequence of images in a round-robin order.
In some embodiments, the client device 130 may also present the corresponding image in the sequence of all-around images based on a side-to-side sliding operation by the user. In this way, the user can view the look-around effect of the target object 202 according to the needs of the user.
Therefore, the embodiment of the disclosure provides a real-time look-around live broadcast scheme based on a distributed shooting system, and the client device can receive the composite video from the central device, and then the client device can present the video to be presented or the look-around effect according to actual needs or user instructions and the like. In this way, a user at a client device can autonomously determine the content being presented, the client device no longer being a single passive recipient of the content, which can greatly enhance the user's interaction experience.
Fig. 8 shows a schematic flow diagram of a video presentation process 800 according to some embodiments of the present disclosure. Process 800 may be performed by client device 130 as shown in fig. 1.
At block 810, the client device 130 receives a composite video from the central device, the ith frame of the composite video being obtained based on the ith frame of the video captured by each of the plurality of capture devices at the same time in the distributed capture system, i being any positive integer. At block 820, the client device 130 determines a video to present based on the composite video, the video to present being associated with at least one of the plurality of capture devices. At block 830, the client device 130 renders the video to be rendered.
In some embodiments, the ith frame of the composite video is obtained by stitching ith frames respectively captured by a plurality of capturing devices at the same time. For example, at the same time, the multiple shooting devices respectively shoot the multiple corresponding frames, and then the central device may splice the multiple corresponding frames, and use the spliced frames as the frames corresponding to the time in the composite video.
In some embodiments, determining, by client device 130, a video to present based on the composite video may include determining each frame of the video to present by: the client device 130 determines an ith frame of the video photographed by a target photographing device, which is a photographing device located at a target position among the plurality of photographing devices, from the ith frame of the composite video; and determining the ith frame of the video to be presented according to the ith frame of the video shot by the target shooting equipment. Alternatively, the target position may be an intermediate position of the plurality of photographing apparatuses.
In some embodiments, determining, by client device 130, the video to present based on the composite video comprises: the client device 130 receives a user input instruction, the user input instruction indicating a target shooting device; and determining each frame of the video to be presented by: determining an ith frame of a video photographed by a target photographing apparatus from the ith frame of the composite video; and determining the ith frame of the video to be presented according to the ith frame of the video shot by the target shooting equipment. Thus, by sequentially taking the value of i as 1, 2 and 8230, each frame of the video to be presented can be obtained frame by frame. Alternatively, the input instruction of the user may be based on a slide operation of the user on the interface, which may be a left slide or a right slide to indicate that the target photographing apparatus is determined by a left shift or a right shift in position, respectively.
For example, for any frame in the composite video, such as the ith frame, the ith frame of the video captured by the target capturing device may be determined from the ith frame of the composite video as the ith frame of the video to be presented. Generally, the image size of the ith frame of the video to be presented is smaller than the image size of the ith frame of the composite video.
Alternatively or additionally, as shown in fig. 8, at block 840, the client device 130 receives a user look-around viewing operation for a current frame of the video to be rendered. At block 850, the client device 130 presents a sequence of look-around images corresponding to the current frame of the video to be presented in response to the look-around viewing operation.
In some embodiments, presenting the sequence of look-around images by the client device 130 in response to the look-around viewing operation may include: the client device 130 determines a frame from the composite video corresponding to a current frame of the video to be presented in response to the look-around viewing operation; splitting a frame corresponding to a current frame of a video to be presented in the determined composite video into a plurality of images respectively corresponding to a plurality of shooting devices; obtaining a sequence of look-around images based on a plurality of images; and presenting the sequence of surround view images. Illustratively, the number of the plurality of images is equal to the number of the plurality of photographing devices.
In some embodiments, obtaining the sequence of around images by the client device 130 based on the plurality of images may include: the client device 130 arranges the plurality of images in order of positions of the plurality of photographing devices to obtain a panoramic image sequence.
In some embodiments, the client device 130 obtaining the sequence of around images based on the plurality of images comprises: the client device 130 arranges the plurality of images in the order of the positions of the plurality of photographing devices; an intermediate frame is inserted between every two adjacent images of the plurality of images through an interpolation operation to obtain a sequence of the all-around images.
Fig. 9 shows a schematic flow diagram of a video presentation process 900 according to some embodiments of the present disclosure. Process 900 may be performed by central device 110 as shown in fig. 1.
At block 910, the hub device 110 receives a video captured by each of a plurality of capture devices in a distributed capture system. At block 920, the center apparatus 110 obtains a composite video based on videos captured by the plurality of capturing apparatuses, respectively, an i-th frame of the composite video being obtained based on the i-th frame of the videos captured by the plurality of capturing apparatuses, respectively, at the same time, i being any positive integer. At block 930, the hub device 110 sends the composite video to the client device.
In some embodiments, the central device 110 obtaining the composite video may include determining each frame of the composite video by: the center apparatus 110 stitches the i-th frame of the video individually photographed by the plurality of photographing apparatuses at the same time to obtain the i-th frame of the composite video. For example, at the same time, the multiple shooting devices respectively shoot the multiple corresponding frames, and then the central device 110 may splice the multiple corresponding frames, and use the spliced frames as the frames corresponding to the time in the composite video.
In some embodiments, it may further include: videos captured by a particular capture device in the distributed capture system are presented at the central device 110. For example, after acquiring videos captured by a plurality of capturing devices in the distributed capturing system, the center device 110 presents the videos captured by the target capturing device. For example, the target photographing device may be a device that interacts with a user in a distributed photographing system, or the target photographing device may be a device located at an intermediate position in the distributed photographing system.
Fig. 10 shows a schematic block diagram of an apparatus 1000 for video presentation according to some embodiments of the present disclosure. The apparatus 1000 may be implemented as or included in the client device 130 of fig. 1.
Apparatus 1000 may include a number of modules for performing corresponding steps in process 800 as discussed in fig. 8. As shown in fig. 10, the apparatus 1000 includes a receiving module 1010, a determining module 1020, and a presenting module 1030. The receiving module 1010 is configured to receive a composite video from a center device, an ith frame of the composite video being obtained based on ith frames of videos respectively captured by a plurality of capturing devices at the same time in a distributed capturing system, and i being any positive integer. The determination module 1020 is configured to determine a video to present based on the composite video, the video to present being associated with at least one of the plurality of cameras. The presentation module 1030 is configured to present the video to be presented.
In some embodiments, the ith frame of the composite video is obtained by splicing the ith frames respectively captured by a plurality of capturing devices at the same time.
In some embodiments, the determination module 1020 may be configured to determine each frame of the video to be presented by: determining an ith frame of a video shot by a target shooting device from the ith frame of the composite video, wherein the target shooting device is a shooting device located at a target position in the plurality of shooting devices; and determining the ith frame of the video to be presented according to the ith frame of the video shot by the target shooting equipment.
In some embodiments, the receiving module 1010 may be further configured to receive a user input instruction, the user input instruction indicating a target capturing device. The determination module 1020 may be configured to determine each frame of the video to be presented by: determining an ith frame of a video photographed by a target photographing apparatus from the ith frame of the composite video; and determining the ith frame of the video to be presented according to the ith frame of the video shot by the target shooting equipment.
In some embodiments, the receiving module 1010 may be further configured to receive a user look-around viewing operation for a current frame of a video to be presented. The presentation module 1030 may also be configured to present a sequence of look-around images corresponding to a current frame of the video to be presented in response to a look-around viewing operation.
In some embodiments, the determination module 1020 may be configured to: responding to the look-around viewing operation, and determining a frame corresponding to a current frame of the video to be presented from the composite video; splitting the determined frame of the composite video into a plurality of images respectively corresponding to a plurality of shooting devices; and obtaining a sequence of all-round images based on the plurality of images.
In some embodiments, the number of the plurality of images is equal to the number of the plurality of photographing apparatuses.
In some embodiments, the determination module 1020 may be configured to: the plurality of images are arranged in order of positions of the plurality of photographing apparatuses to obtain a panoramic image sequence.
In some embodiments, the determination module 1020 may be configured to: arranging a plurality of images in the order of positions of a plurality of photographing apparatuses; and inserting an intermediate frame between every two adjacent images of the plurality of images through an interpolation operation to obtain a panoramic image sequence. The number of images in the sequence of surround view images is greater than the number of the plurality of capture devices.
Illustratively, the apparatus 1000 in fig. 10 may be implemented as the client device 130, or may be implemented as a chip or a chip system in the client device 130, to which embodiments of the present disclosure are not limited. The apparatus 1000 in fig. 10 can be used to implement the above-described processes in connection with the client device 130 in fig. 3 to 9, and for brevity, the description is omitted here.
Fig. 11 illustrates another schematic block diagram of an apparatus 1100 for video presentation in accordance with some embodiments of the present disclosure. The apparatus 1100 may be implemented as or included in the central device 110 of fig. 1.
The apparatus 1100 may include a number of modules for performing corresponding steps in the process 900 as discussed in fig. 9. As shown in fig. 11, the apparatus 1100 includes a receiving module 1110, a determining module 1120, and a transmitting module 1130. The receiving module 1110 is configured to receive videos captured by a plurality of capturing devices in a distributed capturing system, respectively. The determining module 1120 is configured to obtain a composite video based on videos captured by the plurality of capturing devices, respectively, an ith frame of the composite video being obtained based on an ith frame of videos captured by the plurality of capturing devices, respectively, at the same time, and i being any positive integer. The sending module 1130 is configured to send the composite video to the client device.
In some embodiments, the determining module 1120 may be configured to determine that each frame of the composite video "stitches an ith frame of the video respectively captured by the multiple capturing devices at the same time to obtain an ith frame of the composite video by the following process.
In some embodiments, the apparatus 1100 may further include a presentation module configured to present video captured by a particular capture device in the distributed capture system.
Illustratively, the apparatus 1100 in fig. 11 may be implemented as the central device 110, or may be implemented as a chip or a chip system in the central device 110, to which embodiments of the present disclosure are not limited. The apparatus 1100 in fig. 11 can be used to implement the above-described processes in conjunction with the central device 110 in fig. 3 to fig. 9, and for brevity, the description is omitted here.
Fig. 12 shows a schematic block diagram of an example device 1200, which may be used to implement embodiments of the present disclosure. The device 1200 may be implemented as or included in the client device 130 of fig. 1, or the device 1200 may be implemented as or included in the hub device 110 of fig. 1.
As shown, the device 1200 includes a Central Processing Unit (CPU) 1201, a Read-Only Memory (ROM) 1202, and a Random Access Memory (RAM) 1203. The CPU 1201 can perform various appropriate actions and processes in accordance with computer program instructions stored in the RAM 1202 and/or the RAM 1203 or computer program instructions loaded from the storage unit 1208 into the ROM 1202 and/or the RAM 1203. In the ROM 1202 and/or RAM 1203, various programs and data required for the operation of the device 1200 may also be stored. The CPU 1201 and the ROM 1202 and/or RAM 1203 are connected to each other by a bus 1204. An input/output (I/O) interface 1205 is also connected to bus 1204.
Various components in the device 1200 are connected to the I/O interface 1205 including: an input unit 1206 such as a keyboard, a mouse, or the like; an output unit 1207 such as various types of displays, speakers, and the like; a storage unit 1208 such as a magnetic disk, optical disk, or the like; and a communication unit 1209 such as a network card, modem, wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
CPU 1201 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples that may be implemented include, but are not limited to, graphics Processing Units (GPUs), various application-specific Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processors, controllers, microcontrollers, etc., which may be referred to as computing units accordingly. The CPU 1201 performs the various methods and processes described above, such as the processes 800 or 900. For example, in some embodiments, processes 800 or 900 may be implemented as a computer software program tangibly embodied in a computer-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 1200 via the ROM 1202 and/or the RAM 1203 and/or the communication unit 1209. When the computer program is loaded into the ROM 1202 and/or RAM 1203 and executed by the CPU 1201, one or more steps of the processes 800 or 900 described above may be performed. Alternatively, in other embodiments, the CPU 1201 may be configured to perform the processes 800 or 900 in any other suitable manner (e.g., by way of firmware).
Illustratively, the device 1200 in fig. 12 may be implemented as an electronic device (such as the client device 130 or the central device 110), or may be implemented as a chip or a chip system in an electronic device, which is not limited by the embodiments of the present disclosure.
Embodiments of the present disclosure also provide a chip that may include an input interface, an output interface, and a processing circuit. In the embodiment of the present disclosure, the above signaling or data interaction may be completed by the input interface and the output interface, and the generation and processing of the signaling or data information may be completed by the processing circuit.
Embodiments of the present disclosure also provide a chip system, which includes a processor, configured to support the client device 130 or the central device 110 to implement the functions involved in any of the above embodiments. In one possible design, the system-on-chip may further include a memory for storing necessary program instructions and data, which when executed by the processor, cause the device on which the system-on-chip is installed to implement the method according to any of the above embodiments. The chip system may be formed by a chip, or may include a chip and other discrete devices.
Embodiments of the present disclosure also provide a processor for coupling with a memory, the memory storing instructions that, when executed by the processor, cause the processor to perform the methods and functions of any of the above embodiments involving the client device 130 or the central device 110.
Embodiments of the present disclosure also provide a computer program product containing instructions which, when run on a computer, cause the computer to perform the methods and functions related to the client device 130 or the central device 110 in any of the embodiments described above.
Embodiments of the present disclosure also provide a computer readable storage medium having stored thereon computer instructions that, when executed by a processor, cause the processor to perform the methods and functions of any of the above embodiments relating to the client device 130 or the central device 110.
In general, the various embodiments of the disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, microprocessor or other computing device. While various aspects of the embodiments of the disclosure are illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product comprises computer executable instructions, such as instructions included in program modules, which are executed in a device on a real or virtual processor of the target to perform the processes/methods as described above with reference to the figures. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In various embodiments, the functionality of the program modules may be combined or split between program modules as desired. Machine-executable instructions for program modules may be executed within local or distributed devices. In a distributed facility, program modules may be located in both local and remote memory storage media.
Computer program code for implementing the methods of the present disclosure may be written in one or more programming languages. These computer program code may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the computer or other programmable data processing apparatus, causes the functions/acts specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or entirely on the remote computer or server.
In the context of the present disclosure, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus or processor to perform various processes and operations described above. Examples of a carrier include a signal, computer readable medium, and the like. Examples of signals may include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.
The computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More detailed examples of a computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical storage device, a magnetic storage device, or any suitable combination thereof.
Further, while the operations of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Rather, the steps depicted in the flowcharts may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions. It should also be noted that the features and functions of two or more devices according to the present disclosure may be embodied in one device. Conversely, the features and functions of one apparatus described above may be further divided into embodiments by a plurality of apparatuses.
The foregoing has described implementations of the present disclosure, and the above description is illustrative, not exhaustive, and not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terminology used herein was chosen in order to best explain the principles of various implementations, the practical application, or improvements to the technology in the marketplace, or to enable others of ordinary skill in the art to understand various implementations disclosed herein.

Claims (13)

1. A video presentation method, comprising:
the method comprises the steps that client equipment receives a composite video from central equipment, wherein the ith frame of the composite video is obtained based on the ith frame of the video shot by a plurality of shooting equipment in a distributed shooting system at the same moment, and i is any positive integer;
the client device determining a video to present based on the composite video, the video to present being associated with at least one of the plurality of capture devices; and
and the client equipment presents the video to be presented.
2. The method of claim 1, the client device determining the video to present based on the composite video comprising determining each frame of the video to present by:
the client device determines an ith frame of a video shot by a target shooting device from the ith frame of the composite video, wherein the target shooting device is a shooting device positioned at a target position in the plurality of shooting devices; and
and the client equipment determines the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
3. The method of claim 1, the client device determining the video to present based on the composite video comprising:
the client device receives a user input instruction, wherein the user input instruction indicates a target shooting device;
the client device determines each frame of the video to be presented by: determining an ith frame of a video shot by the target shooting device from the ith frame of the composite video; and determining the ith frame of the video shot by the target shooting equipment as the ith frame of the video to be presented.
4. The method of any of claims 1 to 3, further comprising:
the client equipment receives a look-around viewing operation of a user for a current frame of the video to be presented; and
and the client equipment responds to the look-around viewing operation and presents a look-around image sequence corresponding to the current frame of the video to be presented.
5. The method of claim 4, wherein the client device presenting a sequence of look-around images based on the look-around viewing operation comprises:
the client device responds to the look-around viewing operation, and determines a frame corresponding to a current frame of the video to be presented from the composite video;
the client device splits the determined frame corresponding to the current frame of the video to be presented into a plurality of images respectively corresponding to the plurality of shooting devices;
the client device obtaining the sequence of around-the-view images based on the plurality of images; and
the client device presents the sequence of surround view images.
6. The method of claim 5, wherein the client device obtaining a sequence of all-around images based on the plurality of images comprises:
the client device arranges the plurality of images according to the position sequence of the plurality of shooting devices to obtain the all-round-looking image sequence; alternatively, the first and second electrodes may be,
the client device arranges the plurality of images in the order of positions of the plurality of photographing devices; and
the client device inserts an intermediate frame between each two adjacent images of the plurality of images to obtain the sequence of all-around images.
7. The method according to any one of claims 1 to 6, wherein the ith frame of the composite video is obtained by stitching ith frames respectively captured by the plurality of capture devices at the same time.
8. A video presentation method, comprising:
the method comprises the steps that a central device receives videos shot by a plurality of shooting devices in a distributed shooting system;
the central equipment obtains a composite video based on videos shot by the plurality of shooting equipment respectively, wherein the ith frame of the composite video is obtained based on the ith frame of the videos shot by the plurality of shooting equipment respectively at the same moment, and i is any positive integer; and
the central device sends the composite video to a client device.
9. The method of claim 8, wherein the central device obtaining a composite video comprises determining each frame of the composite video by:
and the central equipment splices the ith frames of the videos shot by the shooting equipment at the same moment respectively to obtain the ith frame of the composite video.
10. The method of claim 8 or 9, further comprising:
the central device presents videos shot by a particular shooting device in the distributed shooting system.
11. An electronic device comprising a processor and a memory, the memory having stored thereon computer instructions that, when executed by the processor, cause the electronic device to perform the method of any of claims 1-7 or the method of any of claims 8-10.
12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method of any of claims 1 to 7 or the method of any of claims 8 to 10.
13. A computer program product having computer-executable instructions embodied thereon that, when executed, implement the method of any of claims 1 to 7 or any of claims 8 to 10.
CN202110837120.7A 2021-05-31 2021-07-23 Video presentation method, electronic device, computer storage medium, and program product Pending CN115484486A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/084913 WO2022252797A1 (en) 2021-05-31 2022-04-01 Video presentation method, electronic device, computer storage medium and program product

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110600072 2021-05-31
CN202110600072X 2021-05-31

Publications (1)

Publication Number Publication Date
CN115484486A true CN115484486A (en) 2022-12-16

Family

ID=84419579

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110837120.7A Pending CN115484486A (en) 2021-05-31 2021-07-23 Video presentation method, electronic device, computer storage medium, and program product

Country Status (1)

Country Link
CN (1) CN115484486A (en)

Similar Documents

Publication Publication Date Title
EP1703730A1 (en) Method and apparatus for composing images during video communications
CN111970524B (en) Control method, device, system, equipment and medium for interactive live broadcast and microphone connection
CN112738010A (en) Data interaction method and system, interaction terminal and readable storage medium
CN112738495B (en) Virtual viewpoint image generation method, system, electronic device and storage medium
CN112738534A (en) Data processing method and system, server and storage medium
WO2023169297A1 (en) Animation special effect generation method and apparatus, device, and medium
CN108243318B (en) Method and device for realizing live broadcast of multiple image acquisition devices through single interface
WO2019156819A1 (en) Method and apparatus for processing and distributing live virtual reality content
US11223662B2 (en) Method, system, and non-transitory computer readable record medium for enhancing video quality of video call
CN110730340B (en) Virtual audience display method, system and storage medium based on lens transformation
CN107580228B (en) Monitoring video processing method, device and equipment
WO2015089944A1 (en) Method and device for processing picture of video conference, and conference terminal
JP6821398B2 (en) Image processing equipment, image processing methods and programs
CN110741648A (en) Transmission system for multi-channel portrait and control method thereof, multi-channel portrait playing method and device thereof
CN109862385B (en) Live broadcast method and device, computer readable storage medium and terminal equipment
CN106998457A (en) Stereo photographic device and utilize this image processing method
CN115484486A (en) Video presentation method, electronic device, computer storage medium, and program product
WO2022252797A1 (en) Video presentation method, electronic device, computer storage medium and program product
CN113794844B (en) Free view video acquisition system, method, device, server and medium
CN113938617A (en) Multi-channel video display method and equipment, network camera and storage medium
CN115002345A (en) Image correction method and device, electronic equipment and storage medium
CN115002442A (en) Image display method and device, electronic equipment and storage medium
CN103019912A (en) Processing monitoring data in a monitoring system
CN112738009A (en) Data synchronization method, device, synchronization system, medium and server
JP5927515B2 (en) Image editing method, image editing system, and image editing program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination