US11388215B2 - Multilayer and synchronized communal social media system and method - Google Patents
Multilayer and synchronized communal social media system and method Download PDFInfo
- Publication number
- US11388215B2 US11388215B2 US16/951,404 US202016951404A US11388215B2 US 11388215 B2 US11388215 B2 US 11388215B2 US 202016951404 A US202016951404 A US 202016951404A US 11388215 B2 US11388215 B2 US 11388215B2
- Authority
- US
- United States
- Prior art keywords
- user devices
- multimedia content
- content
- user
- communal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 41
- 230000001360 synchronised effect Effects 0.000 title description 11
- 238000007726 management method Methods 0.000 claims abstract description 48
- 238000012384 transportation and delivery Methods 0.000 claims abstract description 45
- 239000012634 fragment Substances 0.000 claims description 34
- 238000013475 authorization Methods 0.000 claims description 2
- 238000004891 communication Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 10
- 230000008901 benefit Effects 0.000 description 8
- 230000009471 action Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/01—Social networking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
- H04L65/4053—Arrangements for multi-party communication, e.g. for conferences without floor control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/611—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for multicast or broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/765—Media network packet handling intermediate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3226—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using a predetermined code, e.g. password, passphrase or PIN
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3297—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving time stamps, e.g. generation of time stamps
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2463/00—Additional details relating to network architectures or network communication protocols for network security covered by H04L63/00
- H04L2463/101—Additional details relating to network architectures or network communication protocols for network security covered by H04L63/00 applying security measures for digital rights management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/03—Protecting confidentiality, e.g. by encryption
- H04W12/033—Protecting confidentiality, e.g. by encryption of the user plane, e.g. user's traffic
Definitions
- individuals in different locations that are connected via Facebook may view an event such as a concert, movie, or sports match, stream a video, play a video game or attend an education session at the same time, while also being able to communicate, for example via text, with each other over the social media platform.
- individuals in different locations may watch the same movie using, for example, a Chrome teleparty extension, while also being able to communicate, for example via text, during the movie.
- interactions over these platforms are typically not fully synchronized due to the differences in the network configurations of the individuals participating in a social media session. For example, when multiple individuals are simultaneously viewing a video on a platform, there are likely to be different lag times on when the video is received on each user's device due to the differences in, for example, the bandwidth of each user's network.
- FIG. 1 is a block diagram of a communal media system used in accordance with some embodiments.
- FIG. 2 is a block diagram of how packets are structured and delivered in the communal media system in accordance with some embodiments.
- FIG. 3 is a block diagram of a timestamp structure used in accordance with some embodiments.
- FIG. 4 is a flowchart of a method of sharing multimedia content in accordance with some embodiments.
- FIG. 5 is a flowchart of a method of sharing multimedia content generated on a user device in accordance with some embodiments.
- FIG. 6 is another flowchart of a method of sharing multimedia content in accordance with some embodiments.
- FIG. 7 is another flowchart of a method of sharing multimedia content in accordance with some embodiments.
- FIG. 8 is a block diagram of a user device used in accordance with some embodiments.
- FIG. 9 is a block diagram of a system used in accordance with some embodiments.
- Some embodiments are directed to a communal media system including a content management system configured to store multimedia content.
- the communal media system also includes a synchronization system configured to retrieve the multimedia content for delivery to a plurality of user devices, and receive social media content generated by the plurality of user devices, and separate the multimedia content and the social media content into a set of prioritized layers.
- the synchronization system is also configured to transmit the multimedia content and the social media content to the plurality of user devices; continuously receive at least one of timestamp receipts and fragment identifier acknowledgements associated with receipt of the multimedia content from the plurality of user devices; and adjust simultaneous deliveries of the multimedia content and the social media content to the plurality of user devices, based on a priority associated with each layer and at least one of the timestamp receipts and the fragment identifier acknowledgements received from each of the plurality of user devices.
- the multimedia content and the social media content are delivered to each of the plurality of user devices at a same time, without any noticeable lag time during receipt of the multimedia content and the social media content by the plurality of user devices.
- Some embodiments are directed to a method for delivering multimedia content during a communal media session.
- the method includes registering a plurality of user devices to participate in the communal media session.
- the method includes, at a communal media system, registering a plurality of user devices to participate in the communal media session.
- the method further includes, during the communal media session, retrieving, by a synchronization system, the multimedia content to be delivered to the plurality of user devices and receiving social media content generated by at least one of the plurality of user devices; separating, by the synchronization system, the multimedia content and the social media content into a set of prioritized layers; transmitting, by the synchronization system, the multimedia content and the social media content to the plurality of user devices; continuously receiving, by the synchronization system, at least one of timestamp receipts and fragment identifier acknowledgements associated with receipt of the multimedia content from the plurality of user devices; and adjusting, by the synchronization system, simultaneous deliveries of the multimedia content and the social media content to the plurality of user devices, based on a priority with associated each layer and at least one of the timestamp receipts and the fragment identifier acknowledgements received from each of the plurality of user devices.
- the multimedia content and the social media content are delivered to each of the plurality of user devices at a same time without any noticeable lag time during receipt of the multimedia
- the method includes registering, at a communal media system, a plurality of user devices to participate in the communal media session.
- the method further includes: retrieving, by a synchronization system, a recording from the host user device, the host user device being one of a plurality of user devices in the communal media session, wherein the recording includes multimedia content to be delivered to other user devices; receiving, by the synchronization system, social media content generated by at least one of the plurality of user devices; separating, by the synchronization system, the multimedia content and the social media content into a set of prioritized layers; transmitting, by the synchronization system, the multimedia content to the other user devices based on a timestamp offset associated with the host user device and transmitting the social media content among the plurality of user devices; continuously receiving, by the synchronization system, at least one of timestamp receipts and fragment identifier acknowledgements associated with receipt of the multimedia content from
- FIG. 1 is a block diagram of a communal media system used in accordance with some embodiments.
- Communal media system 100 is configured to include one or more systems 102 (shown as, for example, a content management system 102 a , a synchronization system 102 b , a business management system 102 c and a business rules system 102 d ), each of which may be located in the same physical location as components on a single computing device or on different computing devices that are communicatively coupled.
- Systems 102 may also be in remote locations on different computing devices that are communicatively coupled.
- Systems 102 may include additional components (not shown for the sake of simplicity) that may be used to transmit information across communal media system 100 .
- Content management system 102 a may include multimedia content such as live or recorded videos.
- Synchronization system 102 b is configured to synchronize delivery of information to communicatively coupled user devices such that the user devices in a communal session can receive the information at the same time.
- Business management system 102 c is configured to perform, for example, user authorization and authentication, device registrations, entitlement checks, and/or optional digital rights management (DRM) content encryption for licensed content stored on content management system 102 a .
- Business rules system 102 d is configured to confirm access to and/or offer subscriptions for access to the multimedia content stored on content management system 102 a.
- Communal media system 100 is also configured to include multiple user devices 104 (shown as, for example, user devices 104 a - 104 x ) that are communicatively coupled to each other and to systems 102 .
- User devices 104 may be mobile devices including mobile phones or tablets, personal computers, gaming devices or other computing devices that can be used for viewing multimedia content and interacting over social media.
- Each user device 104 may be independently configured.
- user device 104 a may be configured to operate on the iOS platform and on a 5G network having a first set of specifications
- user device 104 b may be configured to operate on the Android platform on a 5G network having a second set of specifications, where the first and second specifications may be different.
- synchronization system 102 b is configured to synchronize simultaneous delivery of multimedia and other social media content to user devices 104 such that each user device 104 receives the delivered content at the same time.
- each user device 104 a - 104 x is configured to download and install an application including communal media system 100 from, for example, a mobile app store or the Internet.
- business management system 102 c is configured to capture system configuration information from the user device and determine if the configuration on the user device meets a predetermined threshold. For example, business management system 102 c may determine if the bandwidth available for each user device 104 is above a predetermined minimum bandwidth threshold. If business management system 102 c determines that the bandwidth for a user device is below the predetermined minimum bandwidth threshold, business management system 102 c may determine that the device with bandwidth below the minimum bandwidth threshold cannot be supported and will prevent the device from participating in future communal media sessions.
- business management system 102 c may capture the user login credentials, such as user ID or social media login, from the user device in order to allow the user of that device to participate in future communal media sessions.
- business management system 102 c is configured evaluate the network configuration of each user device 104 a - 104 x and determine the minimum network bandwidth required for simultaneous transmission over communal media system 100 to user devices 104 - 104 x . If business management system 102 c determines that user device 104 a has a bandwidth of 100 Mbps, user device 104 b has a bandwidth of 75 Mbps, user device 104 c has a bandwidth of 100 Mbps, user device 104 d has a bandwidth of 25 Mbps, and so on, with 25 Mbps being the slowest network bandwidth, business management system 102 c may determine the minimum network bandwidth required for simultaneous transmissions to user devices 104 - 104 x . If business management system 102 c determines that the minimum required network bandwidth is below 25 Mbps, business management system 102 c may capture the user login credentials from user devices 104 - 104 x in order to allow the users of those devices to participate in future communal media sessions.
- business management system 102 c is configured to capture system configuration information from a user device and determine if the configuration on the user device meets a predetermined threshold when the user devices registers to participate in a specific session.
- devices 104 a - 104 d use those devices to register for a session on communal media system 100 .
- Users of devices 104 a - 104 d may register through a user interface on communal media system 100 and may browse media available on content management system 102 a via the user interface.
- a user of one of devices 104 a - 104 d (referred to herein as a host, i.e., the device used to initiate the session and/or used to select media content that is to be shared with the other devices in a session), for example, device 104 a , may select a specific media.
- the media selected by device 104 a may be shared/viewed on devices 104 b - 104 d in a communal setting at an appointed time on communal media system 100 .
- the user of device 104 a may send invitations to devices 104 b - 104 d for the users of devices 104 a - 104 d to view the selected content at the appointed time.
- Each of the users of devices 104 b - 104 d may accept or reject the invitation.
- user devices 104 a - 104 d may be connected via communal media system 100 for the communal viewing session of the content selected via device 104 a.
- business management system 102 c may obtain the configuration information from user devices 104 a - 104 d when the user devices register to participate in a specific session. For example, business management system 102 c may obtain the location, the network carrier, and/or the network type (for example, WiFi, LTE, 5G) of the carrier of each user device 104 a - 104 d . Business management system 102 c may also test the speed of each user device 104 a - 104 d and store the information for each user device 104 a - 104 d along with a timestamp.
- the network carrier for example, WiFi, LTE, 5G
- each device 104 a - 104 d may enter subscription information into the associated device 104 a - 104 d , wherein each device may transmit the subscription information to business rules system 102 d .
- business rules system 102 d may obtain confirmation for access to the licensed content from a content owner or distributor.
- business rules system 102 d may provide access to subscriptions to one or more user devices 104 a - 104 d and is configured to receive payments for the subscription from one or more user devices 104 a - 104 d .
- business management system 102 c may obtain decryption keys for the content from a DRM system.
- Communal media system 100 may then present the decrypted information to user devices 104 a - 104 d for communal viewing.
- synchronization system 102 b is configured to synchronize the simultaneous delivery of multimedia content, stored on content management system 102 a , to user devices 104 a - 104 d , such that each user device 104 a - 104 d receives the delivered content at the same time with no noticeable delivery lag, as though each user is receiving the information on the same device.
- Synchronization system 102 b is also configured to synchronize the simultaneous delivery of live videos captured from forward and/or rear cameras on one or more user device 104 a - 104 d .
- Synchronization system 102 b is further configured to simultaneously transmit video chats, texts, emotive responses, and other social media features generated on user devices 104 a - 104 d while the users of those devices are viewing the delivered multimedia content.
- a user associated with a device 104 a - 104 d may create a profile photo of the user, record an image of the user, or create an avatar that is configured to represent an image of the user, wherein the avatar may mimic the user's actions, including speaking and other actions performed by the user.
- the avatar or other images of users associated with devices 104 a - 104 d , video chats, texts, emotive responses, and other social media features generated on devices 104 a - 104 d may be overlaid on the delivered multimedia content.
- the host may pause, rewind, or fast forward the multimedia content, wherein synchronization system 102 b is also configured to ensure that the pause, rewind, or fast forward action is synchronized across all user devices.
- the host may make the communal session live to the public by activating a live button on the user interface.
- a live button on the user interface.
- users of other devices for example, devices 104 e - 104 x that are accessing communal media system 100 may view and participate in the session by sending texts, emotive responses, and other social media features generated on those uses devices 104 e - 104 x .
- the live session may have a ticker counter to identify how may users are participating in the live session.
- a thumbnail image of the selected content may be provided in communal media system 100 .
- Users of devices 104 a - 104 d may manipulate the thumbnail images by, for example, moving, resizing, or deleting the image on the user interface.
- FIG. 2 is a block diagram of how packets are structured and delivered in the communal media system 100 in accordance with some embodiments.
- Synchronization system 102 b is configured to separate content transmitted over communal media system 100 into multiple layers 200 and to synchronize delivery of the multiple layers to user devices 104 a - 104 d , i.e., those user devices registered to participate in the session on communal media system 100 .
- synchronization system 102 b may configure the multimedia content stored on content management system 102 a in a multimedia content layer 202 , video chats received from user devices 104 a - 104 d in a video chat layer 204 , texts received from user devices 104 a - 104 d in a text layer 206 , and emojis received from user devices 104 a - 104 d in an emojis layer 204 . It should be apparent that synchronization system 102 b may separate content transmitted over communal media system 100 into more or fewer layers than layers 202 - 208 .
- synchronization system 102 b may prioritize delivery of packets. For example, packets in multimedia content layer 202 may have a first priority, packets in video chat layer 204 may have a second priority, packets in text layer 206 may have a third priority, and packets in emojis layer 204 may have a fourth priority. Synchronization system 102 b is also configured to synchronize voice and video packets in video streams in multimedia content layer 202 and voice and video packets in video chat layer 204 so that there is no lag time between the voice and the video packets in each layer.
- synchronization system 102 b is configured to synchronize delivery of packets in the same layer and delivery of multiple layers of content across disparate devices and disparate access networks so that all user devices 104 a - 104 d may receive content delivered over communal media system 100 at the same time.
- synchronization system 102 b has packets (P) in different layers that are to be delivered at times T 1 -T 4 , as shown in FIG. 2 .
- the packets in multimedia content layer 202 are delivered first.
- the packets in multimedia content layer 202 are delivered before the packets in video chat layer 204 .
- the packets in multimedia content layer 202 are delivered first, the packets in video chat layer 204 are delivered second, and the packets in text layer 206 are delivered last.
- the packets in multimedia content layer 202 are delivered first, the packets in text layer 206 are delivered second, and the packets in emoji layer 208 are delivered last.
- FIG. 3 is a block diagram of a timestamp structure used in accordance with some embodiments.
- synchronization system 102 b is configured to continuously receive timestamp receipts from each user device 104 a - 104 d .
- the timestamp receipts include timestamps of when a predefined number of packets is received by a user device. and timestamps of when a predefined number of packets is sent from the device.
- the interval for delivering and/or receiving timestamp receipts may be static or dynamic Static delivery may involve delivering and/or receiving timestamp receipts at predefined intervals (e.g. every 100 mms) and dynamic delivery may involve adapting delivery and/or receipt of timestamp receipts according to the various speeds of the heterogenous networks of user devices 104 a - 104 d.
- Synchronization system 102 b may determine a timestamp offset for each user device 104 a - 104 d and for each type/layer of content (video stream, video chat, text and emojis) delivered.
- the timestamp offset may be based on a predetermined minimum network configuration and/or the timestamp receipts, wherein synchronization system 102 b may adjust packet deliveries to correspond with the timestamp offset of each user device 104 a - 104 d .
- synchronization system 102 b may set the timestamp offset for user devices 104 a - 104 d based on the lowest bandwidth, as shown in FIG. 3 .
- the timestamp offsets for user device 104 a and 104 c are set to four times the timestamp offset of user device 104 d and the timestamp offset for user device 104 b is set to three times the timestamp offset of user device 104 d , This ensures that when packets are delivered using the timestamp offsets, synchronization system 102 b can normalize the delivery of packets to all user devices 104 a - 104 d (i.e., take care of propagation delays or speed up delivery based on the bandwidth).
- all user devices 104 a - 104 d will receive multimedia and social media contents at the same time and without any noticeable delay in receipt of the content among the devices, ensuring that users of devices 104 a - 104 d are viewing and interacting with the content as if they were in the same physical location and viewing the content on the same device.
- synchronization system 102 b may also send and/or receive information associated with frame and/or fragment of the multimedia content to devices 104 a - 104 d .
- host user device 104 a may request, from content management system 102 a , a location of a manifest file including multimedia content for a viewing session.
- Content management system 102 a may provide information including, for example, a Uniform Resource Locator (URL) and/or the name of the manifest file.
- URL Uniform Resource Locator
- Media fragments of, for example, three to six seconds of the multimedia content may be stored in the manifest file.
- each second of the media fragment may include, for example, twenty-four-sixty frames of full motion video and each frame may include a unique frame identifier.
- host device 104 a may send the URL for the manifest file to user devices 104 b - 104 d participating in the communal session.
- synchronization system 102 b may use the frame identifiers in the media fragments transmitted to and from devices 104 a - 104 d to synchronize delivery of the media to devices 104 a - 104 d .
- synchronization system 102 b is to transmit a fragment with twenty-four frames, with each successive frame assigned an identifier from 1-24.
- synchronization system 102 b is to transmit frames 1-10 to devices 104 a - 104 d
- synchronization system 102 b is to transmit frames 11-20 to devices 104 a - 104 d .
- synchronization system 102 b may receive frame identifier acknowledgements of the receipts of frames 1-10 from devices 104 a - 104 d .
- synchronization system 102 b may transmit frames 11-20 to devices 104 a - 104 d , thereby synchronizing delivery of the frames to devices 104 a - 104 b with delivery of the frames to devices 104 c - 104 d.
- business management system 102 c may perform periodic speed tests (for example, on an as needed basis or at predefined intervals) on each user device 104 a - 104 d and may send the results to synchronization system 102 b.
- host device 104 a may start to play the media fragments stored in the manifest file in an order determined, for example, by information obtained from the manifest file.
- host device 104 a may record a timestamp receipt for the start time, for example, in milliseconds or seconds, and may send the timestamp receipt and the frame number of the media fragment being played to synchronization system 102 b .
- host device 104 a may continuously send timestamp receipts and frame identifier acknowledgments of media fragment being played to synchronization system 102 b .
- Synchronization system 102 b may record the frame identifier acknowledgments, timestamp receipts, and the timestamp offset of host device 104 a.
- user devices 104 b - 104 d may receive the start time for playing the media from synchronization system 102 b .
- each user device 104 b - 104 d may provide, to synchronization system 102 b , a timestamp receipt of the start time of the media playback on the device and the frame identifier acknowledgment of the media fragment(s) received by the device.
- each user device 104 b - 104 d may also provide the progress of the media playback on a predetermined basis, for example, as defined by business management system 102 c , to synchronization system 102 b .
- Synchronization system 102 b may review and compare the locations in the media being played on user devices 104 a - 104 d on a pre-determined basis. Synchronization system 102 b may continuously determine that the media being played on user devices 104 a - 104 d are synchronized within a pre-determined period, for example, a pre-determined number of milliseconds or second. Synchronization system 102 b may dynamically adjust the pre-determined period to ensure the quality of the multimedia content. The pre-determined period may be defined by business management system 102 c . Synchronization system 102 b may also ensure that the media being played on user devices 104 b - 104 d is not ahead of the media being played on host device 104 a.
- Each user devices 104 a - 104 d may continuously send its timestamp receipts and/or the frame identifier acknowledgments for fragment(s) played on the device to synchronization system 102 b . If the media being played on any of user devices 104 a - 104 d falls behind a certain threshold, synchronization system 102 b may provide signaling messages to user devices 104 a - 104 d .
- the signaling messages may be defined by business management system 102 c .
- Synchronization system 102 b may make necessary adjustments in transmitting the media to user devices 104 a - 104 d , for example, either by marginally slowing the media being played to one or more devices or by accelerating the media being played to one or more devices, until all of the devices are on the same frame.
- synchronization system 102 b may ensure that the media is transmitted and played on all devices within acceptable tolerances.
- the time offset associated with the host device and/or frame number associated with the content is adjusted continuously until the host device stops rewinding the media and playing the media resumes.
- the time offset and/or frame number is adjusted continuously until the host device stops fast forwarding the media and playing the media resumes.
- the rewind or fast forward offset is sent to synchronization system 102 b for adjustments of the transmissions on the other devices in the communal session.
- FIG. 4 is a flowchart of a method of sharing multimedia content in accordance with some embodiments.
- the user of device 104 a may sign up as a host/moderator in communal media system 100 .
- the user of device 104 a may select a video from content management system 102 a .
- the user of device 104 a may select the users of devices 104 b - 104 d from a contact library and send invitations to user devices 104 b - 104 d .
- the invitations may invite users of devices 104 b - 104 d to view the selected video at a predetermined viewing time.
- the predetermined viewing time may be a current time or a future time.
- the users of devices 104 b - 104 d may accept or reject the invitations.
- synchronization system 102 b may retrieve packets including the video, associate the packets with multimedia content layer 202 and determine timestamp offsets for user device 104 a - 104 d .
- synchronization system 102 b adjusts deliveries of the packets based the timestamp offsets associated with user devices 104 a - 104 d and simultaneously transmits the video packets according to the timestamp offset for each user device 104 a - 104 d .
- synchronization system 102 b may continuously receive social media content and/or timestamp receipts from each user device 104 a - 104 d , wherein the timestamp receipts are for packets delivered to and sent from the user device.
- the packets sent from user devices 104 a - 104 d may include video chats, texts, emojis, or other social media interactions on user devices 104 a - 104 d .
- synchronization system 102 b continues to associate each type of content (video stream, video chat, text and emojis) with an appropriate layer 200 , adjusts simultaneous deliveries of the packets including the multimedia and social media content based on a priority associated with each layer 202 and the timestamp offset associated with user device 104 a - 104 d .
- This allows all the user devices 104 a - 104 d to receive the multimedia content, video chat, texting and emojis at the same time, wherein delivery of the multimedia content, video chat, texting and emojis is fully synchronized across all user devices 104 a - 104 d.
- FIG. 5 is a flowchart of a method of sharing multimedia content generated on a user device in accordance with some embodiments.
- the user of device 104 a may sign up as a host/moderator in communal media system 100 .
- the user of device 104 a may select the users of devices 104 b - 104 d from a contact library and send invitations to user devices 104 b - 104 d .
- the invitations may invite users of devices 104 b - 104 d to view a recording generated from the front and/or rear cameras of user device 104 a at a predetermined viewing time.
- the predetermined viewing time may be a current time or a future time.
- the users of devices 104 b - 104 d may accept or reject the invitations.
- the front and/or rear cameras of user device 104 a are activated and the recording is sent to communal media system 100 .
- the recording may be saved as video fragments on content management system 102 a and synchronization system 102 b may deliver the video fragments from content management system 102 a to user devices 104 b - 104 d .
- host timestamp receipts are sent from user device 104 a to synchronization system 102 b continuously and at predetermined time intervals.
- synchronization system 102 b associates the video being recorded on user device 104 a with multimedia content layer 202 and determines a host timestamp offset for user device 104 a .
- synchronization system 102 b simultaneously transmits packets of the video to user device 104 b - 104 d based on the host timestamp offset for user device 104 a and receives frame identifier acknowledgements from user devices 104 b - 104 d .
- synchronization system 102 b may continuously receive social media content and/or timestamp receipts from each user device 104 a - 104 d for packets delivered to and sent from user devices 104 a - 104 d .
- the packets sent from user devices 104 a - 104 d may include social media content such as video chats, texts, emojis or other social media interactions on user devices 104 a - 104 d .
- synchronization system 102 b continues to associate each type of content (video stream, video chat, text and emojis) with an appropriate layer 200 . sends packets in layer 202 according to the priority assigned to the layer 202 and the host timestamp offset of user device 104 a and sends packets in layers 204 - 208 according to the priority assigned to those layers and the timestamp offset of each user device 104 a - 104 d .
- each type of content video stream, video chat, text and emojis
- synchronization system 102 b may use the frame identifier acknowledgments and/or timestamp receipts received from devices 104 a - 104 d to continuously determine that the media being played on user devices 104 a - 104 d are synchronized. This allows all the user devices 104 a - 104 d to receive the multimedia content being recorded on user device 104 a at the same time, while sharing video chats, texts, and emojis, wherein delivery of the multimedia content, video chats, texts and emojis is fully synchronized across all user devices 104 a - 104 d.
- FIG. 6 is another flowchart of a method of sharing multimedia content in accordance with some embodiments.
- user device 104 a signs up as a host and requests the location of a manifest file including multimedia.
- host device 104 a receives information for the manifest files from content management system 102 a and forwards the information to devices 104 b - 104 d .
- host device starts the media in an order based on information in the manifest file, records a timestamp receipt for the start time, and sends the timestamp receipt and the frame identifier acknowledgment of the media fragment being played to synchronization system 102 b .
- user devices 104 b - 104 d may receive the start time for playing the media from synchronization system 102 b and each user device 104 b - 104 d may provide, to synchronization system 102 b , the timestamp receipt of a start time of the media playback on the device.
- host device 104 a continuously sends timestamp receipts and/or frame identifier acknowledgments of media fragments being played to synchronization system 102 b .
- each user device 104 b - 104 d may also provide timestamp receipts and/or frame identifier acknowledgments on a predetermined basis to synchronization system 102 b .
- synchronization system 102 b may review and compare the locations in the media being played on user devices 104 a - 104 d on a pre-determined basis.
- synchronization system 102 b may use the frame identifier acknowledgments and/or timestamp receipts received from devices 104 a - 104 d to continuously determine that the media being played on user devices 104 a - 104 d are synchronized.
- FIG. 7 is a flowchart of another method of sharing multimedia content in accordance with some embodiments.
- the user of device 104 a may sign up as a host/moderator in communal media system 100 .
- the user of device 104 a may select a video from content management system 102 a .
- the user of device 104 a may select the users of devices 104 b - 104 d from a contact library and send invitations to view the selected video at a predetermined viewing time to user devices 104 b - 104 d .
- the users of devices 104 b - 104 d may accept or reject the invitations.
- synchronization system 102 b may retrieve packets including the video, associate the packets with multimedia content layer 202 and determine timestamp offsets for user device 104 a - 104 d .
- synchronization system 102 b adjusts simultaneous deliveries of the packets based the timestamp offsets associated with user devices 104 a - 104 d and/or based on video fragments and/or frames for differing video delivery formats including, for example, standard definition, high definition, 4K 5K, and other delivery formats.
- synchronization system 102 b may adjust deliveries of the packets based on video fragments and/or frames for differing video delivery formats by synchronizing the fragments and/or frames delivered to user device 104 b - 104 d according to a predetermined (for example, lower) latency interval between user device 104 a and user devices 104 b - 104 d.
- synchronization system 102 b may continuously receive social media content, timestamp receipts, and/or frame identifier acknowledgements from each user device 104 a - 104 d , wherein the timestamp receipts are for packets delivered to and sent from the user device.
- the packets sent from user devices 104 a - 104 d may include video chats, texts, emojis, or other social media interactions on user devices 104 a - 104 d .
- FIG. 8 is a block diagram of a user device 800 , such as user device 104 of FIG. 1 , used in accordance with some embodiments.
- User device 800 may include a communications unit 802 coupled to a common data and address bus 817 of a processor 803 .
- Device 800 may also include an input unit (e.g., keypad, pointing device, etc.) 806 , an output transducer unit (e.g., speaker) 820 , an input transducer unit (e.g., a microphone) (MIC) 821 , and a display screen 805 , each coupled to be in communication with the processor 803 .
- an input unit e.g., keypad, pointing device, etc.
- an output transducer unit e.g., speaker
- an input transducer unit e.g., a microphone
- display screen 805 each coupled to be in communication with the processor 803 .
- the processor 803 may include, that is, implement, an encoder/decoder 811 with an associated code read-only memory (ROM) 812 for storing data for encoding and decoding voice, data, control, or other signals that may be transmitted or received by device 800 .
- the processor 803 may further include one or more of a microprocessor 813 and digital signal processor (DSP) 819 coupled, by the common data and address bus 817 , to the encoder/decoder 811 and to one or more memory devices, such as a ROM 814 , a random access memory (RAM) 804 , and a static or flash memory 816 .
- DSP digital signal processor
- ROM 814 , RAM 804 and flash memory 816 may be included as part of processor 803 or may be separate from, and coupled to, the processor 803 .
- the encoder/decoder 811 may be implemented by microprocessor 813 or DSP 819 , or may be implemented by a separate component of the processor 803 and coupled to other components of the processor 803 via bus 818 .
- Communications unit 802 may include an RF interface 809 configurable to communicate with network components, and other user equipment within its communication range.
- Communications unit 802 may include one or more broadband and/or narrowband transceivers 808 , such as an Long Term Evolution (LTE) or 5G transceiver, a WiMAX transceiver perhaps operating in accordance with an IEEE 802.16 standard, and/or other similar type of wireless transceiver configurable to communicate via a wireless network for infrastructure communications.
- Communications unit 802 may also include one or more local area network or personal area network transceivers such as Wi-Fi transceiver perhaps operating in accordance with an IEEE 802.11 standard (e.g., 802.11a, 802.11b, 802.11g), or a Bluetooth transceiver.
- the transceivers may be coupled to a combined modulator/demodulator 810 that is coupled to the encoder/decoder 811 .
- the one or more memory devices 814 , 816 may store code for decoding or encoding data such as control, request, or instruction messages, channel change messages, and/or data or voice messages that may be transmitted or received by device 800 and other programs and instructions that, when executed by the processor 803 , provide for device 800 to perform the functions and operations described herein.
- FIG. 9 is a block diagram of a system 900 , such as system 102 of FIG. 1 , used in accordance with some embodiments.
- System 900 may include a communications unit 902 coupled to a common data and address bus 917 of a processor 903 .
- the processor 903 may include a code read-only memory (ROM) 912 for storing data for initializing system components of system 900 .
- the processor 903 may further include a microprocessor 913 coupled, by the common data and address bus 917 , to one or more memory devices, such as a read-only memory (ROM) 914 , a random access memory (RAM) 904 , and/or a static or flash memory 916 .
- ROM 914 , RAM 904 and flash memory 916 may be included as part of processor 903 or may be separate from, and coupled to, the processor 903 .
- Communications unit 902 may include a wired or wireless input/output I/O interface 909 configurable to communicate with network components and other user equipment within its communication range.
- Communications unit 902 may include one or more broadband and/or narrowband transceivers 908 , such as a Long-Term Evolution (LTE) transceiver and/or other similar type of wireless transceiver configurable to communicate via a wireless network for infrastructure communications.
- Communications unit 902 may also include one or more local area network or personal area network transceivers such as Wi-Fi transceiver perhaps operating in accordance with an IEEE 802.11 standard (e.g., 802.11a, 802.11b, 802.11g), or a Bluetooth transceiver.
- LTE Long-Term Evolution
- Communications unit 902 may also include one or more local area network or personal area network transceivers such as Wi-Fi transceiver perhaps operating in accordance with an IEEE 802.11 standard (e.g., 802.11a, 802.11b, 802.11g), or a
- the transceivers may be coupled to a combined modulator/demodulator 910 .
- the one or more memory devices 912 , 914 and 916 are configured to store non-transitory computer-executable instructions to perform a set of functions such as described herein.
- Communal media system 100 may be configured to record and store a communal media session for all participants in content management system 102 or on other cloud storage. Communal media system 100 may also be configured to retrieve the recording of the session for viewing at a predetermined time.
- a includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element.
- the terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein.
- the terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%.
- the term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically.
- a device or structure that is “configured” in a certain way is configured in at least that way but may also be configured in ways that are not listed.
- processors such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- processors or “processing devices” such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- FPGAs field programmable gate arrays
- unique stored program instructions including both software and firmware
- an embodiment can be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein.
- Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory.
Abstract
Description
Claims (20)
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/951,404 US11388215B2 (en) | 2019-03-28 | 2020-11-18 | Multilayer and synchronized communal social media system and method |
US17/221,886 US11558441B2 (en) | 2020-11-18 | 2021-04-05 | Integrated system and method for executing a communal media system in a streaming application |
US17/533,474 US11722543B2 (en) | 2020-11-18 | 2021-11-23 | Integrated system and method for executing a communal media session across multiple platforms |
US17/545,693 US11930057B2 (en) | 2020-11-18 | 2021-12-08 | Integrated system and method for executing a communal media session with dual-view sharing |
US18/097,735 US20230156267A1 (en) | 2019-03-28 | 2023-01-17 | Multiview synchronized communal system and method |
US18/130,110 US11924264B2 (en) | 2020-11-18 | 2023-04-03 | Establishing trust networks in a communal media session |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962825212P | 2019-03-28 | 2019-03-28 | |
US16/872,704 US11399050B2 (en) | 2019-10-23 | 2020-05-12 | Communal social media system and method |
US16/951,404 US11388215B2 (en) | 2019-03-28 | 2020-11-18 | Multilayer and synchronized communal social media system and method |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/872,704 Continuation-In-Part US11399050B2 (en) | 2019-03-28 | 2020-05-12 | Communal social media system and method |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/221,886 Continuation-In-Part US11558441B2 (en) | 2019-03-28 | 2021-04-05 | Integrated system and method for executing a communal media system in a streaming application |
US17/221,886 Continuation US11558441B2 (en) | 2019-03-28 | 2021-04-05 | Integrated system and method for executing a communal media system in a streaming application |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210075840A1 US20210075840A1 (en) | 2021-03-11 |
US11388215B2 true US11388215B2 (en) | 2022-07-12 |
Family
ID=74850550
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/951,404 Active US11388215B2 (en) | 2019-03-28 | 2020-11-18 | Multilayer and synchronized communal social media system and method |
Country Status (1)
Country | Link |
---|---|
US (1) | US11388215B2 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150271546A1 (en) * | 2014-03-20 | 2015-09-24 | Sling Media Inc. | Synchronized provision of social media content with time-delayed video program events |
US20180007091A1 (en) * | 2006-01-28 | 2018-01-04 | Blackfire Research Corporation | Synchronous media rendering over a spatially extended wireless network |
US20180255114A1 (en) * | 2017-03-06 | 2018-09-06 | Vyu Labs, Inc. | Participant selection for multi-party social media sessions |
US20200128060A1 (en) * | 2018-10-18 | 2020-04-23 | At&T Intellectual Property I, L.P. | Methods, devices, and systems for encoding portions of video content according to priority content within live video content |
-
2020
- 2020-11-18 US US16/951,404 patent/US11388215B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180007091A1 (en) * | 2006-01-28 | 2018-01-04 | Blackfire Research Corporation | Synchronous media rendering over a spatially extended wireless network |
US20150271546A1 (en) * | 2014-03-20 | 2015-09-24 | Sling Media Inc. | Synchronized provision of social media content with time-delayed video program events |
US20180255114A1 (en) * | 2017-03-06 | 2018-09-06 | Vyu Labs, Inc. | Participant selection for multi-party social media sessions |
US20200128060A1 (en) * | 2018-10-18 | 2020-04-23 | At&T Intellectual Property I, L.P. | Methods, devices, and systems for encoding portions of video content according to priority content within live video content |
Also Published As
Publication number | Publication date |
---|---|
US20210075840A1 (en) | 2021-03-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10057662B2 (en) | Flow controlled based synchronized playback of recorded media | |
US10135887B1 (en) | Shared multimedia annotations for group-distributed video content | |
US20190215542A1 (en) | Processing gaps in audio and video streams | |
KR101643238B1 (en) | Cooperative provision of personalized user functions using shared and personal devices | |
US9066129B2 (en) | Video presentation device and method | |
EP2924998A1 (en) | Method, apparatus and system for acquiring playback data stream of real-time video communication | |
US11558441B2 (en) | Integrated system and method for executing a communal media system in a streaming application | |
CN112291503B (en) | Interaction method and device and electronic equipment | |
KR20130009446A (en) | System and method for contents sharing, device and computer readable medium storing the method | |
US20160100011A1 (en) | Content processing apparatus and content processing method thereof | |
US11399050B2 (en) | Communal social media system and method | |
US11388215B2 (en) | Multilayer and synchronized communal social media system and method | |
CN111837404B (en) | Methods, systems, and media for presenting media content | |
JP2024048339A (en) | Server, terminal and computer program | |
US11792247B2 (en) | Recording system and method of multilayer and synchronized communal social media content | |
US11722543B2 (en) | Integrated system and method for executing a communal media session across multiple platforms | |
US11930057B2 (en) | Integrated system and method for executing a communal media session with dual-view sharing | |
CN110832871B (en) | Method, server system and computer readable medium for real-time incorporation of user-generated content into a broadcast media stream | |
US20230156267A1 (en) | Multiview synchronized communal system and method | |
JP7220859B2 (en) | Systems and methods for exchanging ultra-short media content | |
US11924264B2 (en) | Establishing trust networks in a communal media session | |
US10582158B2 (en) | Synchronization of media rendering in heterogeneous networking environments | |
WO2015131936A1 (en) | System providing web-based online video streaming | |
US11934412B2 (en) | Systems and methods for providing contextual messages for media contents | |
CN114449316B (en) | Video processing method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GROOPVIEW INC., PENNSYLVANIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NEW, DELMOND;RAJAM, KRISHNAN;REEL/FRAME:054408/0212 Effective date: 20201117 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: MICROENTITY Free format text: ENTITY STATUS SET TO MICRO (ORIGINAL EVENT CODE: MICR); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |