US11683285B2 - Enhancing a social media post with content that is relevant to the audience of the post - Google Patents

Enhancing a social media post with content that is relevant to the audience of the post Download PDF

Info

Publication number
US11683285B2
US11683285B2 US17/715,654 US202217715654A US11683285B2 US 11683285 B2 US11683285 B2 US 11683285B2 US 202217715654 A US202217715654 A US 202217715654A US 11683285 B2 US11683285 B2 US 11683285B2
Authority
US
United States
Prior art keywords
content item
metadata
content
control circuitry
social media
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US17/715,654
Other versions
US20220231980A1 (en
Inventor
Madhusudhan Srinivasan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Adeia Guides Inc
Original Assignee
Rovi Guides Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=74043356&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=US11683285(B2) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Rovi Guides Inc filed Critical Rovi Guides Inc
Priority to US17/715,654 priority Critical patent/US11683285B2/en
Assigned to ROVI GUIDES, INC. reassignment ROVI GUIDES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SRINIVASAN, MADHUSUDHAN
Publication of US20220231980A1 publication Critical patent/US20220231980A1/en
Assigned to BANK OF AMERICA, N.A., AS COLLATERAL AGENT reassignment BANK OF AMERICA, N.A., AS COLLATERAL AGENT SECURITY INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ADEIA GUIDES INC., ADEIA IMAGING LLC, ADEIA MEDIA HOLDINGS LLC, ADEIA MEDIA SOLUTIONS INC., ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC., ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC., ADEIA SEMICONDUCTOR INC., ADEIA SEMICONDUCTOR SOLUTIONS LLC, ADEIA SEMICONDUCTOR TECHNOLOGIES LLC, ADEIA SOLUTIONS LLC
Priority to US18/195,264 priority patent/US20230275859A1/en
Application granted granted Critical
Publication of US11683285B2 publication Critical patent/US11683285B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/10Multimedia information
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/52User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking

Definitions

  • the present disclosure relates to modifying a portion of a social media post, and more particularly, to methods and systems for enhancing the social media post with content that is relevant to the audience of the post.
  • Some users may wish to attract more attention to their posts in order to increase their fame and/or have their posts “liked” by more followers to thereby further increase their audience. Such users may wish to add additional content that is relevant to the audience of their social media posts.
  • the operators of the social networks often allow the users to create accounts and post content to the social networks free of charge.
  • the social networks instead derive their revenue from advertisements placed on their websites and applications (apps), and/or interspersed with the posts in the users' social media feeds. While this strategy places the advertisements in front of the users, the users often just scroll or swipe past the advertisements to the posts that attract their attention. Thus, the advertisements are often ignored by the users or given less than a second of time in front of the users to attract the users' attention before being skipped.
  • the embodiments described herein provide for receiving a social media post and extracting, from the social media post, a first content item (e.g., a still image).
  • the methods and systems then identify, in a frame of the first content item, a portion of the frame that is a non-focus portion (e.g., a background portion), and identify a plurality of content items that fit within the non-focus portion of the frame.
  • the methods and systems identify a content preference of an audience of the social media post and determine an estimated length of time that the audience will view the social media post.
  • the methods and systems select a second content item, from the plurality of content items, that matches the content preference of the audience and has a duration that does not exceed the estimated length of time. Thereafter, the methods and systems generate an enhanced social media post by overlaying the second content item onto the non-focus portion of the frame and display the enhanced social media post.
  • the methods and systems identify a focus portion of the frame and a background portion of the frame and select a portion of the frame that is not obscured by the focus portion of the frame as the non-focus portion of the frame.
  • the methods and systems may further identify an object in the frame that extends beyond an edge of the frame and select a portion of the frame that includes the object as the non-focus portion of the frame.
  • the methods and systems may also determine dimensions of the non-focus portion of the frame and identify a plurality of content items having dimensions that are the same as or smaller than the dimensions of the non-focus portion of the frame.
  • the methods and systems retrieve a list of followers of a user from whom the social media post was received, identify a content preference shared by the followers, and select the content preference shared by the followers as the content preference of the audience of the social media post.
  • the methods and systems retrieve, for each of the followers, at least one value corresponding to a duration that a respective follower has previously viewed a social media post, calculate an average of the values, and select the average of the values as the estimated length of time that the audience will view the social media post.
  • the methods and systems described herein provide an improved process for identifying content that is relevant to the audience of a social media post and generating an enhanced social media post including the original content of the post and the additional content that is relevant to the audience.
  • the process increases the efficiency of adding additional content relevant to an audience of a social media post to content included in the post, both for the user and for the computing resources needed to perform the processes described herein.
  • the methods and systems provide for selecting additional content that is both relevant to the audience of the post and that fits into non-focus portions of the content of the post without obscuring the portion(s) of the post to which the user intended to attract the audience's attention.
  • FIG. 1 shows an illustrative scenario for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure
  • FIG. 2 shows an illustrative system for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure
  • FIG. 3 shows a diagram of illustrative devices of the system of FIG. 2 , in accordance with some embodiments of the disclosure
  • FIGS. 4 A and 4 B show a flowchart of an illustrative process for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure.
  • FIG. 5 is a flowchart of a detailed illustrative process for selecting a content item with which to enhance a social media post, in accordance with some embodiments of the disclosure.
  • FIG. 1 shows an illustrative scenario 100 for enhancing the social media post with content that is relevant to the audience of the post.
  • a social media post 110 includes a content item 112 and may optionally include a title 114 and message 116 .
  • Social media post 110 further includes metadata (not shown in FIG. 1 ) describing social media post 110 and/or content item 112 .
  • the metadata may include tags or identifiers describing content 112 .
  • FIG. 1 shows an illustrative scenario 100 for enhancing the social media post with content that is relevant to the audience of the post.
  • a social media post 110 includes a content item 112 and may optionally include a title 114 and message 116 .
  • Social media post 110 further includes metadata (not shown in FIG. 1 ) describing social media post 110 and/or content item 112 .
  • the metadata may include tags or identifiers describing content 112 .
  • FIG. 1 shows an illustrative scenario 100 for enhancing the social media post with content that
  • the metadata may include the tags “father,” “son,” “music,” “violin,” “lesson,” and “teach.”
  • the tags may be determined based on content 112 , such as via image processing and object recognition techniques, and/or based on title 114 and message 116 .
  • Content item 112 may be a still image, a graphics interchange format (GIF) or similar format image, a video, and/or any other type of visual content that may be included in a social media post.
  • Content item 112 includes at least one focus portion 122 and at least one non-focus portion 124 .
  • Focus portion 122 is a portion of content item 112 that includes the subject of the content and the area where an audience of the social media post is intended to focus their attention).
  • Non-focus portion 124 is a portion of content item 112 that does not include the subject and/or includes the background or other objects that are not intended to draw the audience's attention.
  • content item 112 is enhanced by overlaying a second content item 130 onto non-focus portion 124 to generate an enhanced social media post 140 .
  • Second content item 130 may be all or a portion of a video, a still image, an advertisement, a GIF, etc.
  • second content item 130 is a content item that is relevant to content item 112 .
  • content item 112 is video of a father and son playing violin
  • second content item 130 is a video of an orchestra playing the same musical composition as the father and son.
  • FIG. 1 content item 112 is video of a father and son playing violin
  • second content item 130 is a video of an orchestra playing the same musical composition as the father and son.
  • second content item 130 could be an image of the sheet music of the musical composition played by the father and son. In yet another example (not shown in FIG. 1 ), second content item 130 could be an advertisement for violin lessons. In other embodiments, second content item 130 is unrelated to content item 112 and is selected purely based on the content preference of the audience of social media post 110 , as described further below.
  • FIG. 2 shows an illustrative system for enhancing the social media post with content that is relevant to the audience of the post.
  • FIG. 2 shows a system 200 where a media device 210 retrieves second content item 130 and/or the metadata associated with second content item 130 from a content source 240 via a communications path 232 .
  • Content source 240 may be any server or other computing device from which content items (such as second content item 130 ) and/or related metadata associated with content items may be retrieved.
  • content source 240 may be a cloud storage platform by means of which content items are collected, stored, and distributed, and from which media device 210 may retrieve second content item 130 .
  • the communications path 232 may be any network or communication equipment or medium by means of which media device 210 can communicate with (e.g., submit a query to and/or retrieve data or content from) content source 240 .
  • the communications path 232 includes devices connected via the Internet.
  • a content enhancement application may be implemented on any one or a combination of media device 210 , content source 240 , and/or an application server 220 , each of which may function as a stand-alone device or may be part of a network of devices.
  • Various network configurations of devices may be implemented and are discussed in more detail below.
  • system 200 there may be multiple media devices 210 , but only one is shown in FIG. 2 to avoid overcomplicating the drawing.
  • multiple users may each utilize more than one type of media device 210 and also more than one of each type of media device 210 .
  • Media device 210 may be coupled to a communication network 230 .
  • Communication network 230 may be one or more networks including the Internet, a mobile phone network, a mobile voice or data network (e.g., a 4G or LTE network), a cable network, a public switched telephone network, or other types of communication networks or combinations of communication networks.
  • Media device 210 , content source 240 , and application server 220 may be connected to the communication network 230 via one or more communications paths 232 , such as a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IP), free-space connections (e.g., for wireless signals), and/or any other suitable wired or wireless communications path or combination of such paths.
  • IP Internet protocol
  • free-space connections e.g., for wireless signals
  • communications paths 232 are not drawn directly between media device 210 and content source 240 or application server 220 , these devices may communicate directly with each other via communications paths 232 , such as short-range point-to-point communications paths, for example USB cables, IEEE 1394 cables, wireless paths (e.g., BLUETOOTH, infrared, IEEE 802-11x, etc.), or other short-range communication via wired or wireless paths.
  • BLUETOOTH is a certification mark owned by Bluetooth SIG, INC.
  • the devices may also communicate with each other directly through an indirect path via communication network 230 .
  • System 200 may include more than one content source 240 , but only one is shown in FIG. 2 to avoid overcomplicating the drawing.
  • Content source 240 includes a database, table, or other data structure, storing a list of each content item on content source 240 , and/or metadata associated with each content item.
  • the metadata may include descriptions, such as tags or identifiers describing the associated content items.
  • Content source 240 either in the database or in a separate data structure, also stores the content items referred to in the database, or stores links (e.g., hyperlinks, addresses, credentials, etc.) to access the content items referred to in the database on a separate device or server (not shown in FIG. 2 ).
  • the content enhancement application may be, for example, a stand-alone application implemented on media device 210 described above.
  • the content enhancement application may be implemented as software or a set of executable instructions, which may be stored in storage 308 (described below with reference to FIG. 3 ) and executed by control circuitry 304 (described below with reference to FIG. 3 ) of media device 210 .
  • the content enhancement application is a client/server-based application where only a client application resides on media device 210 , and a server application resides on another device, such as application server 220 .
  • the content enhancement application may be implemented partially as a client application on control circuitry 304 of media device 210 and partially on application server 220 as a server application running on control circuitry 304 of application server 220 .
  • the content enhancement application may instruct control circuitry 304 of media device 210 to transmit content item 112 and/or social media post 110 to application server 220 , which then identifies a position of non-focus portion 124 within a frame of content item 112 , determines the dimensions of non-focus portion 124 , and transmits the identified position and dimensions of non-focus portion 124 back to media device 210 .
  • Media device 210 may then query content source 240 for other content items that fit within the dimensions of non-focus portion 124 and are relevant to an audience of social media post 110 .
  • media device 210 may operate in a cloud computing environment to access cloud services.
  • cloud computing environment various types of computing services for content sharing, storage, or distribution are provided by a collection of network-accessible computing and storage resources, referred to as “the cloud.”
  • Media device 210 may be a cloud client that relies on cloud computing for application delivery, or media device 210 may have some functionality without access to cloud resources.
  • some applications running on media device 210 may be cloud applications, that is, applications delivered as a service over the Internet, while other applications may be stored and run on media device 210 .
  • media device 210 uses cloud resources for processing operations, such as the processing operations performed by processing circuitry 306 described in relation to FIG. 3 .
  • media device 210 may rely on application server 220 and/or another cloud resource to identify the position of non-focus portion 124 within content item 112 and/or determine a content preference of the audience of social media post 110 , and retrieve second content item 130 from content source 240 and/or another cloud resource.
  • FIG. 3 shows a generalized embodiment of an illustrative media device 300 .
  • media device 300 may be a smartphone or tablet.
  • Media device 300 may receive content and data via an input/output (I/O) path 302 .
  • I/O path 302 may provide user input, user preferences, social media post 110 , content item 112 , second content item 130 , and/or metadata related to any of the aforementioned to control circuitry 304 , which includes processing circuitry 306 and storage 308 .
  • Control circuitry 304 may send and receive commands, requests, and other suitable data using I/O path 302 .
  • I/O path 302 may connect control circuitry 304 (and specifically processing circuitry 306 ) to one or more communications paths (described below). I/O functions may be provided by one or more of these communications paths, but a single path is shown in FIG. 3 to avoid overcomplicating the drawing.
  • Control circuitry 304 may be based on any suitable processing circuitry, such as processing circuitry 306 .
  • processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, octa-core, or any suitable number of cores).
  • processing circuitry 306 is distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two INTEL CORE i7 processors) or multiple different processors (e.g., an INTEL CORE i5 processor and an INTEL CORE i7 processor).
  • control circuitry 304 executes instructions for a content enhancement application stored in memory (i.e., storage 308 ). Specifically, control circuitry 304 may be instructed by the content enhancement application to identify a position and dimensions of non-focus portion 124 within content item 112 , retrieve second content item 130 , generate enhanced social media post 140 , and/or perform the other functions described above and below.
  • control circuitry 304 includes communications circuitry suitable for communicating with a content enhancement application server or other networks or servers.
  • the instructions for carrying out the above-mentioned functionality may be stored on a server.
  • Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet card, and/or a wireless modem for communications with other equipment, or any other suitable communications circuitry.
  • Such communications may involve the Internet or any other suitable communication networks or paths.
  • the communications circuitry may include circuitry that enables peer-to-peer communication of media devices, or communication of media devices in locations remote from each other.
  • the memory may be an electronic storage device provided as storage 308 that is part of control circuitry 304 .
  • the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, hard drives, optical drives, solid state devices, quantum storage devices, or any other suitable fixed or removable storage devices, and/or any combination of the same.
  • Nonvolatile memory may also be used.
  • Cloud-based storage described in relation to FIG. 2 , may be used to supplement storage 308 or instead of storage 308 .
  • the circuitry described herein may execute instructions included in software running on one or more general purpose or specialized processors. Multiple circuits may be provided to handle simultaneous processing functions. If storage 308 is provided as a separate device from media device 300 , the circuitry may be associated with storage 308 .
  • a user may send instructions to control circuitry 304 using a user input interface 310 of media device 300 .
  • User input interface 310 may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, touchscreen, touchpad, stylus input, joystick, or other user input interfaces.
  • Media device 300 may further include a display 312 and speakers 314 .
  • Display 312 may be a touchscreen or touch-sensitive display. In such circumstances, user input interface 310 may be integrated with or combined with display 312 .
  • a camera, microphone, or other visual or voice recognition interface may also be used to receive user input.
  • Speakers 314 may be provided as integrated with other elements of media device 300 .
  • the content enhancement application may be implemented using any suitable architecture. For example, it may be a stand-alone application wholly implemented on media device 300 .
  • the content enhancement application and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media.
  • Computer-readable media include any media capable of having computer-executable instructions encoded thereon that, when executed by a processor, cause a device, such as media device 300 , to perform the instructed operations.
  • the content enhancement application is a client/server-based application. Data for use by a thick or thin client implemented on media device 300 is retrieved on demand by issuing requests to a server remote from media device 300 , as described above.
  • media device 210 of FIG. 2 may be implemented as media device 300 of FIG. 3 .
  • FIG. 4 is a flowchart of an illustrative process for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure.
  • a process 400 for enhancing the social media post with content that is relevant to the audience of the post may begin at block 402 , where control circuitry, such as control circuitry 304 , receives a social media post.
  • control circuitry 304 of media device 210 may receive an input of social media post 110 and/or a user selection, such as via user input interface 310 , of social media post 110 .
  • control circuitry 304 extracts, from the social media post received at block 402 , a first content item.
  • control circuitry 304 of media device 210 may extract content item 112 from social media post 110 .
  • control circuitry 304 parses the data of social media post 110 and identifies, in the data, data corresponding to content item 112 .
  • Control circuitry 304 then extracts the data corresponding to content item 112 from the data of social media post 110 .
  • control circuitry 304 identifies, in a frame of the first content item, a portion of the frame that is a non-focus portion. For example, control circuitry 304 of media device 210 may identify non-focus portion 124 in a frame of content item 112 . In some embodiments, control circuitry 304 identifies a focus portion, such as focus portion 122 , of the frame of content item 112 , and selects a portion of the frame that is not obscured by focus portion 122 as non-focus portion 124 .
  • control circuitry 304 identifies an object, in the frame of content item 112 , that extends beyond an edge of the frame, and selects a portion of the frame that includes the object as non-focus portion 124 .
  • control circuitry 304 may identify a door in the frame of content item 112 and determine that the door extends beyond an edge of the frame, and thus select the portion of the frame that includes the door as non-focus portion 124 .
  • control circuitry 304 identifies a plurality of content items that fit within the non-focus portion of the frame. For example, control circuitry 304 of media device 210 may identify a plurality of content items that fit within non-focus portion 124 . In some embodiments, control circuitry 304 determines dimensions of non-focus portion 124 and identifies a plurality of content items having dimensions that are the same as or smaller than the dimensions of non-focus portion 124 .
  • control circuitry 304 identifies a content preference of an audience of the social media post.
  • control circuitry 304 of media device 210 may identify a content preference of the audience of social media post 110 .
  • control circuitry 304 retrieves a list of followers of a user from whom social media post 110 was received, identifies a content preference shared by the followers, and selects the content preference shared by the followers as the content preference of the audience of social media post 110 .
  • Control circuitry 304 may identify the user from whom social media post 110 was received based on a device from which social media post 110 was received, and/or based on an account (such as a social network account) from which social media post 110 was received or to which social media post 110 is posted.
  • control circuitry 304 determines an estimated length of time that the audience will view the social media post. For example, control circuitry 304 of media device 210 may determine an estimated time that the audience will view social media post 110 . In some embodiments, control circuitry 304 retrieves, for some or all of the followers in the list retrieved at block 410 , at least one value corresponding to a duration that a respective follower has previously viewed a social media post, calculates an average of the values, and selects the average of the values as the estimated length of time that the audience will view social media post 110 .
  • control circuitry 304 determines whether the plurality of content items, as identified at block 408 , include a given content item that matches the content preference of the audience, as identified at block 410 , and that has a duration that does not exceed the estimated length of time, as determined at block 412 .
  • control circuitry 304 of media device 210 may identify, from the plurality of content items, a first subset of content items that match the content preference of the audience, and identify, from the first subset, a second subset of content items that has a duration that is the same as or shorter than the estimated length of time.
  • control circuitry 304 determines that the plurality of content items include at least one content item that qualifies as the given content item. In response to determining that the plurality of content items does not include the given content item, process 400 ends. In response to determining that the plurality of content items does include at least one content item that qualifies as the given content item, process 400 proceeds to block 416 .
  • control circuitry 304 selects the given content item as a second content item.
  • control circuitry 304 of media device 210 selects the given content item as second content item 130 .
  • each content item of the plurality of content items includes a plurality of frames, and control circuitry 304 selects second content item 130 from among the plurality of content items by identifying a subset of the plurality of frames, of a given content item of the plurality of content items, that include content matching the content preference of the audience, as determined at block 410 .
  • Control circuitry determines whether a segment of the given content item, the segment including the subset of the plurality of frames, has a duration that does not exceed the estimated length of time, as determined at block 412 , and in response to determining that the segment has a duration that does not exceed the estimated length of time, selects the segment as second content item 130 .
  • control circuitry 304 determines whether metadata, of a given content item of the plurality of content items, includes a description of the given content item that matches a description of content item 112 , and in response to determining that the metadata of the given content item includes the description that matches the description of content item 112 , selects the given content item as second content item 130 .
  • control circuitry 304 generates an enhanced social media post by overlaying the second content item, as selected at block 416 , onto the non-focus portion of the frame, as identified at block 406 .
  • control circuitry 304 of media device 210 may generate enhanced social media post 140 by overlaying second content item 130 onto non-focus portion 124 .
  • control circuitry 304 may place second content item 130 over non-focus portion 124 in content item 112 .
  • control circuitry 304 generates for display the enhanced social media post generated at block 418 .
  • control circuitry 304 of media device 210 generates for display enhanced social media post 140 .
  • control circuitry 304 causes display 312 of media device 210 to display enhanced social media post 140 .
  • control circuitry 304 causes enhanced media post 140 to be transmitted to a server of the social network and/or other media devices.
  • FIG. 5 is a flowchart of a detailed illustrative process for selecting a content item with which to enhance a social media post, in accordance with some embodiments of the disclosure.
  • a process 500 for selecting a content item with which to enhance a social media post may begin at block 502 .
  • control circuitry 304 identifies a subset of a plurality of frames included in a given content item of a plurality of content items that include content matching a content preference of an audience of the social media post.
  • control circuitry 304 of media device 210 may identify a subset of a plurality of frames included in a given content item of a plurality of content items that include content matching the content preference of the audience of social media post 110 , as identified at block 410 ( FIG. 4 ).
  • control circuitry 304 determines whether a segment of the given content item, the segment including the subset of the plurality of frames identified at block 504 , has a duration that does not exceed an estimated length of time that the audience will view the social media post. For example, control circuitry 304 of media device 210 may determine whether a segment of the given content item has a duration that does not exceed the estimated length of time determined at block 412 ( FIG. 4 ). In response to determining that the segment has a duration that exceeds the estimated length of time, process 500 ends. In response to determining that the segment has a duration that does not exceed the estimated length of time, process 500 proceeds to block 508 .
  • control circuitry 304 selects the segment as the content item. For example, control circuitry 304 of media device 210 selects the segment as second content item 130 .
  • a computer program product that includes a computer-usable and/or -readable medium.
  • a computer-usable medium may consist of a read-only memory device, such as a CD-ROM disk or conventional ROM device, or a random-access memory, such as a hard drive device or a computer diskette, having a computer-readable program code stored thereon.
  • a computer-usable medium may consist of a read-only memory device, such as a CD-ROM disk or conventional ROM device, or a random-access memory, such as a hard drive device or a computer diskette, having a computer-readable program code stored thereon.

Abstract

Systems and methods are described for enhancing a social media post with a content item. An illustrative method includes receiving a social media post; extracting, from the social media post, a first content item; identifying, in a frame of the first content item, a portion of the frame that is a non-focus portion; identifying a plurality of content items that fit within the non-focus portion; identifying a content preference of an audience of the social media post; determining an estimated length of time that the audience will view the social media post; selecting a second content item, from the plurality of content items, that matches the content preference of the audience and has a duration that does not exceed the estimated length of time; generating an enhanced social media post by overlaying the second content item onto the non-focus portion; and generating for display the enhanced social media post.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation of U.S. patent application Ser. No. 17/170,365, filed Feb. 8, 2021, which is a continuation of U.S. patent application Ser. No. 16/454,627, filed Jun. 27, 2019, now U.S. Pat. No. 10,951,563, which are hereby incorporated by reference in their entireties.
BACKGROUND
The present disclosure relates to modifying a portion of a social media post, and more particularly, to methods and systems for enhancing the social media post with content that is relevant to the audience of the post.
SUMMARY
The development of social media, and particularly the ability for users to share content (e.g., still images, graphics interchange format (GIF) or similar format images, videos, etc.) with their followers on social networks, have resulted in an immense amount of content being generated and posted to such social networks every day. Much of the content generated typically has focus portions (i.e., the portion(s) of the content including the subject of the content and where the audience's attention is intended to be attracted) and non-focus portions (i.e., the portion(s) of the content that does not include the subject and/or includes the background or other objects that are not intended to draw the audience's attention). Some users, for example, social media influencers, may wish to attract more attention to their posts in order to increase their fame and/or have their posts “liked” by more followers to thereby further increase their audience. Such users may wish to add additional content that is relevant to the audience of their social media posts.
On the other side, the operators of the social networks often allow the users to create accounts and post content to the social networks free of charge. The social networks instead derive their revenue from advertisements placed on their websites and applications (apps), and/or interspersed with the posts in the users' social media feeds. While this strategy places the advertisements in front of the users, the users often just scroll or swipe past the advertisements to the posts that attract their attention. Thus, the advertisements are often ignored by the users or given less than a second of time in front of the users to attract the users' attention before being skipped.
To overcome such problems, methods and systems are described herein for enhancing a social media post with a content item. In particular, the embodiments described herein provide for receiving a social media post and extracting, from the social media post, a first content item (e.g., a still image). The methods and systems then identify, in a frame of the first content item, a portion of the frame that is a non-focus portion (e.g., a background portion), and identify a plurality of content items that fit within the non-focus portion of the frame. In order to target the added content to the audience of the social media post, the methods and systems identify a content preference of an audience of the social media post and determine an estimated length of time that the audience will view the social media post. The methods and systems then select a second content item, from the plurality of content items, that matches the content preference of the audience and has a duration that does not exceed the estimated length of time. Thereafter, the methods and systems generate an enhanced social media post by overlaying the second content item onto the non-focus portion of the frame and display the enhanced social media post.
In some embodiments, the methods and systems identify a focus portion of the frame and a background portion of the frame and select a portion of the frame that is not obscured by the focus portion of the frame as the non-focus portion of the frame. The methods and systems may further identify an object in the frame that extends beyond an edge of the frame and select a portion of the frame that includes the object as the non-focus portion of the frame. The methods and systems may also determine dimensions of the non-focus portion of the frame and identify a plurality of content items having dimensions that are the same as or smaller than the dimensions of the non-focus portion of the frame. In further embodiments, the methods and systems retrieve a list of followers of a user from whom the social media post was received, identify a content preference shared by the followers, and select the content preference shared by the followers as the content preference of the audience of the social media post. In still further embodiments, the methods and systems retrieve, for each of the followers, at least one value corresponding to a duration that a respective follower has previously viewed a social media post, calculate an average of the values, and select the average of the values as the estimated length of time that the audience will view the social media post.
The methods and systems described herein provide an improved process for identifying content that is relevant to the audience of a social media post and generating an enhanced social media post including the original content of the post and the additional content that is relevant to the audience. The process increases the efficiency of adding additional content relevant to an audience of a social media post to content included in the post, both for the user and for the computing resources needed to perform the processes described herein. To that end, the methods and systems provide for selecting additional content that is both relevant to the audience of the post and that fits into non-focus portions of the content of the post without obscuring the portion(s) of the post to which the user intended to attract the audience's attention.
BRIEF DESCRIPTION OF THE DRAWINGS
The above and other objects and advantages of the disclosure will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
FIG. 1 shows an illustrative scenario for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure;
FIG. 2 shows an illustrative system for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure;
FIG. 3 shows a diagram of illustrative devices of the system of FIG. 2 , in accordance with some embodiments of the disclosure;
FIGS. 4A and 4B show a flowchart of an illustrative process for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure; and
FIG. 5 is a flowchart of a detailed illustrative process for selecting a content item with which to enhance a social media post, in accordance with some embodiments of the disclosure.
DETAILED DESCRIPTION
FIG. 1 shows an illustrative scenario 100 for enhancing the social media post with content that is relevant to the audience of the post. As shown in FIG. 1 , a social media post 110 includes a content item 112 and may optionally include a title 114 and message 116. Social media post 110 further includes metadata (not shown in FIG. 1 ) describing social media post 110 and/or content item 112. For example, the metadata may include tags or identifiers describing content 112. In the example shown in FIG. 1 , the metadata may include the tags “father,” “son,” “music,” “violin,” “lesson,” and “teach.” The tags may be determined based on content 112, such as via image processing and object recognition techniques, and/or based on title 114 and message 116. Content item 112 may be a still image, a graphics interchange format (GIF) or similar format image, a video, and/or any other type of visual content that may be included in a social media post. Content item 112 includes at least one focus portion 122 and at least one non-focus portion 124. Focus portion 122 is a portion of content item 112 that includes the subject of the content and the area where an audience of the social media post is intended to focus their attention). Non-focus portion 124 is a portion of content item 112 that does not include the subject and/or includes the background or other objects that are not intended to draw the audience's attention.
As described further below with reference to FIGS. 4A, 4B, and 5 , content item 112 is enhanced by overlaying a second content item 130 onto non-focus portion 124 to generate an enhanced social media post 140. Second content item 130 may be all or a portion of a video, a still image, an advertisement, a GIF, etc. In some embodiments, second content item 130 is a content item that is relevant to content item 112. For example, in the illustration shown in FIG. 1 , content item 112 is video of a father and son playing violin, and second content item 130 is a video of an orchestra playing the same musical composition as the father and son. In another example (not shown in FIG. 1 ), second content item 130 could be an image of the sheet music of the musical composition played by the father and son. In yet another example (not shown in FIG. 1 ), second content item 130 could be an advertisement for violin lessons. In other embodiments, second content item 130 is unrelated to content item 112 and is selected purely based on the content preference of the audience of social media post 110, as described further below.
FIG. 2 shows an illustrative system for enhancing the social media post with content that is relevant to the audience of the post. In particular, FIG. 2 shows a system 200 where a media device 210 retrieves second content item 130 and/or the metadata associated with second content item 130 from a content source 240 via a communications path 232. Content source 240 may be any server or other computing device from which content items (such as second content item 130) and/or related metadata associated with content items may be retrieved. For example, content source 240 may be a cloud storage platform by means of which content items are collected, stored, and distributed, and from which media device 210 may retrieve second content item 130. In that regard, the communications path 232 may be any network or communication equipment or medium by means of which media device 210 can communicate with (e.g., submit a query to and/or retrieve data or content from) content source 240. In one illustrative example, the communications path 232 includes devices connected via the Internet.
A content enhancement application may be implemented on any one or a combination of media device 210, content source 240, and/or an application server 220, each of which may function as a stand-alone device or may be part of a network of devices. Various network configurations of devices may be implemented and are discussed in more detail below. In system 200, there may be multiple media devices 210, but only one is shown in FIG. 2 to avoid overcomplicating the drawing. In addition, multiple users may each utilize more than one type of media device 210 and also more than one of each type of media device 210.
Media device 210 may be coupled to a communication network 230. Communication network 230 may be one or more networks including the Internet, a mobile phone network, a mobile voice or data network (e.g., a 4G or LTE network), a cable network, a public switched telephone network, or other types of communication networks or combinations of communication networks. Media device 210, content source 240, and application server 220 may be connected to the communication network 230 via one or more communications paths 232, such as a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IP), free-space connections (e.g., for wireless signals), and/or any other suitable wired or wireless communications path or combination of such paths.
Although communications paths 232 are not drawn directly between media device 210 and content source 240 or application server 220, these devices may communicate directly with each other via communications paths 232, such as short-range point-to-point communications paths, for example USB cables, IEEE 1394 cables, wireless paths (e.g., BLUETOOTH, infrared, IEEE 802-11x, etc.), or other short-range communication via wired or wireless paths. BLUETOOTH is a certification mark owned by Bluetooth SIG, INC. The devices may also communicate with each other directly through an indirect path via communication network 230.
System 200 may include more than one content source 240, but only one is shown in FIG. 2 to avoid overcomplicating the drawing. Content source 240 includes a database, table, or other data structure, storing a list of each content item on content source 240, and/or metadata associated with each content item. The metadata may include descriptions, such as tags or identifiers describing the associated content items. Content source 240, either in the database or in a separate data structure, also stores the content items referred to in the database, or stores links (e.g., hyperlinks, addresses, credentials, etc.) to access the content items referred to in the database on a separate device or server (not shown in FIG. 2 ).
The content enhancement application may be, for example, a stand-alone application implemented on media device 210 described above. For example, the content enhancement application may be implemented as software or a set of executable instructions, which may be stored in storage 308 (described below with reference to FIG. 3 ) and executed by control circuitry 304 (described below with reference to FIG. 3 ) of media device 210. In some embodiments, the content enhancement application is a client/server-based application where only a client application resides on media device 210, and a server application resides on another device, such as application server 220. For example, the content enhancement application may be implemented partially as a client application on control circuitry 304 of media device 210 and partially on application server 220 as a server application running on control circuitry 304 of application server 220. When executed by control circuitry 304 of application server 220, the content enhancement application may instruct control circuitry 304 of media device 210 to transmit content item 112 and/or social media post 110 to application server 220, which then identifies a position of non-focus portion 124 within a frame of content item 112, determines the dimensions of non-focus portion 124, and transmits the identified position and dimensions of non-focus portion 124 back to media device 210. Media device 210 may then query content source 240 for other content items that fit within the dimensions of non-focus portion 124 and are relevant to an audience of social media post 110.
While system 200 is shown in FIG. 2 as including only media device 210, content source 240, and application server 220, those skilled in the art will appreciate that system 200 may further include various other devices. In some embodiments, media device 210 may operate in a cloud computing environment to access cloud services. In a cloud computing environment, various types of computing services for content sharing, storage, or distribution are provided by a collection of network-accessible computing and storage resources, referred to as “the cloud.” Media device 210 may be a cloud client that relies on cloud computing for application delivery, or media device 210 may have some functionality without access to cloud resources. For example, some applications running on media device 210, such as the content enhancement application, may be cloud applications, that is, applications delivered as a service over the Internet, while other applications may be stored and run on media device 210. In some embodiments, media device 210 uses cloud resources for processing operations, such as the processing operations performed by processing circuitry 306 described in relation to FIG. 3 . For example, media device 210 may rely on application server 220 and/or another cloud resource to identify the position of non-focus portion 124 within content item 112 and/or determine a content preference of the audience of social media post 110, and retrieve second content item 130 from content source 240 and/or another cloud resource.
FIG. 3 shows a generalized embodiment of an illustrative media device 300. As depicted, media device 300 may be a smartphone or tablet. Media device 300 may receive content and data via an input/output (I/O) path 302. I/O path 302 may provide user input, user preferences, social media post 110, content item 112, second content item 130, and/or metadata related to any of the aforementioned to control circuitry 304, which includes processing circuitry 306 and storage 308. Control circuitry 304 may send and receive commands, requests, and other suitable data using I/O path 302. I/O path 302 may connect control circuitry 304 (and specifically processing circuitry 306) to one or more communications paths (described below). I/O functions may be provided by one or more of these communications paths, but a single path is shown in FIG. 3 to avoid overcomplicating the drawing.
Control circuitry 304 may be based on any suitable processing circuitry, such as processing circuitry 306. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, octa-core, or any suitable number of cores). In some embodiments, processing circuitry 306 is distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two INTEL CORE i7 processors) or multiple different processors (e.g., an INTEL CORE i5 processor and an INTEL CORE i7 processor). In some embodiments, control circuitry 304 executes instructions for a content enhancement application stored in memory (i.e., storage 308). Specifically, control circuitry 304 may be instructed by the content enhancement application to identify a position and dimensions of non-focus portion 124 within content item 112, retrieve second content item 130, generate enhanced social media post 140, and/or perform the other functions described above and below.
In client/server-based embodiments, control circuitry 304 includes communications circuitry suitable for communicating with a content enhancement application server or other networks or servers. The instructions for carrying out the above-mentioned functionality may be stored on a server. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet card, and/or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communication networks or paths. In addition, the communications circuitry may include circuitry that enables peer-to-peer communication of media devices, or communication of media devices in locations remote from each other.
The memory may be an electronic storage device provided as storage 308 that is part of control circuitry 304. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, hard drives, optical drives, solid state devices, quantum storage devices, or any other suitable fixed or removable storage devices, and/or any combination of the same. Nonvolatile memory may also be used. Cloud-based storage, described in relation to FIG. 2 , may be used to supplement storage 308 or instead of storage 308.
The circuitry described herein may execute instructions included in software running on one or more general purpose or specialized processors. Multiple circuits may be provided to handle simultaneous processing functions. If storage 308 is provided as a separate device from media device 300, the circuitry may be associated with storage 308.
A user may send instructions to control circuitry 304 using a user input interface 310 of media device 300. User input interface 310 may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, touchscreen, touchpad, stylus input, joystick, or other user input interfaces. Media device 300 may further include a display 312 and speakers 314. Display 312 may be a touchscreen or touch-sensitive display. In such circumstances, user input interface 310 may be integrated with or combined with display 312. A camera, microphone, or other visual or voice recognition interface may also be used to receive user input. Speakers 314 may be provided as integrated with other elements of media device 300.
The content enhancement application may be implemented using any suitable architecture. For example, it may be a stand-alone application wholly implemented on media device 300. The content enhancement application and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media. Computer-readable media include any media capable of having computer-executable instructions encoded thereon that, when executed by a processor, cause a device, such as media device 300, to perform the instructed operations. In some embodiments, the content enhancement application is a client/server-based application. Data for use by a thick or thin client implemented on media device 300 is retrieved on demand by issuing requests to a server remote from media device 300, as described above. Those skilled in the art will appreciate that media device 210 of FIG. 2 may be implemented as media device 300 of FIG. 3 .
FIG. 4 is a flowchart of an illustrative process for enhancing the social media post with content that is relevant to the audience of the post, in accordance with some embodiments of the disclosure. A process 400 for enhancing the social media post with content that is relevant to the audience of the post may begin at block 402, where control circuitry, such as control circuitry 304, receives a social media post. For example, control circuitry 304 of media device 210 may receive an input of social media post 110 and/or a user selection, such as via user input interface 310, of social media post 110.
At block 404, control circuitry 304 extracts, from the social media post received at block 402, a first content item. For example, control circuitry 304 of media device 210 may extract content item 112 from social media post 110. In some embodiments, control circuitry 304 parses the data of social media post 110 and identifies, in the data, data corresponding to content item 112. Control circuitry 304 then extracts the data corresponding to content item 112 from the data of social media post 110.
At block 406, control circuitry 304 identifies, in a frame of the first content item, a portion of the frame that is a non-focus portion. For example, control circuitry 304 of media device 210 may identify non-focus portion 124 in a frame of content item 112. In some embodiments, control circuitry 304 identifies a focus portion, such as focus portion 122, of the frame of content item 112, and selects a portion of the frame that is not obscured by focus portion 122 as non-focus portion 124. In another embodiment, control circuitry 304 identifies an object, in the frame of content item 112, that extends beyond an edge of the frame, and selects a portion of the frame that includes the object as non-focus portion 124. For example, in the illustration shown in FIG. 1 , control circuitry 304 may identify a door in the frame of content item 112 and determine that the door extends beyond an edge of the frame, and thus select the portion of the frame that includes the door as non-focus portion 124.
At block 408, control circuitry 304 identifies a plurality of content items that fit within the non-focus portion of the frame. For example, control circuitry 304 of media device 210 may identify a plurality of content items that fit within non-focus portion 124. In some embodiments, control circuitry 304 determines dimensions of non-focus portion 124 and identifies a plurality of content items having dimensions that are the same as or smaller than the dimensions of non-focus portion 124.
At block 410, control circuitry 304 identifies a content preference of an audience of the social media post. For example, control circuitry 304 of media device 210 may identify a content preference of the audience of social media post 110. In some embodiments, control circuitry 304 retrieves a list of followers of a user from whom social media post 110 was received, identifies a content preference shared by the followers, and selects the content preference shared by the followers as the content preference of the audience of social media post 110. Control circuitry 304 may identify the user from whom social media post 110 was received based on a device from which social media post 110 was received, and/or based on an account (such as a social network account) from which social media post 110 was received or to which social media post 110 is posted.
At block 412, control circuitry 304 determines an estimated length of time that the audience will view the social media post. For example, control circuitry 304 of media device 210 may determine an estimated time that the audience will view social media post 110. In some embodiments, control circuitry 304 retrieves, for some or all of the followers in the list retrieved at block 410, at least one value corresponding to a duration that a respective follower has previously viewed a social media post, calculates an average of the values, and selects the average of the values as the estimated length of time that the audience will view social media post 110.
At block 414, control circuitry 304 determines whether the plurality of content items, as identified at block 408, include a given content item that matches the content preference of the audience, as identified at block 410, and that has a duration that does not exceed the estimated length of time, as determined at block 412. For example, control circuitry 304 of media device 210 may identify, from the plurality of content items, a first subset of content items that match the content preference of the audience, and identify, from the first subset, a second subset of content items that has a duration that is the same as or shorter than the estimated length of time. If the second subset of content items includes at least one content item, control circuitry 304 determines that the plurality of content items include at least one content item that qualifies as the given content item. In response to determining that the plurality of content items does not include the given content item, process 400 ends. In response to determining that the plurality of content items does include at least one content item that qualifies as the given content item, process 400 proceeds to block 416.
At block 416, control circuitry 304 selects the given content item as a second content item. For example, control circuitry 304 of media device 210 selects the given content item as second content item 130. In some embodiments, each content item of the plurality of content items includes a plurality of frames, and control circuitry 304 selects second content item 130 from among the plurality of content items by identifying a subset of the plurality of frames, of a given content item of the plurality of content items, that include content matching the content preference of the audience, as determined at block 410. Control circuitry then determines whether a segment of the given content item, the segment including the subset of the plurality of frames, has a duration that does not exceed the estimated length of time, as determined at block 412, and in response to determining that the segment has a duration that does not exceed the estimated length of time, selects the segment as second content item 130. In another embodiment, control circuitry 304 determines whether metadata, of a given content item of the plurality of content items, includes a description of the given content item that matches a description of content item 112, and in response to determining that the metadata of the given content item includes the description that matches the description of content item 112, selects the given content item as second content item 130.
At block 418, control circuitry 304 generates an enhanced social media post by overlaying the second content item, as selected at block 416, onto the non-focus portion of the frame, as identified at block 406. For example, control circuitry 304 of media device 210 may generate enhanced social media post 140 by overlaying second content item 130 onto non-focus portion 124. In some embodiments, control circuitry 304 may place second content item 130 over non-focus portion 124 in content item 112.
At block 420, control circuitry 304 generates for display the enhanced social media post generated at block 418. For example, control circuitry 304 of media device 210 generates for display enhanced social media post 140. In some embodiments, control circuitry 304 causes display 312 of media device 210 to display enhanced social media post 140. In other embodiments, control circuitry 304 causes enhanced media post 140 to be transmitted to a server of the social network and/or other media devices.
FIG. 5 is a flowchart of a detailed illustrative process for selecting a content item with which to enhance a social media post, in accordance with some embodiments of the disclosure. A process 500 for selecting a content item with which to enhance a social media post may begin at block 502. At block 504, control circuitry 304 identifies a subset of a plurality of frames included in a given content item of a plurality of content items that include content matching a content preference of an audience of the social media post. For example, control circuitry 304 of media device 210 may identify a subset of a plurality of frames included in a given content item of a plurality of content items that include content matching the content preference of the audience of social media post 110, as identified at block 410 (FIG. 4 ).
At block 506, control circuitry 304 determines whether a segment of the given content item, the segment including the subset of the plurality of frames identified at block 504, has a duration that does not exceed an estimated length of time that the audience will view the social media post. For example, control circuitry 304 of media device 210 may determine whether a segment of the given content item has a duration that does not exceed the estimated length of time determined at block 412 (FIG. 4 ). In response to determining that the segment has a duration that exceeds the estimated length of time, process 500 ends. In response to determining that the segment has a duration that does not exceed the estimated length of time, process 500 proceeds to block 508.
At block 508, control circuitry 304 selects the segment as the content item. For example, control circuitry 304 of media device 210 selects the segment as second content item 130.
It will be apparent to those of ordinary skill in the art that methods involved in the present invention may be embodied in a computer program product that includes a computer-usable and/or -readable medium. For example, such a computer-usable medium may consist of a read-only memory device, such as a CD-ROM disk or conventional ROM device, or a random-access memory, such as a hard drive device or a computer diskette, having a computer-readable program code stored thereon. It should also be understood that methods, techniques, and processes involved in the present disclosure may be executed using processing circuitry.
The processes discussed above are intended to be illustrative and not limiting. More generally, the above disclosure is meant to be exemplary and not limiting. Only the claims that follow are meant to set bounds as to what the present invention includes. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. It should also be noted, the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods.

Claims (20)

What is claimed is:
1. A method comprising:
determining first metadata of a first content item;
determining second metadata of a second content item;
identifying, in at least one frame of the first content item, a portion of the at least one frame;
determining a relationship between one or more portions of the first metadata and one or more portions of the second metadata;
generating an enhanced content item by integrating one or more determined related portions of the second content item into the portion of the at least one frame of the first content item; and
generating for display the enhanced content item.
2. The method of claim 1, further comprising targeting display of the enhanced content item to at least one entity based on the determined relationship.
3. The method of claim 1, further comprising:
determining user or account metadata of a user or account accessing a device generating and/or displaying the first content item; and
determining the relationship between one or more portions of the user or account metadata and the one or more portions of the first metadata and/or the one or more portions of the second metadata.
4. The method of claim 1, further comprising determining a preference of at least one entity.
5. The method of claim 4, wherein the preference is a shared preference of a group of entities previously or currently consuming the first content item, the method further comprising:
targeting display of the enhanced content item to the group of entities having the shared preference.
6. The method of claim 1, further comprising determining a relevance of one of the first content item to the second content item or the second content item to the first content item.
7. The method of claim 1, further comprising generating the enhanced content item in a social media post.
8. The method of claim 7, further comprising determining one or more objects in the first content item that are intended and/or not intended to draw attention.
9. A system comprising:
control circuitry configured to:
determine first metadata of a first content item;
determine second metadata of a second content item;
identify, in at least one frame of the first content item, a portion of the at least one frame;
determine a relationship between one or more portions of the first metadata and one or more portions of the second metadata;
generate an enhanced content item by integrating one or more determined related portions of the second content item into the portion of the at least one frame of the first content item; and
generate for display the enhanced content item.
10. The system of claim 9, the control circuitry further configured to target display of the enhanced content item to at least one entity based on the determined relationship.
11. The system of claim 9, the control circuitry further configured to:
determine user or account metadata of a user or account accessing a device generating and/or displaying the first content item; and
determine the relationship between one or more portions of the user or account metadata and the one or more portions of the first metadata and/or the one or more portions of the second metadata.
12. The system of claim 9, the control circuitry further configured to determine a preference of at least one entity.
13. The system of claim 12, wherein the preference is a shared preference of a group of entities previously or currently consuming the first content item, the control circuitry further configured to:
target display of the enhanced content item to the group of entities having the shared preference.
14. The system of claim 9, the control circuitry further configured to determine a relevance of one of the first content item to the second content item or the second content item to the first content item.
15. The system of claim 9, the control circuitry further configured to generate the enhanced content item in a social media post.
16. The system of claim 9, the control circuitry further configured to determine one or more objects in the first content item that are intended and/or not intended to draw attention.
17. A non-transitory computer-readable medium having instructions encoded thereon that, when executed by control circuitry, cause the control circuitry to:
determine first metadata of a first content item;
determine second metadata of a second content item;
identify, in at least one frame of the first content item, a portion of the at least one frame;
determine a relationship between one or more portions of the first metadata and one or more portions of the second metadata;
generate an enhanced content item by integrating one or more determined related portions of the second content item into the portion of the at least one frame of the first content item; and
generate for display the enhanced content item.
18. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the control circuitry to:
target display of the enhanced content item to at least one entity based on the determined relationship.
19. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the control circuitry to:
determine user or account metadata of a user or account accessing a device generating and/or displaying the first content item; and
determine the relationship between one or more portions of the user or account metadata and the one or more portions of the first metadata and/or the one or more portions of the second metadata.
20. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the control circuitry to perform one or more of the following:
determine a preference of at least one entity, wherein the preference is a shared preference of a group of entities previously or currently consuming the first content item;
target display of the enhanced content item to the group of entities having the shared preference;
determine a relevance of one of the first content item to the second content item or the second content item to the first content item;
generate the enhanced content item in a social media post; and
determine one or more objects in the first content item that are intended and/or not intended to draw attention.
US17/715,654 2019-06-27 2022-04-07 Enhancing a social media post with content that is relevant to the audience of the post Active US11683285B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/715,654 US11683285B2 (en) 2019-06-27 2022-04-07 Enhancing a social media post with content that is relevant to the audience of the post
US18/195,264 US20230275859A1 (en) 2019-06-27 2023-05-09 Enhancing a social media post with content that is relevant to the audience of the post

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/454,627 US10951563B2 (en) 2019-06-27 2019-06-27 Enhancing a social media post with content that is relevant to the audience of the post
US17/170,365 US11329937B2 (en) 2019-06-27 2021-02-08 Identifying non-focus area of content item and overlaying another content item onto the non-focus area
US17/715,654 US11683285B2 (en) 2019-06-27 2022-04-07 Enhancing a social media post with content that is relevant to the audience of the post

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US17/170,365 Continuation US11329937B2 (en) 2019-06-27 2021-02-08 Identifying non-focus area of content item and overlaying another content item onto the non-focus area

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/195,264 Continuation US20230275859A1 (en) 2019-06-27 2023-05-09 Enhancing a social media post with content that is relevant to the audience of the post

Publications (2)

Publication Number Publication Date
US20220231980A1 US20220231980A1 (en) 2022-07-21
US11683285B2 true US11683285B2 (en) 2023-06-20

Family

ID=74043356

Family Applications (4)

Application Number Title Priority Date Filing Date
US16/454,627 Active US10951563B2 (en) 2019-06-27 2019-06-27 Enhancing a social media post with content that is relevant to the audience of the post
US17/170,365 Active US11329937B2 (en) 2019-06-27 2021-02-08 Identifying non-focus area of content item and overlaying another content item onto the non-focus area
US17/715,654 Active US11683285B2 (en) 2019-06-27 2022-04-07 Enhancing a social media post with content that is relevant to the audience of the post
US18/195,264 Pending US20230275859A1 (en) 2019-06-27 2023-05-09 Enhancing a social media post with content that is relevant to the audience of the post

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US16/454,627 Active US10951563B2 (en) 2019-06-27 2019-06-27 Enhancing a social media post with content that is relevant to the audience of the post
US17/170,365 Active US11329937B2 (en) 2019-06-27 2021-02-08 Identifying non-focus area of content item and overlaying another content item onto the non-focus area

Family Applications After (1)

Application Number Title Priority Date Filing Date
US18/195,264 Pending US20230275859A1 (en) 2019-06-27 2023-05-09 Enhancing a social media post with content that is relevant to the audience of the post

Country Status (1)

Country Link
US (4) US10951563B2 (en)

Citations (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5953076A (en) 1995-06-16 1999-09-14 Princeton Video Image, Inc. System and method of real time insertions into video using adaptive occlusion with a synthetic reference image
US6392664B1 (en) 1998-11-30 2002-05-21 Webtv Networks, Inc. Method and system for presenting television programming and interactive entertainment
US20050031169A1 (en) 2003-08-09 2005-02-10 Alan Shulman Birds eye view virtual imaging for real time composited wide field of view
US20060026628A1 (en) 2004-07-30 2006-02-02 Kong Wah Wan Method and apparatus for insertion of additional content into video
US7015978B2 (en) 1999-12-13 2006-03-21 Princeton Video Image, Inc. System and method for real time insertion into video with occlusion on areas containing multiple colors
US7228280B1 (en) * 1997-04-15 2007-06-05 Gracenote, Inc. Finding database match for file based on file characteristics
US20080040474A1 (en) 2006-08-11 2008-02-14 Mark Zuckerberg Systems and methods for providing dynamically selected media content to a user of an electronic device in a social network environment
US20080040673A1 (en) 2006-08-11 2008-02-14 Mark Zuckerberg System and method for dynamically providing a news feed about a user of a social network
US20080040370A1 (en) 2006-08-11 2008-02-14 Andrew Bosworth Systems and methods for generating dynamic relationship-based content personalized for members of a web-based social network
US20090122196A1 (en) * 2007-11-12 2009-05-14 Cyberlink Corp. Systems and methods for associating metadata with scenes in a video
US20110178854A1 (en) 2008-09-04 2011-07-21 Somertech Ltd. Method and system for enhancing and/or monitoring visual content and method and/or system for adding a dynamic layer to visual content
US20110188836A1 (en) 2008-05-28 2011-08-04 Mirriad Limited Apparatus and Method for Identifying Insertion Zones in Video Material and for Inserting Additional Material into the Insertion Zones
US8024342B2 (en) * 2008-07-31 2011-09-20 Yahoo! Inc. Systems and methods for determining a tag match ratio
US20110321084A1 (en) 2010-06-25 2011-12-29 Kddi Corporation Apparatus and method for optimizing on-screen location of additional content overlay on video content
US8266115B1 (en) * 2011-01-14 2012-09-11 Google Inc. Identifying duplicate electronic content based on metadata
US8264544B1 (en) 2006-11-03 2012-09-11 Keystream Corporation Automated content insertion into video scene
US8281334B2 (en) 2008-03-31 2012-10-02 Microsoft Corporation Facilitating advertisement placement over video content
US8369686B2 (en) 2009-09-30 2013-02-05 Microsoft Corporation Intelligent overlay for video advertising
US8451380B2 (en) 2007-03-22 2013-05-28 Sony Computer Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
US20130201328A1 (en) 2012-02-08 2013-08-08 Hing Ping Michael CHUNG Multimedia processing as a service
US8654255B2 (en) 2007-09-20 2014-02-18 Microsoft Corporation Advertisement insertion points detection for online video advertising
US20140067998A1 (en) 2012-08-31 2014-03-06 David Harry Garcia Sharing Television and Video Programming Through Social Networking
US8903909B1 (en) 2011-09-15 2014-12-02 Google Inc. Detecting and extending engagement with stream content
US20150039603A1 (en) 2013-08-02 2015-02-05 Microsoft Corporation Social snippet augmenting
US9111287B2 (en) 2009-09-30 2015-08-18 Microsoft Technology Licensing, Llc Video content-aware advertisement placement
US20160066024A1 (en) 2014-07-23 2016-03-03 Comigo Ltd. Reducing interference of an overlay with underlying content
US9432703B2 (en) 2014-11-17 2016-08-30 TCL Research America Inc. Method and system for inserting contents into video presentations
US20170279757A1 (en) * 2016-03-22 2017-09-28 Facebook, Inc. Systems and methods for identifying matching content
US20180077455A1 (en) 2016-09-14 2018-03-15 International Business Machines Corporation Attentiveness-based video presentation management
US10121513B2 (en) 2016-08-30 2018-11-06 International Business Machines Corporation Dynamic image content overlaying
US20190146965A1 (en) 2017-11-10 2019-05-16 Our.News Holdings, Inc. Crowdsourced validation of electronic content
US10332159B2 (en) 2014-01-21 2019-06-25 Eleven Street Co., Ltd. Apparatus and method for providing virtual advertisement
US20200028924A1 (en) 2018-07-19 2020-01-23 International Business Machines Corporation Cognitive insight into user activity
US10546318B2 (en) 2013-06-27 2020-01-28 Intel Corporation Adaptively embedding visual advertising content into media content
US10592752B2 (en) 2017-03-24 2020-03-17 Mirriad Advertising Plc Predicting future insertion zone metadata
US20200125671A1 (en) 2018-10-17 2020-04-23 International Business Machines Corporation Altering content based on machine-learned topics of interest
US11044521B2 (en) 2017-12-08 2021-06-22 Google Llc Modifying digital video content
US11076200B2 (en) 2016-12-13 2021-07-27 Rovi Guides, Inc. Systems and methods for minimizing obstruction of a media asset by an overlay by predicting a path of movement of an object of interest of the media asset and avoiding placement of the overlay in the path of movement
US11102549B2 (en) 2017-09-27 2021-08-24 Wurl Inc. Selective video overlay

Patent Citations (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5953076A (en) 1995-06-16 1999-09-14 Princeton Video Image, Inc. System and method of real time insertions into video using adaptive occlusion with a synthetic reference image
US7228280B1 (en) * 1997-04-15 2007-06-05 Gracenote, Inc. Finding database match for file based on file characteristics
US6392664B1 (en) 1998-11-30 2002-05-21 Webtv Networks, Inc. Method and system for presenting television programming and interactive entertainment
US7015978B2 (en) 1999-12-13 2006-03-21 Princeton Video Image, Inc. System and method for real time insertion into video with occlusion on areas containing multiple colors
US20050031169A1 (en) 2003-08-09 2005-02-10 Alan Shulman Birds eye view virtual imaging for real time composited wide field of view
US20060026628A1 (en) 2004-07-30 2006-02-02 Kong Wah Wan Method and apparatus for insertion of additional content into video
US20080040474A1 (en) 2006-08-11 2008-02-14 Mark Zuckerberg Systems and methods for providing dynamically selected media content to a user of an electronic device in a social network environment
US20080040673A1 (en) 2006-08-11 2008-02-14 Mark Zuckerberg System and method for dynamically providing a news feed about a user of a social network
US20080040370A1 (en) 2006-08-11 2008-02-14 Andrew Bosworth Systems and methods for generating dynamic relationship-based content personalized for members of a web-based social network
US8264544B1 (en) 2006-11-03 2012-09-11 Keystream Corporation Automated content insertion into video scene
US8665373B2 (en) 2007-03-22 2014-03-04 Sony Computer Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
US9237258B2 (en) 2007-03-22 2016-01-12 Sony Computer Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
US8451380B2 (en) 2007-03-22 2013-05-28 Sony Computer Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
US9538049B2 (en) 2007-03-22 2017-01-03 Sony Interactive Entertainment America Llc Scheme for determining the locations and timing of advertisements and other insertions in media
US8654255B2 (en) 2007-09-20 2014-02-18 Microsoft Corporation Advertisement insertion points detection for online video advertising
US20090122196A1 (en) * 2007-11-12 2009-05-14 Cyberlink Corp. Systems and methods for associating metadata with scenes in a video
US8281334B2 (en) 2008-03-31 2012-10-02 Microsoft Corporation Facilitating advertisement placement over video content
US20110188836A1 (en) 2008-05-28 2011-08-04 Mirriad Limited Apparatus and Method for Identifying Insertion Zones in Video Material and for Inserting Additional Material into the Insertion Zones
US9477965B2 (en) 2008-05-28 2016-10-25 Mirriad Advertising Limited Apparatus and method for identifying insertion zones in video material and for inserting additional material into the insertion zones
US8024342B2 (en) * 2008-07-31 2011-09-20 Yahoo! Inc. Systems and methods for determining a tag match ratio
US20110178854A1 (en) 2008-09-04 2011-07-21 Somertech Ltd. Method and system for enhancing and/or monitoring visual content and method and/or system for adding a dynamic layer to visual content
US9111287B2 (en) 2009-09-30 2015-08-18 Microsoft Technology Licensing, Llc Video content-aware advertisement placement
US8369686B2 (en) 2009-09-30 2013-02-05 Microsoft Corporation Intelligent overlay for video advertising
US8849063B2 (en) 2010-06-25 2014-09-30 Kddi Corporation Apparatus and method for optimizing on-screen location of additional content overlay on video content
US20110321084A1 (en) 2010-06-25 2011-12-29 Kddi Corporation Apparatus and method for optimizing on-screen location of additional content overlay on video content
US8266115B1 (en) * 2011-01-14 2012-09-11 Google Inc. Identifying duplicate electronic content based on metadata
US8903909B1 (en) 2011-09-15 2014-12-02 Google Inc. Detecting and extending engagement with stream content
US20130201328A1 (en) 2012-02-08 2013-08-08 Hing Ping Michael CHUNG Multimedia processing as a service
US20140067998A1 (en) 2012-08-31 2014-03-06 David Harry Garcia Sharing Television and Video Programming Through Social Networking
US11151606B2 (en) 2013-06-27 2021-10-19 Intel Corporation Adaptively embedding visual advertising content into media content
US10546318B2 (en) 2013-06-27 2020-01-28 Intel Corporation Adaptively embedding visual advertising content into media content
US20150039603A1 (en) 2013-08-02 2015-02-05 Microsoft Corporation Social snippet augmenting
US10332159B2 (en) 2014-01-21 2019-06-25 Eleven Street Co., Ltd. Apparatus and method for providing virtual advertisement
US9432722B2 (en) 2014-07-23 2016-08-30 Comigo Ltd. Reducing interference of an overlay with underlying content
US20160066024A1 (en) 2014-07-23 2016-03-03 Comigo Ltd. Reducing interference of an overlay with underlying content
US9432703B2 (en) 2014-11-17 2016-08-30 TCL Research America Inc. Method and system for inserting contents into video presentations
US20170279757A1 (en) * 2016-03-22 2017-09-28 Facebook, Inc. Systems and methods for identifying matching content
US10121513B2 (en) 2016-08-30 2018-11-06 International Business Machines Corporation Dynamic image content overlaying
US20180077455A1 (en) 2016-09-14 2018-03-15 International Business Machines Corporation Attentiveness-based video presentation management
US11076200B2 (en) 2016-12-13 2021-07-27 Rovi Guides, Inc. Systems and methods for minimizing obstruction of a media asset by an overlay by predicting a path of movement of an object of interest of the media asset and avoiding placement of the overlay in the path of movement
US10592752B2 (en) 2017-03-24 2020-03-17 Mirriad Advertising Plc Predicting future insertion zone metadata
US10909383B2 (en) 2017-03-24 2021-02-02 Mirriad Advertising Plc Predicting future insertion zone metadata
US11102549B2 (en) 2017-09-27 2021-08-24 Wurl Inc. Selective video overlay
US20190146965A1 (en) 2017-11-10 2019-05-16 Our.News Holdings, Inc. Crowdsourced validation of electronic content
US11044521B2 (en) 2017-12-08 2021-06-22 Google Llc Modifying digital video content
US20200028924A1 (en) 2018-07-19 2020-01-23 International Business Machines Corporation Cognitive insight into user activity
US20200125671A1 (en) 2018-10-17 2020-04-23 International Business Machines Corporation Altering content based on machine-learned topics of interest

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Borji, Ali, et al. "Salient object detection: A survey." Computational visual media 5.2 (2019): 117-150.
Ferreira, Tiago Alexandre Fonseca Ribeiro. Entropy-Based Dynamic Ad Placement Algorithms for In-Video Advertising. Diss. Universidade da Beira Interior (Portugal), 2014.
Mei, Tao, et al. "AdOn: Toward contextual overlay in-video advertising." Multimedia systems 16.4 (2010): 335-344.
Mei, Tao, Xian-Sheng Hua, and Shipeng Li. "VideoSense: A contextual in-video advertising system." IEEE Transactions on Circuits and Systems for Video Technology 19.12 (2009): 1866-1879.
Nilizadeh, Shirin, et al. "Cachet: a decentralized architecture for privacy preserving social networking with caching." Proceedings of the 8th international conference on Emerging networking experiments and technologies. 2012. 337-348.

Also Published As

Publication number Publication date
US20230275859A1 (en) 2023-08-31
US10951563B2 (en) 2021-03-16
US20210243144A1 (en) 2021-08-05
US20200412674A1 (en) 2020-12-31
US20220231980A1 (en) 2022-07-21
US11329937B2 (en) 2022-05-10

Similar Documents

Publication Publication Date Title
US9495385B2 (en) Mixed media reality recognition using multiple specialized indexes
US8676810B2 (en) Multiple index mixed media reality recognition using unequal priority indexes
US8369655B2 (en) Mixed media reality recognition using multiple specialized indexes
US8965145B2 (en) Mixed media reality recognition using multiple specialized indexes
CN111277849B (en) Image processing method and device, computer equipment and storage medium
CN105450778B (en) Information transmission system
US10282752B2 (en) Computerized system and method for displaying a map system user interface and digital content
CN110807710A (en) Method and system for managing invitation data of user inviting friends
US8977617B1 (en) Computing social influence scores for users
US20140208213A1 (en) Synchronization of contextual templates in a customized web conference presentation
US9715494B1 (en) Contextually and tonally enhanced channel messaging
WO2016112665A1 (en) Voice data processing method and device
US10764613B2 (en) Video media content analysis
JP2013005445A (en) SYSTEM AND METHOD FOR PROVIDING ADVERTISEMENT USING VOICE RECOGNITION DURING VoIP-BASED VOICE CALL
US20190073410A1 (en) Text-based network data analysis and graph clustering
US20190073411A1 (en) Image-based network data analysis and graph clustering
JP6684894B2 (en) Method and apparatus for push information distribution
US11683285B2 (en) Enhancing a social media post with content that is relevant to the audience of the post
US20170279749A1 (en) Modular Communications
US11599521B2 (en) Systems and methods for providing real-time discrepancies between disparate execution platforms
US20150074073A1 (en) Apparatus, system, and method for event-identified content exchange and management
US20130254331A1 (en) Information processing apparatus, information processing method, program, and information processing system
JP5823668B2 (en) Information processing apparatus, message transmission method, and program
CN111158838B (en) Information processing method and device
US20200111129A1 (en) Dynamic Proponent Targeting Based on User Traits

Legal Events

Date Code Title Description
AS Assignment

Owner name: ROVI GUIDES, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SRINIVASAN, MADHUSUDHAN;REEL/FRAME:059535/0305

Effective date: 20190715

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH CAROLINA

Free format text: SECURITY INTEREST;ASSIGNORS:ADEIA GUIDES INC.;ADEIA IMAGING LLC;ADEIA MEDIA HOLDINGS LLC;AND OTHERS;REEL/FRAME:063529/0272

Effective date: 20230501

STCF Information on status: patent grant

Free format text: PATENTED CASE