WO2015104689A1

WO2015104689A1 - A method and system for providing an asynchronous video conversation

Info

Publication number: WO2015104689A1
Application number: PCT/IB2015/050219
Authority: WO
Inventors: Sagi HIRSH; Gabi BEN AMI
Original assignee: Vidback Ltd
Priority date: 2014-01-10
Filing date: 2015-01-12
Publication date: 2015-07-16

Abstract

A system and method for facilitating an asynchronous media content conversation, the method including the steps of: (a) sending a media item in an asynchronous communication from a sending mobile device to a receiving mobile device via a communications network; (b) receiving the media item on the receiving mobile device; (c) recording a user reaction of a user of the receiving mobile device while the media, item is being played on the receiving mobile device; (d) sending a response asynchronous communication from the receiving device to the sending device, such that the receiving device is now the sending device and the sending device is now the receiving device.

Description

A METHOD AND SYSTEM FOR PROVIDING AN ASYNCHRONOUS VIDEO

CONVERSATION

FIELD OF THE INVENTION

The present invention relates to a system and method of video communication and, more particularly, to an Asynchronous Video Communication.

BACKGROUND

The telecommunications sector has evolved from mere telephone calls to mobile devices -.which allow voice communication while on the move, sending short text messages (SMS), multimedia messages (MMS) and text and/or audio/visual content which can all be received and viewed on mobile devices such as smartphones, mini tablet computers (e.g. iPad mini), laptops, regular tablet computers (e.g. iPad 2), etc. Most mobile devices have a camera for acquiring and storing content as well as a second camera on the 'front' (screen interface side) of the device for making video chat calls among other things. (Laptops only have front facing cameras.)

Video chat is a common way for people to have an audio conversation like a phone call and at the same time stream, live video on one or both sides of the conversation. Video chat on mobile devices has not gained significant traction in the market since both parties have to be connected at the same time which can be inconvenient for one or both parties as. the nature of mobile devices allows for their use while on the go. Live video chat can hamper free movement. For this reason, aside from phone calls, the most common usage on mobile devices is Asynchronous Communication, instant messaging, social networks and most communication apps on mobile devices employ non-real-time communication at their core functionality.

SUMMARY OF THE INVENTION

According to the present invention there is provided a method for facilitating an asynchronous media content conversation, the method including the steps of (a) sending a media item in an asynchronous communication from a. sending mobile device to a receiving mobile device via a communications network; (b) receiving the media itera on the receiving mobile device; (c) recording a user reaction of a user of the receiving mobile device while the media item is being played on the receiving mobile device; (d) sending a response asynchronous communication from the receiving device to the sending device, such that the receiving device is now the sending device and the sending device is now the receiving device- According to further features in preferred embodiments of the invention described below the media item is stored on a remote non-transitory computer- readable storage medium by the sending device and retrieved from the storage medium by the receiving device.

According to still further features in the described preferred embodiments the response asynchronous communication includes the recording.

According to still further features the response asynchronous communication further includes additional media content.

According to still further features the response asynchronous communication includes instructions for ending the asynchronous video conversation.

According to still further features the method further includes the step of presenting a time limit prior to sending the response asynchronous communication.

According to still further features the method further includes the step of detecting whether a face is presented in a field of view of a camera of the receiving device before allowing the media item to be displayed on a display of the receiving device, the display being positioned coplanar to the camera.

According to still further features the method further includes the step of detecting whether a field of view of a camera of the receiving device is obstructed before allowing the media item to be displayed on a display of the receiving device, the display being positioned coplanar to the camera.

According to still further features the method further includes the step of filtering out audio content of the media item while recording the user reaction, for example by applying an Acoustic Echo Cancellation program.

According to another embodiment there is provided a system for facilitating an asynchronous conversation, the system including: (a) a sending mobile device, adapted to send an asynchronous communication, including a media item, via a communications network; (b) a receiving mobile device, including a processor, the processor being programmed to: (i) receive the asynchronous communication, (ii) play the media item on the receiving device, (iii) record a user reaction of a user of the receiving device, while the media item is being played, and (iv) send a response asynchronous communication to the sending mobile device, thereby changing the sending mobile device into the receiving mobile device and changing the receiving mobile device into the sending mobile device.

According to further features of the invention, the system further includes: (c) a remote non-transitory computer-readable storage medium, the remote storage medium adapted to store computer-readable digital media content, wherein the media item is uploaded to the remote storage medium, via the communications network, by the sending device and retrieved from the remote storage medium by the receiving device.

According to still further features the system further includes: (d) an application server, the application server adapted to receive message requests and send message notifications, wherein the asynchronous communication and the response asynchronous communication are sent to the application server over the communications network.

According to still further features the application server and the remote storage medium are collocated.

According to still further features the application server is located in a first location and the remote storage medium is located in a second location.

According to still further features the response asynchronous communication includes the recording.

According to still further features the processor is further programmed to present a time limit prior to sending the response asynchronous communication.

According to still further features the processor is further programmed to detect whether a face is presented in a field of view of a camera of the receiving device before allowing the media item to be played on the receiving device.

According to still further features the processor is further programmed to detect whether a field of view of a camera of the receiving device is obstructed before allowing the media item to be displayed on a display of the receiving device, the display being positioned coplanar to the camera. According to still further features the processor is further programmed to filter out audio content of the media item while recording the user reaction.

According to still further features the processor is programmed to apply an Acoustic Echo Cancellation function to filter out the audio content.

According to another embodiment there is provided a system for facilitating an asynchronous media content conversation, the system including: (a) a sending mobile device, adapted to send an asynchronous communication, including a media item, via a communications network; (b) a receiving mobile device, including: (i) a processor, the processor being programmed to effect computer-readable instructions of a mobile application installed on the receiving device, (ii.) audio and display subsystems in electronic communication with the processor, the audio and display subsystems adapted to play the media item, on a display of the receiving device, (iii) an imaging subsystem, in electronic communication with the processor, the imaging subsystem adapted to record a user reaction of a user of the receiving device, while the media item is being played, and wherein the processor is further programmed to send a response asynchronous communication to the sending mobile device, thereby changing the sending mobile device into the receiving mobile device and changing the receiving mobile device into the sending mobile device.

According to further features the imaging subsystem includes a camera, the camera located coplanar to the display.

According to still further features the processor is programmed to render the recording into a reaction media item, the reaction media item being included in the response asynchronous communication.

According to still further features the reaction media item further includes the media item previously sent by the sending device, such that when the reaction media item is played, both the media items are displayed simultaneously.

According to still further features the previously sent media item is played with reduced audio volume.

According to still further features the processor is further programmed to instruct the audio and imaging subsystems to record a reply media item, the reply media item also being included in the response asynchronous communication and is accessible subsequent to the reaction media item.

According to another embodiment there is provided a system for facilitating an asynchronous conversation, the system including: (a) a sending mobile device, adapted to send an asynchronous communication, including a media item, via a communications network; (b) a remote storage medium, the remote non-transitory computer-readable storage medium adapted to store computer-readable digital media content, wherein the media item is uploaded to the remote storage medium, via the communications network, by the sending device; (c) an application server, the application server adapted to receive message requests and send message notifications, wherein the asynchronous communication is sent to the application server over the coramuni cations network; and (d) a receiving mobile device, adapted to receive a message notification of the asynchronous communication from the application server and retrieve the media item from the remote storage device, the receiving mobile device including: (i) a processor, the processor being programmed to effect computer-readable instructions of a mobile application installed on the receiving device, (ii) audio and display subsystems in electronic communication with the processor, the audio and display subsystems adapted to play the media item on a display of the receiving device, (iii) an imaging subsystem, in electronic communication with the processor, the imaging subsystem adapted to record a user reaction of a user of the receiving device, while the media item is being played, and wherein the processor is further programmed to send a response asynchronous communication to the sending mobile device via the application server, thereby changing the sending mobile device into the receiving mobile device and changing the receiving mobile device into the sending mobile device.

There is provided a method and system for producing a context based Asynchronous Video Communication / conversation as a single video stream on a full screen, in a similar format to a video chat, which makes use of the existing technology in the mobile device to capture a user's reaction to the viewed content (smile, laugh, surprise, sad, etc.). The recorded video is displayed on the same screen (e.g. picture in picture view) as the original video message the user is reacting to (similar to a realtime video chat where a user can see themselves as well as the other person on the video chat).

The complex video stream including the target, video content and the reaction content is termed, within the context of the current document, a "vidback". If a vidback has been created, in response to an initial, target message, then the vidback is presented to the second user before the response message (if such a message was created) is displayed.

This document references terms that are used consistently or interchangeably herein. These terms, including variations thereof, are as follows.

A "computer" includes machines, computers and computing or computer systems (for example, physically separate locations or devices), servers, computer and computerized devices, processors, processing systems, computing cores (for example, shared devices), and similar systems, workstations, modules and combinations of the aforementioned. The aforementioned "computer" may be in various types, such as a personal computer (e.g., laptop, desktop, tablet computer), or any type of computing device, including mobile devices that can be readily transported from one location to another location (e.g., smartphone, personal digital assistant (PDA), mini tablet, laptop, mobile telephone, cellular telephone, "smart" wearable device).

A "mobile device", "mobile", "mobile communication device", "mobile computing devi ce" and any variation and/or combination thereof is understood to be a mobile device discussed above

A server is typically a remote computer or remote computer system, or computer program therein, in accordance with the "computer" defined above, that is accessible over a communications medium, such as a communications network or other computer network, including the Internet. A "server" provides services to, or performs functions for, other computer programs (and their users), in the same or other computers. A server may also include a virtual machine, a software based emulation of a computer.

An "application", includes executable software, and optionally, any graphical user interfaces (GUI), through which certain functionality may be implemented.

A "plug-in" includes software components that add functionalities to the existing software of a computer or to an existing software application.

A "client" is an application that runs on a computer, workstation or the like and relies on a server to perform some of its operations or functionality.

The terms "multimedia", "media", "media content", "recording" are used interchangeably herein to refer to digital content in the form of text, audio, visual and/or any combination thereof (e.g. video).

The terms "multimedia content", "media item", "multimedia item", "media file", "multimedia file" and the refer to a transportable and/or transferable digital media file, which is stored on a computer-readable storage medium. The digital file can be transferred, copied, moved, from a digital storage medium in one location to a digital storage medium in a second location via a computing and/or communications network. The digital media file can be created, accessed (e.g. "played", "viewed", "displayed"), copied, edited etc. The media item may be a picture, text, an audio clip, an audio/visual (video) clip and any combination thereof.

A "reaction" media item refers to a recording (usually audio visual) of a user's reaction to media content that the user is watching and/or listening to.

A "reply" media item, refers, herein, to media content that is generated (e.g. recorded, selected, copied etc.) and attached together with a reaction media item and sent in a response asynchronous communication / message (see below). When this message is receive the reaction media is played first and then the reply media item is played.

The terms "Asynchronous Communication", "Asynchronous Video Communication", "multimedia message", "multimedia communication", "media message", combinations and variations thereof, are used interchangeably herein and. refer to non-real-time communications between a first user and a second user. The message or communication has attached thereto, or embedded therein, digital media described above and includes, but is not limited to: a text message, an audio message, a video message or any combination thereof. For example, user A creates/generates content (e.g. text message, photo, video, email) and sends the content to user B (e.g. via the cloud, communications network) without the need for user B to be available at the same time. User B reads/views the content when desired, without the need for user A to be available at that time. (Another common example of an asynchronous communication is a text message or SMS message where one user sends the message to another user and the second user reads that message at a convenient time, not necessarily in real-time. An example of a synchronous communication is a telephone conversation.)

A "Response Asynchronous Communication" is a message, as defined heretofore, which is sent in response to a preceding or initial message.

The term "asynchronous video conversation", "asynchronous media content conversation", or variations thereof, refers to a set. of asynchronous communications between, at least, a first user and a second user, where each user sends at least one message to the other user. The terms User A and User B are intended to refer to the mobile devices of the users, unless specifically referring to the operator of the device.

Unless otherwise defined herein, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein may be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are herein described, by way of example only, with reference to the accompanying drawings, wherein:

FIG. 1 is a flow chart of a method and system of the invention;

FIG. 2 is a block diagram of various components of an exemplary mobile device;

FIG. 3 is a flow diagram of the asynchronous video conversation of the immediate invention, including the application server and storage medium.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present, invention discloses a system and method to generate context based asynchronous video (and/or other media content) communication conversation.

In preferred embodiments, a first user, User A, creates media content (e.g. records a video, selects a video from the mobile device, edits a video, combines text, audio and/or visual content etc.) and sends the media item to a second user, User B. User B receives a notification (e.g. via a push message) and has the option to choose when to open / view the media item. In some embodiments, when User B accesses the media item (e.g. views the video), the system server sends a notification to User A, that the message is being accessed. Alternatively or additionally, if User A has the application

(mobile app) running, then the notification will appear immediately, indicating that

User B is viewing the video.

While viewing the video, the installed application activates the front camera of the mobile device, and records User B in order to capture the reaction of User B to the media content. User B is aware of the fact that the device is recording his reaction but the user is not able to see a live/real-time visual of the recording.

Once the content has been viewed, User B is given various options for responding. In some embodiments, User B has a predetermined, time limit (e.g. five seconds) during which to select the reply option. In preferred configurations, the following exemplary reply options are offered (e.g. in response to the posted question "Do you want to reply?"):

"Yes" - User B is directed to create a reply video message to user A;

"No" - a response message is sent to User A, indicating that User B does not wish to reply to the message and/or a message to terminate the video conversation, in some embodiments, if the user selects "No", then the aforementioned response is sent together with the recorded reaction (termed herein "vidback");

No selection -^■ referring to the embodiment which includes a counter for limiting the response time, if User B does not select either of the options then the application will make a predefined decision for the user. In some exemplary embodiments, the predefined decision is to send a message to User A, stating that no response was made and/or a message to terminate the video conversation, in another exemplary embodiment, the predefined decision is to send the recorded reaction. In other exemplary embodiments, the predefined decision is to send the recorded reaction together with a message that no response was made and/or a message to terminate the video conversation.

If User B decides to record a reply, then User A receives a complex response messaget^'a vidback). For example, the response message is displayed as a split screen with the original video sent by User A on the bottom part of the screen and the recording of User B's reaction on the top part, of the screen. Both videos are played simultaneously. Once the vidback concludes, the reply message from User B starts to play, in full screen, mode. While the reply message is playing, User A is recorded by the front/facing camera on the same device. Once the reply message finishes playing (and the recording stops), User A is presented with the same options for replying as detailed above (Yes, No, do nothing). The process (conversation) continues until one of the sides decides to terminate the conversation (or fails to respond within the limited response time period).

Asynchronous Video Conversation Process The principles and operation of a method and system according to the present invention may be better understood with reference to the drawings and the accompanying description.

Referring now to the drawings, Figure 1 illustrates a flow chart of a method and system of the invention. It is made clear that while the examples relate to a smartphone, the methods and systems can equally be adapted to other mobile devices such as mini tablets, tablets, smart-watches, other wearable devices and the like. In the first step, User A creates or acquires content on the mobile device. The media content is then sent to User B for viewing.

User B receives notification that a media content file has arrived and is invited to open the file. The innovative system application installed on the mobile device activates various technologies on the device before the user is able to view the content, in order to ensure that the first reaction of User B to the content is recorded. in preferred embodiments, the application runs a Camera Block Detection program that check^'s that the field of view of the front- facing camera (screen-side, facing the user) is not blocked or obstructed, either on purpose or inadvertently (e.g. by the user's finger).

In some embodiments, an additional or alternative feature that is activated is a Face Detection function. The application checks that the user is in front of the camera. before the content is allowed to be displayed. The processor instructs the imaging subsystem to run a face detection program to detect whether a face is presented in the field of view of the camera of the receiving device. Once a face is detected (i.e. the camera is not facing up, down or away from the user), the content is allowed to display. If the camera is not at the correct angle, the screen is preferably dimmed until the correct angle is achieved.

In all embodiments, the front camera begins to record in the background when the content is activated for display. The front camera refers to the camera of the receiving device that is located coplanar to the display screen.

In preferred embodiments, an additional or alternative feature that is provided is audio filtering, hi order to receive the audio feedback, of the User Reaction, as well as the visual feedback, the sound of the playing content is preferably be filtered out of the reaction recording. In order to filter out the sound of the playing content, exemplarily, an Audio Acoustic Echo Cancellation (AEC) program / function is employed to filter out the noise. Once the media content of the received file has run its course, the recording of the User Reaction also ends. Preferably, a decision screen then appears, which gives User B the option of taking various forms of action. In some embodiments, the decision period afforded to User B is limited in time, for example by a five second counter.

Exemplarily, User B can do nothing within the allotted time, in which case the Reaction Video (also termed herein "vidback") will be sent to User A without any additional response. In this case the interaction between User A and User B ends. In another configuration, the default result for not taking any action until the counter runs out can simply be to send a termination message to User A, without the Reaction Video. In either configuration, after sending a Termination Message, the application then exemplarily reverts back to the home screen (e.g. a Friends List screen).

Alternatively, User B can reject the option of replying by selecting the "No" button. By selecting the NO option, User B refuses to send the recoding (i.e. the Reaction Video) or to 'vidback' (using the noun as a verb). In such a case the sender, User A, receives a message that User B has canceled the Vidback or Asynchronous Video Conversation.

In a third option, User B can select the "Yes" button and add and/or record a reply or "response" message (which generally applies directly to the content of the original video User A sent to User B). In such a case User B sends the vidback (Reaction Video) together with a response (Reply Video) to User A. At this point User A, who was the sending user/device, now becomes User B or the Receiving User/device. That is to say that User A goes through the same sequence of events as described for User B. Likewise, User B is now the sending user, such that the reply / "response" content made by User B is regarded in the same manner as the initial content made by User A at the beginning of the process.

Mobile Device Technology

Figure 2 illustrates a block diagram of various components of an exemplary mobile device 200. Only elements and components that are germane to the invention are displayed and/or discussed. Various technologies can be utilized to facilitate the asynchronous video conversation process. The specific technologies described below are merely a preferred exemplary embodiment of the system. At the core of the functionality of the mobile device is an Applications Processor, or simply processor 210. The mobile device, e.g. a smartphone, includes an Audio Subsystem 220 that is used, by the device for both recording and playback of media items, if the items include an audio component. Media items of the immediate invention are displayed by a Display Subsystem 236. The user reaction, initial recorded items and recorded reply items all use an imaging or Visual Subsystem 240 including at least one camera 242. All computer-readable data, logic, instructions and. routines, algorithms etc. are stored one or more of the various non-transitory computer-readable storage mediums included in the device. The various storage media (mediums) are generally delineated in the Figure as Storage 250.

A system mobile application (app) includes computer-readable instructions for programming processor 210 to perfomi the desired functions, by contiOlling the device components. Each User has the mobile app installed on the mobile device. Exemplarily, the computer-readable instructions of the mobile application ("Mobile App Logic") are stored on an SD card 252 device non-transitory computer-readable storage medium. Secure Digital (SD) is a nonvolatile memory card used extensively in portable devices, such as mobile phones. The logic / programming / computer- readable instructions 254, which enable the functionality of the application / service is likewise, exemplarily, stored on the SD card.

Server Side

Figure 3 illustrates a flow diagram of the asynchronous video conversation of the immediate invention, including the application server and storage medium. In the depicted exemplary flow, User A 10 initiates the video message conversation by sending an initial media message to User B 20, via a communications network 50. Therefore User A is the sending user and User B is the receiving user, in the first cycle of the conversation.

Step 1 - User A generates the Original Video File and sends the file to a. cloud storage server 30. The cloud storage server returns the URL of video to User A. The cloud storage server is merely an exemplary embodiment of a remote non- transitory/nonvolatile computer-readable storage medium which is capable of storing digital, computer-readable data / media content.

Step 2 - User A 10 sends a Message Request to the Application server 40, asking the server to send a new message to User B. In some embodiments the application server and the storage server are hosted on the same sever or collocated on a single system. In other embodiments, the storage medium/server is hosted in a first location and the application server is hosted in a second location.

User A attaches the URL/address of the location of the media item on the Storage Server, as well as the recipient (User B) details and any necessary additional metadata to the request. The application server returns a unique Message ID to User A, which is stored locally on User A, together with all the other relevant data.

Step 3 - The application server handles the request from User A by creating a new Message Entity and sending a Push Notification to notify User B that a new message is waiting.

Step 4 - User B accepts the new Message as follows: User B logs into the system and requests a list of new messages. After a successful login, the server returns a list of stored messages waiting for User B on the server. User B watches the original video/multimedia message from User A. While watching, the system app installed on the mobile device records the reaction of User B while viewing the message. After watching the original video, User B is presented with an option of replying to the Original video (e.g. by pressing the "Yes" button) or not (e.g. by pressing the "No" button). In some embodiments, the option to reply is presented for a limited time only. For example, the option to reply can be limited by a 5 second timer.

Step 5 - At this point, User B becomes the sending device and User A becomes the Receiving device. If User B chooses "No" (or, in some embodiments, when the 5 seconds timer ends), then User B uploads the recording of the reaction ("Reaction Video File") to the original video to Storage Server 30. The server returns the URL of the Reaction Video File to User B.

If the user chooses "Yes", then User B is prompted to record / create / generate a new reply message / media item. ("Reply Video File") to User A's Original Video. After recording the reply, User B uploads the media item including both the reaction recording (i.e. the recording of the reaction to the Original Video) and the new reply video to the storage server. Storage server 30 returns the URL of the Reaction and Reply Video File to User B.

Step 6 - User B then initiates a new message request to Application Server 40 and receives a unique message ID back from the server in the same manner as described ..above for User A. in Step 2. The process then continues from step 2 with the roles of User A and User B reversed. In general terms, there is provided a system for facilitating an asynchronous media content conversation. Referring to all the foregoing Figures, a sending mobile device (e.g. User A 10) - which is adapted, to send an asynchronous communication, including a media item, via a communications network 50 (e.g. Internet, Cellular Data Network etc.) - sends a media message (an asynchronous communication) to a receiving mobile device (e.g. User B 20). The receiving device includes a processor (e.g. Applications Processor 210) which is programmed to effect the computer- readable instructions 254 (i.e. programming or logic etc., such as Camera Block. Detection function, AEC, Face Detection, Recording program, Mobile App Logic) of a mobile application installed on the receiving device (and stored, for example, on SD Card 252). The device further includes an audio subsystem 220 and a display subsystem 230 which are in electronic communication with the processor. The audio and display subsystems are adapted to play the media item on the display of the receiving device. The receiving device also has an imaging subsystem 240, which is also in electronic communication with processor, and is configured to record the user reaction of the user viewing the media item when played on the device. The imaging subsystem records the user for the duration of the media item that is being played. The processor is further programmed to send a response message (asynchronous communication) back to the sending mobile device, thereby changing the sending mobile device into a receiving mobile device and changing the receiving mobile device into a sending mobile device.

Preferably, the imaging subsystem includes a camera that is located coplanar to the display (e.g. a forward facing camera on a smartphone). The processor is programmed to render the recording of the user reaction into a "reaction" media item. The reaction media item is included in the response asynchronous communication, in preferred embodiments, the reaction media item further includes the original media item previously sent by sending device so that when the message is accessed the reaction media item is played, while the original video is simultaneously displayed. Preferably, the previously sent media item, (e.g. the original video) is played with reduced audio volume.

Preferably, the processor is further programmed to instruct the audio and imaging subsystems to record a reply to the media item. The reply media item (e.g. a recorded, message) is also included the response asynchronous communication and is accessible after (subsequent to) the reaction media item is played.

Camera and Player

This section describes a specific example of a preferred implementation of the the way the Camera 242 (and any other elements of the imaging subsystem, as in known in the art) and Player (audio subsystem and display subsystem controlled by the processor, programmed with the mobile application logic) components works together. Each of these components can work at the same time, independent of the other, or in any combination.

Player Component

Exemplarily, the smart phone uses AVFoundation framework to play videos. AVFoundation is a well-known framework with an Objective-C interface, which provides essential services for working with time-based autovisual media on Apple Inc. Operating systems: iOS and OS X. AVFoundation was first introduced in iOS 4 and has seen significant changes in both iOS 5 and iOS 6. Starting with Mac OS 10.7, now it is the default media framework for the Mac OS platform.

The Player is preferably integrated with screen dimming functionality which, in turn, is triggered by the face detection function (see discussion about 'face detection' above). Playing a video causes the application to launch the Camera in the background which starts recording the user's reaction using the front (display-side) camera.

Camera Component

Exemplarily, the system uses AVFoundation framework to capture, process, preview and generate video on iOS devices. In preferred embodiments, the framework is integrated with Face Detection, Camera Blocking Detection and/or Audio AEC (Acoustic Echo Cancellation) functionality.

When the System Camera component launches, an A VCaptureSession is created with audio and video device inputs, and outputs for audio and video data. These outputs continuously supply frames of audio and video content to the component, via the captureOutput:didOuipuiSampleBuffer:fromConnection: delegate method.

The component applies a two-step process, one for the video frame and another for the audio frame. Exemplarily, the system uses OpenCV Face Detection and Haar Cascades to analyze each frame in order to detect whether the user is holding the camera in front of his face or not; which in turn will instruct the Device Player component whether to dim the screen or not. That is to say that if the screen/camera is not pointing at the recipient's face, then the screen will be dimmed and the media file will not play.

in a second step, the Audio AEC processes the audio input using bdSES

Library (bdSound.com). The AEC performs echo cancellation, providing a final result (e.g. User B laughing) without the original video sound in the background.

After a frame of video is processed, VCamera component uses OpenGL ES2 to display it on the screen, this step uses the CVOpenGLESTextureCache API (iOS5 and later). The vidback, is shown together with the original movie as a screen within screen (see pictorial depiction in Figure 2). When recording starts, an A VAssetWriter is used to write the processed video and audio to a Meg4 movie file.

Potentially, the vidback (Reaction Video) and/or response (Reply Video) can be reviewed prior to sending. Potentially, each of the definitions can be changed (e.g. duration of count-down, default settings, etc.).

While the invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications and other applications of the invention may be made. Therefore, the claimed invention as recited in the claims that follow is not limited to the embodiments described herein.

Claims

WHAT IS CLAIMED IS

1. A method for facilitating an asynchronous media content conversation, the method comprising the steps of:

(a) sending a. media item in an asynchronous communication from a sending mobile device to a receiving mobile device via a communications network;

(b) receiving said media item on said, receiving mobile device;

(c) recording a user reaction of a user of said receiving mobile device while said media item is being played on said receiving mobile device;

(d) sending a response said asynchronous communication from said receiving device to said sending device, such that said receiving device is now said sending device and said sending device is now said receiving device.

2. The method of claim 1, wherein said media item is stored on a remote non-transitory computer-readable storage medium by said sending device and retrieved from said storage medium by said receiving device.

3. The method of claim 1 , wherein said response asynchronous communication includes said recording.

4. The method of claim 3, wherein said response asynchronous communication further includes additional media content.

5. The method of claim I, wherein said response asynchronous communication includes instructions for ending the asynchronous video conversation.

6. The method of claim 1. further comprising the step of presenting a time limit prior to sending said response asynchronous communication.

7. The method of claim 1 , further comprising the step of detecting whether a face is presented in a field of view of a camera of said receiving device before allowing said media item to be displayed on a display of said receiving device, said display being positioned coplanar to said camera.

8. The method of claim 1 , further comprising the step of detecting whether a field of view of a camera of said receiving device is obstructed before allowing media item to be displayed on a display of said receiving device, said display being positioned coplanar to said camera.

9. The method of claim 1 , further comprising the step of filtering out audio content of said media item while recording said user reaction.

10. The method of claim 9, wherein said step of filtering out said audio content is effected by applying an Acoustic Echo Cancellation program.

1 1. A system for facilitating an asynchronous conversation, the system comprising:

(a) a sending mobile device, adapted to send an asynchronous communication, including a media item, via a communications network;

(b) a receiving mobile device, including a processor, said processor being programmed to:

(i) receive said asynchronous communication,

(ii) play said media item on said receiving device,

(iii) record a user reaction of a user of said receiving device, while said media item is being played, and

(iv) send a response said asynchronous communication to said sending mobile device, thereby changing said sending mobile device into said receiving mobile device and changing said receiving mobile device into said sending mobile device.

12. The system of claim 1 1 , further comprising:

(c) a remote non-transitory computer-readable storage medium, said remote storage medium adapted to store computer-readable digital media content, wherein said media item is uploaded to said remote storage medium, via said communications network, by said sending device and retrieved from said remote storage medium by said receiving device.

13. The system of claim 12, further comprising:

(d) an application server, said application server adapted to receive message requests and send message notifications, wherein said asynchronous communication and said response asynchronous communication are sent to said application server over said communications network.

14. The system of claim 13, wherein said application server and said remote storage medium are collocated.

15. The system of claim 13, wherein said application server is located in a first location and said remote storage medium is located in a second, location.

16. The system of claim 1 1, wherein said response asynchronous communication includes said recording.

37. The system of claim 16, wherein said response asynchronous communication further includes additional media, content.

18. The system of claim 11, wherein said response asynchronous communication includes instructions for ending the asynchronous video conversation.

19. The system of claim 11 , wherein said processor is further programmed to present a time limit prior to sending said response asynchronous communication.

20. The system of claim 11, wherein said processor is further programmed to detect whether a face is presented in a field of view of a camera of said receiving device before allowing said media item to be played on said receiving device.

21. The system of claim 1 1, wherein said processor is further programmed to detect whether a field of view of a camera of said receiving device is obstructed before allowing said media item to be displayed on a display of said receiving device, said display being positioned coplanar to said camera.

22. The system of claim 11, wherein said processor is further programmed to filter out audio content of said media item while recording said user reaction.

23. The system of claim 22, wherein said processor is programmed to apply an Acoustic Echo Cancellation function to filter out said audio content.

24. A system for facilitating an asynchronous media content conversation, the system comprising:

(b) a receiving mobile device, including:

(i) a processor, said processor being programmed to effect computer- readable instructions of a mobile application installed on said receiving device,

(ii) audio and display subsystems in electronic communication with said processor, said audio and display subsystems adapted to play said media item on a display of said receiving device,

(iii) an imaging subsystem, in electronic communication with said processor, said imaging subsystem adapted to record a user reaction of a user of said receiving device, while said media item is being played, and

wherein said processor is further programmed to send a response said asynchronous communication to said sending mobile device, thereby changing said sending mobile device into said receiving mobile device and changing said receiving mobile device into said sending mobile device.

25. The system of claim 24, wherein said imaging subsystem includes a camera, said camera located coplanar to said display.

26. The system of claim 24, wherein said processor is programmed to render said recording into a reaction said media item, said reaction media item being included in said response asynchronous communication.

27. The system of claim 26, wherein said reaction media item further includes said media item previously sent by said sending device, such that when said reaction media item is played, both said media items are displayed simultaneously.

28. The system of claim 27, wherein said previously sent media item is played with reduced audio volume.

29. The system of claim 26, wherein said processor is further programmed to instinct said audio and imaging subsystems to record a reply said media item, said reply media item also being included in said response asynchronous communication and is accessible subsequent to said reaction media item.

30. A system for facilitating an asynchronous conversation, the system comprising:

(b) a remote non-transitory computer-readable storage medium, said remote storage medium adapted to store computer-readable digital media content, wherein said media item is uploaded to said remote storage medium, via said communications network, by said sending device;

(c) an application server, said application server adapted to receive message requests and send message notifications, wherein said asynchronous communication is sent to said application server over said communications network; and

(d) a receiving mobile device, adapted to receive a said message notification of said asynchronous communication from said application server and retrieve said media item from said remote storage device, said receiving mobile device including:

(ii) audio and display subsystems in electronic communication with said processor, said audio and display subsystems adapted to play said media item on a display of said receiving device, (iii) an imaging subsystem, in electronic communication, with said processor, said imaging subsystem adapted to record a user reaction of a user of said receiving device, while said media, item is being played, and

wherein said processor is further programmed to send a response said asynchronous communication to said sending mobile device via said application server, thereby changing said sending mobile device into said receiving mobile device and changing said receiving mobile device into said sending mobile device.