WO2014015080A2 - Procédé et système servant à associer des médias synchronisés en créant un datapod - Google Patents

Procédé et système servant à associer des médias synchronisés en créant un datapod Download PDF

Info

Publication number
WO2014015080A2
WO2014015080A2 PCT/US2013/050960 US2013050960W WO2014015080A2 WO 2014015080 A2 WO2014015080 A2 WO 2014015080A2 US 2013050960 W US2013050960 W US 2013050960W WO 2014015080 A2 WO2014015080 A2 WO 2014015080A2
Authority
WO
WIPO (PCT)
Prior art keywords
media object
datapod
transformations
media
series
Prior art date
Application number
PCT/US2013/050960
Other languages
English (en)
Other versions
WO2014015080A3 (fr
Inventor
Ross Quentin SMITH
Miriam Barbara SEDMAN
Joan Lorraine WOOD
Original Assignee
Jigsaw Informatics, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/553,529 external-priority patent/US20120290907A1/en
Priority claimed from US13/553,562 external-priority patent/US20120284426A1/en
Application filed by Jigsaw Informatics, Inc. filed Critical Jigsaw Informatics, Inc.
Publication of WO2014015080A2 publication Critical patent/WO2014015080A2/fr
Publication of WO2014015080A3 publication Critical patent/WO2014015080A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages

Definitions

  • This invention relates generally to software applications for mobile and other devices, and more particularly to creating and maintaining a synchronized association of objects when displayed on any device, including mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • PCs personal computers
  • game systems automotive and avionics displays
  • digital picture frames digital picture frames
  • TVs set top boxes
  • digital video and still cameras smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • the sender may also wish to communicate particular "navigation" information associated with one or more files. For example, the sender may wish to zoom in on or highlight a particular part of the photo to call the recipient's attention to it. This information would also be lost in the communication of the two files unless the sender took yet another photo of the zoomed in or highlighted portion and communicated the details about the zoomed or highlighted image.
  • Embodiments of the present invention create a "datapod" by associating a media object with a data object or objects so that a synchronized relationship between the media and data objects is formed and preserved.
  • the datapodTM can be shared or communicated which will intrinsically maintain the synchronized relationship between or among the media and data objects. Therefore, the files will play in the intended sequence and with the intended information conveyed precisely as the sender intended.
  • the datapodTM will play with the correct synchronization between the photo and the audio annotation as if the recipient were sitting next to the sender and seeing the same photo and listening to the audio annotation as it was made by the sender.
  • the invention permits the user to play the datapodTM by receiving a datapodTM, unpacking the datapodTM into its synchronously associated media object and data object and playing the datapodTM such that the synchronous association between the media object and the data object are maintained and the playing of the media object and data object is synchronized.
  • Embodiments of the present invention are achieved in a user friendly manner such that senders using a mobile device such as a mobile phone or a tablet computer or a digital camera equipped with the technology can easily create datapodsTM and the synchronized media association is intrinsically preserved on any device playing the associated media.
  • a mobile device such as a mobile phone or a tablet computer or a digital camera equipped with the technology
  • any other device may be used to create or play the datapodTM, for example other mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • Embodiments of the present invention include a method for creating a datapod using a device, comprising: acquiring a first media object, having an original fidelity; preserving the original fidelity of the first media object; annotating the first media object with a series of transformations and a second media object simultaneously; and creating a datapod by synchronously associating the first and second media objects with the transformations of the first media object, wherein temporal and spatial relationships between the first and second media objects and the transformations is precisely preserved, and the original fidelity of the first media object is preserved.
  • the device uses a gesture for controlling the transformations of the first media object.
  • the device has a touchscreen and the touchscreen is used for controlling the transformations of the first media object.
  • the device is a handheld device and the motion of the handheld device is used for controlling the transformations of the first media object.
  • Further embodiments also include sharing the datapod.
  • the sharing is performed by sharing the datapod.
  • the sharing is performed by converting the datapod into a video file and sharing the video file.
  • the sharing the datapod is performed using a media sharing platform.
  • the datapod is created at the same time as at least one of the first and second media objects is acquired.
  • the first media object is a plurality of media objects such that the media objects can be sequenced, rearranged and combined.
  • the second media object is created using a touchscreen.
  • the second media object is created using video.
  • the second media object is created using audio. In one embodiment, the second media object is created using gesturing. In one embodiment, the series of transformations includes repositioning. In one embodiment, the series of transformations includes zooming. In one embodiment, the first media object is a three dimensional data file. In one embodiment, the series of transformations includes three dimensional transformations.
  • Embodiments of the present invention include a system for creating a datapod comprising: a non-transitory memory for storing a first and a second media object and a series of transformations of the first media object, wherein the original fidelity of the first media object is preserved; a user interface for creating a datapod by associating the first media object with the transformations and the second media object simultaneously, wherein temporal and spatial relationships between the first and second media objects and the transformations is precisely preserved and the original fidelity of the first media object is; and a platform for sharing the datapod.
  • the system has a touchscreen and the touchscreen is used for controlling the transformations of the first media object.
  • the system is a handheld device and the motion of the handheld device is used for controlling the transformations of the first media object.
  • the series of transformations includes repositioning.
  • the series of transformations includes zooming.
  • the first media object is a three dimensional data file.
  • the series of transformations includes three dimensional transformations.
  • Embodiments of the present invention include a method for playing a datapod that consists of synchronized associated media, having an original fidelity, and a series of transformations, using a device, comprising: receiving a datapod; unpacking the datapod into a synchronously associated first and second media object and a series of transformations of the first media object; and playing the datapod such that the association between the first and second media objects and the transformations is precisely preserved, wherein the temporal and spatial relationships between the first and second media objects and the transformations is the device is a mobile computing device.
  • the device is a tablet computer.
  • the device is a mobile phone.
  • the device is a personal computer.
  • the device is a gaming system.
  • the device is a camera.
  • the first media object is an image file.
  • the first media object is a video file.
  • the first media object is a three dimensional data file.
  • the series of transformations includes repositioning.
  • the series of transformations includes zooming.
  • the series of transformations includes three dimensional transformations.
  • Embodiments of the present invention include a system for playing a datapod comprising: a platform for receiving the datapod; a user interface for playing a datapod by unpacking a first media object, having an original fidelity, and a second media object and a series of transformations of the first media object, wherein temporal and spatial relationships between the first and second media objects and the transformations is precisely preserved and the original fidelity of the first media object is preserved; and a non-transitory memory for storing the datapod including the first and second media object and the transformations of the first media object.
  • the first media object is a photo file.
  • the first media object is an image file.
  • the first media object is an audio file.
  • the first media object is a three dimensional file.
  • the second media object is a video file.
  • the system is a mobile phone.
  • the system is a tablet computer.
  • the series of transformations includes repositioning.
  • the series of transformations includes zooming.
  • the second media object is an audio file.
  • the series of transformations includes three dimensional transformations.
  • FIG. 1 shows a flowchart of a process to create a synchronized media association or datapodTM, in accordance with various aspects of the present invention.
  • FIG. 2 shows a functional block diagram of a device for creating a datapodTM, in accordance with various aspects of the present invention.
  • FIG. 3 shows a typical user interface for creating and sharing a datapodTM, in accordance with various aspects of the present invention.
  • FIG. 4 shows an embodiment of a user interface for creating a datapodTM, in which a media object (a photo of a crowd) is acquired, in accordance with various aspects of the present invention.
  • FIG. 5 shows an embodiment of a user interface for creating a datapodTM, in which a media object (an image of four geometric shapes) undergoes user navigation to create a datapodTM that contains the media object and navigation, in accordance with various aspects of the present invention.
  • a media object an image of four geometric shapes
  • FIG. 6 shows an embodiment of a user interface for creating a datapodTM, in which a media object (a photo of a crowd) undergoes navigation including zooming, markup with pen and voice audio annotation to create a datapodTM, in accordance with various aspects of the present invention.
  • Fig. 7 shows an embodiment for creating a datapodTM using two media objects with narration, in accordance with various aspects of the present invention.
  • Fig. 8 shows an embodiment for creating a datapodTM using two media objects with pen and narration, in accordance with various aspects of the present invention.
  • Fig. 9 shows an embodiment for creating a datapodTM using two media objects with navigation and narration, in accordance with various aspects of the present invention.
  • FIG. 10 shows a flowchart of a process to play a datapodTM, in accordance with various aspects of the present invention.
  • FIG. 11 shows a functional block diagram of a device for playing a datapodTM, in accordance with various aspects of the present invention.
  • Fig. 12 shows a user interface for playing a datapodTM with a base media object, video and text annotation data objects, in accordance with various aspects of the present invention.
  • Fig. 13 shows an embodiment of a user interface for playing a datapodTM with a base media object (a photo) with navigation including zooming and markup with pen, along with voice audio annotation, in accordance with various aspects of the present invention.
  • Fig. 14 shows a block diagram illustrating the relationship between creating and playing a datapodTM.
  • FIG. 1 is flowchart illustrating a process for creating a datapodTM according to an embodiment of the present invention.
  • Figure 1 shows acquiring a media object 110.
  • This acquisition can be performed using a camera, for example a digital camera or acquisition may be performed by using a digital camera in a mobile phone or tablet computer.
  • the acquisition can also be performed using another device such as a security or traffic camera, other mobile devices, TVs, PCs, Game Systems, Automotive Displays or other devices equipped with digital still or video cameras and/or audio/video capabilities, , etc.
  • the acquiring 110 can also be accomplished by uploading a photo or image already stored on the device or from a networked file storage or the internet.
  • a user takes a picture using the camera built in to a mobile device, which becomes the media object.
  • the media object is edited after it is acquired. Editing is accomplished using known digital image editing techniques.
  • the media object may be another type of file.
  • the media object will be a media file such as a photo, image, text file, document, e.g., word document, pdf, excel, three dimensional (3D) model or file, Visio or other format, audio file or video file.
  • a 3D model or file includes an object, a 3D terrain map, virtual world, synthetic environment, etc.
  • the media object is a collection of files rather than a single file.
  • Additional information may be stored along with the acquired media object. This additional information may be related to the date and time of media object capture, creation or editing or an event time, geo-location information associated with the media object, persons or events related to the media object, or other classification of the media object.
  • Figure 1 also shows annotating the media object with a data object 120.
  • This data object 120 can take the form of an audio recording, text or other data or media object.
  • a voice to text program could be used to create a text data object.
  • sign language could be used or a sign language to text program.
  • a translation program could be used to translate from one language to another in audio or text.
  • the data object can also take the form of an action, for example, navigation information.
  • the navigation information is panning around the image and/or zooming in on a particular part of the media object.
  • the navigation information is entered using a digital pen via touchscreen, stylus or other method to circle or highlight a particular portion of the media object for emphasis.
  • the navigation information is imparted by moving the device or by shaking or gesturing where device capabilities such as accelerometers may be used to record the movement.
  • the navigation information can be input by a user or by the device itself, for example in the case of an automatic zoom feature. Navigation can be accomplished in a number of ways including using a touch screen, buttons, zooming, writing, highlighting, gesturing, voice command or mind control.
  • the media object acquired is a photo of a child's artwork and the data object is an audio recording of the child describing the artwork.
  • the media object is a video of a child's artwork.
  • there is additional information stored with the acquired media object or the annotation such as date information, place information such as where the artwork was created, or information about the acquired media object or navigation information. Navigation information is discussed below with reference to Figures 5, 6, and 7.
  • Figure 1 also shows creating a datapodTM 130.
  • the datapodTM is a media file, such as a video file that may be readily shared and played on other devices.
  • the datapodTM is a collection of media files along with essential association information such that the relationship including synchronization between the media object and the data object is preserved.
  • the resulting datapodTM can be a video file constructed by synchronously combining the audio portion of the child's voice simultaneously with displaying the child's artwork.
  • the datapodTM can be the collection of the media object and the data object along with the synchronized relationship of the objects such that they would play in the proper sequence, synchronization, and with the proper information.
  • Figure 1 also shows sharing the datapodTM 140.
  • This sharing can be accomplished by the user sending the datapodTM as an attachment in a text message, email, instant message, via a link to a website where the media object is stored and "streamed" such as YouTube® for video implementations of the datapodTM.
  • the sharing can be also be accomplished by using a social media site for sharing such as Facebook®, Google+®, Drop Box® or Pinterest®.
  • the sharing can also be accomplished using a removal drive, for example a universal serial bus (USB) drive or memory stick. It can also be accomplished using network drives or cloud drives.
  • the sharing can also be accomplished using web based streaming.
  • One benefit of the present invention is the ease at which information can be shared.
  • it is difficult to share information particularly with multiple media file types.
  • each of the steps depicted in Figure 1 can be conducted in real-time and at the time the media object is acquired to enable real-time sharing or collaboration. Yet another benefit is that each of the steps depicted in Figure 1 can be achieved on a mobile device in a user friendly fashion without knowledge of computers or programming, presentation preparation, non-linear video editing or other complex operations. The steps in Figure 1 can be accomplished as easily as taking a photo with a camera phone.
  • the process shown in Figure 1 has many applications.
  • One application is in maintaining a collection of children's artwork. Many parents are busy and amass a large collection of their children's artwork, school projects, sports pictures and memorabilia, etc.
  • a parent can take a photo of the each item in their collection, annotate the photo with voice, text, video, and/or other actions including navigation and form a synchronous association of the photo and the annotation.
  • Additional information pertinent to the organization of the photo could also be maintained such as the date, the child's name, the child's grade, the subject of the photo, etc. This additional information can also form part of the datapodTM so that this amplifying information could be used as a search string, shared with recipients or otherwise used in the future.
  • the parent could take a photo of their child's artwork as the child is picked up at school and in real-time the child could annotate the photo, or describe the artwork, and the association would be formed between the photo and the annotation. Additionally, in one embodiment other information is captured automatically or manually in real-time as well, such as the date and the location.
  • the artwork is preserved and annotated and stored in such a way that it can be shared easily with others. Also, it is stored in such a way that it can be used in conjunction with other such datapodsTM to create an interactive or video based scrap book that may be shared with family and friends on a wide variety of devices including other mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, etc.
  • PCs personal computers
  • game systems automotive and avionics displays
  • digital picture frames digital picture frames
  • TVs set top boxes
  • digital video and still cameras smart office and home appliances
  • smart office and home appliances smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, etc.
  • FIG. 1 Another application to the process shown in Figure 1 is to inventory items. There are number of reasons inventories are used, such as, for sale using the internet using CraigslistTM or EBay®, to give away to family or for the purpose of a will, for keeping track of items, for communicating a particular item for purchase.
  • photos of items to be put up for sale can be acquired.
  • a video, audio, text description of the items, and/or additional annotation action including navigation and/or markup using pen used to annotate the photo may also be conducted.
  • the resulting datapodTM can be shared via text, email, internet, etc. and may be dispatched automatically to websites such as CraigslistTM or EBay® to ease the process of selling the item(s).
  • a similar process can be used to inventory for the purpose of giving away items or for recording the information for innumerous corporate (e.g., business inventor), professional (e.g., dental supply inventory), governmental (e.g., emergency supply inventory) or consumer purposes (e.g., home owner's inventory).
  • the annotated inventory could also be transcribed to provide a legal, written copy of the inventory as well.
  • Additional applications of the process of Figure 1 will be apparent to one of skill in the art. For example, there are many business applications. In many businesses expense reports are generated or receipts and other information are maintained for tax purposes.
  • the receipts and other items are acquired in a photo image, annotated with video, voice, text, and/or an action and associated to be shared with an accountant or person in charge of expense processing or maintaining the books.
  • the datapodTM may also be readily transcribed into a document form for storage or legal purposes.
  • Other applications that readily come to mind include virtually any avocation or profession where the sharing of annotated media objects is important—such as stamp collecting, teaching, law enforcement, industrial and fashion design, manufacturing quality assurance, scientific collaboration, geneology, etc.
  • the datapod' sTM ready support for transcription with precise clarity provides significant benefit to the users.
  • One of ordinary skill in the art will recognize that other applications not specifically described herein are also applicable.
  • Figure 2 shows a block diagram of a system in accordance with an embodiment of the present invention.
  • Figure 2 shows device 200 which may be used to create and share datapodsTM.
  • device 200 is a mobile phone, for example an iPhone® made by Apple® or any other type of smartphone.
  • device 200 is a tablet computer, for example an iPad® made by Apple® or any other tablet computer.
  • device 200 is any type of computing device such as other mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • Device 200 houses memory 210.
  • Memory 210 stores at least some portion of the acquired media object 110, data object 120 (annotation), and the datapodTM 130. Further memory components may be used in conjunction with memory 210 (not shown). Those memory components can be stored on a different system and/or at a different location such as in a networked device or PC or in a cloud server.
  • Device 200 also has a user interface 220.
  • the user interface 220 is used for acquiring media object 110 and annotating the media object with a data object 120.
  • User interface 220 provides a user friendly means to interact with device 200.
  • User interface 220 includes display, video, audio, and input device such as a touch screen, keyboard, stylus, gesture recognition, etc.
  • Device 200 also has a platform for sharing 230.
  • the user interface 220 is used to interface with the platform for sharing 230 to share the datapodTM 140.
  • the platform for sharing is an email or text message.
  • the platform for sharing may be via a wired or wireless local area network or interface such as Ethernet, high definition multi-media interface (HDMI), Display Port, Thunderbolt®, wireless (WiFi), Bluetooth, universal serial bus (USB) or Zigbee, etc.
  • the platform for sharing may be via removable media such as USB "Stick", Memory Card, subscriber identity module (SIM) Card, compact disc (CD) or digital video disc (DVD) or other such devices.
  • the platform for sharing is a private or public media or social media site for sharing such as Facebook®, Google+®, Pinterest® or YouTube®.
  • Figure 3 shows a typical user interface for creating a datapodTM as might be found on a mobile device.
  • the user interface of Figure 3 shows five areas of the screen, a primary display area for acquisition, display, navigation and markup 360, an area with real or touchscreen buttons related to acquiring a media object 320, an area related to creating an annotation data object 330, an area where datapodTM contents can be implicitly associated 330, and an area where datapodsTM can readily be shared 350 via email, text or web.
  • Figure 4 shows an embodiment of a user interface for creating a datapodTM, in accordance with various aspects of the present invention.
  • Figure 4 shows Figure 3 with the addition of a media object, a photo in this case, in the acquisition area.
  • the user uses media acquisition buttons 420 to acquire or upload a media object.
  • the user has acquired or uploaded a photo that contains images of a crowd with various people.
  • Figure 5 shows an embodiment of a user interface demonstrating navigation information, in accordance with various aspects of the present invention.
  • Figure 5 illustrates the usefulness of capturing navigation information from a touch screen, cursor buttons, gestures or other input mechanism while displaying the image of geometric shapes on the small screen of a mobile device to annotate the image.
  • Figure 5 shows device screen 500 and select acquisition media type buttons 515.
  • One of the select acquisition media type buttons is audio + navigation button 510.
  • a user who wants to annotate a media object with audio and also capture navigation information would use audio + navigation button 510.
  • audio + navigation button 510 Once audio + navigation button 510 is selected the user can navigate through the media object 520 by panning left, right, up or down across the image and/or zooming into or out of a portion of the image, etc, all while narrating the actions.
  • Figure 5 shows media object 520 as a group of geometric shapes; however, the media object could be any media object, as described above.
  • the user can then use the touch screen of the device, buttons on the device or other input mechanism (e.g., gestures) to expand or zoom in on a particular part of the image.
  • the image shown in Figure 5 shows the user zooming in on the square in the image 530.
  • the user can then continue to narrate the audio while zooming on the square 530.
  • the user can also perform other functions, for example, highlighting or circling a portion of the media object. While the user speaks and explains the media object, the user can move around the media object and navigate in or out of the media object. This navigation allows the user to identify something the user is talking about and see it clearly on the small screen.
  • Figure 5 also shows the user continuing to pan around and zoom on image 540. Again, this information is stored as part of the annotated information within the datapodTM.
  • the media object could contain a spreadsheet, pdf or an image of a spreadsheet and the user wants to refer to a particular line item or cell on the spreadsheet, perhaps to highlight an important figure, calculation, result or error, etc.
  • the user can zoom in on and highlight a particular line item on the spreadsheet while discussing it. That navigation information becomes part of the datapodTM.
  • the recipient(s) will see the image which will pan left, right, up and down and zoom in and out via the associated navigation information precisely as recorded by the user (sender) and will simultaneously hear the appropriate, synchronized audio recording.
  • the sender and recipient to communicate as if sitting right next to each other.
  • the datapodTM itself is shared with one or more recipients. The recipients then can use a datapodTM player to play the datapodTM as discussed below in reference to Figures 10-14.
  • the datapodTM is converted to a video and the video is shared with one or more recipients.
  • the media object could contain a child's artwork.
  • the annotation data object could be the child's voice while he describes different portions of the art. As he is describing the art he can pan to that portion and zoom in on it.
  • the annotated media object, the image of the artwork along with the navigation information and the audio forms the datapodTM.
  • the datapodTM can be shared with a recipient, for example, the child's grandparent. The grandparent would see the media object complete with navigation and hear the child's voice as if the grandparent were sitting beside the child describing the artwork.
  • Figure 6 shows an embodiment of a user interface for creating a datapodTM, in accordance with various aspects of the present invention.
  • Figure 6 is another example of using the audio+navigation function shown in Figure 5.
  • the embodiment shown in Figure 6 continues with the example of the media object shown in Figure 4.
  • Figure 4 shows screen 600, including an acquisition area with a photo of a crowd of people that has been uploaded or acquired. While Figure 6 shows a photo as the media object, the media object could be a video or any other media object described above.
  • the user (sender) is looking for a particular person in the crowd.
  • the user (sender) takes a photo using a mobile device and puts that photo on screen 600.
  • the user (sender) would like to indicate a specific person in the crowd so the photo is annotated using navigation button 620 and then by moving the person into the center of the screen 610 using the touch screen, physical buttons, voice command or other input method.
  • the user (sender) then continues to annotate by zooming in to make it easier to identify the face of the person 630.
  • the user (sender) can also be recording audio, for example, "I think this is the person we are looking for. I am going to zoom in further to see.”
  • the user (sender) can also use a pen to annotate the media object 640.
  • the user (sender) can also continue to record audio, for example, "Yes, this is the one we are looking for. See his face here.”
  • the user can continue to zoom in 650.
  • the user can also continue to record audio, for example, "Look at that scarf. It has the logo we are interested in finding.”
  • the audio recording and the navigation including panning, zooming and marking actions are properly synchronized in the resulting datapodTM.
  • the ability to pan, zoom and mark provides ease of communication when communicating to someone who is not co-located with the sender.
  • the resulting collection of annotated media objects becomes an extremely powerful communications capability due to the ability of the datapodTM to have the media object and one or more data objects appropriately synchronized.
  • the concept of Figure 6 could also be applied to multiple media objects. For example different media objects could be compared or contrasted along with their associated annotated data objects.
  • Figure 7 shows an embodiment of a user interface for creating a datapodTM using two media objects with narration, in accordance with various aspects of the present invention.
  • Figure 7 provides an example of a using two images as media objects and using narration as the data object.
  • the image used is of automotive parts. As discussed above, any media object could be used.
  • a first image is loaded as a media object 710.
  • the user interface shown in Figure 3 is used to load the image and to record the data object.
  • the data object is a voice audio recording, "The design features are different in two significant ways.
  • the 997 Bypass replaces the primary muffler and is a crossover design, meaning the left header feeds the right secondary muffler and vice versa.”
  • a second media object is loaded.
  • the second media object is another image of automotive parts 720.
  • another data object audio recording is recorded, "Unlike the 997 Bypass, the GT3 Bypass is installed after the primary mufflers, replacing the single combined secondary muffler. Exhaust gas is redirected through independent air tubes to the centrally located external exhaust tips.”
  • the datapodTM includes two media objects, the two photos 710 and 720 and two data objects, the two voice recordings.
  • the datapodTM can be shared with one or more recipients using the methods described above. Using the datapodTM to compare or contrast two or more annotated media objects can be an extraordinarily powerful communication tool.
  • Figure 8 shows an embodiment of a user interface for creating a datapodTM using two media objects with pen for markup and narration, in accordance with various aspects of the present invention.
  • the embodiment shown in Figure 8 uses the user interface shown in Figure 3 to compare two media objects using pen and narration.
  • the use of two media objects allows a user to compare and contrast the media objects while maintaining the appropriate synchronization of the data objects and media objects.
  • Figure 8 shows first media object 810 which can be loaded using the user interface show in Figure 3.
  • the user can also use the user interface shown in Figure 3 to mark up the media object 820 using the pen.
  • the markup shows the crossover of exhaust gas flow.
  • the user can also use the user interface shown in Figure 3 to record an audio recording, for example, "The design features are different in two significant ways.
  • the 997 Bypass replaces the primary muffler and is a crossover design, meaning the left header feeds the right secondary muffler and vice versa, while the GT3 Bypass employs the primary muffler and uses a central exhaust approach.”
  • the user can use the user interface shown in Figure 3 to load a second media object 830 and create a markup of the media object 840.
  • the user interface of Figure 3 can also be used to record an audio recording, for example, "The GT3 Bypass is installed after the primary mufflers, replacing the single combined secondary muffler. Exhaust gas is redirected through independent air tubes to the centrally located external exhaust tips.”
  • the datapodTM can be shared with one or more recipients using the methods described above. Using the datapodTM to compare two or more annotated media objects can be an extraordinarily powerful communication tool.
  • Figure 9 shows an embodiment of a user interface for creating a datapodTM using two media objects with navigation and narration, in accordance with various aspects of the present invention.
  • the embodiment shown in Figure 9 uses the user interface shown in Figure 3 to compare two media objects using navigation and narration.
  • the use of two media objects allows a user to compare and contrast the media objects while maintaining the appropriate synchronization of the data objects and media objects.
  • Figure 9 shows first media object 910 which can be loaded using the user interface show in Figure 3.
  • the user can also use the user interface shown in Figure 3 to pan around and zoom in on the media object 920.
  • the zoom in shows the crossover of exhaust gas flow.
  • the user can also use the user interface shown in Figure 3 to record an audio recording, for example, "The design features are different in two significant ways.
  • the 997 Bypass replaces the primary muffler and is a crossover design, meaning the left header feeds the right secondary muffler and vice versa.”
  • the user can use the user interface shown in Figure 3 to load a second media object 930 and zoom in on the media object 940.
  • the user interface of Figure 3 can also be used to record an audio recording, for example, "The GT3 Bypass is installed after the primary mufflers, replacing the single combined secondary muffler. Exhaust gas is redirected through independent air tubes to the centrally located external exhaust tips.”
  • the datapodTM can be shared with one or more recipients using the methods described above. Using the datapodTM to compare two or more annotated media objects can be an extraordinarily powerful communication tool.
  • a datapodTM can be sent as a datapodTM or as a video. If it is sent as a video file, there is no need for a datapodTM player to play the video. Any video player can be used to play the video file. However, it can be more efficient to send the datapodTM as a datapodTM rather than a video file.
  • a datapodTM can be smaller than an equivalent video file, requiring less space to store and less bandwidth to send, since it does not need to include resulting video frames, since, depending on the media objects, may only require images and data objects including navigation information and audio files, which collectively may be much smaller than a video with the 24, 30 or 60 frames of video per second typically required for smooth playback.
  • the datapodTM would only include the two (2) still images, the navigation information (pan and zoom) and the audio annotation. Assuming the resulting datapodTM in Figure 9 was 1 minute long in duration, the video version of the datapodTM, if constructed at the same resolution as the base image, could be as much as 30 times larger than the datapodTM itself. In the event where bandwidth or storage is at a premium, it could therefore be very advantageous to send the datapodTM as a datapodTM.
  • the datapodTM preserves the fidelity of the original media objects and data objects since it does not require the same compression levels needed for video transmission and storage.
  • sending datapodsTM in lieu of video may also preserve scarce computing resources and battery power on mobile and other computing devices.
  • Encoding video is a time and compute intensive process, such that creating a 1 minute video on some devices may take substantially longer than 1 minute.
  • the datapodTM is created at the time navigation, narration, etc., the resulting compute resources and battery power required to simply package the datapodTM for transmission is substantially less, thereby saving compute resources and preserving battery life. Transmitting datapodsTM also enables real-time collaboration since it is possible to communicate navigation information to a recipient who can follow along with a live annotation.
  • FIG. 10 shows a flowchart of a process to play a datapodTM, in accordance with various aspects of the present invention.
  • the datapodTM player receives the datapodTM 1010. It then unpacks the datapodTM 1020 into its component media objects and data objects. Finally, the datapodTM player views the datapodTM 1030 by playing the media and data objects maintaining the synchronization between the media and data objects.
  • Fig. 11 shows a functional block diagram of a device for playing a datapodTM, in accordance with various aspects of the present invention.
  • the datapodTM player can reside on any type of computing device 1100.
  • Device 1100 can be a mobile device or other platform including mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • the datapodTM player has a platform for receiving the datapodTM 1100. That platform receives the datapodTM and unpacks the datapodTM.
  • the device 1100 also has a user interface 1120 including video screen and in some cases audio playback and user input capabilities for interfacing with its user (recipient).
  • the device 1100 also has a memory 1130 for storing the datapodTM. Further memory components may be used in conjunction with memory 1130 (not shown). Those memory components can be stored at a different location, on a networked device or in a cloud server.
  • Figure 12 shows a user interface for playing a datapodTM, in accordance with various aspects of the present invention.
  • User interface 220 includes video, audio, and input device such as a touch screen, keyboard, or stylus.
  • Figure 12 shows screen 1200. Contained within screen 1200 are image area 1220, video area 1230, and text area 1240.
  • the image area 1220 is an area of the screen 1200 dedicated to displaying images.
  • Video area 1230 is an area of the screen 1200 dedicated to playing video.
  • Text area 1240 is an area of the screen 1200 dedicated to displaying text.
  • Screen 1200 can be user configurable to provide the various areas 1220, 1230 and 1240 in different locations on screen 1200 or different sizes. Alternatively, a plurality of screen areas of a particular type can also be provided. In addition, audio capabilities and user input areas may also be provided. Thus enabling the recipient to play the datapodTM appropriately such that each media object and data object is shown and shown in the appropriate synchronization.
  • Figure 13 shows an embodiment of a user interface for playing a datapodTM, in accordance with various aspects of the present invention.
  • Figure 13 illustrates how the example shown in Figure 6 could be played using a datapodTM player.
  • the datapodTM player can play the datapodTM in the same way and with the same level of detail as when the datapodTM was created.
  • the media object 1300 would be seen on the player followed by the panning 1310 then the zooming 1320, markup 1330, and further zooming 1340. Meanwhile at the appropriate times the synchronized audio recordings would also be played along with the images, panning, zooming, and marking, replicating with precise fidelity what the sender recorded.
  • Figure 14 shows a block diagram illustrating the relationship between creating and playing a datapodTM.
  • Figure 14 shows a device used to create a datapodTM 1410. Since mobile devices can be carried anywhere one embodiment would use a mobile device to create the datapodTM. However, the datapodTM could also be created on another type of device, such as other mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc. The mobile device can also be used to play the datapodTM 1420.
  • PCs personal computers
  • game systems such as other mobile devices, personal computers (PCs), game systems, automotive and avionics displays, digital picture frames, TVs, set top boxes, digital video and still cameras, smart office and home appliances and lab or industrial devices equipped with displays and audio/visual capabilities, wearable computers, etc.
  • the mobile device can also be used to play the datapodTM 1420.
  • the mobile device used to play the datapodTM can be the same mobile device used to create the datapodTM or it can be another mobile device that received the datapodTM.
  • Figure 14 also shows another device used to play the datapodTM 1430.
  • the datapodTM can be sent to a device other than a mobile device to be played, for example other mobile devices, TVs, PCs, Game Systems, Automotive Displays, etc.
  • Figure 14 also shows using a web server to stream the datapodTM 1440.
  • the datapodTM can be shared by streaming via a web streaming service.
  • the present invention can be implemented as a software application running on a mobile device such as a mobile phone or a tablet computer. It will be apparent to one of ordinary skill in the art that the present invention can be implemented as firmware in an field programmable gate array (FPGA) or as all or part of an application specific integrated circuit (ASIC) such that software is not required.
  • FPGA field programmable gate array
  • ASIC application specific integrated circuit
  • computer readable media includes not only physical media such as compact disc read only memory (CD-ROMs), SIM cards or memory sticks but also electronically distributed media such as downloads or streams via the internet, wireless or wired local area networks or interfaces such as Ethernet, HDMI, Display Port, Thunderbolt®, USB, Bluetooth or Zigbee, etc., or mobile phone system.
  • CD-ROMs compact disc read only memory
  • SIM cards or memory sticks but also electronically distributed media such as downloads or streams via the internet, wireless or wired local area networks or interfaces such as Ethernet, HDMI, Display Port, Thunderbolt®, USB, Bluetooth or Zigbee, etc., or mobile phone system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un système et un procédé servant à créer et à lire un datapod™ qui comprend l'association synchronisée de médias et d'objets de données. Un mode de réalisation de la présente invention implique la création d'un datapod™ en acquérant un objet média, annotant l'objet média avec un objet de données et associant l'objet média avec l'objet de données pour former une relation commandée ou synchronisée entre l'objet média et l'objet de données qui définit implicitement une connexion visuelle/auditive ou expérientielle similaire. La présente invention fournit également ses fonctionnalités à l'aide d'une interface utilisateur simple à utiliser qui permet à l'utilisateur de créer facilement l'association synchrone et de partager le datapod™ résultant avec le public visé.
PCT/US2013/050960 2012-07-19 2013-07-17 Procédé et système servant à associer des médias synchronisés en créant un datapod WO2014015080A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US13/553,562 2012-07-19
US13/553,529 2012-07-19
US13/553,529 US20120290907A1 (en) 2012-07-19 2012-07-19 Method and system for associating synchronized media by creating a datapod
US13/553,562 US20120284426A1 (en) 2012-07-19 2012-07-19 Method and system for playing a datapod that consists of synchronized, associated media and data

Publications (2)

Publication Number Publication Date
WO2014015080A2 true WO2014015080A2 (fr) 2014-01-23
WO2014015080A3 WO2014015080A3 (fr) 2014-04-03

Family

ID=49949369

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/050960 WO2014015080A2 (fr) 2012-07-19 2013-07-17 Procédé et système servant à associer des médias synchronisés en créant un datapod

Country Status (1)

Country Link
WO (1) WO2014015080A2 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030085870A1 (en) * 2000-07-17 2003-05-08 Hinckley Kenneth P. Method and apparatus using multiple sensors in a device with a display
US20070214407A1 (en) * 2003-06-13 2007-09-13 Microsoft Corporation Recognizing, anchoring and reflowing digital ink annotations
US20070256016A1 (en) * 2006-04-26 2007-11-01 Bedingfield James C Sr Methods, systems, and computer program products for managing video information
US20080104503A1 (en) * 2006-10-27 2008-05-01 Qlikkit, Inc. System and Method for Creating and Transmitting Multimedia Compilation Data
US20080263067A1 (en) * 2005-10-27 2008-10-23 Koninklijke Philips Electronics, N.V. Method and System for Entering and Retrieving Content from an Electronic Diary
US20110176179A1 (en) * 2002-07-27 2011-07-21 Archaio, Llc System and method for simultaneously viewing, coordinating, manipulating and interpreting three-dimensional and two-dimensional digital images of structures for providing true scale measurements and permitting rapid emergency information distribution

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030085870A1 (en) * 2000-07-17 2003-05-08 Hinckley Kenneth P. Method and apparatus using multiple sensors in a device with a display
US20110176179A1 (en) * 2002-07-27 2011-07-21 Archaio, Llc System and method for simultaneously viewing, coordinating, manipulating and interpreting three-dimensional and two-dimensional digital images of structures for providing true scale measurements and permitting rapid emergency information distribution
US20070214407A1 (en) * 2003-06-13 2007-09-13 Microsoft Corporation Recognizing, anchoring and reflowing digital ink annotations
US20080263067A1 (en) * 2005-10-27 2008-10-23 Koninklijke Philips Electronics, N.V. Method and System for Entering and Retrieving Content from an Electronic Diary
US20070256016A1 (en) * 2006-04-26 2007-11-01 Bedingfield James C Sr Methods, systems, and computer program products for managing video information
US20080104503A1 (en) * 2006-10-27 2008-05-01 Qlikkit, Inc. System and Method for Creating and Transmitting Multimedia Compilation Data

Also Published As

Publication number Publication date
WO2014015080A3 (fr) 2014-04-03

Similar Documents

Publication Publication Date Title
US20220200938A1 (en) Methods and systems for providing virtual collaboration via network
US20120284426A1 (en) Method and system for playing a datapod that consists of synchronized, associated media and data
US10044778B2 (en) Configuring channels for sharing media
US20170019363A1 (en) Digital media and social networking system and method
US9639545B2 (en) Online binders
TW201545120A (zh) 自動產生編譯視訊
EP3046107B1 (fr) Génération et affichage de vidéo des temps forts associée à des contenus de source
TW201545160A (zh) 編譯視訊的自動產生
TWI522823B (zh) 用於跨多種裝置之智慧型媒體展示技術
US20120290907A1 (en) Method and system for associating synchronized media by creating a datapod
US20160078582A1 (en) Sharing Media
EP3821379A1 (fr) Interface personne-ordinateur efficace en matière de calcul permettant une modification collaborative de contenu
KR101621496B1 (ko) 터치이벤트정보를 이용한 발표자료 재현 시스템 및 방법
TWI514319B (zh) 藉由虛擬物件編輯資料之方法及系統，及相關電腦程式產品
US20140181143A1 (en) File presentation method and apparatus
US20140304659A1 (en) Multimedia early chilhood journal tablet
US20150301725A1 (en) Creating multimodal objects of user responses to media
WO2014015080A2 (fr) Procédé et système servant à associer des médias synchronisés en créant un datapod
BE1021350B1 (nl) Werkwijze en systeem voor het tonen van een opeenvolging van beelden
US20150293678A1 (en) Story board system and method
TWI621954B (zh) 影像檔案分類方法及系統
JP2014110469A (ja) 電子機器、画像処理方法、及びプログラム
Zargham et al. User experience study of multiple photo streams visualization
Broekhuijsen Curation-in-Action: design for photo curation to support shared remembering
McFedries iPad Portable Genius: Covers iOS 8 and all models of iPad, iPad Air, and iPad mini

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13819659

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 13819659

Country of ref document: EP

Kind code of ref document: A2