WO2014072844A1

WO2014072844A1 - Audiovisual recordings

Info

Publication number: WO2014072844A1
Application number: PCT/IB2013/058015
Authority: WO
Inventors: Kevin Shaun CUPIDO
Original assignee: Idealounge (Pty) Ltd
Priority date: 2012-11-09
Filing date: 2013-08-27
Publication date: 2014-05-15

Abstract

Audiovisual recording data and a method for its production are provided. The data produces visual images displayed with the concurrent delivery of audio content, with the visual images being in the form of a plurality of selected still images associated with the audio content in a manner in which the still images are displayed in a sequence that is linked to the audio content. A particular still image is displayed over a time period that is linked to the progress of the audio content. The audio data constituting the audio content and visual image data constituting the still images are packaged together in a container format such as MP4; flash video format FLV or F4V that are suitable for transmission from one communications device to another. The data may be divided into bundles and may be edited prior to conversion to the container format.

Description

AUDIOVISUAL RECORDINGS

FIELD OF THE INVENTION

This invention relates to audiovisual recordings and, more particularly, audiovisual recordings of limited duration that can be formatted with the audio data and the image data together in a container format suitable for transmission and playing on numerous different mobile communications devices. The audiovisual recordings, it is envisaged, will be particularly useful for social, business and educational purposes

At the time of preparing this patent specification, the most attractive container format known to applicant is that known as MP4 but doubtless other container formats will become available as time passes although the likes of Flash video formats FLV and F4V are also in contention.

BACKGROUND TO THE INVENTION

Digital technology has fundamentally changed the way we work and play and two areas where we have seen significant developments in recent years have been in the mobile technology and digital photographic space. Today there is a myriad of mobile devices and digital cameras competing for market share, along with hundreds of thousands of software applications designed as "value-adds" for these useful tools. Digital images and video alone, dominate digital media, so that it comes as no surprise that there are numerous software applications that support these media. In addition to this, social media have played a significant role in driving the usage and sharing of digital media, which in turn, has also assisted the growth of new software applications and product development. Still images are used extensively on the Internet. On Facebook alone, over six billion photos are reportedly uploaded each month.

It is widely accepted that audiovisual recordings are a highly effective way in which information can be communicated from one person to another for various reasons which may simply be interest and involvement in the conduct of social activities or more business orientated such as promotional or sales activities. The most common audiovisual types of recordings of which applicant is currently aware is that in which audio is recorded along with a single image. Other than this, there is the well-known video recording which can be made using numerous different mobile devices ranging from video recorders and camcorders to mobile communications devices such as smart phones, tablets, digital and social cameras, to name just a few appropriate devices. Although video includes both motion and audio, it comes at a price as video is not the ideal delivery platform for many situations, and is typically hampered by file size, and /or image quality when compared to still images. Unlike photos, video typically needs to be edited -something not many people spend time doing!

This renders existing video recordings not suitable for easy communication from one device to another simply by virtue of their rather large file size of recorded data. It is, for example, not practical as a general rule to convert video data to a container format for transmission to other devices on which they can then be easily played by the recipient. It is accordingly, at least at the current stage of development of appropriate recordings, not a suitably convenient way in which information can be communicated from one device to another.

This difficulty seriously restricts the sharing of audiovisual recordings between different communications devices, whether mobile or fixed, and especially restricts the sharing of such recordings over social networks such as Facebook, You Tube, Twitter, Vimeo and the like.

Applicant believes there is a need for an audiovisual recording that may have a slightly different nature, but wherein the data may be configured to be of more manageable size than a conventional video recording.

It is to be noted that in this specification the term communications device is intended to have a broad meaning and includes smart phones, tablets and social cameras as well as computers and personal assistants that have the appropriate sound and image processing facilities.

SUMMARY OF THE INVENTION In accordance with a first aspect of this invention there is provided audiovisual recording data in which playing of the audiovisual recording data produces visual images displayed with the concurrent delivery of audio content, the audiovisual recording being characterised in that the visual images are in the form of a plurality of selected still images associated with the audio content in a manner in which the still images are displayed in a sequence that is linked to the audio content such that a particular still image is displayed over a time period that is linked to the progress of the audio content, and wherein audio data constituting the audio content and image data constituting the still images are packaged together in a container format suitable for transmission from one communications device to another.

Further features of the first aspect of the invention provide for the plurality of selected still images to be created by taking a plurality of successive still images using a digital camera or, alternatively, for the plurality of selected still images to be manually or automatically selected images extracted from a video recording; for the audio content to be selected from concurrent ambient sound at a time during which the plurality of successive still images is created or, in the alternative, for the audio content to be a pre-recorded audio content that may optionally be the audio content of a video recording from which selected still images are extracted for successive display by way of the audiovisual recording data according to the invention; for the container format to be that of MP4 that is discussed further to some extent below, or flash video format FLV or F4V; and for the files to be converted into a MP4 file or other container file online via a web service in an automated roundtrip process. Still further features of the first aspect of the invention provide for the audiovisual recording data to include plural data bundles in which each bundle contains audio data and image data of at least two successive images to be displayed during playing of the audio data; for the bundles to represent recording activity between successive pauses during a recording session; and for the bundles to each be associated with an identification number whereby the order in which the bundles are played, in use, can be varied using a suitable programme.

In accordance with a second aspect of the invention there is provided a method for creating audiovisual recording data as defined above, the method comprising the steps of selecting an audio input type from that of recording concurrent ambient sound and that of utilising a pre-recorded sound, selecting or creating a plurality of still images each of which becomes associated with a particular time slot in the audio data with appropriate interrelationship between the audio data and image data for positioning the display of the image data according to the progress of playing of the audio data, in use; and converting the combined audio data and image data into a container format suitable for transmission from one communications device to another.

Further features of the second aspect of the invention provide for the audio data and image data to be edited prior to conversion into the container format wherein the editing may involve the addition, removal or otherwise of still images in particular, but not exclusively, the addition of promotional or informative still images; for the audiovisual recording data to be divided into bundles each having its own series of plural still images with each bundle being associated with an identification number whereby its order of sequence of playing can be varied; for a new bundle to be started after each time that recording has been "paused" during the creation of the audiovisual recording data; and for the method to be carried out on a single communications device having appropriate sound and image processing facilities with the conversion to the container format actually being carried out at a separate service facility that may be a remote facility with communication optionally being by way of the Internet.

In accordance with a third aspect of the invention there is provided a communications device having appropriate audio and image processing facilities wherein the communications device is programmed to enable audio data to be recorded and played and to enable image data representing plural still images to be taken using a built in camera or plural still images to be extracted from a video recording; the communications device having controls provided thereon for manual operation in order to create the plural still images or extract images from an existing video recording; the communications device further being programmed to enable either on board processing or remote processing of the combined audio data and image data to convert the data to a container format suitable for transmission from one communications device to another.

Further features of the third aspect of the invention provide for the communications device to be programmed to divide the audio data and image data into bundles that are suitable for separate editing; and for the programming of the communications device to include the facility of editing in a manner broadly outlined in relation to the second aspect of the invention. In accordance with a fourth aspect of the invention there is provided a program for a communications device as defined above wherein the program is composed of processing facilities such that when installed on a communications device it enables that device to operate as the communications device defined above.

A data file produced according to the invention can be shared via native social media applications of a communications device. In order that the above and many other features of the invention may be more fully understood an expanded description thereof now follows with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:-

Figure 1 shows a front face of one type of mobile wireless communications device that can be employed in exercising the invention;

Figure 2 shows the rear face thereof with a superimposed block diagram of selected internal components;

Figure 3 is a block diagram broadly illustrating the invention;

Figure 4 illustrates in more detail the recording of audiovisual recording data according to the invention;

Figure 5 illustrates various editing functions that can be applied to data recorded according to the invention; Figure 6 illustrates in more detail the function of a mobile device configured to operate according to the invention; and,

Figure 7 illustrates in more detail the preparation of an audiovisual recording data file according to the invention.

DETAILED DESCRIPTION WITH REFERENCE TO THE DRAWINGS

As indicated above, the invention can be employed as applied to any appropriate communications device and simply by way of example, there is illustrated in Figures 1 and 2, a smart phone (1 ) having controls (2) on one face thereof and, in the alternative or additionally, a suitable touch screen (3); a microphone (4); a loudspeaker or earphone (5); and a camera (6). The smart phone has a processor (CPU) (7) and a data base (8) as well as the usual random access memory (9) and any suitable wireless transmitter/receiver (10). The processor is programmed to function according to the invention.

Any of the software components or functions described in this application, may be implemented as software code to be executed by the processor using any suitable computer language such as, for example, Java, C++ or Objective-C using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands on a computer readable medium, such as the random access memory (RAM), a read only memory (ROM), or a magnetic medium. Any such computer readable medium may reside on or within a single device, or may be present on or within different apparatuses within a system or network.

Referring now more particularly to Figure 3 of the drawings, audiovisual recording data according to the invention may be produced generally as illustrated in that a recording session can be commenced by triggering a "start" command (21 ) by way of appropriate device controls on the mobile device that commences a new session (22) by starting an audio record facility (23). The audio input may be a real-time input by way of the microphone (4) or it could be the audio track of a pre-recorded video (24). The audio input could be a narrative, a commentary, ambient sound such as may be present at a celebration or other function, or it may be the ambient sound in a natural environment such as the sound of birds or another animal; the sound of water running; the ocean; the wind etc. The audio record facility may involve the manual selection of the type of input that is to be made, as may be dictated by the particular setup.

Individual still images may then be captured by operating an "acquire" function, as indicated by numeral (25), either by way of the camera in realtime or by selecting still images from the pre-recorded video. The taking of the succession of images or the selection thereof in each instance may be carried out manually but it is also within the scope of this invention that a somewhat more automated still image taking or extraction process can be built into the program of the processor. Thus, in the instance of the use of the camera, the program may have a timer facility that may be set to automatically take one still image at regular intervals, say of the order of every half to 5 seconds.

The simultaneous audio facility has a pause function (26) and the audio data and still image data are both recorded in the data base (8) with a time correlation so that each still image is associated with a particular part or timeslot of the audio signal. A "save" facility (27) enables multiple sessions to be linked in the data base.

When a recording has been completed, it may be edited as discussed more fully below and thereafter it may be extracted and transmitted by a "build" feature and thence to a remote server (28), typically by way of the Internet (31 ). At the server the file is treated as discussed more fully below in relation to Figure 7 and is returned by way of the Internet in the form of an MP4 container file that can be shared easily with others as indicated at numeral (32).

Figure 4 shows in somewhat more detail the recording procedure and also shows the different facilities and the different actions involved in the different facilities. As will be quite apparent from Figure 4, the audiovisual recordings are preferably made in bundles and various bundles can be associated with each other to create one composite audiovisual recording. Turning now to Figure 5 of the drawings, various editing facilities are shown that clearly indicate that bundles can be deleted, moved or copied. Also the images may be edited, replaced, deleted, or moved to a different position. There are also video titling features for adding titles, a facility to equalise the time spacing between successive still images and even a facility to recapture the audio part of a bundle. In all instances a bundle table is updated as well as a bundle image table each time an editing action is carried out. Titles can be added as may be required and may be superimposed over any existing images. Branding of file images with a watermark can also be achieved as can the addition of other special effects to images.

Figure 6 shows in more detail a mobile device programmed to record the audiovisual recording in a bundle format and using a global timer (35). The program includes a facility for allocating a new incremented bundle number as indicated by numeral (36). Also, the program is enabled to allocate unique image names to each image in a bundle (37) and to record a global time (38) when an image is created or selected. It is to be noted that each time that the recording procedure is paused, a new bundle is automatically created when the system is un-paused. New bundles are inserted into the recorded data in the order in which they are created and that order may, of course, be changed during editing.

Turning now to Figure 7 of the drawings, when an audiovisual recording is to be converted into a container format such as MP4, the relevant metafile is extracted as indicated by numeral (41 ) from the mobile device data base (8) and an XML definition is created with the images in .jpg format and the audio in .ogg format together with the relevant timing information as indicated by numeral (42). The files are zipped as indicated by numeral (43) and the zipped file (44) is uploaded to the server via the Internet as indicated above. The server can process the zipped file in any suitable way that may involve creating a unique processing directory (45); unpacking the zipped file (46); looping through the bundle nodes in the XML file (47); creating a numbered bundle in the processing directory (48); making bundle video frames from the still images (49); applying effects or overlays to the bundle video (50); checking (52) if there are further bundles; and once all the bundles have been processed, joining separate bundles into a single file (53); passing the single file through an ad server (54) for the purpose of associating any advertising video frames with the file; and returning the computer video file in the MP4 container format to the mobile device.

The MP4 container format is currently preferred as the container format as it allows for ease of combination of audio, video, subtitles and still images in one single file. It also allows for advanced content such as 3D graphics, menus and user interactivity. MP4 is regarded as a reliable application that requires a relatively low amount of bandwidth so that many may take advantage of using the resultant file. The enhancement of the speed of various types of Internet connections has helped to make MP4 more accessible to a greater audience and it is commonly used for sharing video files on the Internet.

MP4 is an abbreviated term for MPEG-4 Part 14. It may also be referred to as MPEG-4 AVC (Advanced Video Coding). As the name suggests, this is a format for working with video files. MPEG refers to Motion Pictures Expert Group which is responsible for setting the industry standards regarding digital audio and video. The MPEG-4 video format uses separate compression for audio and video tracks; video is compressed with MPEG-4 or H.264 video encoding; audio is compressed using AAC (Advanced Audio Coding) compression. As indicated above, an audiovisual recording data file according to the invention may be created in which a plurality of selected still images are associated with the audio content in a manner in which the still images are displayed in a sequence that is linked to the audio content such that a particular still image is displayed over a time period that is linked to the progress of the audio content.

A test carried out on a video that had a file size of 3.4Mb (14seconds), resulted in an audiovisual file according to this invention of 220Kb with a 1 second delay between the individual still frames that were captured from the video. It was estimated that the size of a file could be anything up to 10 times smaller which is highly significant.

Use of the invention enables users to produce packages of images and sound in a media-rich format, which by its very nature, is considerably smaller than a standard video file, which is bandwidth intensive. This is achieved by using selected still images only and compressing both the image and audio files and "stitching" them together in a container file such as an MP4 file. The software whereby the invention may be implemented may either be distributed via online mobile app stores or it will ship with specific digital mobile devices as a native application.

Uses of the invention are extensive and unlimited. Simply by way of example, it can be used to capture a toddler's first words; to compile a snag list in the building industry; to showcase a property for sale; to promote a restaurant/gallery/shop; to support an insurance assessor's report; to produce a short instructional presentation; to advertise a product/service/city/town or create other marketing material; or record an interview. It is thus possible to record the audio at birthday parties; the announcement and cheers at a graduation ceremony; record training sequences or walk-throughs; for an estate agent to create a file showing various features of a residence or workplace that could be transferred to a potential customer with great ease;

The invention thus provides the opportunity to use existing technology to package media-rich information in a lightweight file format and share it via Social Media platforms and email.

As will be apparent to those of ordinary skill in the art, the invention applies to audiovisual recordings having an appropriate duration and, whilst not being limiting on the scope of the invention, it is envisaged that audiovisual clips having a duration of not more than about 30 to 40 seconds would be most appropriate.

Numerous variations may be made to the implementation of the invention described above without departing from the scope hereof.

Claims

Audiovisual recording data in which playing of the audiovisual recording data produces visual images displayed with the concurrent delivery of audio content, the audiovisual recording being characterised in that the visual images are in the form of a plurality of selected still images associated with the audio content in a manner in which the still images are displayed in a sequence that is linked to the audio content such that a particular still image is displayed over a time period that is linked to the progress of the audio content, and wherein audio data constituting the audio content and visual image data constituting the still images are packaged together in a container format suitable for transmission from one communications device to another.

Audiovisual recording data as claimed in claim 1 in which the plurality of selected still images is selected from the result of taking a plurality of successive still images using a digital camera and a plurality of manually or automatically selected images extracted from a video recording.

Audiovisual recording data as claimed in either one of claims 1 or 2 in which the audio content is selected from concurrent ambient sound at a time during which a plurality of successive still images is created and a pre-recorded audio content.

Audiovisual recording data as claimed in any one of the preceding claims in which the container format is selected from that of MP4; flash video format FLV or F4V.

Audiovisual recording data as claimed in any one of the preceding claims in which the audiovisual recording data includes plural data bundles in which each bundle contains audio data and image data of at least two successive images to be displayed during playing of the audio data.

Audiovisual recording data as claimed in claim 5 in which the bundles represent recording activity between successive pauses during a recording session.

Audiovisual recording data as claimed in either one of claims 5 and 6 in which the bundles are each associated with an identification number whereby the order in which the bundles are played, in use, can be varied using a suitable programme.

A method for creating audiovisual recording data as defined in any one of the preceding claims comprising the steps of selecting an audio input type from that of recording concurrent ambient sound and that of utilising a pre-recorded sound, selecting or creating a plurality of still images each of which becomes associated with a particular time slot in the audio data with appropriate interrelationship between the audio data and image data for positioning the display of the image data according to the progress of playing of the audio data, in use; and converting the combined audio data and image data into a container format suitable for transmission from one communications device to another.

A method as claimed in claim 8 in which the audio data and image data are edited prior to conversion into the container format wherein the editing involves any one or more of the addition, removal or otherwise of still images and the addition of promotional or informative still images.

10. A method as claimed in either one of claims 8 or 9 in which the audiovisual recording data is divided into bundles each having its own series of plural still images with each bundle being associated with an identification number whereby its order of sequence of playing can be varied.

1 1 . A method as claimed in any one of claims 8 to 10 in which a new bundle is started after each time that recording has been "paused" during the creation of the audiovisual recording data. 12. A method as claimed in any one of claims 10 to 13 in which the method is carried out on a single communications device having appropriate sound and image processing facilities with the conversion to the container format actually being carried out at a separate remote service facility.

13. A communications device having appropriate audio and image processing facilities wherein the communications device is programmed to enable audio data to be recorded and played and to enable image data representing plural still images to be taken using a built in camera or plural still images to be extracted from a video recording; the communications device having controls provided thereon for manual operation in order to create the plural still images or extract images from an existing video recording; the communications device further being programmed to enable either on board processing or remote processing of the combined audio data and image data to convert the data to a container format suitable for transmission from one communications device to another.

14. A communications device as claimed in claim 13 in which the communications device to be programmed to divide the audio data and image data into bundles that are suitable for separate editing.

15. A communications device as claimed in either one of claims 13 or 14 in which the communications device is programmed to enable editing of the audio data and image data by any one or more of the addition, removal or otherwise of still images and the addition of promotional or informative still images.