AU2002301440B2

AU2002301440B2 - System and apparatus for processing and viewing video images

Info

Publication number: AU2002301440B2
Application number: AU2002301440A
Authority: AU
Inventors: Adrian Davis
Original assignee: Individual
Current assignee: Individual
Priority date: 2001-10-16
Filing date: 2002-10-16
Publication date: 2004-03-25
Anticipated expiration: 2022-10-16
Also published as: NZ514809A

Description

16/10/2002 13:06 6444995088 SCHUCH COMPANY PAGE 05/23

AUSTRALIA

Patents Act 1990 COMPLETE SPECIFICATION STANDARD PATENT Application No.: Application Date: Priority Date: Applicant: Address for Service: 16 October 2001 (New Zealand Patent Application No. 514809) Adrian Davis 34 Aotaki Street Otaki NEW ZEALAND Schuch Company PO Box 6411 Baulkham Hills BC NSW 2153 Complete Specification for the invention entitled: SYSTEM AND APPARATUS FOR PROCESSING AND VIEWING VIDEO

IMAGES

The following statement is a full description of this invention, including the best method of performing it known to us: 16/10/2002 13:06 6444995088 SCHUCH COMPANY PAGE 06/23 SYSTEM AND APPARATUS FOR PROCESSING AND VIEWING VIDEO IMAGES TECHNICAL FIELD This invention relates to video image processing methods and systems. More particularly, but not exclusively, the present invention relates to the synthesis of multiple video images processed and streamed over the intemrnet via a suitable website platform and viewable by a user who can have a degree of control over the images being viewed.

BACKGROUND ART Conventionally video images are broadcast to be received by a standard television system displaying a two dimensional image from one camera lens. Such two dimensional displays can restrict viewing pleasure by limiting the depth of the original subject of the image.

Two-dimensional images from one camera lens only allow monocular viewing of an image. As human vision is binocular such imagery from a single lens is limiting when a fuller view of a scene may be desired.

It is an object of the invention to provide a system for video image processing that overcomes at least some of the abovementioned problems, or at least to provide the public with a useful choice.

It is an object of the invention to provide a method of processing video images that overcomes at least some of the abovementioned problems, or at least to provide the public with a useful choice.

16/10/2002 13:06 6444995068 SCHUCH COMPANY PAGE 07/23 SUMMARY OF THlE INVENTION According to a broad aspect of the invention there is provided a method of processing continuous video images from a plurality of camera lenses, the method comprising the steps of: a. capturing at least two visual images from respective camera lenses in synchronisation configured and arranged for panoramic viewing of a scene and recording such images; b. converting the recorded images to a suitable digital format for processing the images; C. processing the images using a digital processing means controlled by a suitable software program; d. combining or stitching the captured images in a predetermined arrangement to form a programme of panoramic images; and e. controlling the interaction with the programme of panoramic images by way of end user controls.

Preferably five horizontal images and three vertical images forming a vertically and horizontally stitched panoramic image is obtained and processed.

Desirably in step d. the plurality of horizontal images captured are stitched substantially along adjacent side edges and then each set of horizontal images are stitched substantially along adjacent tops and bottom edges to form one panoramic image.

Advantageously step c. includes enhancing a said plurality of images by adjusting the brightness, contrast and colourisation of each image captured to form substantially similar images.

Preferably in step e. a user controls the speed and area of viewing of the programme of panoramic images.

16/10/200882 13:06 6444995088 SCHUCH COMPANY PAGE 08/23 Preferably audio recordings are obtained and synchronised with the video images by the processing means.

Preferably in step d. the programme is modified by creating any one or more of the following features including polorised location description panels, voice-overs, credits, advertisements and a title. Advantageously in step d. the programme is enhanced by creating any one or more of the following features including video cassette recorder type animated function button means, a speed control animated remote means, key frame programming at the beginning of each individual video segment, mouse control motion computer program for controlling direction, and a direction program associated with an animated remote control means.

Desirably in step d. the programme is completed and uploaded to a website hosting the programme, and wherein in step e. a user is provided with access means to access the programme via an internet enabled communication means.

According to a second aspect of the invention there is provided a system for processing panoramic video images, the system including a plurality of camera lenses configured and arranged, in use, to capture and record suitable overlapping visual images and generate synchronous output video signals, the video signals are fed into a digital processing means, the digital processing means being provided with suitable computer software to convert the images to a suitable digital format for processing by combining or stitching the video images to form a programme of panoramic images, and a computer server means suitably adapted to store and allow access to the programme of panoramic images via an internet-enabled communication means.

Preferably the computer server means hosts a website allowing access by an internet user to the programme of panoramic images.

Desirably the website is interactive to allow a user to control the panoramic images and control the speed of the video stream and/or the panning motion.

16/10/20~02 13:06 6444995088 SCHUCH COMPANY PAGE 09/23 Advantageously the programme of panoramic images are viewed by an end user using a viewing monitor means adapted to allow a user to control the programme of panoramic images being obtained.

Desirably the plurality of camera lenses include at least two horizontally aligned camera lenses and at least two vertically aligned camera lenses are configured and arranged in a casing, and wherein the horizontally aligned camera lenses are mountable in a circular orientation, and the at least two vertically aligned camera lenses are arranged to capture images above and below a corresponding horizontally aligned camera lens, in use, to allow a panoramic image of the view to be obtainable.

Alternatively five horizontally aligned camera lenses are provided, each horizontal camera lens being configured with at least two vertically oriented camera lenses.

BRIEF DESCRIPTION OF THE DRAWINGS A preferred embodiment of the invention will now be illustrated, by way of example only, with reference to the accompanying drawings in which: Figure 1: Shows a view of an arrangement of main components of the system according to one embodiment of the invention; and Figure 2: Shows a camera unit having a plurality of lenses in a desirable arrangement.

DETAILED DESCRIPTION OF THE DRAWINGS Referring to figure 1, a system for video image processing, generally referred to as 1, according to a preferred embodiment of the invention, is illustrated.

The system I includes a plurality of video camera lenses and associated recorders 2 configured and arranged to overlap areas of an object or subject 3 being captured. The 16/10/2002 13:06 6444995088 SCHLJCH OMPANY PAGE 10/23 arrangement can be by mounting each video camera recorder 2 in different sectors about the subject 3 to allow video images of areas of the subject to be recorded.

Alternatively, a multiple camera lens unit 20 as described with reference to figure 2, may be used.

The output signals of each camera or lens 2 are fed via communications means 4 to a digital image processing means 5. In this embodiment three recorders are shown although it will be appreciated that at least two recorders can be used to capture at l east two video images, or a multiple lens arrangement or camera unit 20 can alternatively be configured and arranged with the system 1.- The image processing means 5 includes a computer processing means program med with suitable computer software to convert the inputted video images captured from the recorders 2 for conversion to a desirable digital format for manipulation or processing.

Conventional video capture software may be used and captured video footage from each lens or recorder 2 can be saved as desirably a .mov, m peg or an .avi file. These files can be desirably converted to swift (.swf) files or any other suitable format using computer conversion software such as, for example, Wifdform ElixN 1.52 software.

Audio signals can be saved as .wav or mp3 files as appropriate.

Manipulation or processing preferably includes combining or stitching the plurality of synchronised video images to form one panoramic view of the subject 3. Initial synchronisation of the images for each recorder 2 may be achieved by sychronising at the commencement of recording of the video recorders 2. Any known synchronisation technique may be applied.

Computer software such as, for example, Macromedia Flash"' 5 and/or Macromedia Director' 8.5 software can be used to set the platform for the manipulation process.

Manipulating the audio and/or visual images in digital format can include fine tuning the visual clarity, stitching and masking, of the various swift video movie files. This process may include importing and incorporating any narration, credits, titles, or otherwise.

16/10/2002 13:06 6444995088 SCHLJCH COMPANY PAGE 11/23 Preferably audio signals are also obtained from the scene alongside the video recorders, and such audio signals are synchronised with the video images captured and forms part of the programme of panoramic images that can be viewed by a user.

Video Cassette recorder (PFVCRP) type player buttons preferably form a part of the functionality of the completed programme of panoramic images. The swift programme files are compiled, and over lap in synch ron isation. Key frame programming can be inserted on time lines within the Macromedia FlashM and/or Macromedia Director' software at the commencement of each video image from each recorder 2 to allow for self-programming by a user and use of the VCI buttons.

A computer program will be used to allow for panoramic viewing of the programme.

Other features of functionality may be incorporated such as computer mouse control operations and speed control operations. Once all the features of the programme are incorporated, a programme File is published and exported or uploaded to a website 7 being hosted by a web server 6. The web sewver 6 can function as a mainframe computer to store all interactive movies accessible from the website.

The programme can preferably be interactive to allow users 8 to have access via a data communication means or internet enabled communication means 9 to select the way they desire to view the programme. A user can view the movie by following a desirable navigation process. It is envisaged that a viewer or user 8 can view a programme or movie via a display means in the form of a head mounted display means or what is commonly known as virtual glasses 10. The glasses 10C are associated with a computer connected to the website 7 via the internet 9. Virtual glasses 10 include miniature screens locatable within the glass lenses and allow a user 8 to see the movie or panoramic images without distraction, and with reasonable quality and effect.

Alternatively it is envisaged within the scope of the invention that as an end user viewer option the programme can be projected onto a display means in the form of a suitable projector screen using at least one projector or multiple projectors as desired (rnot 16/18/2002 13:06 6444995088 SCHUCH COMPANY PAGE 12/23 shown), and wherein such viewing may include a suitable interactive module for controlling and navigating the programme, such navigation may include any one or more of the following control features.

An example of the types of user functionality or control features for a user 8 having access to a movie on the website 7 include any one or more of the following features of functionality: 1 Left to right viewing activated by either a bar at the bottom of a web page frame or separate table and/or a mouse motion activated program attached to the (.swf) media file, Macromedia" Flash 5 built. An interactive animated remote control can provide an alternative panning or scrolling option; 2 Speed control- a viewable interactive speed control animation added to the remote control animation will be created to provide this form of interactivity.

The speed control animation can be accompanied with a program (source code file data) developed within the Macromedia Flash 51M program or Macromedia Director m 8.5 software.

3 Self-selected video segments key frames will be inserted in the Flash 5 T (.fla) project file or Director' software (.dir) project file of a movie production enabling viewers to selected identifiable video segment options. This selfselection process will be revealed as a separate animation developed as a drop-down or pop-up menu attached to the web page or flash panorama movie file, in Macromedia, Flash 5 TM or Macromedia Director M 8.5 software.

4. Activating Video Buttons additionally animated buttons that symbolise the functions of activating a standard VCR player; CD player, tape deck etc will be attached to the animated remote control to provide panoramic movie functionality, for example; buttons to command returning to the beginning of a movie, rewinding to the beginning of the last movie segment, playing, skipping to the next movie segment, and pausing, to name a few.

16/10/2002 13:06 6444995088 SCHUCH OHPANY PAGE 13/23 The following processing steps show a non-limiting embodiment by way of example only. The post-production method can involve two stages. These stages include video post-production techniques and web content programming.

The video post-production techniques may desirably include any one or more of the following features enhancing functionality: 1 Visually enhancing all separate video footage captured of a single panoramic movie production to be substantially similar. For example, brightness, contrast, colourisation all need to be taken into consideration; 2 Creating a panoramic movie title; 3 Creating a polarized location description panels; 4 Creating voice-avers; 5 Creating credits; and 6 Creating commercial advertisements.

The next stage to completing an interactive website and digital panoramic movie can include any one or more of the following features enhancing functionality: 1 Capturing all separate camera footage of a single movie projet, associated title, voiceover, credits and commercials, and save as a ,mov or .avi file using any video capture software. The audio file will need to be captured and saved as a wave (.wav) or mp3 file; 2 Converting all .avi or .mov file to swift (.swO) using Wildform FlixW 1.52 software; 3 Importing each .swf file of a single project into the Macromedia Flash'" or Director'" 8.5 program to begin the programming manipulation process; 4 Fine tuning visual clarity of each .swf file of a single movie project; 5 Stitching and masking all movie files (.swf) together; 6 Importing the title of the panoramic movie project; 7 Importing the voiceovers; I 16/18/2002 13: 06 6444995088 SCHUCH COMPANY PAGE 14/23 8 Importing credits; 9 Importing commercial advertisements; Creating a VCR player animated function buttons; 71 Creating a speed control animated remote; 12 Arranging all swift (,swf) movie files in order on the time line within the Flash' or Directorm software program. The video swift movie files will over lap in synchronisation; 13 Inserting key frame programming on the timeline at the beginning of each individual video segment option enabling self-selection programming and operation of the video buttons attached to the animated remote; 14 Applying the necessary programme that will allow panoramic viewing in motion; Applying a mouse control motion program for direction; 16 Applying a direction program associated with the animated remote control; 17 Applying the speed control program to the panoramic movie file; 18 Applying the activating video button functionality program attached to the animated remote; 19 Publishing a project file; and Exporting as a complete .swf file uploadable to a website.

It is envisaged that the system may include a decoder for access to available movies.

Referring now to figure 2, a camera unit having a plurality of lenses in a desirable arrangement, generally referred to as 20, is illustrated.

The camera unit 20 includes a plurality of image sensors desirably in the form of camera lenses 21a, 21b, 21c mounted in a housing or casing 22. The casing 22 is preferably of a suitable durable and resilient material that is water impervious, thus allowing the camera unit 20 to be used in a variety of weather conditions. For example, metals, plastics material or glass, or a combination thereof, may be applied.

16/10/20082 13:06 6444995088 SCHUCH COMPANY PAGE 15/23 An array of lenses 21a, 21b, 21c are spaced apart about the circular or multi-sided casing 22 such that they are directed to the scene being captured. It is envisaged that the casing 22 may consist of the same number of sides as the number of horizontally aligned lenses. A row of horizontally aligned lenses 21a are preferably aligned with respective vertically aligned lenses 21b, 21c, as shown, such that an alignment of images of a scene in a vertical plane is obtainable. Any suitable mounting means 23 can be incorporated with the casing 22. Desirably a gyroscopic stabiliser device is incorporated with the mounting means 23 to assist with stabilising the camera unit The images captured by the lenses 21 of the camera unit are fed via suitable communication means to a recorder (not shown) which are then fed to the processing means 5 (as seen in figure 1).

The mounting means 23 may be adapted with rotational drive means (not shown) to allow the camera unit 23 to be rotated, if required. The rotational drive means may be activated remotely with suitable remote activation means.

It will be appreciated that further camera lenses or image sensors may be incorporated into the casing.

It is envisaged that the invention can be adapted to be used to capture video footage from any one or more of the following sample of the types of programmes that may be of interest to viewers including travel programmes, sports events, entertainment venues, carnivals, fashion events, real estate listings all allowing interactive viewing by a user of the scene being captured by way of a plurality of images forming a panoramic scene, It is also envisaged that the systems and methods of the invention can be used for remote security monitoring and for remote supervisory work of factory floors and even human and animal operating theatres, when combined with known technologies.

Wherein the aforegoing reference has been made to integers or components having known equivalents, then such equivalents are herein incorporated as if individually set forth. Accordingly, it will be appreciated that changes may be made to the above 16/10/2002 13:06 6444995088 SCHUCH COMPANY PAGE 16/23 described embodiments of the invention without departing from the principles taught herein.

It is to be understood that the above description is intended to be illustrative, and not restrictive. Additional advantages of the present invention will become apparent for those skilled in the art after considering the principles in particular form as discussed and illustrated. Thus, it will be understood that the invention is not limited to the particular embodiments described or illustrated, but is intended to cover all alterations or modifications which are within the scope of the appended claims.

Claims

1. A method of processing continuous video images from a plurality of camera lenses, the method comprising the steps of: capturing at least two visual images from respective camera lenses in synchronisation configured and arranged for panoramic viewing of a scene and recording such images; converting the recorded images to a suitable digital format for processing the images; processing the images using a digital processing means controlled by a suitable software program; combining or stitching the captured images in a predetermined arrangement to form a programme of panoramic images capable of being subsequently viewed by at least one user; and controlling the interaction with the programme of panoramic images by way of end user controls accessible by an end user via data communication means, and wherein the end user selects a panoramic image for viewing, and in the case of a pre-recorded programme of panoramic images, controls the speed and sequence of segments of the programme being viewed.

2. A method according to claim 1 wherein five horizontal images and three vertical images forming a vertically and horizontally stitched panoramic image is obtained and processed.

3. A method according to claim 2 wherein in step d. the plurality of horizontal images captured are stitched substantially along adjacent side edges and then each set of horizontal images are stitched substantially along adjacent tops and bottom edges to form one panoramic image.

4. A method according to any one of the preceding claims wherein step c. includes enhancing a said plurality of images by adjusting the brightness, contrast and colourisation of each image captured to form substantially similar images.

5. A method according to any one of the preceding claims wherein in step e. a user controls the speed and area of viewing of the programme of panoramic images.

6. A method according to any one of the preceding claims wherein audio recordings are obtained and synchronised with the video images by the processing means.

7. A method according to any one of the preceding claims wherein in step d. the programme is modified by creating any one or more of the following features including polorised location description panels, voice-overs, credits, advertisements and a title.

8. A method according to any one of the preceding claims wherein in step d. the programme is enhanced by creating any one or more of the following features including video cassette recorder type animated function button means, a speed control animated remote means, key frame programming at the beginning of each individual video segment, mouse control motion computer program for controlling direction, and a direction program associated with an animated remote control means.

9. A method according to any one of the preceding claims wherein in step d. the programme is completed and uploaded to a website hosting the programme, and wherein in step e. a user is provided with access means to access the programme via an internet enabled communication means.

10. A system for processing panoramic video images, the system including a plurality of camera lenses configured and arranged, in use, to capture and record suitable overlapping visual images and generate synchronous output video signals, the video signals are fed into a digital processing means, the digital processing means being provided with suitable computer software to convert the images to a suitable digital format for processing by combining or stitching the video images to form a programme of panoramic images, and a computer server means suitably adapted to store and allow access to the programme of panoramic images via an internet-enabled communication means, and end user control means configured and arranged, in use, allowing an end user to select a panoramic image for viewing, and in the case of a pre-recorded programme of panoramic images, to control the speed and sequence of segments of the programme being viewed.

11. A system according to claim 10 wherein the computer server means hosts a website allowing access by an internet user to the programme of panoramic images.

12. A system according to either claim 10 or claim 11 wherein the website is interactive to allow a user to control the panoramic images and control the speed of the video stream and/or the panning motion.

13. A system according to any one of claims 10 to 12 wherein the programme of panoramic images are viewed by an end user using a viewing monitor means adapted to allow a user to control the programme of panoramic images being obtained.

14. A system according to any one of claims 11 to 14 wherein the plurality of camera lenses include at least two horizontally aligned camera lenses and at least two vertically aligned camera lenses are configured and arranged in a casing, and wherein the horizontally aligned camera lenses are mountable in a circular orientation, and the at least two vertically aligned camera lenses are arranged to capture images above and below a corresponding horizontally aligned camera lens, in use, to allow a panoramic image of the view to be obtainable.

A system according to any of claims 11 to 13 wherein five horizontally aligned camera lenses are provided, each horizontal camera lens being configured with at least two vertically oriented camera lenses.

16. A method according to claim 1 substantially as herein described.

17. A system substantially as herein described with reference to any one of the accompanying drawings. Dated: this 5th day of February 2004. ADRIAN DAVIS By his Attorneys It-' SCHUCH COMPANY