JP2008537854A - Mobile imaging applications, device architecture, service platform architecture and services (MOBILEIMAGEMAGPLICATION, DEVICEARCHITECTURE, SERVICEEPLATFORMARCHITECTURE and SERVICES) - Google Patents

Abstract

  Systems and methods for still image and video image compression and decompression in mobile devices and monitoring applications are provided. Equivalent to mobile device and camera architecture and service platform architecture for transmission, storage, editing and transcoding of still and video images over wireless and wired networks and viewing on viewable devices To do.

Description

<Related invention>
This application is accompanied by a priority claim of a provisional application in the United States filed Feb. 16, 2005, with application number 60 / 654,058.
This application is also fully referenced and related to U.S. Application 11 / 232,165, filed on September 20, 2005; U.S. Application 11 / 232,726, filed on September 21, 2005. US application 11 / 232,725 filed September 21, 2005; US application 11 / 249,561 filed October 12, 2005; and US application filed October 13, 2005 No. 11 / 250,797.
This application incorporates all of the following for all purposes:
Sweldens, Wim's The Lifting Scheme: Building custom designs for bi-orthogonal wavelets. Appl. Comput. Harmon. Anal. 3 (2): 186-200, 1996;
US patent application 10 / 418,363, filed April 17, 2003, under the names WAVELET TRANSFORM SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT, the inventors have William C. Lynch, Krasimir D. Kolarov and Steven E. Saunders;
US patent application 10 / 447,514, filed May 28, 2003, CHROMA TEMPORAL RATE REDUCTION AND HIGH-QUALITY PAUSE SYSTEM, AND METHOD, inventor Steven E. Saunders, Krasimir D. Kolarov and William C. Lynch;
US patent application 10 / 447,455, filed May 28, 2003, PILE PROCESSING SYSTEM AND METHOD FOR PARALLEL PROCESSOR with inventors C Williams, Krasimir D. Kolarov and Steven E. Saunders;
Golomb, SW (1966). "Run-length encoding" IEEE Transaction Information Theory, IT-12 (3): 399-401;
RF Rice's "Practical Universal Noise Recording Techniques" Jet Propulsion Laboratory, Pasadena, California, JPL Publishing, 79-22 March 1979;
J. Teuhola's "Clustered Bit-Vector Compression Method" Information Processing Letter, October 1978, 7 vol. Pp. 308-311;
US patent application 10 / 447,455, PILE PROCESSING SYSTEM AND METHOD FOR PARALLEL PROCESSORS filed May 28, 2003, with William C. Lynch, Krasimir D. Kolarov and Steven E. Saunders.
The present invention
A “Multiple Technique Entropy Coding System and Method” filed in the United States on September 21, 2005 with application number 11 / 232,726, and a US patent provisional application filed on September 22, 2004 with application number 60 / 612,652. A continuation-in-part of an application with priority claim for the application;
"Permutation Procrastination" filed in the United States on September 21, 2005, with application number 11 / 232,725, claiming priority of US provisional application filed on September 22, 2004, with application number 60 / 612,651 A continuation-in-part of an application with
"Compression Rate Control System and Method with Variable Subband Processing" filed in the United States on September 20, 2005 with application number 11 / 232,165, filed on September 21, 2004 with application number 60 / 612,311 A continuation-in-part of a US patent provisional application with priority claim;
"System and Method for Temporal Out-of-Order Compression and Multi-Source Compression Rate Control" filed in the United States on Sep. 29, 2004 with application number 10 / 955,240. Currently, US Patent Publication No. US 2005 / No. 0105609, published on May 19, 2005, application number 60 / 612,311 and US patent provisional application on September 22, 2004, both filed on September 30, 2003, application number 60 / 507,148 And a continuation-in-part of an application with priority claim from a provisional US patent application with application number 60 / 507,147;
"Multiple Codec-Imager System and Method" filed in the United States on Sep. 16, 2004 with application number 10 / 944,437 and currently published on May 19, 2005 under US Patent Publication No. US 2005/0104752. With an application claim 60 / 390,380 with a US patent provisional application on June 21, 2002 and an application number 60 / 374,061 with a US patent provisional application filed on April 19, 2002 with priority claims A continuation-in-part of an application that is a continuation of US Pat. No. 6,825,780 issued on November 30, 2004;
"Pile-Processing System and Method for Parallel Processors" filed in the United States on May 28, 2003 with application number 10 / 477,455, now published on December 11, 2003 as US Patent Publication No. US 2003/0229733 A continuation-in-part application of a priority application from a US provisional application with application number 60 / 385,253 and application number 60 / 385,250, both of which were filed on May 28, 2002;
"Chroma Temporal Rate Reduction and High-Quality Pause System and Method" filed in the United States on May 28, 2003 with application number 10 / 447,514. Currently, US Patent Publication No. US 2003/02355340, December 2003 Applications that have been published on the 25th and are part of a continuation-in-part of an application claiming priority from the US provisional application with application number 60 / 390,345 and application number 60 / 390,492 filed on June 21, 2002 ,;
"System, Method and Computer Program Product for Image and Video Transcoding" filed in the United States on April 17, 2003, with application number 10 / 418,649, and currently in US Patent Publication No. US 2003/0206597, November 2003 A continuation-in-part of an application published on 6th and claiming priority from a US provisional application of application number 60 / 374,069 filed on April 19, 2002;
"Wavelet Transform System, Method and Computer Program Product" filed in the United States on April 17, 2003, with application number 10 / 418,363, and currently on US Patent Publication No. US 2003/0198395 on October 23, 2003 From published US application provisional application 60 / 373,974 and application number 60 / 373,966 on April 19, 2002 together with US patent application 60 / 385,254 filed June 21, 2002 This is a continuation-in-part of an application with a priority claim.
This application is fully referenced and related to “System and Method for a Dyadic-Monotonic (DM) Codec” in US Pat. No. 6,847,317, issued on January 25, 2005, and November 2004. This is “Multiple Codec-Imager System and Method” of US Pat. No. 6,825,780 issued on the 30th.
Directly digitized images and videos require many bits. Therefore, images and videos are generally compressed for storage, transmission, and other uses. Several basic compression methods are known and many of these variants exist. The general method consists of three steps including transform, quantization, and entropy code. Many image and video compressors possess these basic configurations and their variations.
The purpose of the transformation stage in a video compressor is to combine the energy and information of the source image into as compact a form as possible, using the similarities and patterns of the image or sequence. There is no compressor that can compress all possible inputs. Therefore, we developed a compressor that works well with typical inputs and does not respond to random or pathological inputs. Discrete cosine transform (DCT) is used as a conversion process in many image compression methods and video compression methods such as MPEG-2 and MPEG-4. In the MPEG-4 static texture compression process, wavelet transform is used as the transform process (stage).
In the non-quantized information after the transformation process (stage), the reconstructed restored information is not an accurate reproduction of the original. Entropy coding is generally a lossless process, where information is stored after quantization and coding so that it can be accurately reproduced in the decoder. Therefore, the determination of which information is ignored does not affect the entropy coding stage described below.
In the studio, where a highly complex encoder can be run on a workstation computer, video content encoding is used in limited DCT (originally developed for video broadcast and streaming applications). This is a video codec (compression decompression) technique based on discrete cosine transform. Such a computer-complex encoder (encoding device) is installed in a consumer's playback device, which is a computer-simple and relatively inexpensive decoder (player). However, asymmetric encoder and decoder technology does not match well with mobile media devices, and video messages are captured and played back by mobile phones (portable transceivers) in real time. As a result, video images of mobile devices are smaller in size than other consumer products and are limited to lower frame rates.

<Background of the Invention and Summary of Invention>
The present invention relates to a method, apparatus, system and architecture for still and video images recorded by a device. The present invention includes the following. That is, for mobile devices, related mobile device architectures, service platform architectures, transmission, storage, editing, sharing, and marketing, conversion of still and video images through wireless and wired network systems, and viewing on display devices Methods and services, including but not limited to networks and other system services related to the foregoing. The invention also relates to improvements in image recording technology and related mobile device and service platform architecture improvements.

  The present invention is composed of a video codec (compression decompression) and a camcorder (video camera) application by executing software alone for compression and / or decompression of moving images or still images. The scope of the present invention also includes infrastructure-related products and methods and processes, as well as video message deployment for video codec (compression decompression) and camcorder (video camera) technology software for portable transceivers. Includes multi-function mobile services (MMS) infrastructure technology to deploy shared services and editing and exchange technologies to support full interoperability with commonly deployed general purpose and dedicated video formats. In addition, the present invention establishes an innovative mobile video blog and marketing for video content created by mobile users of mobile devices, enabling an innovative MMS (multifunctional mobile service) to deliver and deliver it. It consists of methods, processes and business processes to be implemented.

<Brief description of drawings>
FIG. 1 illustrates video image size limitations for mobile image communication.

  FIG. 2 shows a system diagram of information source / channel coding (joint source-channel cording). (A) Encoder (b) Decoder

  FIG. 3 shows the configuration of the mobile image transceiver.

  FIG. 4 shows the configuration of the mobile image service platform.

  FIG. 5 shows a comparison diagram of conventional video codec techniques.

  FIG. 6 shows a system diagram of improved information source / channel coding (joint source-channel cording). (A) Encoder (b) Decoder

  FIG. 7 shows an improved mobile image transceiver platform configuration.

  FIG. 8 shows a comparative diagram of the effect of the video codec.

  FIG. 9 shows an improved mobile image transceiver platform configuration.

  FIG. 10 shows an improved mobile image transceiver platform configuration.

  FIG. 11 shows the configuration of an improved mobile image service platform.

  FIG. 12 shows a deployed (OTN) upgrade over a network of MMSC video gateways.

  FIG. 13 shows self-contained playback where video MMS does not require transcoding.

  FIG. 14 illustrates the reduction in complexity, cost and number of video editing servers required for the deployment of media production services.

  FIG. 15 shows a mobile video service platform.

  FIG. 16 illustrates the faster and lower cost development and deployment of a higher quality multimedia portable transceiver (cell phone) service.

  FIG. 17 illustrates the mobile video service of the present invention.

  FIG. 18 shows an application to the broadband multimedia device and service of the present invention.

  FIG. 19 shows implementation options for the software image application of the present invention.

  FIG. 20 illustrates implementation options for the hardware-accelerated image application of the present invention.

  FIG. 21 illustrates implementation options for a software image application accelerated with hybrid hardware of the present invention.

  FIG. 22 shows a simplified multimedia portable transceiver (cell phone) platform architecture application.

  FIG. 23 shows elements of a mobile video communication demo over a GSM / GPRS network.

  FIG. 24 shows a specific MMS function according to the present invention.

<Detailed Description of the Invention>
<Wavelet image processing>
The wavelet transform consists of a continuous specification of a pair of wavelet filters for one data set in one dimension or a dimension above one dimension. In still image compression, a two-dimensional (horizontal and vertical) wavelet transform is used. The video codec according to the present invention uses a three-dimensional (horizontal, vertical, and time) wavelet transform. According to the improved video codec (codec = compression reduction) device with symmetrical 3D wavelet transform, it is more desirable than the device using DCT codec to improve the complexity and power consumption of the mobile device on the computer, In addition, it is possible to simultaneously support still image processing and video image processing using a single codec. This simultaneous support of still images and video images with a single codec reduces or eliminates the required MPEG (for video) or JPEG codec for still images, or compression effects and resulting motion The memory characteristics of JPEG have been greatly improved. An improved symmetric 3D wavelet transform video processing device provides an MMS platform that supports automatic or manual editing of user-created videos and storage, retrieval and restoration of user-created video databases. Allows reduction of device complexity and power consumption on the computer.

<Mobile video communications and services>
The present invention relates to new methods, services, and schemes related to the acquisition, compression, transmission, editing, storage, and sharing of improved video content installed on mobile devices. The present invention applies to data and multi-function controllers including telecommunications (including wireless and wired providers), the Internet, cable (telephone) and other stationary or mobile wireless service providers. . The present invention provides rich content, advanced usage areas, and average revenue per user (ARPU). Mobile multimedia service (MMS) is an evolution of text-based short communication service (SMS). In accordance with the present invention, the promising new Mobile Media Services (MMS) specification presented is a revolution that enables the need to communicate with video communications, sharing, and targeted viewers' personal information. Mobile image communication and sharing further requires a digital camera function (for still images) and / or a camcorder (video camera) function on the mobile handheld device, thereby capturing the video information that the subscriber wants to send (encord = sign The received video information can be reproduced (decord = restoration). These functions, which are the features of the present invention, could not be realized even if possible from the prior art.

  In accordance with the present invention, mobile devices can evolve into an integrated user multifunction entertainment platform. Industry investments have been devoted to technology and platforms. The technologies and platforms are repackaged TV broadcast programs (such as news clips, sports highlights and popular programs such as special "mobisodes"), and other studio video content (such as movie trailers and music videos). ) Can be sent to a mobile device for viewing. In the latter case, mobile device users have been exploited as a new field of video consumers. However, most of the latter cases use compressed video content from large broadcasters. However, the present invention provides the world's mobile operators with significant new opportunities to support subscribers as media creators (enabled by this invention) rather than just media recipients. Photo and video acquisition and sharing functions similar to the stand-alone digital cameras and camera recorders of mobile devices enabled by the present invention will enable new services along with the advancement and aggregation of high-speed cellular and fixed wireless data networks. Basic technology. The present invention also allows for a reduction in development costs and retail prices for both camera-recorder phones and video transmission sharing infrastructure devices, which can be used in existing markets and associated mobile media / data services. The key to large-scale commercial adoption in the future market.

  Traditional mobile video communication / sharing services and applications are typically smaller in size and lower frame rate video than televisions, personal computers, recorders with digital video cameras, personal media players that receive and play on other multifunction devices. The image was strictly limited (see FIG. 1). Mobile image transmission services and applications can support VGA (or larger) video frame rates that are desired to be 30fps or higher than provided by the present invention.

<ADAPTIVE JOINT SOURCE-CHANNEL CODING>
Mobile network video transmission is very interesting because it requires a high data rate compared to other data / media type transmissions such as text, audio and still images. In addition, channel bandwidth changes and limitations due to variations in mobile network noise and error characteristics impose constraints and difficulties on video transport. In the present invention, various source coding techniques are used to apply video bitstreams to different channel conditions (see FIG. 2). This information source / channel coding combination according to the present invention is scalable (can be scaled) to adapt to changes in channel bandwidth and error characteristics. In addition, the present invention supports scalability for scalability scenarios, and the computer output and display capability for decoding at the receiving end of the video stream of different devices are in different limits.

  As shown in FIG. 2, according to the present invention, the video information source string 30 is first information source encoded 32 (compressed) and channel encoded 34 with error correction code (ECC). In mobile networks, DCT-based compression techniques such as H.263, MPEG-4, and motion JPEG are used as typical information source coding. As channel coding methods, there are a Reed-Solomon code, a BCH code, an FEC code, and a turbo code. The video bitstream with both source and channel coded meets the channel bandwidth requirements while achieving the best reconstructed video quality through rate control 36. Rate control computes the discrete rate-distortion of the compressed video stream before the video stream is transmitted over channel 38. Due to computer limitations of mobile devices, typical rate control only considers the channel bandwidth and does not explicitly consider the error characteristics of the transmission channel.

  It is desirable and feasible for the present invention to use improved adaptive source / channel coding based on highly computer efficient algorithms. This maximizes both the instantaneous average quality (video rate vs distortion) of the reconstructed video signal for all three of source coding 32, channel coding 34 and rate control 36. Channel bandwidth and error conditions are available that can be instantaneous and predicted to control.

  A further advantage of improved adaptive source / channel coding technology is that, from the point of view of the present invention, provides and uses a large range of communication service quality (QoS) to wireless mobile and MMS service providers. It maximizes the revenue generated by using the wireless network infrastructure by providing price levels to consumers and business users.

  Multicast scenarios require a single adaptive video bit stream that can be decoded by many users. This is a modern, large, heterogeneous network, such a network cannot send a large number of simultaneous broadcast video signals specifically tailored to individual users due to bandwidth limitations, Then it is particularly important. Multicasting a single adaptive video bit stream greatly reduces the required bandwidth. On the other hand, a large number of users, including broadband wireless or line-connected high-end users and mobile phone users with limited bandwidth and error-prone communication means, require the creation of a decodable video bit stream To do. Due to the limited computer capabilities in mobile devices, the accuracy of the adaptation rate control device is generally very poor. For example, only a two-layer bitstream consisting of a base layer and one enhancement layer can be generated.

  Algorithm with high computational efficiency to enable support for higher levels of heterogeneous networks in terms of channel type (wireless and wired), channel bandwidth, channel noise / error characteristics, user equipment, and user services It is desirable, and enabled by aspects of the present invention, to utilize improved adaptive joint-source channel coding based on.

<Mobile Image Transceiver (MOBILE IMAGING HANDSET) Architecture>
Adding a digital camcorder (video camera) function to a portable transceiver (mobile phone) usually requires the addition of the following functions to either hardware, software, or a combination of hardware and software. (See FIG. 3).
An imaging device array (typically a CMOS or CCD pixel array) equivalent to a preamplifier and analog-to-digital (A / D) signal conversion network.
Image processing functions such as pre-processing, encoding / decoding (codec), and post-processing.
• Buffering of processed images for non-simultaneous transmission or simultaneous streaming over a wireless or wired network.
• One or more image display screens.
• Built-in or removable local image storage.

  When using a codec based on DCT conversion such as MPEG-4, portable transceivers (cell phones) that can use commercial images will be able to capture and display other multimedia devices such as televisions, personal computers, etc. Compared to computers, digital camcorders (video cameras) and personal media players, it is limited to capturing video images of small size and low frame rate. These latter devices are typically stored / displayed in VGA format (640 × 480 pixels) or larger video images at 30 frames per second (fps) or higher display speeds. On the other hand, portable transceivers (cell phones) with commercial image availability are limited to save video images in QVGA format (320x240 pixels), QCIF format (176x144 pixels) or smaller at 15fps or lower display speed (See, for example, FIG. 1). This low video capture performance is due to excessive computer requirements, processor power consumption, and the total number, format, and sequence of procedures on the computer related to video compression / uncompression using DCT conversion. Depending on the required buffer memory.

  Using commercial video codecs and microprocessor technology is very complex and power-saving for mobile image transceivers aimed at capturing VGA (or larger) video at 30fps or higher frame rates. Necessary and costly architecture. The architecture of such portable transceivers (cell phones) includes software programs, reduced instruction set (RISC) processors, digital signal processors (DSPs), along with larger buffer memory blocks (typical memory capacity of 1 megabyte or more). ), Requires a codec that utilizes a combination of both hardware-accelerators that run on a combination of application specific integrated circuits (ASICs) and reconfigurable processing units (RPDs). These codec functions can be implemented by using such RISC processors, DSPs, ASICs, multimedia processors and RPDs as separate integrated circuits (ICs). Alternatively, one or more RISC processors, DSPs, ASICs, multimedia processors and RPDs integrated in a system in package (SIP) or system on chip (SoC) can be combined.

Codec functions that run on RISC processors, or DSPs, are typically software routines with the advantage that they can be modified to correct programming errors or upgrade functions. Disadvantages of implementing certain complexes, repetitive codec functions as software, as a result, overall processor resources and power requirements typically exceed those of mobile communication devices. is there.
Codec functions that run on ASICs and multimedia processors are typically complex, fixed hardware implementations, with iterative computer procedures, usually specially tuned hardware acceleration, the overall codec. With the advantage that the power consumption can be greatly reduced. Disadvantages of implementing specific codec functions on fixed hardware are the long-term and more expensive design phase, the risk of expensive product recalls when errors are found in a fixed silicon implementation, and When a newly created function is added to the image application, the fixed silicon function upgrade deployed in the product cannot be performed.
Codec functions that operate with RPD are typically routines that require both the hardware acceleration of the final mobile image transceiver and the ability to add or change functionality. The disadvantage of implementing specific codec functions in the RPD is that it has a higher number of silicon gates and higher power consumption compared to a fixed ASIC to allow hardware reconfiguration. .

  Image applications that reduce or remove complexity simplify the architecture described above, and repetitive codec functions make all software image transmitters and receivers compatible with mobile software transceivers (cell phones) in large commercial deployments. It is desirable to have a VGA (or larger) video acquisition capability at a frame rate of 30 fps in the configuration. The present invention is the first technique that successfully accomplishes and enables these objectives.

  Multimedia mobile transceivers (cell phones) not only support photo and video communication capabilities, but also a variety of additional multimedia capabilities (voice, music, images) and fixed and mobile wireless access Requires a variety of modes, including but not limited to 2.5G and 3G cellular access, WiBro, HSDPA, WiFi, WiFi, and Bluetooth. The complexities and dangers involved in developing, deploying and supporting such equipment help to deploy services and applications that generate new benefits more effectively and avoid recalling expensive equipment , Over-the-air (OTA) distribution and many functions and applications will benefit greatly.

  It is desirable for image applications to be executed by software alone to enable over-the-air (OTA) distribution and operation of image applications by mobile transceiver (mobile phone) products, mobile phone operators, and other MMS service providers.

<Mobile JAVA application>
Java technology brings a wide range of devices from desktop servers to mobile devices with one language and one technology. Although applications in this range of equipment are different, Java technology works to overcome these important differences, allowing talented developers in one field to leverage their technology across a wide range of devices and applications. enable.

  First introduced to the Java community by Sun Microsystems in June 1999, J2ME (Java2, Micro Edition) was part of a broader preemptive system that met the diverse needs of Java developers. Along with the Java2 platform, Sun has redefined the Java technology architecture in three editions. Standard Edition (J2SE) was offered as a practical solution for desktop deployment and low-cost business applications. Enterprise Edition (J2EE) was for developers specializing in applications for the corporate environment. Micro Edition (J2ME) is a limited hardware resource, such as PDAs, mobile phones, pagers, television set top boxes, remote information handling devices and many other consumer electronics and embedded devices Published for developers working on equipment, etc.

  J2ME was aimed at machines with only 128KB of RAM and a processor that was significantly inferior to the typical desktop and server computers on which they were used. J2ME is essentially composed of a set of profiles. Each profile defines a specific type of device, such as a mobile phone, PDA, etc. It consists of a set of minimum class libraries required for the specification of a Java virtual machine that supports specific types of devices and devices. The virtual machines specified in all J2ME profiles are not necessarily the same as the virtual machines used in Java2 Standard Edition (J2SE) and Java2 Enterprise Edition (J2EE).

Due to differences in processor power, memory, persistent storage, and user interface, it is very difficult to define a single J2ME technology that is optimal or near optimal for all the devices listed above.
To address this issue, Sun shared and reshared the definition of equipment conformity to classify J2ME. Initially, Sun ignored the intended use and shared it into two broad categories based on processing power, memory and storage capacity.
The company then defined a Java language version, with at least the minimum Java language features, but with the extra equipment in each category that would work within device constraints.

  Next, Sun identified a class of equipment with similar roles in each of these two categories. For example, all mobile phones were included in one class regardless of product. In cooperation with the Java Community Process (JCP), Sun has since defined additional functionality specific to each device class.

  The initial sharing created two J2ME configuration environments, a connected device configuration (CDC) and a connected limited device configuration (CLDC). The assumed environment can be a set of minimum class libraries and APIs that define the runtime environment for a selected group of Java virtual machines (JVMs) and devices. The assumed environment can identify the minimum common criteria subset of the Java language that fits within the resource constraints imposed by the type of equipment deployed. Because of the great variability in user interface, functionality, and usage, typical assumption environments do not define critical parts such as user interface toolkits and fixed storage APIs. Instead, the definition of the function belongs to a so-called profile.

  A J2ME profile can be a set of JavaAPIs defined by an industry-driven group that deals with specific classes of devices such as pagers and mobile phones. Each profile is built on top of the minimal common criteria subset of the Java language provided by that assumed environment, and is designed to supplement that assumed environment. Two important profiles for portable transceivers (cell phones) are the Foundation Profile that assists CDC and the Mobile Information Device Profile (MIDP) that assists CLDC. More profiles are in progress, specifications and reference implementations are being developed and released.

  The Java Technology for the Wireless Industry (JTWI) specification, JSR185, defines an industry standard platform for mobile phones that enables the next generation of Java technology. JTWI was founded by the Java Community Process (JCP) by a specialized group of leading mobile device manufacturers, wireless operators and software vendors. JTWI is included in all JTWI compatible devices, CLDC1.0 (JSR30), MIDP2.0 (JSR118) and WMA1.1 (JSR120) as well as applicable CLDC1.1 (JSR139) and MMAPI (JSR135) Define the technology. Two additional JTWI specifications that define technologies and interfaces for mobile multimedia devices are JSR-135 ("Mobile Media API") and JSR-234 ("Advanced Multimedia Supplements").

The JTWI specification raises the functional constraints of high-capacity devices while extending the real standards of applications already developed for mobile phones with minimal API fragmentation.
The benefits of JTWI include:
• Interoperability: The goal of this effect is to deliver a predictable environment for application developers and a deliverable feature set for device manufacturers. Its purpose is two advantages by conforming to the JTWI standard. Manufacturers benefit from a wide range of compatible applications, and software developers benefit from a wide range of devices that support those applications.
-Clarification of safety specifications: The JSR185 specification introduces some clarifications for untrusted applications with respect to "Recommended Security Policy for GSM / UMTS-Compliant Devices" defined in the MIDP 2.0 specification. It extends the base of the base MIDlet suite security framework defined in MIDP 2.0.
Roadmap: An important feature of the JTWI specification is the roadmap that is the key to the common functionality that software developers can expect from JTWI-compliant equipment. In January 2003, the first feature of the roadmap series, which described additional functionality in harmony with the evolution of mobile phones, was expected to appear at intervals from June to September. The roadmap allows all parties to plan for a more reliable future: operators can make better plans for application deployment strategies, and equipment manufacturers can Better decisions can be made in the plan, and content developers can foresee clear guidelines for application development efforts. In particular, carriers will extract / protect underlying wireless communications / network functions from security breaches such as viruses, worms, and other “attacks” that are now prevalent in the public Internet in the future. Will rely on JavaVM to do.

  Java-based image applications are portable across all Java-compatible transceivers, "portable in any environment, once written," and virus, worm, and other mobile network security "attacks" Desirable for competing JavaVM security and portable transceiver (cell phone) / network structure stability, and simplified wireless codecs and application download procedures. Such Java-based image applications conform to the JTWI specification, JSR-135 ("Mobile Media API") and JSR-234 ("Advanced Multimedia Supplements"). The form of the present invention provides these advantages.

<Mobile Image Service Platform Architecture>
The key structure of the mobile image service platform architecture according to aspects of the present invention includes the following (see FIG. 4):
・ Mobile phone 60
・ Mobile Base Station (BTS) 62
Base station controller / wireless network controller (BSC / RNC) 64
・ Mobile Switching Center (MSC) 66
・ Gateway Service Node (GSN) 68
・ Mobile Multimedia Service Controller (MMSC) 70

Typical functions included in the MMSC according to aspects of the invention include (see FIG. 4):
・ Video gateway 72
・ Communication company server 74
・ MMS application server 76
Storage server 78

  The MMSC video gateway according to aspects of the present invention is useful for transcoding between different video formats supported by the image service platform. Transcoding is also utilized by wireless operators that support different voice codecs used in cellular networks, and corresponding voice transcoders are incorporated into the RNC. Upgrading such a mobile imaging service platform with the architecture shown in Figure 4 would typically involve deploying new transceivers and manually adding new hardware to the MMSC video gateway. I need. Some mobile video communications and shared use applications are desirable to eliminate the cost and complexity associated with transcoding. One of the aspects of the present invention is the ability to incorporate a software decoder with each transmitted video stream to enable "self-playing" functionality in public handsets and PV video players There is.

  The MMS application server in MMSC can support applications such as automatic or manual user-generated video editing as well as user-generated video database storage, retrieval, and modification. The complexity of computers that require the implementation of such functions is more common than simple software applications that run on inexpensive, low-performance CPU chips used in standard personal computers (PCs) and servers It requires specialized servers installed by mobile operators, along with corresponding video processing functions that require expensive and high performance application specific integrated circuits (ASICs) and digital signal processors (DSPs).

As provided by the aspects of the present invention, the mobile image application service platform by means of software standalone execution according to aspects of the present invention is an automated wireless upgrade of deployed mobile transceivers (cell phones), deployed MMSC automatic Desirable for support of wireless upgrades, support for mobile video applications used on standard PCs and servers.
A Java implementation of a portable transceiver (cellphone) image application according to aspects of the present invention is desirable with respect to improved transceiver / network structural stability against viruses, worms, and other "attacks" Enable mobile network operators to provide the quality and reliability of services required by national coordination agencies

<Problem>
The deployment of mobile video communication and shared usage services is subject to the inherent limitations of video compression technology.

  On the other hand, such mobile video services are now on the same market as home cinema quality broadcasts—30 frames per second full-size image formats like VGA. On the other hand, performing such large data processing using existing video technology originally developed for broadcast and streaming applications is a real-time video capture (encoding) for mobile phones. Significantly exceed available computer resources and battery power. Broadcast and streaming applications rely on encoding video content in a studio environment where a high complexity encoder can operate on a computer workstation. Since video messages are captured in real time by a portable transceiver (cell phone), they are limited to smaller sizes and lower frame rates.

  As a result, today's mobile video services are primitive; photos are small compared to features that have been replicated to the video phone functions that subscribers have come to expect from digital camcorders (video cameras) (QCIF, QVGA), coarse (15 fps or less). The primitive video image quality offered to mobile subscribers is far from the clear high-definition video (HDV) advertised in lifestyle advertising by industry. Mobile subscribers are demanding a full VGA with 30 fps performance (ie, similar to a camcorder) before paying a premium for a widely adopted camera-integrated (camcorder) phone and associated mobile video services.

  Aside from the inventor, competing video codec providers are very expensive and vast, trying to deliver VGA30fps performance, with overall cost and power consumption far exceeding commercial business requirements and technical capabilities Despite the development time-consuming development plan, it still provides a complex mixed software codec + hardware accelerator solution. Portable transceivers (cell phones) are limited to such small, coarse images or expensive high power consumption structures. Service adoption is very expensive and the quality of service is very low to be adopted in the mass market.

  If new or specific hardware is needed, upgrading the MMSC infrastructure is equally expensive. Software stand-alone application and service platform should enable automatic wireless upgrade of mobile transceiver (cell phone), MMSC video gateway network communication upgrade, and support mobile video applications using standard PC and server . The need for transcoding between different video formats also adds additional cost and complexity

<Solution>
In particular, the solution to this problem according to aspects of the present invention reduces the complexity of the mobile transceiver (mobile phone) structure and the complexity of the mobile image service platform structure, and the mobile transceiver (mobile phone) as a software standalone application. It is a very simple image application (codec) that can be implemented.
In accordance with aspects of the present invention, a software-only video codec solution significantly reduces or eliminates the cost and requirements of baseband processors and video accelerators in multimedia portable transceivers (cell phones). Combined with the ability to install codec post-production via wireless download, this software standalone execution solution significantly reduces the complexity, risk and cost of mobile transceiver (mobile phone) development and video communication service structure and deployment To do.
Software video transcoders according to the present invention and modification applications that edit, save and retrieve, as well as the use of standard PCs and servers running in such applications, are automatically based on deployed MMS control (MMSC). In addition, the wavelet transcoder of the present invention allows operators to fully interoperate between wavelet video formats and other standards-based and proprietary video formats. provide. The present invention also allows a software decoder to be captured with each transmitted video stream to enable "self-playing" functionality in public portable transceivers (cell phones) and PV video players, transcoding Eliminate overall costs and complexity.
The video platform with single software execution of the present invention enables rapid deployment of new MMS services, and some of the embodiments of the present invention also achieve objective processing speed and video product accuracy that cannot be obtained with other existing technologies. Enable. Such a new MMS service is a feature of the present invention.
The wavelet codec of the present invention stands out for its ability to effectively process both still images and video, with a single low cost and low power solution that can support both mobile image mail and video communications services simultaneously. Can be replaced with MPEG and JPEG codecs.

  In this embodiment, the precise aspects, figures or examples are referred to as “Droplet” aspects or embodiments. In this application, “Droplet” is understood with reference to embodiments of the present invention.

<Improved wavelet-based image processing>
Aspects of the present invention utilize a three-dimensional (3D) wavelet transform in a video compression / uncompression (codec) device with very low computer complexity compared to a DCT-based codec (in FIG. 5, a conventional DCT encoder). Providing a comparison of relative computer requirements between the technology and the typical technology of the present invention). The application of the wavelet transform stage also allows the design of quantization and entropy coding stages, with greatly reduced computational complexity. Further effects of the 3D wavelet codec of the present invention on mobile image applications, devices, and services include:
Symmetrical, low-complexity video encoding and decoding Low processor power requirements for both software and hardware codec implementations With existing commercial portable transceivers (cell phones) as both native code and Java applications VGA (or larger) video encoding and decoding at compatible 30fps (or higher) frame rate by software single execution Low gate count ASIC core for SoC integration Low buffer memory requirements Single codec support for both still images (~ JPEG) and video (~ MPEG) Simplified video editing (cut, insert, text overlay) with short GOP (Group of Pictures)
・ Simple synchronization with video codec with short GOP ・ Low latency for improved video streaming with short GOP ・ Adaptation rate control, multicasting, and fine particles for joint source-channel coding ( Fine grain) Extensibility • Low complexity performance scaling of emerging HDTV video formats • Compact software decoders (eg 40KB or less) are compatible with public mobile transceivers (cell phones) and PC video players It can be integrated with each video stream capable of having a “self-playing” video message.

  The above effects are achieved by the following features included in the present invention.

  Wavelet transform application using 16-bit / 32-bit short dyadic integer filter coefficients in lifting configuration: In the implementation of the present invention, Haar, 2-6, and 5-3 wavelet transforms and various wavelet transforms are used. These only require addition, division and small fixed shifts, no multiplication or floating point operations.

  Calculation with lifting scheme: These filters are calculated using a lifting scheme, allowing in-place calculations. This minimizes register usage and temporary occupancy of the RAM, and makes the query destination local, thereby increasing the cache usage efficiency.

  Pyramidal Wavelet Transform with Customized Pyramid Structure: In the practice of the present invention, each stage is calculated based on half of the data resulting from the previous stage in the wavelet transform sequence. As a result, the overall calculation is almost independent of the numbers at each stage. In the practice of the present invention, this pyramid is customized to take advantage of the lifting scheme described above, further increasing the efficiency of register usage and cache memory processing.

  Block structure: In contrast to many wavelet compressions, the present invention shares an image into rectangular blocks and proceeds with each block independently of the other blocks. This allows the memory reference to be done locally and the entire transformation pyramid can be executed using the data remaining in the processor. Therefore, in many cases, the amount of data that moves within the processor can be significantly reduced. This block structure is particularly useful for implementation in hardware. This is because a large capacity is not required for the intermediate storage in order to send a signal.

  Block Boundary Filter: In the present invention, clear noise at the boundary of each block can also be prevented by performing a correction filter calculation detailed in US patent application Ser. No. 10 / 418,363, incorporated herein by reference.

  Temporary desaturation: The features of the present invention also avoid the processing of chroma signal from field to field, and instead are described in detail in US patent application Ser. No. 10 / 447,514, incorporated herein by reference. Use a single field for the saturation of GOP (Group of Pictures).

  Temporal compression using 3D wavelets: The implementation of the present invention does not use motion detection and compensation operations that are very expensive in conventional video compression techniques such as MPEG. Instead, the implementation of the present invention computes a field-to-field temporal wavelet transform. This can greatly reduce the calculation cost. In this process, a 16-bit / 32-bit (short integer) filter may be used together with the lifting scheme.

  Binary Quantization: In an embodiment of the present invention, the quantization step in the compression process is accomplished by performing a binarization operation uniformly throughout the coefficient distribution domain. This avoids sample-by-sample multiplication / division required in conventional quantization.

  Piling: In an embodiment of the present invention, the amount of data handled by the entropy coding method described below is reduced through continuous zero conversion. In the practice of the present invention, the method and invention disclosed in US patent application Ser. No. 10 / 447,455 is used to account for consecutive zeros on a parallel processing architecture.

  Cycle-efficient entropy coding: In an embodiment of the present invention, the entropy coding step in the compression process is performed by superimposing a conventional table lookup and a direct calculation of an input symbol. Achieved by technology. Since symbol assignment is characterized, simple entropy coding techniques such as Rice-Gorom coding, exponential Golomb coding, and Dyadic Monotonic can be used. What is selected for the entropy coding method is frequently changed according to the capability of the processor platform.

<Improved adaptive joint-source channel coding>
The fine extensibility of the wavelet-based codec according to the features of the present invention allows for improved adaptive rate control, multicasting, and joint-source channel coding. Reduces complexity on the computer, and with the computational efficiency of the wavelet algorithm of the present invention, information about the predicted channel bandwidth and error conditions at any given moment, source coders, channel coders, and rate controllers. And optimizes both instantaneous and average image quality (video rate versus distortion) management of the reconstructed video signal (see FIG. 6). With the improved adaptive joint-source channel coding according to the present invention, wireless carriers and MMS service providers are more efficient in quality of service (QoS). It can be reduced to the price demanded by individual consumers and business customers. Improved adaptive information source / channel coding based on algorithms with higher computational efficiency can be used to support higher levels of heterogeneity on the network It becomes. The term “heterogeneity” as used herein refers to the viewpoint of communication format (wireless and wired), communication band, characteristics of noise or error in communication, user equipment, and user service.

<Improved mobile image transceiver platform architecture>
FIG. 7 depicts a mobile image transceiver platform architecture that is improved in accordance with features and implementations of the present invention. The image processing application according to the features of the present invention is implemented as a software single execution application, and is executed as a native code (machine language program) or a Java application in a RISC processor. Java coding acceleration can be implemented in the RISC processor itself, or can be done using a separate Java accelerator IC. The Java accelerator can be implemented as a stand-alone IC, or it can be integrated with other functions in either a SIP or SoC.

  The improved mobile image transceiver platform architecture shown in FIG. 7 is a separate DSP, ASIC, multimedia processor (which would be required in previous devices and systems) for mobile video applications, Alternatively, the RFD processing block is unnecessary. This architecture also significantly reduces the buffer memory required for image processing in mobile phones.

  FIG. 8 shows that the calculation performance required for full video (VGA / 30 fps) encoding according to the present invention can be kept low. This required performance reduction is in the sense that it is in contrast to the capability values achieved by the current technology based on MPEG-4 and H-264 video codecs after the priority filing date of this application. is there.

  FIG. 9 represents a part that implements the features of the present invention in a GSM mobile phone platform with a commercially available camcorder (video camera). The existing GSM baseband / multimedia SoC (Texas OMAP850 shown in FIG. 9) requires a hardware accelerator, DSP, and RISC processor for the QCIF / 15fps video camera function. In contrast, the present invention realizes the VGA / 30 fps video camera function on this platform. This implementation is possible using only software running on the RISC processor, and does not require a hardware accelerator or DSP.

  FIG. 10 represents a part that implements the features of the present invention in a commercially available CDMA mobile phone platform with a camcorder (video camera). The existing CDMA baseband / multimedia SoC (Qualcomm MSM6500 shown in FIG. 10) requires a hardware accelerator, DSP, and RISC processor for the QCIF / 15fps camcorder (video camera) function. In contrast, the present invention realizes a VGA / 30 fps camcorder (video camera) function on this platform. This implementation is possible using only software running on the RISC processor, and does not require a hardware accelerator or DSP.

<Platform architecture of image service through improved mobile devices>
Components of an image service platform architecture through an improved mobile device that is an embodiment of the present invention (see FIG. 11):
・ Mobile phone 160
-Mobile phone relay base (BTS) 162
・ Relay base controllers / wireless communication controllers (BNS / RNC) 164
・ Mobile Phone Exchange Center (MSC) 166
・ Gateway service connection point (GSN) 168
・ Mobile Multimedia Service Controller (MMSC) 170
・ Image service download server 171

Typical functions of MMSC (see Figure 11):
・ Video Gateway 172
・ Communication company server 174
・ MMS application server 176
・ Storage server 178

  The development of an improved image service platform based on the features of the embodiment of the present invention takes the following steps.

  Step 1

  A signal is sent to the network that a video gateway transcoder application and / or video communication / sharing application for use in updating the deployed MMSC is available. The update file is installed automatically over the network or manually.

  Step 2

  The video gateway transcoder application and / or the video communication / sharing application are installed and set automatically through the network or manually (see FIG. 12).

  Step 3

  A signal that the mobile video file can be downloaded and installed is transmitted to the registered mobile phone.

  Step 4

  When the registration device receives and the transaction settlement is successfully completed, the mobile video file is downloaded and installed.

  Step 5

  Send a signal to the network that the mobile phone upgrade is complete. Make services and related applications available. The record is updated in order to record the price of the mobile video file in the registrant's monthly usage fee bill.

  FIG. 13 shows as a feature of the present invention that the functionality of a video MMS for “self-reproducing” is achieved by integrating the software decoder and the transmitted video stream.

  FIG. 14 shows as a feature of the present invention that it reduces the complexity, cost and number of video application servers. This server is required for the deployment of media production services, as well as storing, retrieving, and restoring user-created videos, as well as automatic or manual editing of user-created videos.

  As a feature of the present invention, FIG. 15 shows functional elements of a platform for transmitting, sharing, and calling a video. This platform incorporates wavelet-based codec / camcorder (video camera) applications, joint-source channel coding, and video editing and data storage, retrieval, and restoration according to the present invention. Is.

  FIG. 16 shows the feature of the present invention from the viewpoint that a higher-quality multimedia mobile phone and service can be developed and deployed faster and at a lower cost. Benefits include the ability to deploy an innovative personal multimedia market platform that allows users to create “soft” copies (downloads) or “hard” copies (DVDs) of user-created audio or video content. , Users will be able to audition, share, and buy and sell. The present invention also enables more efficient video “tagging”. This tagging is used to index databases, provide network (RSS) information, and support bridging with existing wave markets such as E-bay, Google, Yahoo, Microsoft and other portal sites.

  As an aspect of the present invention, FIG. 17 shows an innovative mobile video service. This service is based on wavelet-based codec / camcorder (video camera) application according to the present invention, information source and joint-source channel coding, and video editing and data storage, retrieval and restoration Yes.

  As a feature of the present invention, FIG. 18 shows a wavelet-based codec / camcorder (video camera) application according to the present invention, joint-source channel coding, and video editing and data Application software for the above-mentioned platform that transmits, shares, and calls video that incorporates storage, retrieval, and restoration. This application will be used to deploy new video services on fixed, mobile, and wired networks as well as “centralized” networks that combine elements of fixed, mobile and wired architectures .

<Performance>
By incorporating wavelet-based mobile video video processing application, joint-source channel coding, mobile transceiver architecture and service platform architecture, improved by the present invention, The objectives of higher quality mobile video images, cost reduction of portable transceivers (cell phones), simplification of architecture and reduction of service deployment costs are achieved.

<Various Examples>
Implementation of the features of the present invention enhances the mobile image transceiver architecture. For example, several implementation options can be considered as an image processing application based on a wavelet executed by software alone (see FIG. 19). The image processing application is transmitted to the baseband multimedia processing unit, the removable storage device 402b, or the image processing module 402c of the portable transceiver (cell phone) 402a through an over-the-air (OTA) download (400a, 400b, 400c). Installed. If desired, the image processing application can be installed on the baseband multimedia processing unit, removable storage device, or image processing module of the portable transceiver (cell phone) during the manufacturing process or at the point of sale at the store. Additional implementation options can also be incorporated as an evolution of the mobile device architecture.

  The features of the present invention further improve the performance of the mobile image transceiver, further reducing cost and electricity consumption. This relies on accelerating computational elements through hardware-based processing resources. The reason for this is to utilize mobile device computing hardware (ASIC, DSP, multimedia processor, RPD) and integrated circuit technology (SoC, SIP) that are currently being developed. In order to concentrate hardware-based processing resources in these mobile phones, an option by hardware alone can be executed (see FIG. 20). This hardware option includes a baseband multimedia processing unit, a removable storage device, and an image processing module of a portable transceiver (mobile phone).

  As shown in FIG. 21, the composite architecture provided by the image processing application according to the present invention is a hardware intensive, iterative, fixed function execution and after software manufacturing. Improvement is desired and contributes to the improvement of processing capacity through the execution of required functions.

  FIG. 22 illustrates the possibility that features of the present invention can simplify the architecture, placement and maintenance of mobile camcorder (video camera) equipment.

<Effect>
The image processing by software execution in the present invention greatly reduces the cost and necessary components of the baseband processor and video accelerator in the multimedia mobile phone. The combined complexity of both mobile phone development and video communication service deployment through image processing through the execution of this software alone, together with the post-shooting editing codec installation and maintenance function via over-the-air (OTA) download Enables significant risk and cost savings.

  The present invention allows mobile phone carriers to use the finest mobile video communication and sharing platforms, and in terms of video image quality, mobile transceiver (mobile phone) retail prices and service deployment costs, personal consumers and businesses in the mass market. It is possible to meet customer demand. The present invention provides the highest quality software standalone application that enables real-time shooting of full-size movies (VGA: 640 x 480 pixels) at 30 frames per second (fps) on a mobile phone with a camcorder (video camera). To do. This feature allows the use of a standard RISC processor already built into an overwhelming number of multimedia portable transceivers (cell phones). The mobile phone communication company integrates the video processing / distribution technology simplified according to the present invention with a new software execution platform, and the existing mobile transceiver (mobile phone) and mobile multimedia message service (MMS). It is possible to enable comprehensive deployment using existing infrastructure. As a complement to the above software standalone mobile video camera application, mobile phone carriers can use the features of the content management platform according to the present invention to integrate compressed images and videos with audio and text according to this technology. And complete mobile multimedia messages and "ringtone" modules are available. This content management platform not only provides these modules, but also on-the-fly (no intermediate output required) editing, thumbnail preview, multimedia mailbox, online storage, sharing, marketing services and registration management. Make it available.

In Example 1 below, the demonstration components, settings and operations are described, but this demo will show the features and benefits gained by implementing the software-only mobile video message platform features of the present invention. It is. The demo used a commercially available GSM / GPRS multimedia portable transceiver (cell phone) and was intended to be operated in any commercial GSM / GPRS network. The demonstration was successful enough. This demonstration can also be easily performed using a CDMA portable transceiver (cell phone) and operated in a commercial CDMA network. In the demo of Example 1, the code name “Droplet” is given, and the demo and a series of files so labeled in Example 1 are executed.
Example 1
As illustrated in FIGS. 23 and 24, the demonstration includes the following five elements.
1. Download the "DTV" video codec application that runs the software according to the present invention from the MMS service provider's server computer via wireless communication (OTA: over the air) and install it on the multimedia portable transceiver (mobile phone) To do.
2. A DTV video codec is used to compress high-quality (VGA, 30 fps) video messages in a multimedia portable transceiver (cell phone).
3. Send the compressed video message (DVT format according to the present invention) to the MMS server. This is the same for full-size resolution (VGA / 30fps) for transfer / playback to a computer or TV, or for low resolution (QCIF / 15fps) reduced for transfer / playback to a portable transceiver (cell phone).
4). A notification that the compressed video image is available for download is sent from the MMS server through an email alert to a computer connected to the network and an SMS alert to a mobile transceiver (cell phone).
5. The compressed video message is downloaded with the DTV decoder according to the present invention. This is a "self-playing" function, that is, decoding on a networked computer (VGA / 30fps) or mobile transceiver (mobile phone) (QCIF / 15fps) This is to enable playback using general video player software already installed on the mobile phone.

<Devices used for video message demonstration>
2.1 Mobile transceiver (mobile phone)

  Description of the experiment described in Example 1. The following two types of commercially available GSM / GPRS multimedia portable transceivers (cell phones) are used. Many other similar multimedia portable transceivers (cell phones) can be substituted.

2.1.1 T-Mobile MDA-II PDA-Phone (manufactured by HTC)
・ Able to use on local GSM / GPRS network ・ Available to access data on the Internet ・ MDA-II Phone cradle (attached USB2.0 cable to connect mobile phone and PC)
・ OS: Windows Mobile 2003 Pocket PC with Phone Edition

2.1.2 O2 Xphone Smartphone (HTC)
・ Able to use on local GSM / GPRS network ・ Available to access data on the Internet ・ USB2.0 cable to connect portable transceiver (mobile phone) and PC ・ OS: Windows Mobile 2003 Smartphone Edition

2.2. Computer

2.2.1 Personal computer (laptop or desktop)

Description of the experiment described in Example 1. Use a laptop computer and make the following settings.
・ Model: Sony Vaio PCG-K33P
Processor / Memory: Pentium 4 3.06GHz, 512MB RAM
・ Graphics: ATI Radeon IGP 345M (64MB RAM)
・ Storage: 60GB
・ Connection port: IEEE1394 port x1, USB2.0 port x2
・ OS: Windows XP
・ Preinstalled video application: Windows Movie Maker

2.2.2 Remote server

  Description of the experiment described in Example 1. The following MMS functions are demonstrated using a UNIX standard remote server.

Save DTV codec for download by MDA-II portable transceiver (mobile phone)-Save DTV player application for download by Xphone-Receive compressed video message from MDA-II machine and save or transfer it.
-Send SMS message to inform Xphone device that video message is available for download.
・ Access via Xphone browser allows both DTV player application and QCIF / 15fps video messages to be downloaded.

2.3 Video shooting (video capture)

Description of the experiment described in Example 1. The following commercially available camcorder (video camera) is used to convert the external material into a high-quality moving image sequence, which is compressed by the DTV codec of the present invention that is executed by a portable transceiver (mobile phone).
-Canon ZR65DV Camcorder (IEEE1394 cable is used to connect camcorder (camcorder) and PV)

  The advantage of the DTV codec and mobile video communication capability of the present invention is more effective by using an external camcorder (video camera) instead of the low-performance image module currently used in most commercial camera-equipped mobile phones It is possible to explain an example.

  Digital video movie files shot and compressed by a camcorder (video camera) are first extended and converted to a UYVY movie format on a personal computer. Thereafter, it is transferred to an MDA-II portable transceiver (cellular phone) and encoded / compressed by the DTV codec of the present invention. The UYVY format is a typical video format and is expected to be loaded into a video codec in a multimedia portable transceiver (cell phone).

<Equipment settings for experiments>
3.1 PC settings

3.1.1 Downloading the Droplet Demo Package Using the Present Invention Open the URL “http://droplet-tech.com/demo_partner_access.html” with a web browser on a personal computer.
-Select the link "Demo Package-January 2005" (WinZip compressed file).
・ Click this link to start downloading the demo package file to your computer.
・ Place a Zip file in the selected folder.
・ When you unzip the Zip file, the following files / folders are expanded.
○ Demo package "read me" document file (Demo_package_readme.html)
○ The following folders ・ MDA_DTV: MDA-II client Ul application software ・ Xphone_DTV: XphoneUl client application software ・ PC_player: A file that executes DTV files on a PC ・ Virtual_Dub1.6.3: PC application software that converts different video formats to each other ・MMS_server: Sample monitoring script program for server computers-Canon_driver: Video camera driver software-PHMRegEditor: Registry editor (for installing MDA-II and Xphone)
・ Ewesoft: Java disguise machine for Xphone

3.1.2 Download the J9 JVM (Java Virtual Machine) installation package.
・ Open the URL "http://droplet-tech.com/ftp_accss.html" with a PC web browser.・
• Select the “J9-JVM” (WinZip archive) link • Click this link to start downloading the J9 JVM package to your PC • Place the zip file “weme57prod_pp_1.zip” in the selected folder.

3.1.3 Download and install DirectX 9.0 SDK (used by DTV PC player).
・ A free SDK can be downloaded from the Microsoft website.
・ Http://msdn.microsoft.com/library/default.asp?url=/downloads/list/direct.asp
・ This download is large (~ 230MB), so be careful.
・ Check if the DirectX Utilities> GraphEdit function is enabled.

3.1.4 The Droplet DTV PC player according to the present invention is installed on a personal computer (PC).
• The software package can be found in the demo package in the PC_player directory. The file is called PC_player.zip.
-Open (unzip) the package as "C: \ DTV_PCplayer_Demo \" on the "C:" drive of the test PC. If the actual path is different, edit RegisterFilter.bat and UnRegisterFilter.bat.
-Open the folder and double-click the batch file "RegisterFilter.bat" to register the DirectShow filter.

3.1.5 Download and install Virtual Dub on your PC.
In the current version of the demo, the Virtual Dub was converted to a compressed DV video file as it was shot with a camcorder (video camera) and placed in a decompressed (decompressed) UYVY video format on the PC . The restored video file is then input to the MDA-II mobile phone for encoding / compression with the DTV codec of the present invention. UYVY is a typical video format that is input to the video codec of a multimedia mobile phone.
• The software package can be found in the demo package in the Virtual_Dub directory.
On the other hand, software packages can also be found at the following URL:
http://jaist.dl.sourceforge.net/sourceforge/virtualdub/VirtualDub-1.6.3.zip
In Virtual_Dub, the directory is a file called VirtualDub.exe. This is selected and the application is confirmed to work.

3.1.6 Install Canon ZR65 camcorder driver on PC
• The Canon driver package can be found in the demo package in the Canon_driver directory.
On the other hand, the driver can also be found at http://www.canon.com.
The Canon_driver directory is opened, and the subfolder "ZR65WI503EN" and the subfolder "English" are selected, and then the executable program "SETUP.EXE" is installed to install the Canon camcorder driver Operate.

3.2 Configuration of remote MMS server In the demonstration, the remote MMS server can download video codec files to the FTP server [mobile phone (mobile transceiver) and record video from the mobile phone (mobile transceiver). Enables network storage of files, as well as mail servers [email / SMS notification of video messages from mobile phones (mobile transceivers), and networked computers and other mobile phones (mobile transceivers) It also functions as “allowing downloads via”. Functionally, the server must be able to send SMS messages to send SMS notifications to other mobile phones (mobile transceivers) of the video message being sent.
Create an FTP site to have the following directories:
○ public_html
-Install the DTV codec (file DtvMDADemo.exe) on the server in advance for later download by the MDA-II mobile phone (mobile transceiver). This file is transferred to the FTP server in top-level binary mode.
-Install the DTV decoder (file DtvXphoneDemo.exe) on the server in advance for the next download by Xphone. This file is transferred to the FTP server in top-level binary mode.
-The sample monitoring script is in the Droplet demo package in the MMS_server directory (file name: monitor.php).
This monitoring script is crontab on Unix-based servers. This script monitors for the presence of new files (ending in .Ink) in the ftp: / public_html directory every minute.
If the user decides to use the above sample monitoring script on the server other than the Droplet demo server, it is customized as a local server / network environment.

3.3 Configuration of MDA-II Connect MDA-II to a PC (preconfigured as in section 3.1 above) with a cradle / USB cable.

3.3.1 Registry entry modification for stable GPRS connections To ensure a stable GPRS connection, increase the timeout period to be greater than the default setting of 60 seconds. Mobile phone (handset transceiver) manufacturers (HTC) provide recommended registry changes. If the registry editor is not installed on the device, first install the registry editor from the "PHMRegEditor" directory included in the Droplet demo package.
-Install the file PHMRegEdit.msi in a temporary directory on the PC.
・ Install regedit.Mrln_ARM.CAB which is CAB file in the same directory of PC ・ Double-click to activate PHMRegEdit.msi, and follow the instructions.
・ Synchronize with MDA-II ・ Copy regedit.Mrln_ARM.CAB file to Windows directory on MDA-II.
• Double-clicking on a file in MDA-II will install the CAB file properly.
-Run PHMRegEdit.msi and follow the instructions.
The resulting program PHMEditor is installed in the MDA-II directory \ program files \ PHM tool.
・ Start the program regedit (registry editor) in the above directory ○ Select
HKEY_LOCAL_MACHlNE \ SOFTWARE \ Microsoft \ ConnMgr \ Planner \ Settings
○ Change the CacheTime setting to 300 (5 minutes is the timeout period).
○ Change the SuspendResume setting to ~ GPRS (do not consider timeout)-Exit from Registry Editor.
・ Activate the soft reset so that the changed settings will take effect.

3.3.2 Install the IBM J9 JVM.
・ Open (unzip) the “weme57prod_pp_wm_1.zip” file downloaded on the PC in 3.3.1 above. There are five files.
○ inst_pp_wm.html
○ readme_pp_wm.html
○ weme-wm2003-arm-ppro10-5.7.1-P-20040723-1833.bin
○ weme-wm2003-arm-ppro10-5.7.1-P-20040723-1833.exe
○ weme57_ppc_pp_1.pdf
-Open the inst_pp_wm.html. File. Call the installation instructions and follow the instructions.

3.3.3 Install Ul application of MDA-II mobile phone (mobile transceiver).
This application can be found in files already opened (uncompressed) on the PC as already described in 3.1.1.
The demo package in the directory MDA_DTV contains a subfolder called "mms_client". These consist of three files.
○ "application.properties" for the demo package
○ "Droplet.jar"
○ "Droplet.lnk"
-Copy the files "application.properties" and "Droplet.jar" from the PC to MDA directory \ ProgramFiles \ J9 \ PPRO10 \ examples.
-Copy the file "Droplet.lnk" to the MDA directory \ windous \ StartMenu.
• Click on the Start menu to confirm that MDA-II was formed correctly. An icon called Droplet becomes available.
-To check functionality, select the Droplet icon and a new window will appear. It consists of three buttons.
○ Codec Download / install
○ Video shooting ○ Transmission of video message Note: Information code for MDA-II mobile phone (mobile transceiver) Ul application is available under directory MDA_DTV \ mms_client_src, for example.
This creates MDA-II as a demo.

3.3.4 Install Pictpocket movie / video player (optional).
This step is optional because the default video player above MDA-II can be viewed in the decoded Droplet video file. The installation of a third party video player demonstrates that the decoded video file according to the present invention is compatible with a multifunction mobile device video player.
-The "Pictpocket movie" application from Digisoft is installed on the MDA-II mobile phone (mobile transceiver).
・ A 14-day trial version of the Pictpocket movie can be downloaded from the vendor's website: www.digisoftdirect.com.

3.4 Form an Xphone.
Connect Xphone to PC with USB cable.

3.4.1 Modifying registry entries to a stable GPRS connection To guarantee a stable GPRS connection, the timeout period needs to be increased to be greater than the default setting of 60 seconds. A mobile phone (handheld transceiver) manufacturer (HTC) suggested a registry change that could be recommended. If no registry editor is installed on the device, first install the registry editor included in the Droplet demo package in the "PHMRegEditor" directory.
Install the file PHMRegEdit.msi in the Temp directory on the PC.
-Install the CAB file regedit.Mrln_ARM.CAB in the same directory on the PC.
• Run PHMRegEdit.msi by double clicking and follow the instructions.
・ Synchronize Xphone with PC.
-Copy the file regedit.Mrln_ARM.CAB to the Windows directory on the Xphone.
・ Double-click the file in Xphone to install the CAB file properly.
-Run PHMRegEdit.msi and follow the instructions.
The resulting program PHMEditor is installed on Xphone in directory \ program files \ PHM tools.
Start the program regedit (registry editor) from the above directory.
○ Select
HKEY_LOCAL_MACHlNE \ SOFTWARE \ Microsoft \ ConnMgr \ Planner \ Settings
○ Change the CacheTime setting to 300 (5 minutes is the timeout period).
○ Change the SuspendResume setting to ~ GPRS (do not consider timeout)-Exit from Registry Editor.
• Activate the soft reset so that the changed settings will take effect.

3.4.2 Install Ewesoft JVM.
• The demo package in the directory Ewesoft has a file called Ewe143-CAB-SmartPhone.zip. Unzip this file on your PC. A file called Ewe143-SmartPhone2003.arm.CAB- is used.
Copy the Ewe-SmartPhone2003.arm.CAB file to the Xphone directory called Storage \ windows \ Start Menu \ Accessories.
・ Start Accessories. CAB files are displayed as menu items
Select the CAB file and then install the VM.
• The installation creates a new Ewe folder that appears in the Start menu.
In that folder, find the Ewe VM itself and the "Solitaire" demo application that confirms the correct installation.

3.4.3 Installation of Xphone mobile phone (mobile transceiver) Ul application • The demo package in the directory Xphone_DTV has a subfolder called "mms_client". These consist of three files.
○ "application.properties"
○ "Droplet.jar"
○ "Droplet.lnk"
-Copy the files "application.properties" and "Droplet.jar" from the PC to the Xphone directory / Storage / windous / StartMenu / Ewe /.
Copy the file "Droplet.lnk" to the Xphone directory: / Storage / windous / StartMenu /
Click the Start menu to confirm that the Xphone has been correctly formed. An icon called Droplet becomes available.
-To check functionality, select the Droplet icon and a new window will appear. It consists of three buttons.
○ Codec Download / install
○ Video shooting ○ Video message transmission MDA-II is formed as a demo.

<Demonstration (execution)>
4.1 Download and install video codec on MDA-II
-From MDA, click on the Start menu and the Droplet icon will appear in the list.
-Select the Droplet icon and a new window will appear. It consists of a selection of three buttons.
○ Codec Download / install
○ Video recording ○ Send video message ・ Click the Download / install codec button.
・ The following message appears.
○ FTP host: Connect to xx.xx.xx.xxx
(This is the server's IP address)
○ Connection ○ Login to server ○ Login completed ○ Codec file download started ○ Download completed ○ Video codec installed successfully. End of connection-Go to My Documents directory on MDA to confirm successful codec download. A file called DtvMDADemo.exe is presented with the date and time stamp it was downloaded.

4.2 Recording video messages with a mobile phone (mobile transceiver) In this version of the demo, the MDA-II mobile phone (mobile transceiver) is an uncompressed high quality video from an external video camera (video camcorder). Live broadcast video was input and encoded / compressed (encoded / compressed). On the other hand, MDA-II is equipped with a camera that can shoot VGA, but it only takes still images at that resolution. Video shooting with MDA-II is limited to QCIF (10fps video automatically compressed in 3GPP format on the device).

4.2.1 Connecting the camcorder (camcorder) to the PC-Connect the Canon video camera (camcorder) to the PC using the Firewire (1394) cable. A video camera (camcorder) has a 4-pin connector terminal.
-Once the Firewire cable is connected and the video camera (camcorder) is adjusted to "camera" mode, the PC OS (Windows XP Professional in this demo version) offers several options for shooting.
・ Select "Windows Movie Maker" to install a movie maker application.

4.2.2 Obtain a video sequence from a camcorder (video camera) on a PC.
• Select “Get from video device” from the movie maker and select the name of the acquisition file and location to store it on the PC.
• The next screen asks for video settings; select “Digital Device Format (DV-AVI)”. It is a video recording format in DV format and a high quality (almost lossless) video recording format.
• The next screen displays a preview window of the video that was filmed with a “shooting start” and “shooting end” interface. Many wireless mobile phones (portable transceivers) do not have large-capacity on-board storage elements, so 2-3 seconds of full-motion video (60-90 frames) were suggested as faster computerized recordings. .
• Once a video sequence is recorded, you can exit the Windows Movie Maker program in .AVI format.
• For this document, the recorder file name is assumed to be "testDV.avi". (The file is in .AVI format and all the interleaved audio and video files with individual frames are trimmed one after another.)

4.2.3 Uncompressed DV video sequence on PC-Video captured in the previous step is in DV format (720 x 480 pixels, 30 fps at approximately 28 Mbps) as integrated audio Is compressed. To simulate full motion shooting, the video is in the normal uncompressed UYVY format, scaled down to VGA size (640x480 pixels), and the audio information is separated.
For this demo, you need to create two source files to encode to MDA: one is VGA (640 x 480) @ 30fps and the other is QClF (160 x 144) @ 15fps .
-Runs "VirtualDub.exe" from the VirtualDub1.6.3 directory.
a. Use "Open video file" in "File" and optionally use "testDV.avi"
b. Select "No audio" file as an option in "Audio"
c. Select “Filters” as an option in “Video”. Select “Add”, scroll down the menu and set the “resize” filter. Set 640 as "New width" and 480 as "New height". Set “Precise Bicubic (A = 0.75)” as “Filter Mode”.
d. Select “Color depth” in “Video”. Set the output format in the left column to "4: 2: 2 YCbCr (UYVY)" format.
e. Make sure that the “Compression” option in “Video” is set to “Uncompressed RGB / YC bCr”.
f. If a very long video sequence is filmed, it can be shortened by selecting the "Select range" option in "Video". The large scale of video that can typically be reasonably manipulated is 60-90 frames (2-3 seconds video). The start may be from the beginning ("Start offset" of 0) or somewhere in the sequence, depending on the video being filmed. For example, a 2 second (60 frame) uncompressed UYVY video VGA (640x480) sequence is 36 MB of data on a computer.
g. Select "File" and "Save as AVI ..." to save the uncompressed file. Call the "testVGA_UYVY.avi" file. A raw (uncompressed) UYVY VGA input file to be encoded and encoded / compressed on an MDA-II mobile phone (mobile transceiver).
Create a file for QCIF (recommended for sending video clips to another mobile phone (mobile transceiver) via GPRS connection):
In step c above, enter 160 as “New width” to enter 160. Enter 144 as "New height".
○ In step g above, name the file "testVGA_UYVY.avi".

4.2.4 Transfer of uncompressed video sequence to MDA-II mobile phone (mobile transceiver)
・ Transfer uncompressed video files "testVGA_UYVY.avi" and "testQcif_UYVY.avi" from PC to MDA.
Copy these files to the MDA-II My Documents directory for compression.
For intermediate storage, the above large source file (uncompressed video input) is placed in the “Storage Card” of the device and then copied to the My Documents directory for compression.
・ Separate MDA from PC.
At this point, an uncompressed video sequence is present in the mobile phone (mobile transceiver) and the encoder software according to the present invention, which is the encoder software previously downloaded and installed in the mobile phone (mobile transceiver). Is ready to be compressed.

4.2.5 Compression of conventional video sequences in MDA-II mobile phones (mobile transceivers) Video codecs and uncompressed video sequences are successfully downloaded to MDA-II mobile phones (mobile transceivers) Mobile phones (portable transceivers) are now ready to perform encoding / compression.
-Go to the MyDocuments directory on MDA.
-Find and select the "DtvMDADemo" application.
A UI window appears and the user enters information sequentially:
O Source file: The name of the file contains the video that is not compressed in UYVY format (this is the file generated in the previous step).
○ Destination (designated) file: This file stores the compressed video sequence. (The compressed file has an extension that ends with .dtv.) In this demo, the file name is bitstream.dtv.
○ Horizontal and vertical frame sizes ○ Parameters for encoding VGA video are 640 and 480 respectively
○ The parameters for encoding QCIF video are 160 and 144 respectively.
○ Input file type: YUV 4: 2: 2, default (standard setting)
○ Compression rate: 12 levels, default (standard setting)
O Range of frames to be compressed: typically set to "all". (Note: If the user chooses to change this, DTV will process two frames as a group of images, so the total number of frames specified will be an even number.)
○ Select "Encode" or "Decode" or both.
○ Select Start.
• If completed successfully, a specific compressed bitstream.dtv file in the destination (designated) file area will be created in the “My Documents” directory on MDA-II.
It is important to maintain the .dtv file extension if the file name is changed.
Note: The above encoding is performed twice. Once for QCI F / 15fps and once again for VGA / 30fps.
-The restored QCIF file is also saved in "My Documents \ DTV Output" of MDA-II for MMS transmission to other mobile phones (mobile transceivers) via the GPRS network.

4.3 Transfer / playback of VGA / 30 fps video message on PC
To execute an encoded VGA file on a PC, it is necessary to transfer the file from MDA-II to the PC. At current GPRS data transfer rates (~ 20-40kb / s), it takes approximately 16 seconds to send 1 second of compressed VGA video over a mobile network. High-speed 3G or WiFi network allows for more efficient transfer of full VGA / 30 fps video files for the time being. In this demonstration, the USB connection between the MDA-II mobile phone (mobile transceiver) and the PC is used to facilitate file transfer.
・ Connect MDA-II to PC with cradle / USB. (Note that this is the standard way that current digital cameras and camcorders (video cameras) are connected to the home PC to transfer digital photos and videos.)
・ VGA file encoded or compressed from MDA-II to PC
Copy (\ MyDocument \ bitstream.dtv). Files are copied to the following directory: "C: \ DTV_PCplayer_Demo \"
The MDA-II device is equipped with a USB1 port, allowing faster transfer of VGA / 30 fps files to USB2 with a PC, copying them to the MDA-II removable memory card first and then It can be realized by removing the card, connecting it directly to the USB2 port of the PC and using a USB2 card reader for transfer.
Once the file transfer is complete, go to the PC directory "C: \ DTV_PCplayer_Demo \", double-click the file "dtvplayer.grf" or "dtvplayer_Win2K.gif" (included with the OS), and then VGA / 30 fps Click the “Play” button (or select the “Play” menu item) to watch the video.
• To stop a clip, simply exit the application (GraphEdit).

4.4 Sending QCIF / 15 fps video as MMS message by GPRS
This section shows the ability to send compressed QCIF / 15fps video from MDA-II mobile phone (mobile transceiver) to MMS server via GPRS. An SMS notification is sent to the targeted mobile phone (portable transceiver) (in this case Xphone), indicating that the video MMS is ready for download and playback. On the other hand, if the targeted receiving device is a networked computer, an email notification is sent.

4.4.1 Sending QCIF / 15 fps video to MMS server via GPRS
-Clicking on the MDA-II start menu will bring up a menu list for the Droplet icon.
-Select the Droplet icon and a new window will appear. It consists of three selection buttons.
○ Download / install codec ○ Video recording ○ Send video message ・ Click “Send Video Message” button.
A new window for selecting a file opens.
○ Select the file you want to send (in this case a compressed QCIF / 15 fps video file) ○ A new window will open to enter the target phone / email address.
○ When an email address is entered (confirmed by the presence of the @ symbol), the selected file is sent with an email notification.
○ If a character string without the "@" symbol is entered, a file / SMS notification will be sent to mailto: "string" @ tmomail.net and the corresponding phone number will be entered as "string" in this case. Sent to T-Mobile subscribers.
・ To send compressed video as an MMS message to Xphone, enter the phone number of the mobile phone (mobile transceiver).
• The user is prompted to establish a GPRS connection. This is done, for example, by reaching Internet Explorer and going to some known URL.
-Once the GPRS connection is established successfully, click “OK”.
A new window opens where the next status message should appear.
○ FTP host: Connect to xx.xx.xx.xxx
(This is the server's IP address)
○ Connection ○ Login to the server ○ Login completed ○ Change directory to: public_html ○ Change completed ○ Start uploading video file <selected video file name> .dtv ○ Complete upload
○ Start uploading link file <selected video file name>. Lnk ○ Complete upload
○ Successful video file transmission. End of connection.
Note: Two new files (<selected video file name> .dtv and <selected video file name>. Lnk) appear in the public_html directory on the ftp server.

4.4.2 SMS notification is sent from MMS server.
A script running on the server calls the ftp location public_html (polls) and checks for the existence of a new file.
When a new file exists, the server script parses the <new file> .lnk file and extracts the name of the video file to be sent and the destination (instructed) mobile phone (mobile transceiver) number or email address.
• The script sends an SMS notification message to the target point targeted by either email or mobile SMS.

4.4.3 Download mobile phone (mobile transceiver) video message and DTV decoder This section receives SMS notifications on Xphone, connects to MMS server and downloads QCIF / 15fps video file with DTV decoder Demonstrate ability to download. After receiving the video file and decoder, the file is decoded and demonstrated on the Xphone.
・ Xphone receives SMS messages.
・ Open an SMS message. Inside is the location to download video files.
• Download DTV video files.
○ Open Internet Explorer.
○ Type in the URL where the video file is located (type in).
○ The video file is downloaded to the Xphone.
○ In addition, you need to download an Xphone video decoder (if it is not already on the Xphone). Once downloaded, the Xphone decoder file (DtvXphoneDemo.exe) is placed in the Xphone My Documents directory.
-Run DtvXphoneDemo.exe.
• An Ul window appears to allow the user to enter information (the application defaults to processing the bitstream.dtv file).
○ Horizontal and vertical frame sizes.
○ The parameters are 160 and 144, respectively, for decoding QCIF video.
O Range of frames to be compressed: typically set to "all". (Note: if the user chooses to change this, DTV will process two frames as a group of images, so the total number of frames specified will be an even number.)
○ Ignore all other fields.
○ Select “Decode”.
○ Select Start.
O Form an uncompressed AVI file, which can be executed by most video players found on mobile cell phones (portable transceivers).
To run a video file on an Xphone, simply click on the formed file and the Microsoft media player inside will play the video.

4.4.4 Using other FTP connections on Xphone
In order to disclose the flexibility enabled by the Droplet according to the present invention, all software video transmission platforms are disclosed. Also disclosed is the use of a simple Smartphone FTP application to download QCIF / 15fps video messages, and software video to download a Droplet decoder to an Xphone mobile phone (mobile transceiver).
Orneta FTP application for Windows Smartphones devices on Xphone is used.
Orneta FTP application installer can be downloaded to your PC from:
http://www.handango.com/PlatformProductDetail.jsp?productType=2&platformld=11&siteld=1&Sectionld=0&catalog=1&productld=87548
-Follow the instructions to install Orneta FTP using a PC connected to Xphone.
・ Get a registration code for free from:
http://x.msmobiles.com/free-smartphone-software/default.aspx
Orneta FTP application on Xphone to download video message and download Droplet decoder from MMS server:
○ Start GPRS connection ○ Set Orneta FTP application from Start menu ○ Select choose Windows from Menu / Settings / Set Download Folder ○ Connect to specified MMS server (eg Section 4.1) ○ Download: Droplet video・ Select and download the encoder “DtvXphoneDemo.exe” ○ Select \ Storage \ My Documents \ from the download folder in Menu / Settings / Set ○ Reconnect to MMS server ○ Video file QCIF / 15 "bitstream dtv "Download ○ Exit
This completes the detailed description of the first embodiment.

  The feature of the present invention is that a camera recorder (camcorder) phone application consisting entirely of software is capable of real-time shooting of a full (VGA) size image (640 × 480) at 30 frames per second (fps). It is possible to use a single standard RISC processor that is used in the majority of multimedia mobile phones (mobile transceivers). In contrast, the battery power of mobile cell phones (portable transceivers) is constrained, and current MPEG based camera recorder (camcorder) phones have QCIF or CIF sizes (1 / 16th or 1/16) at 4-15 fps. / 4, VGA size) support for real-time image capture is limited. However, even these small and variable video clips have become a complex and expensive mobile phone (portable transceiver) platform design, in which video functions are implemented as a combination of hardware and software, a multi-function processor You must share the RISC processor, ASIC and DSP.

  For mobile operator environments, the video processing and transmission technology that avoids the complexity of the present invention utilizes the infrastructure of existing mobile cell phones (mobile transceivers) and mobile multimedia communications service controllers (MMSCs). However, it will be integrated into a powerful new creative software stand-alone video communication platform that enables turnkey deployment. In the management platform of the embodiment of the present invention supplemented with the above-mentioned mobile camcorder (video camera) application, a module for integrating compressed images and video into a complete mobile multimedia / integrated text / sound Delivered with messages, on-the-fly editing, thumbnail preview multimedia mailbox, online repository service and subscription management.

<Comparison of operation (efficiency) (PERFORMANCE COMPARISON)>
When the optimized MPEG-2 / MPEG-4 codec and the video codec according to the present invention are compared, the user can use both software and hardware in terms of power consumption, as shown in Table 1 below). A 40X reduction could be provided. Hardware product implementation costs have been significantly reduced, and the number of required CMOS gates has been reduced by a factor of 10 from about 1,000,000 to 100,000, as does the corresponding silicon real estate demand.
For video processing at full size (VGA) and full frame rate (30 fps), the improved video codec design of the present invention reduces the amount of memory required internally from several megabytes to 128 kilobytes, The memory resources installed in the transceiver (cell phone) were released for other revenue-generating elements and applications. Since the codec according to the present invention can efficiently process both still images and video, the codec, which is separated into MPEG and JPEG, can be replaced with one inexpensive and low-power solution.

表１、コーデック実行効率比較：モバイル送受信機（携帯電話）アプリケーション Although fully available in VGA deployment and support, 30 fps camcorder (video camera) phones, and related services, the unique mobile video platform technology of the present invention further extends to a wide range of others by combining: Sufficient benefits are shown through mobile video services: Measureable image size: QCIF (176 x 144)-D1 (720 x 480), simplified video editing (cut, insert, text E.g. overlay), simplified synchronization with audio codecs, and low-potential enhanced video streaming efficiency.

Table 1, Codec execution efficiency comparison: Mobile transceiver (cell phone) application

The present invention also provides an MMS infrastructure that enables deployment of premium video delivery services related to patented software video codec / camcorder (codec / camcorder) applications for mobile transceivers (cell phones). It consists of structure products. A further feature of the present invention consists of advanced transcoding applications supporting full interoperability with other commonly deployed standards-based or proprietary video formats. Also included is a complete mobile multimedia message and "ring-tone" with corresponding compressed image and video and sound and text of the present invention and corresponding MMS message management A content management platform that provides modules for integrating together a set of capabilities.
This content management platform can be used by both wireless communicators and MMS service providers as a set of software modules, as a quick and cost-effective upgrade to existing MMS infrastructure, or as a stand-alone for new MMS controller devices. Can be used as Aaron's server. Patented MMS infrastructure products will include:
Product and description
> DTV-VGT Video Gateway Transcoder:
Upgrade existing MMS video gateways to support conversion of video content between DropletDTV formats and video formats like MPEG-2, MPEG-4, Motion-JPEG, Microsoft Media and RealVideo Software transcoder application for.
> DTV-CMP Software Content Management Platform:
A complete content management software module for upgrading existing MMS messaging application servers: sound and text, on-the-fly editing, thumbnail preview, multimedia mailbox, on-line repository service and Generation of MMS messages and "ring-tones" that integrate the compressed images and videos of the present invention along with subscription management.
> DTV-CMS content management server:
Server-based integrated software content management platform for new MMSC deployments.

The present invention further includes a content management service platform software module or stand-alone server that includes:
> Mobile Multimedia Composer: Combines the improved wavelet-compressed images and video of the present invention with sound and text into a single message.
> Preview Player: Provides a “thumbnail” preview with MMS messages integrated with wavelet compressed images and video of the present invention.
> Mobile Multimedia Editor: Enables on-the-fly editing with wavelet-compressed images and videos of the present invention and MMS messages integrated with tools and filters.
> Multimedia ring-tone creator: Wavelet-compressed images and videos combined with multi-tone ring-tones and other sounds allow users to create a personal multimedia “ring-tone” (ring-tone) tone) ".
> Mobile Multimedia Album or "Mblog": Repository (information database) for wavelet compressed images, videos and integrated MMS messages of the present invention.
> Mobile Multimedia Subscription Management: Copy / forward MMS messages integrated with compressed wavelet image and video of the present invention. : Purchase additional storage devices: Purchase DVD hard copies.
> Mobile Multimedia Mailbox: SMS managed in and out boxes for MMS integrated in the present invention.
> Mobile multimedia address book: Mobile multimedia contact management.

It should be noted that embodiments of the present invention are provided:
> Flexible, quick and cost effective OTA (wireless) / OTN (over the net) upgrades increased ROI from installed MMS infrastructure.
> Advanced transcoding supports full interoperation between other commonly developed standards-based and proprietary video formats.
> A content management platform that can be used as a set of software modules to upgrade an existing MMS infrastructure or as a stand-alone server for new MMS devices

<JSR-135: MOBILE MEDIA API SPECIFICATION>
The DTV-JVC Java video codec of the present invention generates a restored video image that supports all the player functionality defined in the Java community process JSR-135, including:
Get Int getDisplayHeight ()
Returns the actual height of the current flowing through the video.
Get Int getDisplayWidth ()
Returns the actual width of the current flowing in the video.
Get Int getDisplayX ()
Returns the video X coordinate for the GUI object when the video was aired.
Int getDisplayY ()
When the video is aired, it returns the video's Y coordinate for the GUI object.
byte [] getSnapshot (java.lang.String imageType)
Get a snapshot of the displayed content.
Int getSourceHeigh ()
Returns the height of the source video.
Int getSourceWidth ()
Returns the width of the source video.
java.lang.Object initDisplayMode (int mode, java.lang.Object arg)
The mode is initialized according to the video broadcast status.
Void setDisplayFullScreen (Boolean fullScreenMode)
Set the size of the related area so that the video clip is fullscreen.
Void setDisplayLocation (int x, int y)
If the video is aired, set the canvas position as the video location.
Void setDisplaySize (int width, int height)
Change the size of the video image.
Void setVisible (boolean, visible)
Whether the video will be broadcast or disconnected.

<JSR-234: ADVANCED MULTIMEDIA SUPPLEMENTS>
The DTV-JVC Java video codec of the present invention generates a restored video image that supports all of the player effect controls defined in the Java community process JSR-234, including:
ImageFilterControl
ImageFilterControl is an image effect that can be used to set various image filters, monochrome and not.
IimageTonalityControl
ImageTonalityControl is an effect that can be used to set various image settings such as brightness, contrast and gamma (brightness).
ImageTransforrnControl
ImageTransformControl is used to crop, zoom, mirror, flip, stretch, and rotate an image.
OverlayControl
OverlayControl controls the setting of the overlay image of the first screen of the video and the still image,
WhiteBalanceControl
WhiteBalanceControl is the image / video effect of changing the white balance.

  The present invention further comprises products, methods and processes for establishing, providing and operating a mobile video blog service. This service provides all users with a video phone capable of shooting, editing, saving, sharing, and “publishing” personal videos, as well as online movies.

For users, a mobile video blog service, code-named "Mobedia", is offered as a product of the present invention:
1. A Mobedia software camcorder application that can be pre-installed on a mobile cell phone (mobile transceiver) that users can download and install on any video phone that supports Java.
2. Mobedia software's camcorder application allows users to shoot full VGA / 30 fps (or even higher quality) video using their mobile cell phone.
3. Using the movie application of Mobedia software, users can browse / edit, add titles, etc. If you take "splice" in multiple (duplicate), you can use the software camcorder application of the present invention. Can be used to make personal movies.
4). A simple version of the movie application in Mobedia software will be distributed to users free of charge, while a richer "Chinema-Pro" version will be purchased by the user.
Features of the present invention are provided in a server named Mobedia server:
5. Using the Mobedia mobile phone (mobile transceiver) client software, the user can transmit the captured video to the Mobedia server via a mobile, fixed, wireless, or wired connection.
6). The Mobedia subscription service allows users to archive videos and movies taken on a server (paid storage), and to bring out and download additional editions online, as well as a variety of popular You can save the video in a format or request a copy on DVD (paid).
7). The Mobedia subscription service allows users to create movie albums on the Mobedia site and invite friends, family, colleagues, etc. to watch those movies or copy them to a DVD (paid) as a gift. Can be ordered.
For friends, family and others, the Mobedia type according to the present invention is provided:
8). Friends, family, colleagues, etc. can receive email invitations to watch the user's movies, downloads and stored copies (paid), and can order a copy of the DVD (paid) for themselves.
For general viewers (movie models), Mobedia according to the present invention provides:
9. Droplet's Mobedia subscription service allows their movies to be "published" to the general public and can be viewed on the Mobedia movie site.
10. Public viewers can search public archives of movies with rankings, themes, categories, etc. on the Mobedia movie site and free previews.
11. Ordinary viewers can pay, watch, download, or copy to DVD.

The following methods and processes include features of the present invention and can be implemented exclusively with current technology.
Source of income / business model:
Movie Maker: Pay for server space, file downloads, DVD orders. Pay for the enhanced version of the editing software.
• Friends, family, etc .: Pay for file downloads and order DVDs.
• General audience: Pay for viewing the entire movie (free for previews), ordering or downloading DVDs.
-Moviemakers get a share of the fees paid by regular viewers.
• Mobile operators: pay for revenue sharing models according to increased data traffic.
The service component of the present invention features:
-Mobile video messaging platform for personal video content capture, transmission and management according to the present invention (server and mobile phone (mobile transceiver) software application)
The video camcorder application of the Mobedia software of the present invention enables video recording on any java / video phone.
The Mobedia software video movie client application of the present invention includes basic video production and editing and viewing techniques (simple and professional versions).
The Mobedia software video movie wave-based content management application of the present invention supports Mobedia movie movie albums, personal moving sharing, and “publishing” of Mobedia movies. .

  Shown here is an improved mobile image application and mobile transceiver (cell phone) architecture and service platform architecture, essentially high quality still and video images for mobile subscribers. Is a combination of technical complexity and cost reduction for providing. Improved adaptive joint-source channel coding technology enables wireless carriers and MMS service providers to achieve a greater range of quality of service (QoS) efficiencies and price levels for consumers and enterprise customers. The ability to accommodate presentations, thereby maximizing the revenue generated by using this wireless network infrastructure. Improved adaptive joint-source channel coding techniques are based on algorithms with higher computational efficiency, including channel type (wireless or wired), channel bandwidth, channel noise / error characteristics, It makes it possible to support a higher level of network homogeneity in terms of user equipment and user services. Further provided are methods, apparatus, processes and business methods that provide enhanced and innovative services in the field of still images and video in mobile phones.

  Further, the following summary is provided as a feature of the present invention.

  A mobile image application that uses an improved wavelet-based codec. This application takes the form of implementation by single software execution, hardware only implementation, or software + hardware hybrid implementation.

  It also provides a system and method comprising improved joint-source channel coding with the improved wavelet-based codec described above using fine grain stretchability, and the instant of the recovered video signal And the instantaneous value of all three channel bandwidths and error conditions of the source coder, channel coder and adaptive rate controller for maximum control of both quality and average quality (video rate vs. distortion) And information on both scheduled values are used. In addition, the provided systems and methods can apply a greater range of quality of service (QoS) efficiency and price levels for consumers and enterprise MMS customers, channel types (fixed wireless communication, mobile Wireless and wired communications), channel bandwidth, channel noise / error characteristics, user equipment, and much higher levels of support for network heterogeneity in user services including improved multicasting.

  Mobile camcorder (video camera) applications are also provided. This application combines the features described in Section 2 above with related image pre-processing and post-processing functions and audio recording for full camcorder capability on mobile devices. Hardware-only implementation, or software + hardware hybrid implementation.

  A mobile image application is also provided. This application uses a modified wavelet-based codec and is implemented as a Java application. This implementation takes the form of implementation by software single execution, implementation of hardware only, or hybrid implementation of software + hardware.

  Also provided is a mobile camcorder application that combines the applications described in the previous section with image pre- and post-processing functions and voice recording for full camcorder capabilities on mobile devices. This application takes the form of implementation by single software execution, hardware only implementation, or software + hardware hybrid implementation.

  Also provided is an architecture for mobile transceivers (cell phones) that can be used as images and used as pects. The mobile image application featured in the previous section of this summary is the mobile transceiver (cell phone). The handset is integrated into the baseband's multimedia processing section, image module, or removable storage media.

  Further provided is a wireless delivery or upgrade with the above features of the above summary in a portable transceiver (cell phone) capable of using the above images.

  Further provided is a system that allows the above features and POS installation or upgrade of the system to a portable transceiver (cell phone) capable of using images.

  Also provided are mobile image transcoders, all of which are software applications that make the above-mentioned features of this summary universally compatible with standards-based image formats owned by others, via automated networks. Delivered or installed into the MMSC video gateway by upgrade or manual operation.

  Further provided is a mobile image service platform architecture and method and system that combine all the features of this summary.

  The above is a feature of embodiments of the present invention, and various alternatives, modifications and equivalents may be used. Therefore, the above description should not be taken as a limitation on the scope of the invention as defined in the appended claims.

Indicates the video image size limit for mobile image communication. A system diagram of information source / channel coding (joint source-channel cording) is shown. (A) Encoder (b) Decoder The structure of a mobile image portable apparatus is shown. The configuration of the mobile image service platform is shown. A comparison diagram of conventional video codec technologies is shown. FIG. 2 shows a system diagram of improved information source / channel coding (joint source-channel cording). (A) Encoder (b) Decoder 2 shows the configuration of an improved mobile image transceiver platform. A comparison diagram of the effect of the video codec is shown. 2 shows the configuration of an improved mobile image transceiver platform. 2 shows the configuration of an improved mobile image transceiver platform.

1 shows the platform configuration of an improved mobile image service. (OTN) upgrade over a network of deployed MMSC video gateways. Video MMS shows self-contained playback that does not require transcoding. Demonstrates the complexity, cost, and number of video editing servers required to deploy media production services. Shows mobile video service platform. Demonstrate faster and lower cost development and deployment of higher quality multimedia portable transceivers (cell phones) services. Shows the mobile video service of the present invention. Shows applications to the broadband multimedia device and service of the present invention. Fig. 4 illustrates implementation options for the software image application of the present invention. Fig. 5 illustrates implementation options for an image application accelerated with the hardware of the present invention.

Fig. 5 illustrates an implementation option for a software image application accelerated with hybrid hardware of the present invention. Fig. 2 shows an application of a simplified multimedia portable transceiver (cell phone) platform architecture. Demonstrates the elements of a mobile video communication demo on a GSM / GPRS network. Fig. 4 shows a specific MMS function according to the present invention.

Claims (2)

The image service platform development method consists of the following steps:
Providing a transcoder application to a networked download server,
Send to the network that the transcoder application is available for deployment,
The transcoder application is deployed through the network from the download server to the video gateway on the network.   The method of claim 1 further includes automatically installing the deployed transcoder application on the video gateway.

Images