JP2000312360A - Information service system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a means that reproduces and displays diversified contents data such as high image quality full color still picture data and three- dimensional computer graphics by using a reproduction terminal such as a mobile information terminal and a mobile phone having capability of displaying compressed moving picture data but not having capability of displaying data such as JPEG still picture data. SOLUTION: The system comprises a servicing unit 109 that stores a plurality of contents data 110 and transmits them via a network interface 116ab and a communication channel 112 and a service relay unit 101 that transmits them to a reproduction terminal 106 via a network interface 115ab and a channel 111. The reproduction terminal 106 having capability of displaying compressed moving picture data but not having capability of displaying desired contents data can display the desired contents data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a computer,
The present invention relates to an information providing apparatus configured by a digital network such as a LAN (Local Area Network) or a public telephone network.

[0002]

2. Description of the Related Art A personal computer and a LAN (Local A)
Consider an information providing apparatus having a server-client type structure constituted by a digital network such as a public network (REA network) or a public telephone network. FIG. 28 shows a configuration diagram of a general information providing apparatus. In FIG. 28, reference numeral 2801 denotes a service providing device, 2802, a reproduction terminal, and 2803, a communication path.

[0003] The service providing apparatus 2801 includes a large-capacity storage device such as a hard disk, stores content data in the storage device, and stores the content data in the playback terminal 280.
2 reproduces the content data in response to a request from
Send to 02. The reproduction terminal 2802 extracts desired content data from the service providing apparatus 2801 via the communication path 2803, and reproduces and displays the content data. As a matter of course, it is necessary for the playback terminal 2802 to have a capability of decoding each data constituting the content data.
One example of such information provision is WWW (World Wide Wealth).
The content data described in HTML (Hyper Text Markup Language) in b) can be reproduced and displayed by Microsoft's "Internet Explorer" which is widely used to browse homepages on personal computers. Can be At this time, a personal computer is generally used as the reproduction terminal 2802.

[0004]

With the spread of mobile devices, portable information terminals (PDs) will be used as reproduction terminals 2802 in the future.
A: Personal Digital Assistants) and systems that provide information using mobile phones are expected to be required.
These playback terminals 2802 generally have lower capabilities of a CPU (central processing unit) and a storage device than personal computers, and it is completely impossible to play back and display content data described in HTML or the like. Although not possible, it must be used under many restrictions, such as displaying only character data or displaying black and white still image data.

[0005] W-CDMA (Wide-band Code Division Multip
le Access) and other new mobile communication networks (IMT-2
000: International Mobile Telecommunications-200
With the spread of (0), the data transmission capacity per unit time of the communication path 2803 is improved, and the content data used for providing information is diversified, such as full-color still image data and moving image data accompanied by audio.

For example, JPEG (Joint Photographic Exper
Consider displaying a full-color still image (hereinafter referred to as a JPEG still image) compressed in a ts group) format on a portable information terminal or a mobile phone. One way to realize this is to provide a portable information terminal or a mobile phone with a function of decoding a JPEG still image, but there are some problems.

[0007] The first problem is that the time from when a display is requested to when the display is completed is long. For example, 768 pixels vertically and 1024 pixels horizontally obtained by an electronic still camera
The data amount of a JPEG still image is about 100 Kbytes. Here, K means 1024. This JPEG still image is
In order to take out the data via a mobile communication network having a data transmission capacity of 4 kbps (64000 bits per second) and display it on a mobile phone, it is necessary to wait for the data transfer to be completed for about 13 seconds.

[0008] The second problem is that the number of components of a portable information terminal or a portable telephone increases. This is because a JPEG still image decoder, a storage device for temporarily storing a JPEG still image, and the like must be provided in a portable information terminal or a mobile phone. In addition, a storage battery becomes large in order to cope with an increase in power consumption due to the implementation of such a function, which is an unacceptable constraint for these terminals which are small and lightweight.

[0009] A third problem is that it is difficult to cope with diversification of content data. Each time content data in a new data format appears, it is very difficult to provide a display function for the content data in a portable information terminal or a mobile phone. That is, it is very difficult to carry out the work of incorporating the content data reproducing software as necessary, such as the practice of a general personal computer, with a portable information terminal or a portable telephone.

The present invention relates to a method for compressing moving picture data or multiplexing.
Although it has the ability to display AV data, its data processing ability is lower than that of a personal computer, and it does not have the ability to display data such as JPEG still images. Full color still image data and 3D computer graphics
An object is to provide means for reproducing and displaying various content data.

[0011]

SUMMARY OF THE INVENTION In order to solve this problem, according to the present invention, first, a plurality of content data can be stored and the content data can be transmitted via a network interface in response to a request. A service relay device that acquires data from a service providing device, converts the data into a format that allows the data to be reproduced without any problem even in the user environment (performance of the reproduction terminal), and sends it out. It runs on a general personal computer. The content data reproduction software can be used as reproduction means, and the means relays still image data derived from the content data obtained by reproducing the content data obtained from the service providing device via the network interface to the moving image encoding means. Stationary with data relay means and stepwise image quality complementary coding Moving image encoding means for generating high-quality compressed moving image data from the image data and transmitting the compressed moving image data to the reproducing terminal via the second network interface; Control means for determining an area of the still image data to be transmitted and transmitting the determined area to the moving image encoding means. As a result, it is possible to display the desired content data on a playback terminal that has a display capability of the compressed moving image data but does not have a capability of displaying the desired content data. Any part of the data can be enlarged and displayed.

Secondly, in the first invention, the control means may receive a button operation on the reproduction terminal, generate a control signal for changing a display state of the reproduction means, and send the control signal to the reproduction means. Also, the area of the still image data to be encoded into the compressed moving image data is determined and transmitted to the moving image encoding means, and the moving image encoding means changes the encoding setting as needed, so that the contents are sequentially changed. This generates compressed video data. As a result, it is possible to display the desired content data on a playback terminal that has a display capability of the compressed moving image data but does not have a capability of displaying the desired content data. It is possible to display related content data by enlarging and displaying an arbitrary part of the data, moving the displayed content up, down, left and right, and arbitrarily selecting the displayed link information.

Thirdly, in the first aspect, the moving image encoding means can also generate audio data in a specified data format from audio data included in the content data, and And audio data are multiplexed to generate multiplexed AV data, which can be transmitted to the playback terminal via the second network interface. This allows the desired content data to be displayed with audio playback on a playback terminal that has the ability to display multiplexed AV data but does not have the ability to display desired content data. Thus, any part of the content data displayed on the reproduction terminal can be enlarged and displayed.

Fourthly, the service providing device and the service relay device according to the first invention are integrated, thereby having a display capability of compressed moving image data.
It is possible to display desired content data on a playback terminal that does not have the ability to display the desired content data, and it is also possible to enlarge and display an arbitrary part of the content data displayed on the playback terminal. Become.

Fifthly, the service providing device and the service relay device according to the second invention are integrated with each other, thereby having a capability of displaying compressed moving image data.
The desired content data can be displayed on a playback terminal that does not have the ability to display the desired content data. In addition, any part of the content data displayed on the playback terminal can be enlarged and displayed, and the display content can be displayed. Is moved up, down, left, and right, and by arbitrarily selecting the displayed link information, the related content data can be displayed.

Sixth, the service providing device and the service relay device according to the third aspect of the present invention are integrated with each other, thereby having a display capability of multiplexed AV data.
The desired content data can be displayed along with the reproduction of audio on a playback terminal that does not have the ability to display the desired content data. In addition, any part of the content data displayed on the playback terminal can be displayed. Can be enlargedly displayed.

[0017]

Embodiments of the present invention will be described below with reference to FIGS. 1 to 27. The present invention is not limited to these embodiments at all, and can be implemented in various modes without departing from the gist thereof.

(Embodiment 1) A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 shows a configuration diagram of the first embodiment. In FIG. 1, 101 is a service relay device, 102 is a playback device, 103 is an image data relay device, 104 is a moving image encoding device, 105 is a control device,
06 is a multimedia communication terminal as a reproduction terminal, 1
07 is a moving image reproducing device, 108 is a content browsing device,
109 is a service providing apparatus, 110 is content data, 111 is a communication path between the terminal 106 and the service relay apparatus 101, 112 is a communication path between the service providing apparatus 109 and the service relay apparatus 101, and 113 is the playback apparatus 1
02, a control signal 114 for the video encoder 104, 115a and 115b for the network interface between the service relay device 101 and the multimedia communication terminal 106, and 116a and 116b for the service relay device 101 and the service providing device, respectively. Reference numeral 109 denotes a network interface, and 117 denotes operation buttons.

First, a mode of use assumed in this embodiment will be described. The service providing apparatus 109 stores a digital still image (hereinafter, referred to as a digital still image) as content data 110.
(Described as a still image). The multimedia communication terminal (hereinafter, referred to as a terminal) 106 acquires and displays arbitrary content data 110 (here, a still image) in a form that can be displayed on its own display means. Terminal 106
Is assumed to be about the size of a mobile phone. Therefore, the display device is a liquid crystal display having a small screen size, for example, a diagonal length of about 2 inches.

On the other hand, a still image is assumed to be data shot by a digital still camera or the like.
Is very large as compared with the size of the display device, for example, 1024 pixels × 768 pixels. When such content data 110 is displayed on the terminal 106, the entirety of the content data 110 is first reduced and displayed. As a matter of course, the entire image of the content data 110 displayed on the terminal 106 has low visibility of details. Therefore, an arbitrary part of the content data 110 is enlarged to an arbitrary magnification by the operation button 117 provided on the terminal 106, and the details of the content data 110 are displayed.

Next, the operation of the present embodiment and its realizing means will be described. The terminal 106 includes a moving image reproducing device 107
And a content browsing device 108 and operation buttons 117, for example, for establishing a communication path such as the International Telecommunication Union Telecommunication Standardization Sector (hereinafter referred to as ITU-T) recommendations H.320, H.323 or H.324. Communication conforming to the protocol for communication. The content browsing device 108 is a service relay device 1
01, a device for displaying the content data 110 obtained from the service providing device 109 via the communication channel 111 and the communication channel 112, and decoding and displaying the compressed moving image data using the moving image reproducing device 107 if necessary. I do. In addition, the content browsing device 108 operates the operation button 11
7 is used to provide a function for selecting content data 110. In the present embodiment, the terminal 106 displays character data representing a menu or the like included in the content data 110 by the function of the content browsing device 108 itself, and displays the compressed moving image data by the moving image reproducing device 107.

The moving image reproducing device 107 decodes and displays the compressed moving image data under the control of the content browsing device 108. In addition, when the compressed moving image data being received is stopped, the moving image reproducing device 107 stops displaying at that time, and keeps the display state until newly receiving the compressed moving image data.

The service relay device 101 is located between the terminal 106 and the service providing device 109, and transmits content data 1 composed of character data to the content browsing device 108.
10 is a device that converts the content data 110 that cannot be displayed by the content browsing device 108 into compressed moving image data and sends it to the moving image reproducing device 107.

The operation of the service relay apparatus 101 will be described. Control device 10 for controlling service relay device 101
5 is shown in FIG. In FIG. 2, 201 is a relay control unit, 202 is a communication control unit, 203 is a communication path with the network interface 116b, and 204 and 205 are communication paths with the network interface 115a.

Basic control of communication between the terminal 106 and the service relay apparatus 101 is performed by the communication control unit 202 according to a protocol for establishing a communication path, such as a method defined in ITU-T Recommendation H.245. . When the communication between the terminal 106 and the service relay apparatus 101 is started, the communication control unit 202 exchanges the data display capability of the terminal 106 by performing a capability exchange based on a protocol for establishing a communication path such as ITU-T Recommendation H.245. And until the communication is completed, it is stored in its own storage device. After communication between the terminal 106 and the service relay device 101 is established by the communication control unit 202, the main functions of the control device 105 are provided by the relay control unit 201.

The relay control unit 201 appropriately refers to the data display capability of the terminal 106 stored in the communication control unit 202, and sends the content data 110 to the terminal 106 as it is or after converting it into compressed moving image data. Terminal 106
When a certain content data 110 in the service providing apparatus 109 is designated, the relay control unit 201 extracts the designated content data 110 from the service providing apparatus 109 via the communication path 203 and the network interface 116ab.

Next, the relay control unit 201
If the content browsing device 108 of the terminal 106 can display all the data of the specified content data 110 with reference to the data display capability of the terminal 106 stored in the
04 to the terminal 106.

If the content browsing device 108 of the terminal 106 cannot display all data of the designated content data 110, the relay control unit 201 controls the playback device 102 and the moving image encoding device 104 to 110 is converted into compressed moving image data and sent to the terminal 106.

If the content browsing device 108 of the terminal 106 can display a part of the specified content data 110, the relay control unit 201 prompts the terminal 106 to select a display method. The terminal 106 is the content browsing device 1
08 is selected, the relay control unit 2
01 extracts only data that can be displayed by the content browsing device 108 from the designated content data 110 and sends it to the terminal 106. When the terminal 106 selects display using the moving image reproducing apparatus 107, the relay control unit 201 converts the designated content data 110 into compressed moving image data and sends it to the terminal 106.

Processing for converting content data 110 into compressed moving image data and sending it to terminal 106 will be described. In the present embodiment, the content data 110 is stored in a JPEG (Joint
(Photographic Experts Group) format, and software for displaying still image data, such as JPEG format, which operates the playback device 102 on a general personal computer.

The playback device 102 transmits the content data 110 specified by the control signal 113 from the relay control unit 201 in the control device 105 to the service providing device 1 via the communication path 112 and the network interface 116ab.
09 and displayed on the image data relay device 103. Note that the “display” here does not need to be actually displayed on a display or the like, but means that the playback device 102 sends data to the same interface as the display or the like, as described later.

[0032] The moving picture coding device 104 includes the control device 10
5 from the image data relay apparatus 103 based on the control signal 114 from the relay control unit 201 in the content data 11.
The compressed moving image data is generated by extracting still image data derived from 0, and sent to the terminal 106 via the communication channel 111 and the network interface 115ab.

Here, the realizing means and operation of the image data relay apparatus 103 will be described. The image data relay apparatus 103 according to the present invention must satisfy the following two conditions.

The first condition is that the playback device 102 can use easily available content playback software such as a commercially available product. Although the content data 110 in the present embodiment is still image data in the JPEG format or the like as an example, it is expected that the content data 110 in various formats will be handled in actual use. For example, HTML (Hyper Text Markup Language)
And VRML (Virtual Realit)
3D computer graphics data described in (y Modeling Language).

The various contents data 110
In order to cope with this, the playback device 102 must be provided with a number of playback functions corresponding to the types of the content data 110. In this case, if easily available content reproduction software such as a commercially available product can be used as a means for reproducing the respective content data 110, the service relay apparatus 101 can be easily and inexpensively realized. In addition, it is possible to quickly respond to the content data 110 in a new format.

The second condition is that the content data 110 displayed on the reproducing apparatus 102 be transmitted to the moving picture encoding apparatus 104 without deteriorating the image quality. The image data relay device 103 in the present embodiment is realized by a video memory (hereinafter, referred to as V-RAM) used in a general personal computer or the like. By using the V-RAM, the image data relay device 103 can use a CRT (Cathode
The same interface as displaying data on a display device such as a ray tube can be provided to the playback device 102. That is, transmission to the interface is regarded as “display”.

In addition, an operating system (hereinafter referred to as OS) generally used in personal computers, for example, Microsoft Windows-N
T provides the application program with a function of extracting data from the V-RAM, so that the moving image encoding device 104 can easily use this function to extract the still image data from the V-RAM. Image data can be obtained.

By realizing the image data relay device 103 by a V-RAM, the above two conditions can be satisfied without newly providing a special control program. That is, the still image data display software that is the playback device 102 displays a still image on a display device such as a CRT without any special change (or sends data transmission that can be regarded as the same as “display” to the V-RAM). By doing so, the still image data derived from the content data 110 can be
4 can be told.

Next, the moving picture coding apparatus 104 will be described. FIG. 3 shows the configuration of the moving picture coding device 104.
3, reference numeral 301 denotes a data acquisition unit, 302 denotes a shared memory, 303 denotes a moving image encoding unit, 304 denotes an operation control information memory, 305 denotes still image data from the image data relay device 103, 306 denotes compressed moving image data, and 307 denotes This is a control signal.

The moving picture coding apparatus 10 according to the present embodiment
No. 4 has the following two features. The first feature is that, when the still image data to be encoded changes over time like an animation, general moving image encoding is performed, and the still image data to be encoded is In the case where there is no time change like image data, moving image encoding is performed by stepwise image quality complementing processing to enable decoding of high quality still image data.

A second feature is that a method of generating compressed moving image data, for example, a method of generating compressed moving image data in accordance with characteristics of still image data to be coded, such as characters, natural images, computer graphics, and the like. Pre-filtering,
Is adaptively changed to generate high-quality compressed moving image data.

The moving picture coding apparatus 10 in the present embodiment
Operation 4 and means for realizing the above two features will be described. The operation control information memory 304 stores information defining the operation of the video encoding device 104 via the control signal 114. In the present embodiment, the information stored in the operation control information memory 304 is the position of the still image data derived from the encodeable content data 110 stored in the image data relay device 103, for example, the upper left pixel of the rectangular area. Address on the V-RAM, the size of the still image data derived from the content data 110,
For example, the number of pixels in each of the horizontal and vertical directions, the type of still image data derived from the content data 110, for example, characters,
The position of the area to be actually encoded in the still image data derived from the content data 110 such as a natural image or a computer graphic, for example, the address of the upper left pixel of the rectangular area on the V-RAM, the still image data derived from the content data 110 , The size of the area to be actually encoded, for example, the number of pixels in the horizontal and vertical directions of the rectangular area, the number of bits per unit time (bit rate) of the compressed moving image data, the frame per unit time of the compressed moving image data Number (frame rate), resolution of compressed video data, mode of encoding process,
In other words, gradual image quality complement mode or normal video mode,
An operation permission flag of the moving image encoding device 104, and the like.

FIG. 4 is a schematic diagram showing the relationship between the respective areas stored in the operation control information memory 304. FIG.
Reference numeral 401 denotes a display memory area of the image data relay apparatus 103; 402, an origin of the memory area 401;
Reference numeral 3 denotes still image data derived from the encodeable content data 110 displayed on the image data relay device 103;
4 is still image data 40 derived from the content data 110
The reference point 405 is a representative point for specifying the position of the region 405 actually encoded in the still image data 403 derived from the content data 110, and the reference point 406 is a representative point for specifying the position of the region 405 to be actually encoded.

In FIG. 4, Xm and Ym are the numbers of horizontal and vertical pixels of the display memory area 401, respectively, and Xd and Yd are the numbers of horizontal and vertical pixels of the still image data 403, respectively (xd0, yd0 ) Indicates the display memory area 4 of the representative point 404.
01, Xt and Yt are the number of pixels in the horizontal and vertical directions of the coding area 405, respectively, and (xt0, yt0) is the representative point 406.
In the display memory area 401.

FIG. 5 is a flowchart showing the operation of the data acquisition unit 301. The data acquisition unit 301 refers to the operation permission flag of the operation control information memory 304 at a certain cycle according to a timer provided therein (step 50).
1) If the operation permission flag for permitting the generation of the compressed moving image data is valid (step 502), the image data relay device is operated based on the information on the still image data to be encoded stored in the operation control information memory 304. The still image data is acquired from 103 (step 503).

Next, the data acquisition unit 301 enlarges or reduces the acquired still image data based on the information on the size and type of the still image data stored in the operation control information memory 304 and the resolution of the compressed moving image data. Processing is performed to create still image data of a predetermined resolution. If necessary, a filter process or the like is performed according to the type of still image data, and preprocessing for obtaining high-quality compressed moving image data is performed.

After performing such processing, the obtained still image data is stored in the shared memory 302 (step 50).
3), through the control signal 307, the video encoding unit 303
To start the encoding process (step 504), and waits until the encoding process in the moving image encoding unit 303 ends (step 505).

The data acquisition section 301 is a moving picture encoding section 3
03, the end of the encoding process is detected, and if the operation mode is the stepwise image quality complement mode, the operation permission flag of the operation control information memory 304 is invalidated (step 506).
Thereafter, step 501 is started again according to the timer.

When the mode of the encoding process stored in the operation control information memory 304 is the ordinary moving image mode, the cycle of a series of processes (steps 501 to 505) performed by the data acquisition unit 301 is determined by the operation control information memory 504
The timer generates a signal so as to match the frame rate of the compressed moving image data stored in. In this case, the moving image encoding unit 303 generates and outputs one frame of video data every one cycle. That is, the data acquisition unit 301 maintains the frame rate of the compressed moving image data.

On the other hand, when the mode of the encoding process stored in the operation control information memory 304 is the step-wise image quality complement mode, the moving image encoding unit 303 While maintaining the data frame rate, a plurality of frame data is generated by the stepwise image quality complementary coding process.

The moving picture coding unit 303 includes a data acquisition unit 3
When a process start command is received from 01, it refers to information on the bit rate, frame rate, and resolution of the compressed moving image data stored in the operation control information memory 304, and generates and outputs desired compressed moving image data 306. When the encoding process is completed, the moving image encoding unit 303 sends the control signal 30
7 to the data acquisition unit 301.

When the encoding mode is the ordinary moving image mode, the moving picture coding unit 303 performs general moving picture coding. The general moving image coding is, for example, MPEG-1 (Moti
on Picture Image Coding Experts Group-1: ISO / IEC11
172) means a method that combines intra-frame encoding and inter-frame predictive encoding like compressed video data of the format.

One of the features of this embodiment is the operation of the moving picture coding unit 303 when the coding processing mode is the stepwise image quality complement mode. FIG. 6 shows an outline of the stepwise image quality complementary coding processing. FIG. 6A shows the concept of a moving image encoding process in a general MPEG-1 format or the like, and FIG. 6B shows the concept of a stepwise image quality complementary encoding process. In FIG. 6A, reference numerals 601, 602, and 603 denote uncompressed image frames;
605 and 606 are compressed image frames, 607 and 608
And 609 are reference image frames. In FIG. 6B, 610, 611, and 612 are uncompressed image frames, and 613
, 614 and 615 are compressed image frames, 616 and 617
Reference numerals 618 and 618 denote reference image frames, and the processing changes from left to right with the passage of time in both figures.

First, a general MPEG will be described with reference to FIG.
An overview of moving picture coding processing in the -1 format and the like will be given. In the encoding process for the uncompressed image frame 601, intra-frame encoding including discrete cosine transform (DCT) and quantization is performed, and as a result, a compressed image frame 604 is obtained. Next, an uncompressed image frame 602
To encoding. The encoding of the uncompressed image frame 602 is performed by inter-frame predictive encoding. This encoding requires a reference image frame for the motion compensation processing. In this case, the reference image frame is the compressed image frame 60 obtained earlier.
4 is used. A motion compensation process is performed on the uncompressed image frame 602 and the reference image frame 607 to obtain a compressed image frame 605. Next, the process proceeds to encoding of the uncompressed image frame 603. Encoding of the uncompressed image frame 603 is also inter-frame predictive encoding. In this case, a reference image frame 608 created using the compressed image frame 605 and the reference image frame 607 obtained earlier is used as the reference image frame. A motion compensation process is performed on the uncompressed image frame 603 and the reference image frame 608 to obtain a compressed image frame 606. Hereinafter, the same encoding process is repeated.

Next, the step-wise image quality complementing type encoding process will be outlined with reference to FIG. Most of the step-wise image quality complementing encoding processing is the same as the general moving picture encoding processing in the MPEG-1 format or the like, but the difference lies in the uncompressed image frame and quantization step value to be used. Uncompressed image frame 61
In the encoding process for 0, intra-frame encoding is performed, and as a result, a compressed image frame 613 is obtained. At this time, a coarse (large) quantization step value is used to reduce the data amount of the compressed image frame 613.

Next, the process proceeds to the encoding of the uncompressed image frame 611. Since the gradual image quality complementing process is performed on data that does not change over time, such as still image data, the uncompressed image frame 611 used here is the uncompressed image frame 6.
The use of the same data as in FIG. 10 is one of the differences from the moving image encoding processing in the general MPEG-1 format or the like. The encoding of the uncompressed image frame 611 is performed by inter-frame prediction encoding. This encoding requires a reference image frame for the motion compensation processing. In this case, the reference image frame 616 generated from the compressed image frame 613 obtained earlier is used. Uncompressed image frame 611
From the reference image frame 616 to the compressed image frame 614
Get. Since the uncompressed image frame 611 uses the same data as the uncompressed image frame 610, it is known that the motion vector between the uncompressed image frame 611 and the reference image frame 616 is zero. Instead, quantization processing is performed on the difference between the uncompressed image frame 611 and the reference image frame 616. At this time, the compressed image frame 613 is used as the quantization step value.
By using a value smaller (smaller) than the quantization step value used when generating the compressed image frame 614, the compressed image frame 614 having more detailed information than the compressed image frame 613 is used.
Can be obtained.

Next, the process proceeds to the encoding of the uncompressed image frame 612. The encoding of the uncompressed image frame 612 is also inter-frame predictive encoding. In this case, the reference image frame is the compressed image frame 614 obtained earlier and the reference image frame 616.
Is used. The uncompressed image frame 612 is the same as the uncompressed image frame 610, and the encoding process is the same as that performed when the compressed image frame 614 is generated.
Get 5. However, as the quantization step value, a value smaller (smaller) than the quantization step value when the compressed image frame 614 is generated is used. Thus, the compressed image frame 615 becomes a compressed image frame having more detailed information than the previously obtained compressed image frame 614. Hereinafter, the quantization step value is gradually reduced (smaller).
Then, the same encoding process is repeated, and the encoding process ends when the quantization step value becomes sufficiently small.

The manner in which the image quality is gradually improved in the stepwise image quality complementary coding processing will be described using mathematical expressions.
FIG. 7 shows a series of mathematical expressions. The notation of each item in FIG.
Unless otherwise noted, it is derived from FIG.

The reproduced image frame F1 obtained from the first compressed image frame 613 can be expressed by the following equation (1). In Equation 1, x1 represents a deterioration in image quality due to a quantization error, and R1 represents a reference image frame 616. Strictly speaking, the reproduced image frame F1 and the reference image frame R1 are not equal because decoding processing such as inverse DCT processing is performed by different devices. However, since the difference between the reproduced image frame F1 and the reference image frame R1 does not significantly affect the essence of the description here, the difference is ignored. The relationship between the reproduced image frame F2 and the reference image frame R2 is the same.

The second compressed image frame 614, P1 in FIG. 7, can be expressed as in Equation 2. In Expression 2, x2 represents a deterioration in image quality due to a quantization error.
The reproduced image frame F2 obtained from the second compressed image frame 614 can be expressed as in Equation 3.

The third compressed image frame 615, P2 in FIG. 7, can be expressed as in Equation 4. In Expression 4, x3 represents a deterioration in image quality due to a quantization error.
The reproduced image frame F3 obtained from the third compressed image frame 615 can be expressed as in Equation 5.

Since the quantization step value used in generating each compressed image frame is gradually made smaller (smaller), the deterioration of the image quality due to each quantization error, x
1, x2, and x3 have a relationship as shown in Expression 6. Therefore, when the quantization step value becomes sufficiently fine (small), the quantization error becomes negligibly small, and the quality of the reproduced image frame obtained as a result is uncompressed image frame 6.
Very close to 10.

In addition to this, the data amount of the compressed image frames 614 and 615 obtained by the inter-frame predictive coding is also adjusted so as to obtain a desired bit rate by appropriately selecting the quantization step value. Is possible. By converting a still image into compressed moving image data by the method described above, it is possible to display an image with an image quality very close to the original image.

All information stored in the operation control information memory 304 can be rewritten at any time via the control signal 114. The data acquisition unit 301 and the moving image encoding unit 303 The information stored in the operation control information memory 304 is acquired every time, and the details of the processing can be changed based on the information.

Next, a method of enlarging and displaying a part of the content data 110 will be described. The operation buttons 117 of the terminal 106 have a configuration as shown in FIG. In FIG. 8, reference numeral 801 denotes direction buttons for instructing up, down, left, and right directions;
802 is a menu button, 803 is a start button, 804
Is a clear button, 805 is an end button, 806 is a dial button composed of numbers from 0 to 9 and symbols * and #. When each button is pressed, a corresponding signal is generated.

FIG. 9 shows a display example of a still image as content data 110 on terminal 106. In FIG.
Reference numeral 901 denotes a rectangular frame representing an area to be enlarged and displayed, 902 denotes a rectangular frame representing a desired area to be enlarged and displayed, and 903 denotes a rectangular frame.
2 is a rectangular frame representing a desired enlarged display area.

An example of a button operation and an example of a display state when a part of the content data 110 is enlarged and displayed will be described with reference to FIG. FIG. 9A shows an example in which the content data 110 (still image in the present embodiment) stored in the service providing apparatus 109 is displayed in full size. FIG. 9B shows the entire content data 110 displayed on the display device of the terminal 106. That is, the entire still image is displayed in a reduced size according to the size of the display device of the terminal 106. In this embodiment, this state is set as the initial state of the still image display. FIG.
(C) is a display state immediately after the direction button 801 is pressed to specify one of the up, down, left, and right directions after the initial state. Reference numeral 901 denotes a rectangular frame representing an area to be enlarged and displayed, and after pressing the direction button 801,
For example, if there is no further pressing of the direction button 801 or pressing of the start button 803 for 3 seconds, the direction button 801 is automatically deleted. The display and deletion of the rectangular frame are performed by the terminal 106 in accordance with the depression of the operation button 117 or the like. In FIG. 9D, after the direction button 801 is pressed to display the frame 901, the direction button 801 is continuously pressed an arbitrary number of times.
This is a state when the frame 901 is moved to a desired enlarged display area 902. In FIG. 9E, a region 902 to be enlarged and displayed by a frame 901 is designated, and a start button 803 is displayed.
The state is enlarged by pressing. In FIG. 9 (f), the direction button 801 is further depressed and the frame 901 is displayed.
Is displayed, and a desired enlarged display area 903 is designated. FIG. 9G shows a state where the enlarged display area 903 is enlarged and displayed. By the button operation as described above, a part of the content data 110 displayed on the terminal 106 is enlarged and displayed.

Next, means for realizing such an enlarged display will be described. When the direction button 801 is pressed an arbitrary number of times from the initial state as shown in FIG. 9B to specify a desired enlarged display area, and then the start button 803 is pressed, the moving image reproducing apparatus 107 displays on the display of the terminal 106. Is sent to the relay control unit 201.

The relay control unit 201 transmits the content data 1
The position and size of the 10 newly encoded areas are
From the position and the size of the designated area sent from the controller 06, the information in the operation control information memory 304, that is, the position and the size of the area to be actually encoded in the bitmap data are updated. Enable the permission flag.

The moving picture coding apparatus 104 generates new compressed moving picture data in accordance with the updated information in the operation control information memory 304. As a result, the terminal 106 can enlarge a desired display area.

The format of the compressed moving image data that can be handled by the moving image encoding device 104 and the moving image reproducing device 107 in this embodiment is the MPEG-4 standard (ISO / IEC1449).
6), but MPEG-1 (ISO
/ IEC11172), MPEG-2 (ISO / IEC13818), ITU-T Recommendation H.26
It is possible to use compressed moving image data obtained by an encoding method that uses both intra-frame encoding and inter-frame prediction encoding, such as 1 or H.263.

The image data relay device 1 of the present embodiment
03, it is also possible to use a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to the video memory. FIG. 10 shows a configuration diagram of an image data relay device using a semiconductor memory such as a DRAM.

In FIG. 10, 1001 is a write control unit, 1002 is a read control unit, 1003 and 1004
Is a memory independent of each other, 1005 is a shared memory, 1006 is a signal line for still image data derived from the content data 110 from the playback device 102, 1007 is still image data derived from the content data 110 to the video encoding device 104 Signal line.

The use state of the memories 1003 and 1004 at the current time is recorded in the shared memory 1005, and the write control unit 1001, the read control unit 1002,
It is readable and writable from both sides. Write control unit 1001
Knows a memory (memory 1003 or memory 1004) to which still image data can be written with reference to the shared memory 1005, and transfers the still image data obtained from the playback device 102 via the signal line 1006 to the memory. Write. The read control unit 1002
5 (memory 1003 or memory 10) in which the latest still image data readable with reference to
04), the still image data is read from the memory, and sent to the moving image encoding device via the signal line 1007.

The writing control unit provides the reproducing device 102 with an interface as if it were a display device such as a video memory. Thus, a commercially available still image display device or the like that can be easily obtained can be used as the playback device 102.

Note that the image data relay device 103 is
It is also possible to realize the above by using only one of the memory 1003 and the memory 1004. Also,
The image data relay apparatus 103 can be realized by using three or more memories having the same function as the memory 1003 or the memory 1004 in FIG.

As described above, in the present embodiment, the content data 110 in the service providing apparatus 109, ie,
The still image data in the JPEG format or the like is transferred to the service relay device 1
By converting the compressed moving image data into compressed moving image data at 01, a still image can be displayed on the terminal 106 having no still image display function.

Further, by generating compressed moving image data by stepwise image quality complementation, high image quality (about:
A still image can be displayed with the same image quality as when the acquired content data 110 is reproduced / transmitted by the reproducing device 102). Further, since the data amount of the compressed moving image data is smaller than that of the JPEG still image data, the data transfer completion waiting time can be shortened, and the content data can be displayed on the terminal 106 quickly.

Further, by using a video memory (V-RAM) for the image data relay device 103, easily available still image display software such as a commercially available product can be used, and the content data 110 in various data formats can be used. Can be easily realized. Further, by using a semiconductor memory such as a DRAM in the image data relay device 103, the display memory area 401 can be enlarged as compared with the case where a V-RAM is used, and as a result, a larger content data 110 can be stored. It can be displayed, and its practical effect is great.

(Embodiment 2) A second embodiment of the present invention will be described with reference to FIGS. FIG. 11 shows a configuration diagram of the second embodiment. In FIG. 11, 1101 is a service relay device, 1102 is a reproducing device, 1103 is a control device, and 1104 is a control signal of the reproducing device 1102.
The other components are the same as the components in the first embodiment. The main difference between the configuration of the second embodiment and the configuration of the first embodiment is a playback device 1102 and a control device 1103 in the service relay device 1101.

First, a mode of use assumed in this embodiment will be described. The terminal 106 is the same as the terminal 106 in the first embodiment, but differs in the content data 110 to be displayed. The content data 110 assumed in the second embodiment is HTML (Hyper Text M).
This is a homepage described in arkup language (arkup language) and is composed of character data and still image data. When such content data 110 is displayed on the terminal 106, the entirety of the content data 110 is first reduced and displayed. As a matter of course, the content data 110 displayed on the terminal 106 has low detail visibility. Therefore, the operation button 1 provided in the terminal 106
At 17, an arbitrary part of the content data 110 is enlarged to an arbitrary magnification, and details of the content data 110 are displayed. In addition, the displayed home page is moved up, down, left and right, or another home page associated with the URL (Universal Resource Locator) is selected and displayed.

Next, the operation of this embodiment and the means for realizing it will be described. As the playback apparatus 1102, homepage browsing software that runs on a general personal computer, for example, “Internet Explore by Microsoft”
r ".

Next, the control device 1103 will be described.
FIG. 12 shows the configuration of the control device 1103. 12, reference numeral 1201 denotes a relay control unit; 1202, a communication path with the network interface 116b; and 1203, a communication path with the network interface 115b. Other components are the control device 105 according to the first embodiment.
Is the same as the component of

Basic control of communication between the terminal 106 and the service relay device 1101 is performed by the communication control unit 202 as in the first embodiment. The communication control unit 202 controls the terminal 1
After communication with the service relay device 1101 has been established, the main function of the control device 1103 is the relay control unit 1201
Provided by As in the case of the first embodiment,
The relay control unit 1201 selects data to be sent to the terminal 106, that is, character data or compressed moving image data.

The process of converting the content data 110 into compressed moving image data and sending it to the terminal 106 will be described. The playback device 1102 is a relay control unit 1 in the control device 1103.
The content data 110 specified by the control signal 1104 from the server 201 is extracted from the service providing apparatus 109 via the communication path 112 and the network interface 116ab, and displayed on the image data relay apparatus 103.

The moving picture coding apparatus 104 includes a control unit 11
The still image data derived from the content data 110 is extracted from the image data relay device 103 based on the control signal 114 from the relay control unit 1201 in the communication device 03 to generate compressed moving image data, and the terminal 106 is transmitted via the communication path 111 and the network interface 115ab. Send to

In the second embodiment, the operation related to the display of the content data 110 on the terminal 106, that is, the home page described in HTML is performed by the content data 1
10 whole display and enlarged display of an arbitrary part, movement of the content data 110 in the vertical and horizontal directions (scroll display), link information (URL) in the content data 110
The main change is the display change of the content data 110.

Hereinafter, the operation of this embodiment in each operation and the realizing means will be described. Content data 11
Since the entire display of 0 and the enlarged display of an arbitrary part are the same as the method of enlarging and displaying a part of the content data 110 described in the description of the first embodiment, the description is omitted.

The operation of this embodiment and the means for realizing the movement of the content data 110 in the up, down, left and right directions will be described. The relay control unit 1201 stores the operation control information memory 3 in the video encoding device 104 in the internal storage device.
04 relating to the content data 110, that is, the position (xd0,
yd0), the sizes Xd and Yd, and the position (xt0, yt0) and the sizes Xt and Yt of the area 405 to be actually encoded in the still image data 403. Number of pixels representing one unit of movement in the direction, area 405
Of the pointing cursor, and the number of pixels representing one unit of movement of the pointing cursor.

When the direction button 801 is pressed on the terminal 106 whose operation mode is "scroll operation", the moving image reproducing apparatus 107 generates a direction signal indicating the designated direction, and the network interface 115ab
And the service relay device 110 via the communication path 111
1 to the relay control unit 1201. The terminal 106 generates this direction signal each time the direction button 801 is pressed, and sends it to the service relay device 1101.

Upon receiving the direction signal, relay control section 1201 performs the processing shown in FIG. 13, and as a result, moving picture coding apparatus 1103 converts the designated area of content data 110 into compressed moving picture data, and Send to 106.

The operation of relay control section 1201 will be described with reference to FIG. When receiving the direction signal from the terminal 106 (step 1301), the relay control unit 1201 sets the operation mode in the operation control information memory 304 to the normal moving image mode and makes the operation permission flag valid (step 1301).
2) The positions Xt and Yt after moving the area 405 in the designated direction by one movement unit are calculated (step 1303).

If the new position of the region 405 is within the region 403 (step 1304), the position of the region to be actually coded in the still image data is rewritten to the value calculated at step 1303 (step 1305). It waits for reception of a direction signal (step 1301).

If the new position of the area 405 exceeds the area 403 (step 1304), the control signal 1104
Is sent to the reproducing apparatus 1102, the position of the area 403 is moved in the designated direction by one movement unit (step 1306), and the reception of a new direction signal is waited (step 1301).

If the direction signal has not been received (step 1301), the direction signal reception wait time is increased by one unit time (step 1307). If the wait time exceeds a predetermined threshold (step 1308), the operation control is performed. The operation mode in the information memory 304 is set to the stepwise image quality complement mode, the operation permission flag is made valid (step 1309), and the reception of a new direction signal is waited (step 1301). By this processing, the moving picture coding apparatus 1 according to the present embodiment
04 automatically transmits the high-quality content data 110 to the terminal 10 when the reception waiting time of the direction signal exceeds a predetermined time.
6 and the processing is stopped.

The operation and the realizing means of this embodiment in changing the display of the content data 110 by the link information (URL) in the content data 110 will be described. The terminal 106 changes the display of the content data 110 by specifying the link information in the content data 110 using the pointing cursor. The pointing cursor provides a function similar to a mouse cursor in a personal computer, and is displayed on the display device of the terminal 106 at the same time as the content data 110. The display of the pointing cursor on the display device is performed by the terminal 106.
The terminal 106 itself moves the pointing cursor in accordance with the pressing of the direction button 801.

When the direction button 801 is pressed while the pointing cursor is displayed on the terminal 106, the pointing cursor moves by a predetermined moving unit in the designated direction, and the terminal 106 is connected to the network interface 115ab and the communication path 111 via the communication path 111. Then, the moving direction and the moving amount of the pointing cursor are
Tell 1

FIG. 14 shows an operation flow of the relay control unit 1201 regarding the movement of the pointing cursor. Relay control section 1201 receives a signal related to the movement of the pointing cursor from terminal 106 (step 140).
1), the operation mode in the operation control information memory 304 is set to the normal moving image mode, and the operation permission flag is made valid (step 1402), and the information held by itself, that is, the position of the area 403, the position of the area 405, the area From the position of the pointing cursor within 405, the number of pixels representing one unit of movement of the pointing cursor, and information on the movement of the pointing cursor from the terminal 106, that is, the direction and amount of movement of the pointing cursor, the position of the new pointing cursor is determined. Calculate (Step 1
403).

The position of the pointing cursor in the area 401 is the position of the pointing cursor in the area 405 by xd0 + xt0 and yd0 + y in the X-axis direction and the Y-axis direction, respectively.
It can be obtained by adding t0.

If the position of the new pointing cursor is within area 403 (step 1404), relay control section 1201 sends control signal 1104 to reproducing apparatus 1102, and updates the position of the pointing cursor on reproducing apparatus 1102 (step 1404). Step 1405).

If the position of the new pointing cursor is not within area 403 (step 1404), relay controller 1201 moves content data 110 in the direction of the movement of the pointing cursor (step 14).
06), Step 1405 is executed.

If the mobile signal has not been received (step 1401), the mobile signal reception wait time is increased by one unit time (step 1407). If the wait time exceeds a predetermined threshold (step 1408), the operation control is performed. The operation mode in the information memory 304 is set to the stepwise image quality complement mode, the operation permission flag is made valid (step 1409), and the reception of a new movement signal is waited (step 1401). By this processing, the moving picture coding apparatus 1 according to the present embodiment
04 automatically transmits the high-quality content data 110 to the terminal 10 when the reception waiting time of the mobile signal exceeds a predetermined time.
6 and the processing is stopped.

After moving the pointing cursor to the position of the desired link information on the terminal 106, when the start button 803 indicating the determination of the link information selection is pressed,
The terminal 106 confirms the selection via the network interface 115ab and the communication path 111,
Tell 1

FIG. 15 shows an operation flow of the relay control unit 1201 regarding selection of link information. Relay control unit 12
01 receives a signal signifying selection of link information (step 1501), sets the operation mode in the operation control information memory 304 to the ordinary moving image mode, activates the operation permission flag (step 1502), and presses the left mouse button. A signal similar to the pressing of the playback device 1
201 (step 1503). Thus, the new content data 11 associated with the link information
0 is displayed on the image data relay device 103 by the playback device 1201 so that the new content data 1
10 is displayed.

If the confirmation signal is not received (step 1501), the reception waiting time of the confirmation signal is increased by one unit time (step 1504). If the waiting time exceeds a predetermined threshold (step 1505), the operation control is performed. The operation mode in the information memory 304 is set to the stepwise image quality complement mode, the operation permission flag is made valid (step 1506), and the reception of a new confirmation signal is waited (step 1501). By this processing, the moving picture coding apparatus 1 according to the present embodiment
04 automatically transmits the high-quality content data 110 to the terminal 10 when the reception wait time of the confirmation signal exceeds a predetermined time.
6 and the processing is stopped.

Reproducing device 1102 based on control signal 1104
The following describes how to change the display state of, such as moving the display up, down, left, and right, selecting link information and changing the display content using a pointing cursor. In the present embodiment, it is assumed that the playback device 1102 and the control device 1103 operate on the same operating system, and each form an execution unit of a program called a process.

As described above, the relay control unit 1201 in the control device 1103 generates the control signal 1104 for instructing the change of the display state of the reproduction device 1102 based on the button operation on the terminal 106, and Send to 1102. The control signal 1104 at this time has the same content as the event signal generated by pressing the keyboard or operating the mouse in the playback device 1102. The relay control unit 1201 transmits the control signal 1104 to the playback device 1102 using the inter-process communication function provided by the operating system.
Send to

If the content reproduction software used by the reproduction apparatus 1102 is operating independently of the present embodiment, an event signal generated by pressing a keyboard or operating a mouse is a process provided by the operating system. Or sent to the content reproduction software using the communication function.

Also in this embodiment, since the control signal 1104 is sent to the content reproduction software in the reproduction apparatus 1102 by the same inter-process communication function, the reproduction apparatus 110
2 operates by interpreting the control signal 1104 from the relay control unit 1201 as if it were an actual keyboard press or mouse operation. By such means, the control device 1103 can change the display state of the playback device 1102.

Note that the format of the compressed moving image data that can be used in the present embodiment is the same as the format of the compressed moving image data in the first embodiment. Further, in the image data relay apparatus 103 of the present embodiment, similarly to the first embodiment, the image data relay apparatus 103 can be realized using a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to the video memory. It is possible.

As described above, in the present embodiment, the content data 110 in the service providing device 109, that is, the homepage described in HTML is converted into compressed moving image data by the service relay device 1101, so that the data is The content data 110 can be displayed on the terminal 106 having no (HTML) display function. Also, for the displayed content data 110,
Operations such as displaying the entire contents data 110 and enlarging and displaying an arbitrary part thereof, moving the contents data 110 up and down and left and right (scroll display), and changing the display of the contents data 110 based on link information (URL) in the contents data 110 are performed. This can be performed by the terminal 106, and the practical effect is large.

(Embodiment 3) A third embodiment of the present invention will be described with reference to FIGS. FIG. 16 shows a configuration diagram of the third embodiment. In FIG. 16, reference numeral 1601 denotes a service relay device, and 1602 denotes a control device. The other components are the same as the components in the second embodiment. The main difference between the configuration of the third embodiment and the configuration of the second embodiment is the control device 1602 in the service relay device 1601.

First, a mode of use assumed in this embodiment will be described. The terminal 106 is the same as the terminal 106 in the second embodiment, but differs in the content data 110 to be displayed. The content data 110 assumed in the third embodiment is HTML (Hyper Text M).
This is a home page described in arkup language (arkup Language), which is composed of character data, still image data, and animation using computer graphics.

When such content data 110 is displayed on the terminal 106, first, the content data 11 is displayed.
0 is reduced and displayed. As a matter of course, the content data 110 displayed on the terminal 106 has low detail visibility. Therefore, an arbitrary part of the content data 110 is enlarged to an arbitrary magnification by the operation button 117 provided on the terminal 106, and the details of the content data 110 are displayed. In addition, you can move the displayed home page up, down, left and right, or use the URL (Universal Resource Loca
Select and display another home page associated with tor).

Next, the operation of this embodiment and the means for realizing it will be described. The control device 1602 will be described.
FIG. 17 shows the configuration of the control device 1602. 17, 1701 is a relay control unit, 1702 is a communication path with the network interface 116b, and 1703 is a communication path with the network interface 115b.
Other components are the control device 11 in the second embodiment.
It is the same as the component of No. 03.

Basic control of communication between the terminal 106 and the service relay device 1601 is performed by the communication control unit 202 as in the second embodiment. The communication control unit 202 controls the terminal 1
After communication with the service relay device 1601 has been established, the main function of the control device 1602 is the relay control unit 1701
Provided by As in the case of the second embodiment,
The relay control unit 1701 selects data to be sent to the terminal 106, that is, character data or compressed moving image data.

[0117] The process of converting the content data 110 into compressed moving image data and sending it to the terminal 106 will be described. Here, only the parts different from the second embodiment will be described. The difference between this embodiment and the second embodiment is that
The point is that the content data 110 of the present embodiment is accompanied by movement. That is, the content data 110 in the second embodiment is an HT composed of character data and still image data.
Although the homepage is described in ML, the content data 110 in this embodiment is a homepage described in HTML, but includes animation using computer graphics and the like.

The same operation as in the second embodiment is performed on such content data 110, that is, the entire display of the content data 110 and the enlarged display of an arbitrary part, and the movement of the content data 110 in the vertical and horizontal directions (scroll display) The operation of each component and a means for realizing the display change of the content data 110 by the link information (URL) in the content data 110 will be described.

The entire display of the content data 110 and the enlarged display of an arbitrary part are similar to the method of enlarging and displaying a part of the content data 110 described in the description of the first embodiment. The difference is that the moving picture coding apparatus 104 always operates in the normal moving picture mode. As described above, since the content data 110 assumed in the present embodiment involves movement, it must always be in the normal moving image mode. Therefore, the relay control unit 1701 always sets the operation mode in the operation control information memory 304 to the normal moving image mode to operate the moving image encoding device 104. By doing so, the content data 11 with motion
The entire display of 0 and the enlarged display of an arbitrary part can be realized.

Next, the operation and the realizing means of this embodiment in moving the content data 110 in the up / down / left / right directions and changing the display of the content data 110 by the link information (URL) in the content data 110 will be described.

As in the second embodiment, the relay control unit 1701 stores the operation control information memory 3 in its internal storage device.
04, that is, the position (xd0, yd0) and the size Xd, Yd, and bitmap data of the coded bitmap data 403 stored in the image data relay device 103. (Xt0, yt0) of area 405 to be actually encoded within
And the size Xt, Yt, and in addition, the playback device 1102
Number of pixels representing one unit of movement in the up, down, left, and right directions, area 4
The position of the pointing cursor in 05 and the number of pixels representing one movement unit of the pointing cursor are held. Since most operations of the terminal 106 and the relay control unit 1701 are the same as those of the second embodiment, only the differences will be described here.

The moving picture coding apparatus 10 according to the present embodiment
The operation mode of No. 4 is always a normal moving image mode. Therefore, the operation flow of the relay control unit 1701 in moving the content data 110 in the up, down, left, and right directions is as shown in FIG. The operation flow of the relay control unit 1701 in moving the pointing cursor and confirming the selection of the link information is as shown in FIGS. 19 and 20, respectively.

Each step in FIG. 18 corresponds to each step in FIG. 13 as follows. Step 18
01 is the same process as step 1301, step 1802
The same processing as step 1303. step 1803 the same processing as step 1304. step 1804 the same processing as step 1305. step 1805 the same processing as step 1306.

Each step in FIG. 19 corresponds to each step in FIG. 14 as follows. Step 19
01 is the same processing as step 1401, step 1902
Step 1903 Same processing as Step 1903 Step 1404 same processing Step 1904 Step 1405 same processing Step 1905 Step 1406 same processing.

Each step in FIG. 20 corresponds to each step in FIG. 15 as follows. Step 20
01 is the same processing as step 1501, step 2002
This is the same processing as step 1503. That is, the relay control unit 1701 in the present embodiment
04 always works in normal video mode.

Note that the format of the compressed moving image data that can be used in the present embodiment is the same as the format of the compressed moving image data in the first embodiment. Further, in the image data relay apparatus 103 of the present embodiment, similarly to the first embodiment, the image data relay apparatus 103 can be realized using a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to the video memory. It is possible.

As described above, in the present embodiment, the content data 110 in the service providing apparatus 109, that is, a home page having data described in HTML and accompanied by movement (such as animation by CG) is transmitted to the service relay apparatus 1601. The content data 110 can be displayed on the terminal 106 having no display function of the data by converting the data into compressed moving image data. Also,
For the displayed content data 110, the entire display of the content data 110 and an enlarged display of an arbitrary part thereof, movement of the content data 110 in the vertical and horizontal directions (scroll display), link information (URL) in the content data 110 , The operation of changing the display of the content data 110 can be performed on the terminal 106, and the practical effect is great.

(Embodiment 4) A fourth embodiment of the present invention will be described with reference to FIGS. FIG. 21 shows a configuration diagram of the fourth embodiment. In FIG. 21, reference numeral 2101 denotes a service relay device; 2102, a video encoding device;
3 is a control device, 2104 is audio data included in the content data 110, 2105 is a moving image encoding device 210
2, a control signal 2106 is a multimedia communication terminal,
Reference numeral 2107 denotes a moving image reproducing device. The other components are the same as the components in the first embodiment. The main difference between the configuration of the fourth embodiment and the configuration of the first embodiment is that the moving picture coding apparatus 2102 and the control apparatus 2103 in the service relay apparatus 2101 and the multimedia communication terminal 2106 (hereinafter referred to as the terminal ) Is a moving image reproducing apparatus 2107.

First, a mode of use assumed in the present embodiment will be described. As in the first embodiment, the content data 110 and the digital still image stored in the service providing apparatus 109 are displayed on the terminal 2106. At this time, the content data 110 has audio data in addition to the still image data. Therefore terminal 210
In step 6, the audio data is reproduced simultaneously with the display of the still image.

Next, the operation and the realizing means of this embodiment will be described. Video playback device 210 in terminal 2106
Reference numeral 7 has a function of decoding and displaying compressed moving image data, which is the same as that of the moving image reproduction device 107 in the first embodiment, and a function of decoding and reproducing simultaneously transmitted audio data.

FIG. 2 is a block diagram of the moving picture coding apparatus 2102.
It is shown in Fig.2. In FIG. 22, 2201 is an audio codec unit, 2202 is a multiplexing unit, 2203 is an operation control information memory, 2204 is multiplexed AV data, 2205 is a control signal,
It is. Other components are the same as the components of the video encoding device 104 in the first embodiment.

The operation control information memory 2204 stores information specifying the operation of the moving picture coding apparatus 2102 via the control signal 2105. In the present embodiment, the information stored in the operation control information memory 2204 is the operation control information memory 304 in the first embodiment.
, The storage location and data amount of the audio data 2104, the encoding format and bit rate of the audio data 2104, the encoding format and bit rate of the audio data output by the audio codec unit 2201, and the multiplexing. Part 22
02 is the bit rate of the multiplexed AV data 2204 to be output.

The operation of the moving picture coding apparatus 2102 will be described. Since the generation of the compressed moving image data in the moving picture coding apparatus 2102 is the same as that in the first embodiment, only the part different from the operation of the moving picture coding apparatus 104 in the first embodiment will be described here.

Upon receiving a control signal 2205 indicating the start of operation from the data acquisition unit 301, the audio codec unit 2201 extracts information necessary for generation of audio data from the operation control information memory 2204, and converts the specified audio data. The data is taken out from the service providing device 109 via the communication path 112 and the network interface 116ab, converted into designated voice data, and output.

The multiplexing unit 2202 includes the data acquisition unit 301
Upon receiving a control signal 2205 indicating the start of an operation from the CPU, information necessary for the multiplexing process is extracted from the operation control information memory 2204, and the compressed moving image data from the moving image encoding unit 303 and the audio data from the audio codec unit 2201 are extracted. To generate multiplexed AV data 2204,
Send to 06.

The control device 2103 receives the control signal 2105 from the video encoding device 2102 as described above.
The control process is performed to operate the. The control device 2103
Is similar to the control device 105 in the first embodiment,
Basic control of communication between the service relay apparatus 2101 and the terminal 2106 is performed.

The format of the compressed moving image data that can be used in this embodiment is the same as the format of the compressed moving image data in the first embodiment. Further, in the image data relay apparatus 103 of the present embodiment, similarly to the first embodiment, the image data relay apparatus 103 can be realized using a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to the video memory. It is possible.

As described above, in the present embodiment, the service relay device 2101 converts the content data 110 in the service providing device 109, that is, still image data with audio data, into multiplexed AV data. The content data 110 can be displayed on the terminal 2106 that does not have the data display function.
Its practical effect is great.

(Embodiment 5) A fifth embodiment of the present invention will be described with reference to FIG. FIG. 23 shows a configuration diagram of the fifth embodiment. In FIG. 23, reference numeral 2301 denotes a service providing device. The other components are the same as the components in the first embodiment. The main difference between the configuration of the fifth embodiment and the configuration of the first embodiment is that the configuration does not require the service relay apparatus 101.
First, the mode of use assumed in this embodiment is the first
This is the same as the form of use assumed in the embodiment.

Next, the operation of this embodiment will be described. The playback device 102 and the control device 105 directly transmit the content data 1 without using the network interface 116ab and the communication path 112 as in the first embodiment.
10 is taken out and the subsequent processing is performed.

The format of the compressed moving image data usable in this embodiment is the same as the format of the compressed moving image data in the first embodiment. Also in the present embodiment, as in the first embodiment, the image data relay device 103 can use a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to a video memory. It is.

As described above, in the present embodiment, the content data 110 in the service providing apparatus 109, that is, still image data in JPEG format or the like is converted into compressed moving image data, so that a still image display function is not provided. Terminal 1
At 06, a still image can be displayed. Also, by generating compressed video data by stepwise image quality complementation,
Still images can be displayed with high image quality comparable to the original image quality. Further, by using a video memory (V-RAM) for the image data relay device 103, easily available still image display software such as a commercial product can be used, and the content data 110 of various data formats can be used. The service relay apparatus 101 can be easily realized, and its practical effect is great.

(Embodiment 6) A sixth embodiment of the present invention will be described with reference to FIG. FIG. 24 shows a configuration diagram of the sixth embodiment. In FIG. 24, 2401 is a service providing device. The other components are the same as the components in the second embodiment. The main difference between the configuration of the sixth embodiment and the configuration of the second embodiment is that the configuration does not require the service relay device 1101. First, the form of use assumed by this embodiment is:
This is the same as the form of use assumed in the second embodiment.

Next, the operation of this embodiment will be described. The playback device 1102 and the control device 1103 are connected to the network interface 116ab as in the second embodiment.
Then, the content data 110 is directly extracted without using the communication path 112 and the subsequent processing is performed.

It is obvious that the present embodiment can be easily applied to the use form assumed in the third embodiment. Further, the format of the compressed moving image data that can be used in the present embodiment is the same as the format of the compressed moving image data in the first embodiment. Also in the present embodiment, as in the first embodiment, the image data relay device 103 can use a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to a video memory. It is.

As described above, in the present embodiment, by converting the content data 110 in the service providing apparatus 109, that is, the homepage described in HTML, into compressed moving image data, it has no display function of the data. The terminal 106 can display the content data 110. Also, the displayed content data 110
In response to this, the entire display of the content data 110 and an enlarged display of an arbitrary part, the movement of the content data 110 in the vertical and horizontal directions (scroll display), the content data 1
Content data 1 by link information (URL) in 10
The operation of changing the display of 10 can be performed on the terminal 106.

(Embodiment 7) A seventh embodiment of the present invention will be described with reference to FIG. FIG. 25 shows a configuration diagram of the seventh embodiment. In FIG. 25, reference numeral 2501 denotes a service providing device. The other components are the same as the components in the fourth embodiment. The main difference between the configuration of the seventh embodiment and the configuration of the fourth embodiment is that the configuration does not require the service relay apparatus 2101.
First, the mode of use assumed in this embodiment is the fourth mode.
This is the same as the form of use assumed in the embodiment.

Next, the operation of the present embodiment will be described. The playback device 102 and the control device 2103 directly extract the content data 110 without using the network interface 116ab and the communication path 112 as in the fourth embodiment, and perform the subsequent processing.

The format of the compressed moving image data that can be used in the present embodiment is the same as the format of the compressed moving image data in the first embodiment. Also in the present embodiment, similarly to the first embodiment, the image data relay device 103 can use a semiconductor memory such as a DRAM or a storage device such as a magnetic disk device as an alternative to the video memory. is there.

As described above, in the present embodiment, the display function of the data is converted by converting the content data 110 in the service providing device 109, that is, the still image data with audio data, into multiplexed AV data. The content data 110 can be displayed on the terminal 2106 that does not have it, and its practical effect is great.

In the present invention, Embodiments 1 to 4
In the first embodiment, the service providing apparatus 109 (server S) and the service relay apparatus 101 (gateway GW) are separated from each other. In the fifth to seventh embodiments, they are integrated. Such a case is assumed. In other words, as a form, there are a case where it is considered that it is inefficient to provide a GW for each of a number of servers and each has a cost, or a case where a network infrastructure provider installs a GW by itself (FIG. 26). Examples of the form include a case where each server has a GW, and a case where a provider of network infrastructure does not have a GW by itself (FIG. 27). It is possible to provide an optimal system form according to each of these situations.

[0152]

As described above, according to the present invention, first,
(A plurality of content data are stored, and upon request, the content data is
A service providing device capable of transmitting, and) a moving image capable of generating high-quality compressed moving image data from still image data by stepwise image quality complementary coding and transmitting the generated moving image data to a playback terminal via a second network interface An encoding device and content data playback software that runs on a general personal computer can be used as a playback device, and the content data is obtained by the device playing content data obtained from a service providing device via a network interface. An image data relay device that relays still image data to a moving image encoding device, and receives a button operation on a playback terminal, determines a region of still image data to be encoded into compressed moving image data, and transmits the region to the moving image encoding device. Equipped with a service relay device comprising a control device capable of Thereby, the desired content data can be displayed on a playback terminal that does not have the ability to display the desired content data, and in addition, any part of the content data displayed on the playback terminal can be enlarged and displayed. Becomes possible.

Second, (a service providing apparatus capable of storing a plurality of pieces of content data and transmitting the content data via a network interface in response to a request) High-quality compressed video data can be generated from the data, and by changing the setting of the encoding process as needed, the compressed video data is generated from the still image data whose contents change sequentially, and the second network A moving picture encoding device that can be transmitted to a playback terminal via an interface and content data playback software running on a general personal computer can be used as a playback device, and the device can be obtained from a service providing device via a network interface. Content obtained by playing back content data An image data relay device for relaying still image data derived from content data to a moving image encoding device, and a playback device that receives a button operation on a playback terminal and generates a control signal for changing a display state of the playback device. And a service relay device including a control device capable of determining an area of the still image data to be encoded into the compressed moving image data and transmitting the determined region to the moving image encoding device. Desired content data can be displayed on a playback terminal that does not have the ability to display. In addition, any part of the content data displayed on the playback terminal can be enlarged and displayed, and the displayed content can be moved up, down, left, or right. Then, by arbitrarily selecting the displayed link information, it becomes possible to display related content data.

Third, (a service providing device capable of storing a plurality of content data and transmitting the content data via a network interface in response to a request) and still image data by gradual image quality complementary coding Can generate high-quality compressed video data, and can generate audio data of a specified data format from the audio data included in the content data. A video encoding device that can generate multiplexed AV data and send it to a playback terminal via a second network interface, and content data playback software running on a general personal computer can be used as the playback device. , From the service providing device via the network interface Image data relay device that relays still image data derived from content data obtained by playing back content data to a moving image encoding device, and a still image that receives a button operation on a playback terminal and encodes the compressed moving image data By providing a service relay device including a control device capable of determining a data area and transmitting the data to the video encoding device, a playback terminal having no ability to display desired content data can transmit desired content data. Can be displayed with the reproduction of audio, and in addition, any part of the content data displayed on the reproduction terminal can be enlarged and displayed.

Fourth, a moving picture coding apparatus which can generate high-quality compressed moving picture data from still picture data by stepwise picture quality complementary coding and send it to a playback terminal via a network interface, An image data relay device that can use content data reproduction software running on a personal computer as a reproduction device, and relays still image data derived from content data obtained by the device reproducing the content data to a moving image encoding device; A control device that receives a button operation on the playback terminal, determines a region of still image data to be encoded into compressed moving image data, and transmits the region to the moving image encoding device,
A service providing device capable of storing a plurality of content data and transmitting the content data via a network interface in response to a request, the playback terminal having no ability to display desired content data Thus, desired content data can be displayed, and in addition, any part of the content data displayed on the playback terminal can be enlarged and displayed.

Fifth, high-quality compressed moving image data can be generated from still image data by stepwise image quality complementary coding, and the contents can be sequentially changed by changing the setting of the encoding process whenever necessary. Uses a moving image encoding device that can generate compressed moving image data from changing still image data and sends it out to a playback terminal via a network interface, and content data playback software running on a general personal computer as the playback device A video data relay device that relays still image data derived from content data obtained by reproducing the content data to the moving image encoding device, receives a button operation on the reproduction terminal, and displays the display of the reproduction device. A control signal for changing the state can be generated and sent to a playback device, and a compressed moving image can be generated. A control device that determines the area of the still image data to be encoded into the data and transmits the data to the moving image encoding device, and stores a plurality of content data, and sends out the content data as required via the network interface. And a service providing device capable of displaying the desired content data on a playback terminal that does not have the ability to display the desired content data. It is possible to display related content data by enlarging and displaying an arbitrary part of the displayed content data, moving the displayed content up, down, left and right, and arbitrarily selecting the displayed link information.

Sixth, high-quality compressed moving image data can be generated from still image data by the stepwise image quality complementary coding, and the specified data format of the audio data included in the content data can be obtained. A moving picture encoding device capable of generating audio data, multiplexing compressed moving image data and audio data to generate multiplexed AV data, and transmitting the multiplexed AV data to a playback terminal via a network interface; An image data relay device that can use content data reproduction software running on a computer as a reproduction device, and relays still image data derived from content data obtained by reproducing the content data to the moving image encoding device, Still image data to be encoded into compressed video data in response to button operations on the playback terminal A control device capable of determining an area of the content and transmitting the content to the video encoding device; a service providing device capable of storing a plurality of content data and transmitting the content data according to a request via a network interface; Is provided, the desired content data can be displayed along with the reproduction of the sound on the playback terminal that does not have the ability to display the desired content data. An arbitrary part of the content data can be enlarged and displayed.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a first embodiment of the present invention.

FIG. 2 is a configuration diagram of a control device according to the first embodiment of the present invention.

FIG. 3 is a configuration diagram of a moving image encoding device according to the first embodiment of the present invention.

FIG. 4 is a schematic diagram showing a relationship between content display areas according to the first embodiment of the present invention.

FIG. 5 is a flowchart illustrating an operation of a data acquisition unit according to the first embodiment of the present invention.

FIG. 6 is a conceptual diagram of an MPEG format video encoding process according to the first embodiment of the present invention.

FIG. 7 is an example diagram of a mathematical expression representing a change in reproduction image quality by the stepwise image quality complementation type encoding process according to the first embodiment of the present invention.

FIG. 8 shows an operation button 1 according to the first embodiment of the present invention.
Configuration diagram of 17

FIG. 9 is a diagram illustrating an example of display of a still image on a terminal according to the first embodiment of the present invention.

FIG. 10 is a configuration diagram of an image data relay device using a semiconductor memory such as a DRAM according to the first embodiment of the present invention.

FIG. 11 is a configuration diagram of a second embodiment of the present invention.

FIG. 12 is a configuration diagram of a control device according to a second embodiment of the present invention.

FIG. 13 is a flowchart illustrating an operation when a relay control unit receives a direction signal according to the second embodiment of the present invention.

FIG. 14 is a flowchart illustrating an operation related to movement of a pointing cursor performed by a relay control unit according to the second embodiment of the present invention.

FIG. 15 is a flowchart showing an operation related to selection of link information performed by the relay control unit according to the second embodiment of the present invention;

FIG. 16 is a configuration diagram of a third embodiment of the present invention.

FIG. 17 is a configuration diagram of a control device according to a third embodiment of the present invention.

FIG. 18 is a flowchart illustrating an operation related to movement of content data in up, down, left, and right directions performed by a relay control unit according to the third embodiment of the present invention.

FIG. 19 is a flowchart illustrating an operation related to movement of a pointing cursor performed by a relay control unit according to the third embodiment of the present invention.

FIG. 20 is a flowchart illustrating an operation related to selection of link information performed by the relay control unit according to the third embodiment of the present invention.

FIG. 21 is a configuration diagram of a fourth embodiment of the present invention.

FIG. 22 is a configuration diagram of a video encoding device according to a fourth embodiment of the present invention.

FIG. 23 is a configuration diagram of a fifth embodiment of the present invention.

FIG. 24 is a configuration diagram of a sixth embodiment of the present invention.

FIG. 25 is a configuration diagram of a seventh embodiment of the present invention.

FIG. 26 is a diagram showing an example of a network infrastructure according to the first to fourth embodiments of the present invention;

FIG. 27 is a diagram illustrating an example of a network infrastructure according to the fifth to seventh embodiments of the present invention;

FIG. 28 is a configuration diagram of a general information providing apparatus.

[Explanation of symbols]

101 Service Relay Device 102 Playback Device 103 Image Data Relay Device 104 Video Encoding Device 105 Control Device 106 Multimedia Communication Terminal as Reproduction Terminal 107 Video Reproduction Device 108 Content Browsing Device 109 Service Providing Device 110 Content Data 111 Terminal 106 Communication path between the service relay apparatus 101 and the service providing apparatus 109 and the service relay apparatus 1
01 113 Control signal of playback device 102 114 Control signal of video encoding device 117 Operation button 201 Relay control unit 202 Communication control unit 203 Communication channel with network interface 116b 204 Communication with network interface 115a Path 205 communication path with network interface 115a 301 data acquisition unit 302 shared memory 303 moving image encoding unit 304 operation control information memory 305 still image data from image data relay device 103 306 compressed moving image data 307 control signal 401 image data relay device The display memory area 402 of the memory area 103 The origin of the memory area 401 403 Still image data 404 derived from the coded content data 110 displayed on the image data relay apparatus 103 404 A representative point 405 for specifying the position of the still image data 403 derived from the source data 110 A region 406 to be actually encoded in the still image data 403 derived from the content data 110 406 A representative point 601 for specifying the position of the region 405 to be actually encoded Compressed image frame 602 Uncompressed image frame 603 Uncompressed image frame 604 Compressed image frame 605 Compressed image frame 606 Compressed image frame 607 Reference image frame 608 Reference image frame 609 Reference image frame 610 Uncompressed image frame 611 Uncompressed image frame 612 Uncompressed Image frame 613 Compressed image frame 614 Compressed image frame 615 Compressed image frame 616 Reference image frame 617 Reference image frame 618 Reference image frame 1001 Write control unit 1002 Read Content data 11 from the control unit 1003 Memory 1004 Memory 1005 shared memory 1006 reproduction apparatus 102
0-based still image data signal line 1007 Still image data-based signal line derived from content data 110 to the moving image encoding device 104

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) H04N 7/173 610 H04L 11/20 101Z (72) Inventor Toshio Oka 1006 Odakadoma, Kadoma, Osaka Matsushita Electric Within Sangyo Co., Ltd. (72) Inventor Akino Inoue 1006 Kazuma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (72) Inventor Tsutomu Uenoyama 1006 Oji Kadoma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. Person Daisaku Komiya 1006 Kazuma Kadoma, Kadoma-shi, Osaka Matsushita Electric Industrial Co., Ltd. (72) Inventor Kazunori Yamada 1006 Kadoma Kadoma, Kadoma-shi, Osaka Pref. GA11 GA25 GA31 GB04 HA01 HA06 JB04 JB06 JB21 KA09 KA11 KH11 KH28 LB04 LB14 LB17 LB21 5C059 KK34 KK38 KK 39 MA00 MA23 MC15 PP01 PP04 PP19 RB14 RC32 RC33 RE09 RE16 RE20 SS08 SS10 TA46 TC37 TC45 5C064 BA01 BB03 BC06 BC16 BD02 BD08 5K030 GA10 GA18 HA06 HB01 HB02 HC01 HC09 HC14 JA01 JT09 KA19 LA07

Claims (6)

[Claims] An apparatus for accumulating content data, acquiring data from a service providing apparatus capable of transmitting content data in response to a request, and providing the data to a user terminal in its original form or in a converted form. Yes,
Reproducing means for reproducing the content data; image data relay means for relaying still image data derived from the content data reproduced by the reproducing means to the moving image encoding means;
A moving image encoding unit that generates high-quality compressed moving image data from still image data by step-wise image quality complementing encoding, which is an encoding method that complements image quality in stages, and sends it to the user terminal; A control unit that receives an operation, determines a region of the still image data to be encoded into the compressed moving image data, and transmits the region to the moving image encoding unit. 2. The control means receives an operation of a user terminal, generates a control signal for changing a reproduction state of the reproduction means, and sends the control signal to the reproduction means. 2. The information providing apparatus according to claim 1, wherein the setting of the processing is changed as needed to generate the compressed moving image data from the still image data whose contents change sequentially. 3. A moving image encoding means generates audio data in a specified data format from audio data included in content data, and multiplexes compressed moving image data and audio data to generate multiplexed AV data. 2. The information providing apparatus according to claim 1, wherein the information is transmitted to a user terminal. 4. An apparatus for providing data to a user terminal in response to a request, a storage unit for storing content data, a reproducing unit for reproducing the content data, and a still image derived from the content data reproduced by the reproducing unit. Image data relaying means for relaying data to moving image encoding means, and high-quality compressed moving image data from still image data by stepwise image quality complementing coding, which is an encoding method for complementing image quality stepwise, Moving image encoding means for sending to the user terminal, control means for receiving an operation from the user terminal, determining an area of still image data to be encoded into compressed moving image data, and transmitting the determined area to the moving image encoding means; Information providing device provided with. 5. The control means receives an operation of the user terminal, generates a control signal for changing a reproduction state of the reproduction means, and sends the control signal to the reproduction means. 5. The information providing apparatus according to claim 4, wherein the setting of the processing is changed as needed, thereby generating the compressed moving image data from the still image data whose contents change sequentially. 6. A moving image encoding means generates audio data in a specified data format from audio data included in content data, and multiplexes compressed moving image data and audio data to generate multiplexed AV data. The information providing apparatus according to claim 4, wherein the information is transmitted to a user terminal.