JPH11205786A - Animation data housing form, animation data transferring system, animation displaying method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a data transferring quantity and to properly select resolution or a frame rate according to a device environment of a client by efficiently change zooming, pan (changing a display region) or the change of the frame rate at the time of reproducing animation. SOLUTION: Animation data consists of plural images of stepwise different resolution corresponding to one frame of animation and is constituted of pixel group element pieces constituted in common. This animation data transferring system consists of a server 100 housing this image, a client 200 for downloading an image corresponding to one frame of animation by the unit of the pixel group element piece from the storage device of the server by designating at least one of resolution, a displaying range and a frame rate, and a communication line 300 the server 100 and the client 200 are connected with.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is suitable for application on a network such as the Internet, in which the amount of data transfer can be reduced and zooming, panning, or changing the frame rate can be efficiently performed at the time of reproducing a moving image. , A moving image data storage system, a moving image data transfer system, a moving image display method, and a storage medium.

[0002]

2. Description of the Related Art Hitherto, various methods have been known in which a receiving apparatus can receive a high-resolution video signal or a low-resolution video signal. For example, when the video signal is a high-definition television signal (HDTV), the transmitting apparatus samples the video signal at an appropriate rate in the horizontal and vertical directions and transmits the signal.
It is known that a receiving-side apparatus obtains a video signal having a resolution corresponding to the sampling rate. For example, Qua
drawing Mirror Filter (QM
In the process according to F), the transmitting device can generate a multi-resolution video signal and change the transmission bit rate to transfer the video signal having a resolution corresponding to the bit rate to the receiving device. However, since this method is a so-called broadcast type, the resolution of an image related to a video signal received by the receiving apparatus is uniformly determined. Therefore, the user of the receiving apparatus cannot freely zoom in or zoom out as desired. of course,
The user cannot freely change (pan) the display area of the image or change the frame rate as desired.

There is also known a method in which a plurality of cameras at different positions with respect to the subject simultaneously photograph the subject, and moving picture data composed of a plurality of bitmaps for each frame is prepared in a transmitting apparatus. In this method, the receiving apparatus generates an image corresponding to the user's viewpoint in response to the user's request, so that the user can obtain an image corresponding to zooming in, zooming out, or panning. However, in this method, the receiving apparatus must generate a video signal according to the user's viewpoint position (that is, it is necessary to extract video information in a three-dimensional space), so that the amount of calculation becomes enormous. It is difficult to play a moving image at high speed.

On the other hand, in the field of networks such as the Internet, for example, a receiving device (terminal computer) reads image data provided from a site using a browser such as a WWW (World Wide Web) browser. A method of downloading and displaying this is known.

[0005] Some browsers can display moving images in a predetermined display area of a display. The moving image data is stored in a storage device of a transmission device (hereinafter, referred to as a “server”) such as a site, and a terminal computer (hereinafter, referred to as a “client”) downloads the moving image data for each frame, Play the video using a browser. By applying such a moving image reproduction method, moving images having different resolutions can be displayed on the terminal computer. In this method, on the server side,
A set of low-resolution data and high-resolution data for the same moving image is stored for each frame. The user (client) can download any of the resolution data from the server and display the moving image in a small area of the browser using low-resolution data or in a large area using high-resolution data. . In addition, the user can zoom in on a part of the moving image displayed on the browser or zoom out on the moving image displayed on the browser by using the moving image data of different resolutions.

For example, as shown in FIG. 11, a client 52 sends a low-resolution moving image data M
D _LR is downloaded via the communication line 50, and the moving image data MD _LR is used to display the display area 521 of the browser.
The video is displayed in advance. When the user instructs to zoom in, the client 52 downloads the high-resolution moving image data MD _HR from the server 51 and displays the zoomed-in moving image in the display area 521 using the moving image data MD _HR . When the user gives an instruction to zoom out of the zoomed-in moving image, the client 52 downloads the low-resolution moving image data MD _LR from the server 51 and uses the moving image data MD _LR to zoom out to the display area 521. Is displayed.

However, in the method shown in FIG. 11, when the user zooms in on a moving image, the client 52
Must download the entirety of each frame constituting the moving image to be downloaded, and perform a clipping process for matching this frame to the size of the display area 521 of the browser. That is, the client 52 also downloads the moving image data of a portion of each frame that is not displayed in the display area 521. For this reason, there is a problem in that extra time is required for downloading, and the capacity of a buffer for temporarily storing downloaded moving image data is increased. In addition, the load on the image processor increases due to the clipping process, and it becomes impossible to smoothly rewrite the frame buffer.

[0008]

It is an object of the present invention to (1) reduce the amount of data transfer and efficiently perform zooming, panning (change of display area) or change of frame rate during reproduction of a moving image, or (2) a moving image data storage format, a moving image data transfer system, which can appropriately select a resolution and a frame rate according to a device environment on a client side;
A moving image display method and a storage medium are provided.

[0009]

SUMMARY OF THE INVENTION In the moving image data storage format of the present invention, a plurality of images having different resolutions are prepared for one frame of a moving image. The plurality of images having different resolutions are stored in the moving image data storage device. When this storage format is applied to a network environment such as the Internet, the moving image data storage device is a large-capacity storage device such as a hard disk provided in a server. When this storage format is applied in an environment not connected to a network, the moving image data storage device is a large-capacity storage device such as a hard disk drive or a CDROM. In these moving image data storage formats, each image corresponding to each frame is composed of a pixel group element having a common configuration.
For example, such a pixel group element can be a square composed of pixels having the same number of vertical and horizontal pixels (for example, 64 × 64).

[0010] the above-mentioned resolution is different images, the resolution is the highest original image, resolution (original resolution of the image) / 2 of the image, the resolution (the original image resolution) / 2 ^second image, ... An image whose resolution is (original image resolution) / 2 ^K−1 (where K is a positive integer). The above original image having the highest resolution can be generally an integer multiple of the tile size (for example, 64 × 64).

Further, the moving image data transfer system of the present invention provides a server having a storage device in which moving image data is stored in the above storage format, a transmitting device for transmitting the moving image data, a resolution and a display range. A client having a receiving device capable of downloading at least one frame rate of an image corresponding to one frame of a moving image from a storage device of the host computer in pixel group unit units, the server and the client And a communication line to which is connected.

Further, the moving picture display method of the present invention is used in the above moving picture data transfer system, wherein the client executes a "download step" and a "copy step" according to the following initial setting conditions: A "request detection step" and, when a user request is made, a "download step" and a "copy step" according to the request are executed.

When the system starts up, the client executes a "download step" according to the initial setting conditions, and stores only an image portion of the moving image data necessary for displaying the moving image in the pixel group unit in the server. Download from the device to a predetermined buffer. Also,
In the “copy step”, the downloaded image portion of the pixel group unit is copied to the frame buffer. By repeatedly executing these “download step” and “copy step”, a moving image is displayed in the display area of the display. Thereafter, the client enters a “waiting for request” state, and the client displays a moving image while
Execute the "request detection step". That is, in this “request detection step”, the user requests a zoom-in or zoom-out (hereinafter, collectively referred to as “zooming”), a pan request, or a frame rate change of the moving image displayed in the display area of the display. Perform request detection.

[0014] The zoom-in request may be accompanied by designation of a zoom-in area in the display area or zoom-in center. Further, the zoom-out request can be accompanied by designation of the zoom-out center. When the zooming request is accompanied by designation of the zoom-in center or the zoom-out center, the resolution can be changed by one step by one zooming operation by the user, or the resolution can be changed once for a plurality of steps. It can be changed to When the resolution is changed at once for a plurality of steps, the user may be able to specify the zooming magnification. The pan request involves specification of the moving direction (vertical direction, horizontal direction, or oblique direction) of the display portion. The request for changing the frame rate can be made, for example, by the user specifying a number of speeds (for example, specifying whether to display at a normal rate, a high rate, or a slow rate).

The client executes a "download step" when a zooming request, a panning request, or a frame rate change request is received from the user. For example, when a zooming request is made, an image portion corresponding to the display area on the browser, out of the images of the predetermined resolution, is downloaded from the storage device of the server to the buffer for temporary storage in the unit of the pixel group unit. At the time of downloading, the pixel group fragments that are not used for image display are not downloaded, so no extra time is required for downloading, and a wasteful use of the buffer capacity for temporarily storing downloaded video data occurs. Absent. In addition, when displaying a moving image as a set of pixel group pieces in the display area, clipping processing is not required when copying a frame from the temporary storage buffer to the frame buffer, so that the load on the image processing processor is heavy. Instead, the frame buffer is smoothly rewritten.

[0016] Thereafter, the client executes a "copy step", copies the image portion of the downloaded pixel group unit into a frame buffer, and displays a moving image with a specified resolution in a predetermined area of the display. I do.

The program for executing each of the above steps can be stored in a computer-readable storage medium such as a floppy disk, CDROM, or hard disk.

According to the present invention, since zooming, panning, or changing the frame rate can be performed by a user operation, an interactive moving image display environment is provided.
For example, the user can select an appropriate resolution to display a moving image in accordance with a resolution constraint caused by a client-side device environment (for example, the performance of a video circuit). In addition, by saving information (URL, zooming magnification, etc.) necessary for video playback as a script,
It is also possible to automatically reproduce a moving image accompanied by zooming processing and panning processing. Further, the display area can be changed according to the device environment on the client side. For example, when the image processing device on the client side has high performance (for example, when the capacity of the buffer memory is large and the performance of the processor for image processing is high), the display area is enlarged, and conversely, the image processing device on the client side has high performance. When the performance is not good, the display area can be reduced.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing an entire moving image data transfer system of the present invention. A server 100 is a client (only one is shown in FIG. 1) 200 via a network 300.
It is connected to the. The server 100 has a storage device 110 and a transmission device 120. In the storage device 110,
A plurality of moving image files (indicated by MF in FIG. 1) are stored according to the storage format of the present invention. The transmitting device 120
The image data of the predetermined moving image file MF can be output to the receiving device 230 of the client 200 via the network 300. This image data is configured in units of pixel group pieces (hereinafter, referred to as “tiles”) constituting each frame. The client 200 has an image processing device 210 and a display 220. The image processing device 210 includes a temporary storage buffer 212 and a frame buffer 214, and the moving image is displayed in a display area 222 of the display 220.

FIG. 2 shows the storage device 110 of the server 100.
FIG. 3 is a diagram showing a storage state of moving image data in FIG. In FIG. 2, an image (hereinafter, referred to as a “frame image”) corresponding to one frame of a moving image has a plurality of different resolutions (here, [4], [3], [2] in descending order of resolution). ,
(For four levels of [1]). In each frame, the frame images for the four stages form a so-called pyramid as a whole (that is, each of the four frame images is a constituent element of the pyramid). Many pyramids corresponding to each frame constitute one moving image file MF. It is needless to say that a plurality of moving image files can be stored in the storage device 110 of the server 100. In this embodiment, the pyramid is represented by P, where n is the frame number.
(N) (n = 0, 1,..., N ₀ −1, n ₀ , n ₀ +
,...), And four frame images constituting each pyramid P (n) are FI ₄ (n), FI ₃ (n),
It is represented by FI ₂ (n) and FI ₁ (n). FIG.
Three pyramids are shown when = n ₀ -1, n ₀ , n ₀ +1.

As shown in FIGS. 3A to 3D, the frame image FI at the resolution level [4] (highest resolution) is used.
₄ (n) is composed of 1024 × 512 (the number of horizontal pixels × the number of vertical pixels) pixels, and the frame image FI ₃ (n) of the resolution level [3] is composed of 512 × 256 pixels. Frame image FI ₂ (n) at resolution level [2]
Is composed of 256 × 128 pixels, and the frame image FI ₁ (n) at the resolution level [1] (lowest resolution) is
It consists of 128 × 64 pixels.

These frame images FI ₄ (n), FI _4
3 (n), FI ₂ (n), and FI ₁ (n) are each constituted by a square tile composed of 64 pixels vertically and 64 pixels horizontally, and FI ₄ (n) is 16 pixels horizontally × vertically. FI ₃ (n) is composed of 8 × 4 tiles, FI ₂ (n) is composed of 4 × 2 tiles, and FI ₁ (n) is composed of 8 tiles. It consists of 2 tiles x 1 vertical tile. A technique for forming a plurality of images having different resolutions into a pyramid configuration is based on FlashPix (Flash Pi).
x). For example, as a protocol for transferring this image data on the Internet, IIP (Internet Imaging Prot)
ocol) is used.

FIG. 4 is a flowchart showing a moving image display method applied to the moving image data transfer system shown in FIG. First, predetermined initial settings are performed on the client 200 side. That is, the user starts the browser and
An address ADD of a moving image data storage destination such as RL, a moving image file name (here, a moving image file MF), a resolution level RL, a display range DR, a frame rate FR, and the like are designated. As a result, the client 200 communicates with the server 10
The storage device 110 accesses the moving image file MF of the storage device 110 (step S1).

The user can individually specify items such as an address ADD, a moving image file MF, a resolution level RL, a display range DR, a frame rate FR, and the like. All of the items can be registered as defaults, and when the user starts the browser, the registered items can be collectively specified according to the defaults. In this embodiment, the address ADD and the moving image file MF are individually specified by the user, but the resolution level RL, the display range DR, and the frame rate FR are assumed to be registered as defaults on the client 200 side.

Here, the default of the resolution level RL is [1]. The display area 222 has a size in which two tiles are arranged horizontally (128 pixels horizontally and 64 pixels vertically). Therefore, the default of the display range DR is the entire range of the image. In the present embodiment, the frame rate FR is “normal rate” (a setting in which frame images with consecutive frame numbers n are sequentially loaded) and “high rate” (frame images with alternate frame numbers n are sequentially taken in). ) Can be selected, and the normal setting is selected by default.
Note that the moving image need not be displayed from the frame image FI ₁ (0) whose frame number n is “0”. For example, the moving image can be displayed from a frame image whose frame number n is an arbitrary value. In the example, the moving image is displayed from the frame image FI ₁ (0) having the frame number 0.

After performing the initial setting in step S1, the client 200 converts the tiles of the resolution level RL and the display range DR specified by default to the frame rate FR (normal rate in this case) specified by default.
To download. That is, the client 200
Downloads all (two) of the tiles constituting FI ₁ (0) from the moving image file MF stored in the storage device 110 of the server 100, and downloads the image processing device 210
Is transferred to the temporary storage buffer 212 in tile units (step S2). The temporary storage buffer 212
Image data of only one frame may be stored, or image data of a plurality of frames may be stored. The client 200 is executing a data transfer monitoring process, and optimizes the download of the moving image file MF on the network 300 (step S3).

Next, the image processing device 210 copies the contents of the image data (FI ₁ (0)) in tile units stored in the temporary storage buffer 212 to the frame buffer 214 in the image processing device 210. The image is reconstructed (step S4). Display area 2 of display 220
22 displays an image of two tiles of FI ₁ (0) according to the default (step S5).

The browser detects whether or not the user has requested the end of the moving image display during the display of the moving image (step S6). When the end of the moving image display is requested, the browser ends the moving image display. Then, if there is no request to end the moving image display (step S7), the process proceeds to step S8. In step 8, it is detected whether or not there is a zooming request or a panning request from the user. When there is no zooming request or pan request from the user, it is detected whether or not there is a frame rate change request (step S9). When a zooming request or a panning request is made from the user, the resolution level RL and the display range D
R is reset (step S10), and thereafter, the detection in step S9 is performed. At this time, since there is no zooming request or pan request, the resolution level RL and the display range DR in step S10 are not reset.

If there is no request for changing the frame rate in step S9, the frame number n is incremented according to the frame rate FR (step S1).
1). Here, since the frame rate FR is “normal”, when there is no frame rate change request,
The frame number n is incremented by one. If there is a frame rate change request in step S9, the frame rate FR is reset (step S1).
2) In response, the frame number n is incremented by an appropriate value (for example, 2) (step S11). In this embodiment, it is assumed that there is no request for changing the frame rate, and thus the frame rate FR is not reset. In step S11, after the frame number n is incremented, the process returns to S2. Less than,
By repeating the above processing, FI ₁ (0), FI _1
1 are sequentially displayed in the display area 222.

After FI ₁ (n ₀ ) is displayed in step S5, the user designates, for example, a certain area or point on the display area 222 so that the resolution is increased by one step (that is, the resolution level is increased). Suppose that a zoom-in request is made. The image processing apparatus 210, the zoom request, upon detecting in step S8, and re-sets the resolution level RL in step S10, _{n 0} the frame number _{n 0} in step S11
Increment to +1. In this case, step S
In 2, the image processing device 210 reads the FI of the pyramid P (n ₀ +1) from the storage device 110 of the server 100.
₂ (n ₀ +1), a predetermined tile is specified and downloaded, and the buffer 212 for temporary storage is specified.
Transfer to Note that a method of specifying a tile in the zooming process will be described later.

Thereafter, in step S4, the tiles of FI ₂ (n ₀ +1) stored in the temporary storage buffer 212 are copied to the frame buffer 214, and in step S5, the images of these tiles are displayed in the display area 222 of the display 220. Is displayed. Hereinafter, in step S11, the frame number n is sequentially incremented by 1 to obtain FI ₂ (n ₀ +2), FI ₂ (n ₀ +3),.
The images of the resolution level [2] corresponding to the display range DR are sequentially displayed in the display area 222.

Here, it is assumed that a user has made a zoom-in request to increase the resolution by two steps (ie, change the resolution level from [2] to [4]). It is assumed that the frame image displayed at the time of the zoom-in request is FI ₂ (n ₁ ). This zoom-in request is detected in step S8. Then, in step S10, the resolution level RL is reset to [4].
Step S11 the frame number _{n 1} in _n 1 +1
To increment. Hereinafter, in the same manner as described above, the image processing device 210
0, a predetermined tile of FI ₄ (n ₁ +1) of the pyramid P (n ₁ +1) is downloaded from the storage device 110 and stored in the temporary storage buffer 212. In step S5, the image corresponding to the display range DR The data is copied to the frame buffer 214, and a moving image of the resolution level [4] is displayed on the display area 222 of the display 220. Hereinafter, in step S11, the frame number n is sequentially
FI ₄ (n ₁ +2), F
The images corresponding to the display range DR of I ₄ (n ₁ +3),... Are sequentially displayed in the display area 222.

As described above, the image processing device 210
However, it is assumed that a pan request is made by the user when a frame image of the resolution level [4] is displayed. The frame image displayed at the time of this pan request is FI ₄ (n
₂ ) Suppose that. The bread request is detected in step S8, and re-sets the display range DR in step S10, increments the frame number _{n 2} to _n 2 +1 in step S11. Hereinafter, as described above, the image processing device 210 stores the pyramid P (n ₂₎ from the storage device 110 of the server 100 in step S2.
+1), a predetermined tile of FI ₄ (n ₂ +1) is downloaded and stored in the temporary storage buffer 212. In step S5, the image data corresponding to the new display range DR is copied to the frame buffer 214 and displayed. The panned image of the resolution level [4] is displayed in the display area 222.

As described above, the image processing device 210
However, when a frame image of resolution level [4] is displayed, a user issues a zoom-out request to lower the resolution by one step (ie, change the resolution level from [4] to [3]). Let's say It is assumed that the frame image displayed at the time of this zoom-out request is FI ₄ (n ₃ ). This zoom-out request is sent in step S8.
Is detected. Then, re-set the resolution level RL in [3] in step S10, increments the frame number _{n 3} to _n 3 +1 in step S11. Hereinafter, as described above, the image processing device 210
Is the storage device 1 of the server 100 in step S2.
10, FI ₃ (n ₃ +) of pyramid P (n ₃ +1)
The predetermined tile of 1) is downloaded and stored in the temporary storage buffer 212, and in step S5, the display range DR
Is copied to the frame buffer 214 and the moving image of the resolution level [3] is displayed on the display area 222 of the display 220.

[0035] Figure 5, how the change in resolution of the above video by zooming and panning, (1) Display of video resolution level [1] (in Fig. 5, indicated by _{f 1),} (2)
By zooming in (FIG. 5, denoted by _{f 2)} video resolution level [2] Display, (3) by zoom, display video resolution levels [4] (in FIG. 5, denoted by _{f 4),} ( 4) Display of a moving image (f ₄ ) of resolution level [4] by moving the display area DR by panning, and (5) Moving image of resolution level [3] by zooming out (f in FIG. 5)
₃ ).

Hereinafter, a method of specifying a tile in a zooming request and a zooming process will be described. (A) [In the case of zooming in on the basis of the middle point of two points in the display area] When zooming in, as shown in FIG. By specifying, the zoom-in center and the zoom-in magnification can be specified. For these designations, an appropriate user interface (for example, an arrow key, a pointer device such as a mouse, etc.) is used. For example, if the resolution level is k (k = 1, 2, 3)
Are displayed in the display area 222, two points (x ₁ , y ₁ ), (x ₂ ,
By designating y ₂ ), it is possible to zoom in on the basis of a rectangle having a line segment connecting these two points as a diagonal line. In this case, the zooming reference point (the point that becomes the center of the display area 222 after zooming) can be the midpoint of the line connecting the above two points. If the aspect ratio of a rectangle whose diagonal is a line connecting the two specified points is the same as the aspect ratio of the display area, the display area after zooming will correspond to the rectangle whose diagonal is that line. You only need to zoom. When the first point is specified, the next point can be specified only so that the aspect ratio of a rectangle having a line segment connecting these two points as a diagonal line is equal to the aspect ratio of the display area 222.
Such zooming is possible. If the aspect ratio of a rectangle whose diagonal is a line connecting two specified points is different from the aspect ratio of the display area 222, the zooming magnification is set to ((diagonal length of display area 222) / (specified). The length of the diagonal line is the line connecting the two points, and the length of the vertical side of the display area 222 / (the length of the vertical side of the rectangle whose diagonal is the line connecting the designated two points) , (Display area 2
Zooming can be performed based on the length of the side of 22) / (the length of the side of the rectangle having the line segment connecting the designated two points as the diagonal).

Hereinafter, the first point (x₁, Y₁) Is specified
When the next point (x₂, Y₂) Connects these two points
The display area 2 is a rectangle whose aspect ratio is a diagonal line segment.
Can only be specified to be equal to the aspect ratio of 22
Will be described below. The above two points (x₁,
y₁), (X₂, Y₂) Indicates the frame image FI_k(N)
At two points (p_{k, 1}, Q_{k, 1}), (P_{k, 2}, Q
_{k, 2}). Where the subscript k is the p coordinate and
The q coordinate is the frame image FI_kImage in space (n)
Indicates that the coordinates are in units of prime. Increase resolution
The number of bells s is determined by the length of the diagonal line of the display area 222 and the display area.
Two points (x₁, Y₁), (X₂, Y₂)
It depends on the ratio r to the length of the segment. For example, the resolution
The number of increase levels s is s = int (log₂r) (just
And int (X) is a function that takes the largest integer not exceeding X.
Number). After zooming in, the video is a frame image
FI_{k + s}(N) is represented using the tiles.
Frame image FI_{k + s}(N) two points (p_{k + s, 1},
q_{k + s, 1}), (P _{k + s, 2}, Q_{k + s, 2})
A rectangular area whose diagonal line is the diagonal line is the frame
Image FI_kTwo points in (n) (p_{k, 1},
q_{k, 1}), (P_{k, 2}, Q_{k, 2}) Connect the diagonal lines
Corresponding to a rectangular area. And zoom in
After the frame image FI_{k + s}(N) two points (p
_{k + s, 1}, Q_{k + s, 1}), (P_{k + s, 2}, Q
_{k + s, 2}) Is a diagonal line on the indication area 222
Will respond.

[1] Frame image FI after zoom-in
_{k + s} (n) (p _{k + s, 1} , q _{k + s, 1} ),
The coordinate values of the middle point of ( _{pk + s, 2} , qk _{+ s, 2} ) are two points ( _{pk, 1} , qk _{, 1} ) before zooming in,
Taking into account that the coordinate value of the middle point of ( _{pk, 2} , qk _{, 2} ) is 2 ^s times and [2] the size of the display area is invariable, the following relationship is established.

[0039]

## EQU00001 ## ( _{pk + s, 1} + _{pk + s, 2} ) / 2 = 2 ^s.multidot. ( _{Pk, 1} + _{pk, 2} ) / 2 (1) (qk _{+ s, 1} + qk _{+ s, 2} ) / 2 = 2 ^s · (q _{k, 1} + q _{k, 2} ) / 2 (2) _{pk + s, 2-} pk _{+ s, 1} = _{pk, 2-} pk _{, 1} (3) q _{k + s, 2−} q _{k + s, 1} = q _{k, 2−} q _{k, 1} (4)

By solving the above equation, the following relational expression is obtained.

[0041]

P _{k + s, 1} = {(2 ^s +1) / 2} · _{pk, 1} + {(2 ^s −1) / 2} · _{pk, 2} (5) q _{k + s, 1} = {(2 ^s +1) / 2} · q _{k, 1} + {(2 ^s −1) / 2} · q _{k, 2} ... (6) p _{k + s, 2} = {(2 ^s −1) / 2 ｝ · _{Pk, 1} + ｛( ^2s + 1) / 2｝ · _{pk, 2} ... (7) q _{k + s, 2} = {( ^2s− 1) / 2｝ · qk _{, 1} + ｛( 2 ^s +1) / 2｝ · q _{k, 2} ... (8)

When s = 1, that is, when the resolution is increased by one step, the following relational expression holds.

[0043]

P _{k + 1,1} = (3pk _{, 1} + pk _{, 2} ) / 2 (9) q _{k + 1,1} = (3qk _{, 1} + qk _{, 2} ) / 2 (10) _{pk + 1,2} = (pk _{, 1} + 3pk _{, 2} ) / 2 (11) qk _{+ 1,2} = (qk _{, 1} + 3qk _{, 2} ) / 2 (12)

As described above, if the coordinates of the frame image FI _{k + s} (n) after zooming in can be known,
The tile required for display can be known.

(B) [When zooming in or out based on an arbitrary point in the display area] A point C (x ₀ ,
A method of displaying the frame image FI _{k + s} (n) by increasing the resolution by s steps based on (y ₀ ) will be described below.

As shown in FIG. 7, a frame image FI corresponding to a point C (x ₀ , y ₀ ) in the display area 222 is displayed.
A point on _k (n) is _defined as C _k (c _{k, p} , c _{k, q} ). Point C _{k + s} on zoomed-in frame image FI _{k + s} (n) corresponding to C _k (c _{k, p} , c _{k, q} )
The coordinates are ^{_{(2 s c k, p,}} 2 s c k, q). Upper left corner and lower right corner of the front of the display area zoom corresponds to the frame image _FI k (n) on the _{(p k, 1, q k} , 1) and _{(p k, 2, q k} , 2), The upper left and lower right corners of the display area after zooming in are frame images FI
_Assuming that (p _{k + s, 1} , q _{k + s, 1} ) and ( _{pk + s, 2} , q _{k + s, 2} ) on _{k + s} (n) correspond, the following relational expression is derived.

[0047]

Equation 4] ^{_{p k + s, 1 = 2}} s c k, p - (p k, 2 -p k, 1) / 2 ··· (13) q k + s, 1 = 2 s c k, q - (q k _{, 2 -q k, 1) /} 2 ··· (14) p k + s, 2 = 2 s c k, p + (p k, 2 -p k, 1) / 2 ··· (15) q k + s, ^{_{2 = 2 s c k, q}} + (q k, 2 -q k, 1) / 2 ··· (16)

From the equations (13) to (16), it is possible to know tiles required for display after zooming in. Of course, at the time of the display after zooming in, the frame image FI _{k + s is set} so that the upper left corner and the lower right corner of the display area are (pk _{+ s, 1} , qk _{+ s, 1} ) and ( _{pk + s, 2} , qk _{+ s, 2} ). The moving image according to (n) is displayed.

[0049] In general, the display area, next to _{t p} individual, vertical _{t q}
Pieces in a size arranged tiles, vertical 64 each tile, when composed of pixels in the lateral 64, the upper left vertex of the coordinates of the display area after ^{zooming, (2 s c k, p} -64 · t ^{_{p / 2,2 s c k, q}} -64
* _Tq / 2). When n is a frame number and u and v are coordinate elements of tile coordinates (that is, (u, v) is coordinates indicating a tile position in a frame image), a frame image FI _k (n) having a resolution level of k is provided. The tile is T
_k (n, u, v). Therefore, the tile at the upper left corner including the upper left vertex of the display area after the above zoom-in is represented as follows.

[0050]

[Number 5] _{T k + s (n, int} (α1), int (α2)) _{^{However, α1 = 2 s / 64 ·}} c k, p -t p / 2 α2 = 2 s / 64 · c k, q -t _q / 2

Where int {α1}, int {α2}
Is the largest integer not exceeding α1, α2. Video tiles necessary for reproducing, t _p pieces worth next (right) from the tile, the t _q pieces worth of tiles in vertical (down). By replacing s in the above equations (13) to (16) with -s, when the resolution level is [k], any point C _k (c
_{k, p} , c _{k, q} ) are required for display after zooming out by knowing the coordinates of the center of the display area in the frame image after zooming out even when zooming out by s steps around the center. You can know the tile.

For example, as shown in FIG. 8A, when a moving image using two tiles is displayed in the display area 222 in the frame image FI ₁ (n), the point C ₁ (70, Assume that there is a zoom-in request to increase the resolution by one step based on 30). In this case, as shown in FIG.
₁ (n) _C 1 of the coordinates _{C 2} points on the frame image _FI 2 corresponding (n) to (70,30) points are (140,60), to the point _{C 2} diagonal midpoint Coordinates at both ends of the diagonal of the display area (here, coordinates of each of the upper left corner and the lower right corner of the display area 222) are (76, 28),
(203, 92). From these coordinate values, the required tile for the frame image FI ₂ (n) is T
₂ (n, 1, 0), T ₂ (n, 2, 0), T ₂ (n,
3,0), T ₂ (n, 1, 1), T ₂ (n, 2, 1),
T ₂ (n, 3, 1).

In the examples of FIGS. 7 and 8A and 8B,
Although the case where the display area 222 extends over the tile has been described, the display area 222 always has a fixed number (2 in the following example).
It is also possible to perform zooming such that the tiles are displayed.

For example, as shown in FIG. 9A, tiles T ₁ (n, 0, 0) and T of frame image FI ₁ (n) having the lowest resolution (resolution level is [1])
_{In the} case where the moving images of ₁ (n, 1, 0) are sequentially displayed, the user sets the resolution to 1 based on a certain point on the display area 222 (point (50, 10) in the frame image FI ₁ (n)). It is assumed that a zoom-in request to increase the level is made (see step S8 in FIG. 4). Due to this zoom-in request, the tile T of the frame image FI ₂ (n) shown in FIG. 9B with a higher resolution by one step (the resolution level is [2])
₂ (n, 1, 0) and a moving image by T ₂ (n, 2, 0) are sequentially displayed in the display area 222. Further, the same point on the display area 222 (the frame image FI
₁ (n) is represented by ( _{100, 20} )), and when a zoom-in request is made to further increase the resolution by one step, a frame image FI whose resolution level is [3] shown in FIG. ₃ (n) tiles T ₃ (n, 3,0) and T ₃ (n, 4,0) are sequentially displayed on the display area 222 as moving images.

(C) [In the case of zooming in or out on the basis of the diagonal midpoint of the display area] An example of the case of performing zooming on the basis of the diagonal midpoint of the display area will be described below. That is,
As shown in FIG. 10, the resolution level is in the frame image _FI k (n) of the [k], lateral _{t p} pieces, the vertical direction _{t q}
T _k (n, i)
_p , _jq ). As described above, Here, n is the frame _number, i p, _{j q} is tile coordinate elements in the frame image _FI k (n), p is 1,
2,..., Any value of _{t p} can take, q is 1, 2, ..., can take any value of _{t q (however,} t p, _{t q} is , The number of horizontal and vertical tiles of the frame image FI _k (n)).

In general, tiles of a group of tiles corresponding to a case where a tile of a frame image FI _k (n) having a resolution level of [k] is zoomed in s steps are represented by T _{k + s} (n, i ′,
j '), i' and j 'are expressed as follows.

[0057]

(Equation 6)

Here, int {(t _p -1) / 2}, i
nt {(t _q -1) / 2} is (t _p -1) / 2, (t
_q- 1) / 2 is the largest integer not exceeding 2.

The same applies to zoom-out. When zooming out in s steps, if the above s is replaced by -s, the equations (17) and (18) can be applied as they are.

In the above-described embodiment, a case is described in which, when a request for zooming in, zooming out, or panning is made, a frame of the moving image whose resolution or display portion has been changed is displayed immediately after the frame of the moving image at the time of the change request. However, when a change request is issued, a predetermined number of frames can be displayed using the image data (tile) before the change request already stored in the temporary storage buffer 212.

[0061]

According to the present invention, the amount of data transfer can be reduced, and zooming and display area change (viewpoint movement) can be efficiently performed during moving image reproduction. Also, display at an appropriate resolution is possible according to the device environment on the client side.

[Brief description of the drawings]

FIG. 1 is a diagram showing an entire moving image data transfer system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a storage state of moving image data in the moving image data transfer system of FIG. 1;

FIGS. 3A to 3D are explanatory diagrams showing frame images having different resolutions.

FIG. 4 is a flowchart illustrating a moving image display method used in the moving image data transfer system illustrated in FIG. 1;

FIG. 5 is an explanatory diagram showing how a resolution changes in zoom-in, pan, and zoom-out processing.

FIG. 6 is an explanatory diagram showing a relationship between coordinates on a display area, coordinates of a certain frame image, and another frame image.

FIG. 7 is an explanatory diagram of a method of displaying a frame image by increasing the resolution based on an arbitrary point in a display area.

FIG. 8A is a diagram illustrating a frame image before being zoomed in, and FIG. 8B is an explanatory diagram illustrating a frame image after being zoomed in;

9A is a diagram illustrating a state in which a moving image is displayed by a frame image having the lowest resolution, FIG. 9B is a diagram illustrating a state in which a moving image is displayed by a frame image having a next higher resolution, and FIG. (C) is a diagram showing a state in which a moving image is displayed by a frame image having a higher resolution by one step.

FIG. 10 is an explanatory diagram showing a display example of each tile constituting a frame image (tile group).

FIG. 11 is an explanatory diagram showing a conventional moving image display method in a network environment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 server 110 storage device 120 transmission device 200 client 210 image processing device 212 temporary storage buffer 214 frame buffer 220 display 222 display area 230 reception device 300 network

Claims (6)

[Claims] 1. A storage format of moving image data for storing a plurality of images having different resolutions in stages corresponding to one frame of a moving image, wherein each image corresponding to each frame is: A storage format of moving image data, comprising a pixel group element having a common configuration. 2. The method according to claim 1, wherein the plurality of images having different resolutions are an original image having the highest resolution, an image having a resolution of (original image resolution) / 2, and a resolution having a resolution of (original image resolution) / 2. ² ... The moving image data storage according to claim 1, wherein the resolution is (original image resolution) / 2 ^K−1 images (where K is a positive integer). format. 3. A server having a storage device in which moving image data is stored in the storage format according to claim 1 and a transmitting device for transmitting the moving image data, a resolution, a display range, and a frame rate. At least one of
And a client having a receiving device capable of downloading an image corresponding to one frame of a moving image from the storage device of the server in units of pixel groups, and a communication line connecting the server and the client. , A moving image data transfer system. 4. A moving image display method used in the moving image data transfer system according to claim 3, wherein the client is configured to execute an image portion necessary for displaying a moving image in accordance with an initial setting condition required for displaying an image. Downloading only the pixel group unit from the storage device of the server to the predetermined buffer from the storage device of the server, copying the downloaded image portion of the pixel group unit to the frame buffer, Detecting a request for a zoom-in or a zoom-out of a moving image displayed in the display area of the display, a request for a pan, or a request for a change in a frame rate, when the request from the user is received, Of the corresponding images, only the image portions necessary for displaying moving images are stored in the server in units of the pixel groups. From 憶 device, the step of downloading a predetermined buffer, moving image display method characterized by performing the step of copying the image portions of said downloaded pixel group segment units in the frame buffer. 5. The moving image display method used in the moving image data transfer system according to claim 4, wherein an image portion required for displaying the moving image is zoomed-in specified by a user when requesting zoom-in or zoom-out. A moving image display method characterized by being specified by a center and a zoom-out center, or a zoom-in area and a zoom-out center. 6. A computer-readable storage medium storing a program for executing the method according to claim 4. Description: