JP2004199291A - Scroll display method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a scroll display method with which a user response during the scroll display of image data can be improved by reducing communication traffic between a server and a client. <P>SOLUTION: A portion of coded data stored in a server is fragmentarily received through a network, and the fragmentary coded data are cached. Then, only the cached coded data are decoded during the scroll operation, and decoded image data are displayed so as to be enlarged. After the scroll operation of the cached coded data is ended, the coded data are decoded with higher resolution and higher layer than the previous time, and the decoded image data are displayed so as to be enlarged. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to scrolling decoded images while receiving fragmented data over a network.
[0002]
[Prior art]
On the Internet, accessing a WWW server from a Web (Web) browser to browse document data, image data, and the like is actively performed. As described above, in a system environment including a WWW server that publishes information on the Internet and a client for browsing the information, each client uses a Web browser to browse information published by the WWW server. be able to.
[0003]
The WWW server stores a document, which is generally called a home page, in which information for disclosure is described in HTML, and the client-side web browser accesses the document and displays it on the client-side computer. Further, the web browser on the client side can obtain necessary information by following the link in the displayed page. Further, as a method of downloading a file managed by the WWW server, there is a method called File Transfer Protocol (hereinafter abbreviated as "FTP"). The FTP is a mechanism for simultaneously transferring a file on a WWW server to a client computer via a network.
[0004]
Also, there is Flashpix / IIP as a protocol for fragmentarily accessing and displaying an image file. The Internet Imaging Protocol (IIP) is an optimal protocol for the format of a Flashpix image data file, and can request a part of image data through a network. The access at this time is performed for each Flashpix tile.
[0005]
On the other hand, considering the case where this Flashpix / IIP is applied to JPEG2000 as it is, in JPEG2000, the coded data of each scalability (Scalability) is composed of difference data from the scalability data one level below the scalability. . Therefore, the fragmentary encoded data received on the client side is cached, and all the encoded data is transferred to the decoder and re-decoded from the beginning. Is passed to the decoder, and decoding is performed from the continuation of the previous time.
[0006]
Among such methods, in a system that requests a server for fragmented data of only necessary data from image encoded data having multi-resolution data and receives and displays the data, a client side Various methods have been conventionally proposed as a method of displaying image data on a computer, in particular, a scroll display method.
[0007]
For example, in a technique related to an image editing apparatus that displays an image cut out from a base image and a base image, in particular, a technique for synchronizing the above two types of images, performing enlargement / reduction / scroll processing at high speed, and performing high-speed display. Has been proposed (for example, see Patent Document 1).
[0008]
Also, a technique related to a still image reproducing apparatus that performs hardware configuration and control so that read / write to an image memory and control signals to an image processor for image processing do not cause bus contention on a local bus. Has also been proposed (for example, Patent Document 2).
[0009]
Further, there has been proposed a technology relating to an image scrolling device for decoding run-length encoded data and performing scroll display, wherein processing is performed using a line memory instead of a memory for one screen. Reference 3).
[0010]
Furthermore, when data is transferred from the display memory to the display control means, a portion not to be displayed is set in advance, and a drawing display control device and a drawing display control for controlling not to read the data of the portion not to be displayed from the memory. A technique relating to a drawing display device provided with the device has also been proposed.
[0011]
[Patent Document 1]
JP-A-05-055411
[0012]
[Patent Document 2]
JP-A-06-004067
[0013]
[Patent Document 3]
JP-A-06-019452
[0014]
[Patent Document 4]
JP-A-11-052942
[0015]
[Problems to be solved by the invention]
However, every time the display area is scrolled on the screen in accordance with the user's request, the code data of the portion that is newly required to be displayed is intended to be displayed at the finally required resolution and SNR (SNR). However, when the request is issued, the communication time of the code data occurs, and there is a problem that it takes time to display. In such a case, the user response during scroll display of the image data deteriorates.
[0016]
On the other hand, in the case of Flashpix / IIP, since the encoded data is independent for each tile, there is no need to cache the data of the tile received once on the client side. However, in order to display low-resolution data by scroll display, it is necessary to newly request low-resolution code data from the server, and there is a problem that communication time and traffic increase by performing communication again. . Further, in the case where the image data once displayed is cached, there is a problem that a memory capacity for caching the image data is further required.
[0017]
Further, when the encoded data is cached by Flahspix / IIP, it is impossible to decode while adjusting the image quality because the encoding method is the JPEG method. Therefore, there is a problem that a decoding time proportional to the number of tiles is fixedly required to decode low-resolution code data.
[0018]
Furthermore, when only the code data that has been cached on the client and that has already been received and compressed by the JPEG2000 method is used to decode and display the part newly required for display, the code data is present. It is conceivable to decode and display the portion that is not used, assuming that the discrete wavelet transform (DWT) coefficient is 0, but in this case, there is a problem that decoding takes time.
[0019]
Furthermore, there is a method of using a zero length packet as a method of supplementing the coded data of the unreceived portion. In this case, similarly to the above, it takes time to decode the image data with the finally required resolution and SNR. There is such a problem.
[0020]
Here, as described above, the “image editing device” described in Patent Literature 1 synchronizes two types of images, the image cut out from the background image and the background image, and performs the enlargement / reduction / scrolling processing at high speed. Things. Therefore, the control is carried out with information for managing the clipped image data portion, and the entire displayed area is scrolled. That is, scroll display is performed at high speed while decoding JPEG2000, and scrolling is not performed after holding all image data, and the above problem cannot be solved.
[0021]
Further, the “still image reproducing device” described in Patent Document 2 described above prevents read / write to / from an image memory and control signals to an image processor for image processing from causing bus contention on a local bus. It performs hardware configuration and control. That is, it is not a technology relating to a display device (hardware) but a technology relating to a configuration combining a JPEG2000 decoder and IIP2k (image communication), and cannot solve the above-mentioned problems.
[0022]
Further, the “image scrolling device” described in Patent Document 3 described above is a technology for decoding run-length encoded data and performing scroll display. In particular, processing is performed using a line memory instead of a memory for one screen. Is what you do. That is, when the image data larger than the display area is decoded and the screen is scrolled, the code data of the currently displayed resolution is not decoded and then scrolled, but is scrolled while decoding. Therefore, the decoding time is shortened by controlling how far the JPEG2000 decoding is performed, and the above problem cannot be solved.
[0023]
Furthermore, the “drawing display control device and the drawing display device including the drawing display control device” described in Patent Document 4 described above, when transferring data from the display memory to the display control means, as described above. This is a technique for setting a portion not to be displayed in advance, and controlling so that data of the portion not to be displayed is not read from the memory. However, in the embodiment of the present invention described below, there is no particular region corresponding to the non-displayed portion, and the entire display region can be displayed. It is not necessary to perform processing such as superimposition of the image data.
[0024]
The present invention has been made in view of such circumstances, and provides a scroll display method capable of reducing communication traffic between a server and a client and improving a user response during scroll display of image data. The purpose is to do.
[0025]
[Means for Solving the Problems]
In order to solve the above problems, the present invention is a scroll display method in a client that receives a part of encoded data stored in a server via a network in a fragmentary manner and scrolls and displays decoded image data. ,
Caching the fragmented encoded data received by the client;
A first decoding step of decoding only the encoded data cached in the cache step during a scroll operation;
A first display step of enlarging and displaying the image data decoded in the first decoding step;
After a scroll operation, a second decoding step of decoding the encoded data cached in the cache step with a higher resolution and a higher layer than the first decoding step;
A second display step of enlarging and displaying the image data decoded in the second decoding step;
A determination step of determining whether or not display of the encoded data is possible at the highest resolution and the highest Layer using the cached encoded data,
A requesting step of newly requesting the server to transmit the missing encoded data;
A third decoding step of decoding the encoded data cached in the cache step and the encoded data newly transmitted from the server;
A third display step of displaying the image data decoded in the third decoding step;
It is characterized by having.
[0026]
That is, as described in the following embodiments, when displaying image data on a client, a part of the image data is displayed, and the display area is changed according to a user's request. Here, when the screen is scroll-displayed, only the JPEG2000 coded data already received by the client is used, and only the SNR portion of the coded data having a lower resolution than the resolution to be originally displayed is decoded at high speed. indicate.
[0027]
Then, at the end of the scroll, the coded data closest to the resolution and SNR that must be finally displayed and can be decoded with the coded data at hand is decoded and displayed. At the same time, if the image data displayed at this time is not all the encoded data finally required, the server further requests the server for the encoded data of the missing part, Decodes and displays complex image data. In this way, even at the end of scrolling, an image closest to the image data that must be finally displayed with the encoded data at hand is displayed once, and is displayed again after all the encoded data has been transmitted.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
[0029]
The present embodiment described below relates to a method of scrolling and displaying image data while receiving fragmentary code data via a network. In particular, by using a communication protocol such as the Internet Imaging Protocol (IIP) for JPEG2000 encoded data conforming to ISO / IEC-15444, the client can transmit a fragmented area or fragment of image code data from the server. This is to improve the perceived speed of scroll processing on the client by receiving coded data.
[0030]
<First embodiment>
FIG. 1 is a conceptual diagram showing a configuration of a network system including a server and a client according to the first embodiment of the present invention. That is, it is a conceptual diagram showing a state where a plurality of computers including a server computer and a client computer are connected on a network represented by the Internet.
[0031]
In FIG. 1, reference numeral 100 denotes a network represented by the Internet. Reference numerals 101 and 103 denote server computers, each of which includes software necessary for a WWW server function, such as a JPEG2000 IIP server for transmitting image data. Further, large-capacity hard disks 101a and 103a are connected to the servers 101 and 103, respectively, and a large amount of image data is stored in each of the hard disks. Reference numerals 102a and 102b denote client computers, each having software required on the client side, such as a Web browser and a JPEG2000 decoder.
[0032]
FIG. 2 is a block diagram illustrating a hardware configuration of each computer such as the server computers 101 and 103 and the client computers 102a and 102b according to the first embodiment of the present invention.
[0033]
In FIG. 2, reference numeral 201 denotes a CPU, which controls the entire apparatus such as a computer. Reference numeral 202 denotes a keyboard, which is used together with the mouse 202a to input various data and commands to the device. A display unit 203 includes, for example, a CRT and a liquid crystal display. Reference numeral 204 denotes a ROM, and 205 denotes a RAM, which constitutes a storage unit in the device, and stores a program executed by the device, data used by the device, and the like.
[0034]
Reference numeral 206 denotes a hard disk, which constitutes an external storage device used in the device. Reference numeral 207 denotes a flexible disk, which is detachable via an interface of the device as an external storage device used in the device. Reference numeral 208 denotes a printer. Further, reference numeral 209 denotes a network interface (I / F) for controlling a network, from which a resource on the network such as a server computer connected to the network 100 such as the Internet is accessed (in the case of a client); An encoded image or document data can be transmitted in response to a request from a client (in the case of a server).
[0035]
In the present embodiment, the already generated JPEG2000 code file is managed by the server computers 101 and 103. On the other hand, the client computers 102a and 102b request only the necessary portions in the JPEG2000 code file such as the image display / print application from the server computers 101 and 103 using the JPEG2000 IIP. Then, the coded data transmitted in response thereto is fragmentarily received, and the received fragmented coded data is cached, for example, on the hard disk 206 or the like.
[0036]
In the client computers 102a and 102b, the fragmented JPEG2000 code data cached on the hard disk 206 or the like may be directly processed by a JPEG2000 decoder, or the cached code data may be temporarily converted to JPEG2000 compliant JPEG2000 code data. And may be passed to a JPEG2000 decoder. In the present embodiment described below, for simplicity of description, it is assumed that JPEG2000 encoded data directly cached is processed by a decoder.
[0037]
The user of the client computer 102a, 102b opens a homepage using, for example, a Windows (registered trademark) machine, and clicks a link to a JPEG2000 image described therein. As a result, a JPEG2000 image is obtained as fragmentary data necessary for displaying at an image size and resolution suitable for the use of the clients 102a and 102b, cached, and displayed after a necessary portion is decoded. Is done.
[0038]
In the following, it is assumed that the displayed portion is changed while being scrolled in response to a request from the user.
[0039]
Here, the general JPEG2000 code data will be described with reference to FIG. FIG. 3 is a conceptual diagram showing a configuration of a JPEG2000 code file recorded along the layer-resolution level-component-position progression (SNR progression) in the first embodiment. When conforming to the SNR Progression, recording is performed in the order of Layer (layer) / Resolution (resolution) / Component (component) / Position (position).
[0040]
FIG. 4 is a view for explaining the resolution scalability of JPEG2000 in the first embodiment. That is, the relationship between the resolution (image size) and the Resolution number is shown. In FIG. 4, the resolution number (resolution level) of the image with the smallest resolution is set to “0”, and the width and height of the decoded image are both doubled each time the Resolution number increases by one. In addition, each layer (Layer) is assigned to each resolution in ascending order of the Layer number. The layer number corresponds to the S / N ratio of the restored image to the original image, and the smaller the layer number, the worse the S / N ratio.
[0041]
The maximum values of the Resolution number, the Layer number, and the Component number in one JPEG2000 code file are set in advance by the encoder and are encoded according to the parameters, and the information is stored in the encoded data. Each packet (packet) is composed of a packet header (packet header) section that manages information of a code block (code-block) stored in the packet, and encoded data of each code block. I have.
[0042]
By using such JPEG2000 code data, the clients 102a and 102b can receive only necessary data from the server 101 without acquiring all the image data stored in the server 101. . In the embodiment described below, the case of the server 101 will be described as an example. However, the server 103 can be similarly executed.
[0043]
As a unit of the data received by the client, a packet of JPEG2000 encoded data or a code block unit which is an encoding unit smaller than the packet can be considered. In the present embodiment, a packet unit is assumed as a data unit received by the client from the server. However, the present invention can be executed in a code block unit in the same manner as in the present embodiment.
[0044]
That is, the scroll display method according to the present embodiment is characterized in that encoded data is transmitted from the server in packet units or code block units via the network.
[0045]
FIG. 5 is a conceptual diagram for explaining a data request and response in a packet unit between the server and the client in the first embodiment.
[0046]
In FIG. 5, a client 102a requests data by designating a tile number (Tile), a resolution number (resolution level), a layer (Layer), a component, and a position number of an image stored in the server 101. In the example shown in FIG. 5, the tile number (Tile) is “1”, the resolution is “0”, the layer is “0”, the component is “0”, and the position number is “0”. As a result, the server 101 analyzes the code stream of the image 503, extracts packet data corresponding to the specified tile number, Resolution level, Layer, component, and position number, and sends it back to the client 102a as packet 0. .
[0047]
FIG. 6 is a conceptual diagram of tile division in the first embodiment. The example shown in FIG. 6 is a case where the image data is divided into eight tiles in both the X and Y directions, that is, the entire image is divided into 64 tiles. The numbers in the figure indicate tile numbers. In the present embodiment, the image size of the maximum resolution of the image data is 1024 × 1024 pixels. In this case, the number of pixels of each tile at the maximum resolution is 128 × 128 pixels.
[0048]
FIG. 7 is a diagram illustrating an example of a JPEG2000 code file managed (stored) by the server 101 used in the first embodiment.
[0049]
In FIG. 7, reference numeral 700 denotes a configuration of the entire code data of an example encoded by SNR Progression. The code file shown in FIG. 7 shows codes in the case of being divided into 64 tiles shown in FIG. In FIG. 7, reference numeral 701 denotes an example of code data of each tile. Tile part headers 702 and packet data 703 constitute code data of tiles.
[0050]
In the example shown in FIG. 7, there are three types of layers, “0” to “2”, six types of Resolution levels, “0” to “5”, and one type of Component and Position. For the sake of simplicity, it is assumed that the maximum resolution image size of JPEG2000 managed by the server 101 is 1024 × 1024 pixels as described above. That is, when there are six types of Resolution levels, the image size at each resolution is 1024 × 1024 for resolution number 5, 512 × 512, 256 × 256, 128 × 128, and 64 × 64 in the following order. The number 0 is 32 × 32 pixels.
[0051]
FIG. 8 is a flowchart illustrating an outline of processing performed by an application in the client 102a according to the first embodiment. This flowchart shows a display processing procedure of image data after a JPEG2000 application is activated on the client 102a and a JPEG2000 file to be displayed is specified by another means. Note that a similar process is performed in another client (for example, the client 102b).
[0052]
First, a thumbnail image of a specified file is displayed by the client application (step S800). Thereafter, the display is enlarged (step S801) and scrolled (step S802).
[0053]
Specifically, assuming that the size of the image display area in the client application is 128 × 128 pixels, in the display of the thumbnail image in step S800, decoding is performed up to the maximum SNR code data of resolution number 2 of all tiles. indicate. Next, in the enlarged display in step S801, for example, when the image displayed in step S800 is enlarged by two steps, the resolution becomes resolution number 4. In this case, since the size of the display area is 128 × 128 pixels, the entire image cannot be displayed. Therefore, for example, if the center of the displayed image is fixed and enlarged in step S800, four tiles with tile numbers 27, 28, 35, and 36 indicated by 600 in FIG. 6 are displayed.
[0054]
FIG. 9 is a diagram for explaining the communication protocol for displaying thumbnails and the received code data in step S800 in FIG. That is, FIG. 9 shows the cache state after the processing of step S800 is executed. 9, the same numbers are assigned to code data having the same contents as in FIG. In step S800, since the same request has been issued to all tiles, they are in the same cache state.
[0055]
In FIG. 9, reference numeral 901 denotes a cache of each tile. In this embodiment, since all the code data up to the resolution number 2 are received, the received code data portion is described in the expression “Layer numerical value / Reso numerical value / Comp numerical value / Pos numerical value”. The parts not yet received use the expression "ZLP (Zero Length Packet)".
[0056]
FIG. 10 is a diagram for explaining the communication protocol and the received code data after the flow proceeds from step S800 to step S801 in FIG. In step S801, the server stores six code data indicated by a portion "Layer [0-2] / Reso [3-4] / Comp0 / Pos0" in which only four tiles of tile numbers 27, 28, 35, and 36 in FIG. , The data of this part is filled. Other tiles are in the same state as that shown in FIG.
[0057]
Next, the scroll display processing in step S802 will be described. In the scroll display according to the present embodiment, from the state in which the resolution number 4 and the maximum SNR of the four tiles of tile numbers 27, 28, 35, and 36 are displayed, the area indicated by 602 in FIG. A case will be described in which scrolling and display are performed continuously to the area indicated by 601 (the area of tile numbers 45, 46, 53, and 54) via the areas 44 and 45). FIG. 11 is a flowchart for explaining the scroll process (step S802) according to the first embodiment in detail.
[0058]
First, a scroll operation is started by a user's pointing device such as a mouse, and the amount of movement of the mouse when displaying image data at a corresponding position is calculated according to the amount of movement (step S802a). Next, an area of the image data to be displayed is specified based on the movement amount obtained in step S802a (step S802b). Here, a particularly applicable tile group is calculated. That is, the scroll display method according to the present embodiment specifies an area of image data to be scroll-displayed on the display screen based on the amount of movement of the pointer on the display screen on which the decoded image data is displayed. And
[0059]
Thereafter, the tile group obtained in step S802b is decoded with the lowest resolution and the lowest layer (step S802c). Then, the image data obtained by decoding in step S802c is subjected to enlargement processing as necessary and displayed (step S802d). Thereafter, it is determined whether or not the scroll request from the user is continuous (step S802e). This determination method is, for example, if the scroll instruction method is a method of moving while pressing the left mouse button, the state of the mouse button is determined, and whether the button is kept pressed or not is determined. Judge. As a result, if the button is kept depressed, it is determined that the operation is completed, and the process returns to step S802a. On the other hand, if the mouse button has been released, the process moves to step S802f.
[0060]
That is, the scroll display method described in the present embodiment is a scroll display method in a client that receives a part of encoded data stored in a server in a fragmentary manner via a network and scrolls and displays decoded image data. . In this scroll display method, first, the fragmentary encoded data received by the client is cached (cache step). Next, only the encoded data cached in the cache step is decoded during the scroll operation (first decoding step), and the decoded image data is enlarged and displayed (first display step). Then, after the scroll operation is completed, the encoded data cached in the cache step is decoded with a higher resolution and a higher layer than in the first decode step (second decode step). Then, the image data decoded in the second decoding step is enlarged and displayed (a second display step).
[0061]
Also, in the scroll display method according to the present embodiment, the first display step obtains an enlargement ratio of the image data from the resolution and the layer of the encoded data in the first decoding step, and enlarges and displays the image data. It is characterized by. Further, in the scroll display method according to the present embodiment, the second display step obtains an enlargement ratio of the image data from the resolution and the layer of the encoded data in the second decoding step, and enlarges and displays the image data. It is characterized by.
[0062]
A specific example from steps S802a to S802e will be described. For example, assume that the mouse has moved to 602 in FIG. 6 in step S802a. In this case, it is determined in steps S802a and S802b that the area to be displayed this time is the area indicated by the tile numbers 36, 37, 44, and 45. In step S802c, the JPEG2000 decoder is instructed to decode the resolution number 0 and the layer number 0 of the four tile groups of the tile numbers 36, 37, 44, and 45.
[0063]
Thereafter, in step S802d, coarse image data obtained by decoding in step S802c is displayed. In the case of the present embodiment, since the decoding of the lowest resolution is performed, it is necessary to enlarge the resolution to the final display, and this time, the display is performed while performing the enlargement processing of 16 times. Further, in the case of the present embodiment, since scrolling is performed thereafter, the process returns to step S802a if it is not completed in step S802e.
[0064]
Thereafter, scrolling is further performed according to the user's designation, and the areas indicated by the tile numbers 45, 46, 53, and 54 are displayed. This determination is made in step S802b. In step S802c, the four tiles (tile numbers 45, 46, 53, 54) are decoded with the resolution number 0 and the layer number 0 in the same manner as described above, and in step S802d, the tile number 45 is decoded. , 46, 53, and 54 are displayed while being enlarged. Then, in this embodiment, since the user releases the left mouse button, in step S802e, it is determined that the process has ended, and the process moves to step S802f. That is, the present embodiment is characterized in that the first decoding step decodes the encoded data with the lowest resolution and the lowest layer.
[0065]
Next, processing in the case of scroll end after step S802f will be described in detail. First, decoding is performed up to the closest resolution and SNR that can be reproduced with the cached code data at hand and must be finally displayed (step S802f). Then, the image data decoded in step S802f is displayed while being enlarged (step S802g). Further, it is determined whether or not the image data decoded in step S802f has the resolution that must be finally displayed and the SNR (step S802).
[0066]
As a result, if the final resolution is SNR (that is, if it is not a new request) (No), the process ends. On the other hand, if it is different (that is, if it is a new request) (Yes), it is necessary to reproduce the image of the already received code data currently at hand and the resolution and SNR that must be finally displayed. The server requests the server for code data that is different from the code data (step S802i). Then, the code data requested in step S802i is received (step S802j). Further, decoding for final display is performed (step S802k), and final display is performed (step S802l). In this case, it is unnecessary to perform the process of expanding the decoded image data.
[0067]
That is, in the scroll display method according to the present embodiment, in addition to the above-described procedure, it is further determined whether or not the encoded data can be displayed at the highest resolution and the highest layer using the cached encoded data. (Judgment step). Then, the server newly requests the server to transmit the missing encoded data (request step), and decodes the encoded data cached in the cache step and the encoded data newly transmitted from the server. (Third decoding step). Then, the image data decoded in the third decoding step is displayed (third display step).
[0068]
Here, the processing after step S802f will be described using specific numerical values. First, in step S802f, it is determined how much code data of tile numbers 45, 46, 53, and 54 has been filled. In the case of this embodiment, the resolution and the layer displayed before scrolling are respectively the resolution number 4 and the layer number 2, and the code data that the tile numbers 45, 46, 53, and 54 currently have are the resolution numbers. 2, SNR number 2. Therefore, the above four tiles are decoded with the resolution number 2 and the layer number 2. Then, in step S802g, display is performed while enlarging from resolution number 2 to resolution number 4, that is, quadrupling. As a result, image data having a higher resolution than the image data displayed in step S802d can be displayed.
[0069]
Next, in steps S802h and S802i, the code data of the “Layer [0-2] / Reso [3-4] / Comp0 / Pos0” portion of the above four tiles (tile numbers 45, 46, 53, 54) is sent to the server. Request to. Then, in step S802j, the requested code data is received, and in step S802k, the four tiles are decoded up to the resolution number 4 and the layer number 2. In step S802l, the image data decoded in step S802k is displayed without being enlarged, and the scroll display process ends.
[0070]
As described above, according to the scroll display method in the server / client system according to the first embodiment, during the scroll processing, only the code data already received before the scroll processing is started (the code data being cached in the client). Code data only). That is, during the scrolling, the display is repeated at the lowest resolution and the lowest SNR, and at the end of the scrolling, the image data is displayed at the highest resolution that can be reproduced by the code data at hand, and the part to be finally displayed (final display) Only the code data of the display area) is further requested from the server and displayed. Therefore, it is possible to minimize communication traffic between the server and the client.
[0071]
Also, at the end of scrolling, even if there is no image data to be finally displayed, the image data closest to the final display that can be reproduced with the code data at hand is once displayed, thereby improving the bodily sensation speed of the user. be able to.
[0072]
Furthermore, by dynamically controlling the image quality and the like of the image displayed during scrolling, display suitable for the user's sensitivity can be performed. Furthermore, in each step, since the image data is displayed in small increments, the user can operate the image display without feeling stress.
[0073]
In this embodiment, JPEG2000 encoded data is decoded directly from cached encoded data. However, application of the present invention is not limited to this. A method of creating a data file and decoding the data file may be used.
[0074]
<Second embodiment>
In the first embodiment described above, decoding is performed at the lowest resolution and the lowest SNR during scrolling, and enlarged display is performed. In the second embodiment, a case will be described in which the resolution is not limited to the lowest resolution but is displayed at the resolution closest to the resolution that must be finally displayed in the already received code data.
[0075]
That is, in this case, the processing in step S802c of FIG. 11 is different from that of the first embodiment. Specifically, for example, resolution number 2 is decoded instead of resolution number 0. The SNR is the same as in the first embodiment.
[0076]
In the present embodiment, since the resolution to be decoded in step S802c changes, the enlargement ratio at the time of display in the next step S802d is 16 times in the first embodiment, but is 4 times in the present embodiment. Double. The other processing is the same as in the first embodiment.
[0077]
<Third embodiment>
In the first and second embodiments, when decoding is performed in step S802c in FIG. 11, the scalability in the SNR direction is controlled by the layer. In such a case, it depends on the code data in the server. That is, if the layer is not specified in advance in the encoding condition, it cannot be used.
[0078]
Thus, in the present embodiment, coding-path is used for control in the SNR direction. By using this, it is possible to control the SNR even for coded data whose layers are not cut.
[0079]
Specifically, at the time of decoding in step S802c in FIG. 11, the JPEG2000 decoder is provided with a parameter indicating how far the coding-path should be processed instead of the layer number. For example, if coding-path is specified up to 3, the JPEG2000 decoder decodes only the MSB (most significant bit) when restoring DWT coefficients. As a result, it is possible to control the SNR more finely than the layer. The other processing is the same as in the first and second embodiments.
[0080]
<Other embodiments>
In the embodiments described above, the basic processing procedure of the scroll display using JPEG2000 and IIP has been described. Here, as another embodiment, a display method according to the moving speed of the mouse when the user operates the scroll display will be described. This means that, for example, when the mouse is moved quickly, an image with a low SNR is displayed, and when the mouse is slowly moved, an image with a high SNR is displayed. That is, in the present embodiment, the SNR of the image data to be scroll-displayed on the display screen is specified based on the moving speed of the pointer on the display screen on which the image data is displayed.
[0081]
In order to perform the above operation, the processing from step S802b to S802d in FIG. 11 (hereinafter, referred to as step S802m) may be changed as shown in FIG.
[0082]
That is, FIG. 12 is a flowchart illustrating an example of a processing procedure according to another embodiment. Note that, in the drawing, the same reference numerals are given to portions performing the same operation as the above-described embodiment. Hereinafter, a newly added portion will be described.
[0083]
First, the maximum (Max) coding-path of each tile of the currently displayed image is acquired (step S1200). Then, the moving amount is calculated in step S802a, and at the same time, the moving speed (moving speed) is calculated (step S1201). This can be calculated by using a timer, such as the amount of movement per unit time. Next, the tile group to be displayed this time is obtained (step S802b), and the SNR to be decoded is calculated from the moving speed calculated in step S1201 and the Max coding-path obtained in step S1200 using coding-path (step S1202). .
[0084]
Furthermore, decoding is performed by designating the coding-path and the resolution calculated in step S1202 (step S1203). Then, the image decoded in step S1203 is displayed while being enlarged according to the magnification calculated from the decoded resolution (step S802d).
[0085]
Here, the operation flow described above will be described in detail using specific numerical values. First, in step S1200, the value of the Max coding-path is obtained from the code data for each tile for all tiles of the current code data. Since this cannot be obtained from the code data cached in the client, a request is made to the server.
[0086]
Thereafter, scroll display is started by a user's operation of a mouse or the like, and the amount of mouse movement is calculated in step S802a. Next, in step S1201, the relationship between the amount of movement in step S802a and the unit time is classified into, for example, "high speed", "medium speed", and "low speed" as shown in the table of FIG. I do. FIG. 13 is a diagram showing the determination of the mouse speed and the coding-path corresponding thereto. In the classification method shown in FIG. 13, since the image size of the current display area is 128 × 128 pixels, “high speed” is “128 pixels or more” and “medium speed” is “ “64 to 128 pixels” and “low speed” are determined by “not more than 64 pixels”. Therefore, in this step, the speed is classified into three types.
[0087]
Next, in step S802b, a tile group corresponding to an area to be scrolled and displayed is specified. Thereafter, step S1202 is performed for each tile. In step S1202, a coding-path for determining the SNR is calculated from the speed obtained in step S1201. Here, for example, as shown in FIG. 13B, coding-paths to be decoded for three types of speeds are associated.
[0088]
In the example of FIG. 13B, when the Max coding-path acquired in step S1200 is “26”, “High” is “4”, “Medium” is “10”, and “Low” is “19”. As described above, the higher the speed, the lower the SNR, thereby shortening the decoding time. Then, in step S1203, the coding-path and resolution, for example, resolution number 2 obtained in step S1202 are decoded.
[0089]
In step S802d, an enlargement ratio is obtained from the resolution decoded in step S1203 and the resolution currently required to be displayed. Incidentally, in the case of the present embodiment, it is four times. Then, display is performed while enlarging at this enlargement ratio. Next, in step S802e, it is determined whether or not the mouse button has been released. If it has not been released (No), the process returns to step S802a, and if it has been released (Yes), the process proceeds to the next step. The following steps are the same as in the above-described embodiment.
[0090]
In the case of the present embodiment, the displayed SNR is changed depending on the speed at which the mouse of the user is moved. In particular, control is performed such that the faster the mouse is moved, the lower the SNR of the displayed image. That is, the JPEG2000 decoder can perform high-speed decoding as the SNR is lowered, so that it is possible to perform scroll display according to the movement of the mouse of the user.
[0091]
It should be noted that the method of classifying the speed related to the mouse movement, the number of classes to be classified, and the number of coding-paths to be decoded for the classification are not limited to those described above. The resolution for decoding is not limited to this.
[0092]
Furthermore, as the decoding method during scrolling, the description has been given of decoding with lowering of SNR and resolution. However, the present invention is not limited to this. For example, in the case of full color, the number of color components is reduced, that is, decoding is performed in black and white. Is also good.
[0093]
Note that the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), but a device including one device (for example, a copying machine, a facsimile machine, etc.). May be applied.
[0094]
Further, an object of the present invention is to supply a recording medium (or a storage medium) recording a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or a CPU or a CPU) of the system or the apparatus. Needless to say, the present invention can also be achieved by the MPU) reading and executing the program code stored in the recording medium. In this case, the program code itself read from the recording medium implements the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention. When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing.
[0095]
Further, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.
[0096]
When the present invention is applied to the recording medium, the recording medium stores program codes corresponding to the flowcharts described above.
[0097]
When the present invention is executed by a program, the present invention is not limited to only the processing flow described in the above-described embodiment, and it is easily inferred that the user response can be further improved by effectively using a multi-thread or the like. can do.
[0098]
【The invention's effect】
As described above, according to the present invention, it is possible to shorten the communication traffic between the server and the client and improve the user response during the scroll display of the image data.
[Brief description of the drawings]
FIG. 1 is a conceptual diagram illustrating a configuration of a network system including a server and a client according to a first embodiment of the present invention.
FIG. 2 is a block diagram illustrating a hardware configuration of each computer such as server computers 101 and 103 and client computers 102a and 102b according to the first embodiment of the present invention.
FIG. 3 is a conceptual diagram showing a configuration of a JPEG2000 code file recorded along a layer-resolution level-component-position progression (SNR progression) in the first embodiment.
FIG. 4 is a diagram illustrating the resolution scalability of JPEG2000 in the first embodiment.
FIG. 5 is a conceptual diagram illustrating a data request and response in packet units between a server and a client according to the first embodiment.
FIG. 6 is a conceptual diagram of tile division in the first embodiment.
FIG. 7 is a diagram illustrating an example of a JPEG2000 code file managed (stored) by the server 101 used in the first embodiment.
FIG. 8 is a flowchart illustrating an outline of processing performed by an application in a client 102a according to the first embodiment.
FIG. 9 is a diagram for explaining a communication protocol for thumbnail display and received code data in step S800 in FIG. 8;
FIG. 10 is a diagram for explaining a communication protocol and received code data after moving from step S800 to step S801 in FIG. 8;
FIG. 11 is a flowchart for explaining a scroll process (step S802) according to the first embodiment in detail;
FIG. 12 is a flowchart illustrating an example of a processing procedure according to another embodiment.
FIG. 13 is a diagram illustrating determination of a mouse speed and a coding-path corresponding thereto.

Claims (1)

A scroll display method in a client that receives a part of encoded data stored in a server in a fragmentary manner via a network and scrolls and displays decoded image data.
Caching the fragmented encoded data received by the client;
A first decoding step of decoding only the encoded data cached in the cache step during a scroll operation;
A first display step of enlarging and displaying the image data decoded in the first decoding step;
After a scroll operation, a second decoding step of decoding the encoded data cached in the cache step with a higher resolution and a higher layer than the first decoding step;
A second display step of enlarging and displaying the image data decoded in the second decoding step;
A determination step of determining whether or not display of the encoded data is possible at the highest resolution and the highest Layer using the cached encoded data,
A requesting step of newly requesting the server to transmit the missing encoded data;
A third decoding step of decoding the encoded data cached in the cache step and the encoded data newly transmitted from the server;
A third display step of displaying the image data decoded in the third decoding step.

Images