JP3897691B2 - Scroll display method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to scrolling display of decoded images while receiving fragmentary data via a network.
[0002]
[Prior art]
On the Internet, browsing a document data or image data by accessing a WWW server from a Web (web) browser is actively performed. As described above, in a system environment including a WWW server for publishing information on the Internet and a client for browsing the information, each client uses a web browser to browse the information published by the WWW server. be able to.
[0003]
The WWW server normally stores a document describing information to be disclosed, which is called a home page, in HTML, which is accessed by a client-side web browser and displayed on a client-side computer. Also, the client-side web browser can obtain necessary information by following the links in the displayed page. Furthermore, as a method for downloading a file managed by the WWW server, there is a method called File Transfer Protocol (hereinafter abbreviated as “FTP”). FTP is a mechanism for transferring files on a WWW server to a client computer at once through a network.
[0004]
Further, there is Flashpix / IIP as a protocol for accessing and displaying an image file in pieces. The Internet Imaging Protocol (IIP) is an optimal protocol for the format of a Flashpix image data file, and can request a part of image data through a network. Access at this time is performed in units of Flashpix tiles.
[0005]
On the other hand, considering the case where this Flashpix / IIP is applied to JPEG2000 as it is, in JPEG2000, the encoded data of each scalability is composed of differential data with the data of scalability one level lower than the scalability. . Therefore, a method of caching the fragmented encoded data received on the client side, delivering all the encoded data to the decoder and re-decoding from the beginning, and the encoded data received this time while stopping the decoder halfway. To the decoder and decode from the previous continuation.
[0006]
Among such methods, in the system that requests the server for fragmented data of only the necessary data from the encoded image data having multi-resolution data, and receives and displays the data, the client side Conventionally, various methods have been proposed for displaying image data on such computers, particularly as a scroll display method.
[0007]
For example, in a technique related to an image editing apparatus that displays an image cut out from a background image and the background image, in particular, a technique for performing high-speed display by performing high-speed enlargement / reduction / scroll processing by synchronizing the two types of images. Has been proposed (see, for example, Patent Document 1).
[0008]
In addition, a technology related to a still image reproducing apparatus that performs hardware configuration and control so that read / write to an image memory, a control signal to an image processor for image processing, and the like does not cause bus contention on a local bus. Has also been proposed (for example, Patent Document 2).
[0009]
In addition, a technique related to an image scrolling apparatus that decodes run-length code data and performs scroll display and performs processing using a line memory instead of a memory for one screen (for example, patents) has been proposed. Reference 3).
[0010]
Further, when data is transferred from the display memory to the display control means, a drawing display control device and drawing display control for setting a portion not to be displayed in advance and controlling the data not to be displayed to be read from the memory. A technique related to a drawing display device including the device has also been proposed.
[0011]
[Patent Document 1]
Japanese Patent Laid-Open No. 05-054111
[0012]
[Patent Document 2]
Japanese Patent Laid-Open No. 06-004067
[0013]
[Patent Document 3]
Japanese Patent Laid-Open No. 06-019452
[0014]
[Patent Document 4]
JP 11-052942 A
[0015]
[Problems to be solved by the invention]
However, every time the display area is scrolled on the screen in accordance with the user's request, the code data of the part that is newly required to be displayed is finally stored in the server in order to display at the necessary resolution and SN ratio (SNR). If the request is issued, the communication time of the code data occurs, and there is a problem that it takes time until display. In such a case, the user response during scroll display of image data becomes poor.
[0016]
On the other hand, in the case of Flashpix / IIP, since the code data is independent for each tile, it is not necessary to cache the tile data received once on the client side. However, in order to display low-resolution data by scroll display, it is necessary to newly request the server for low-resolution code data, and there is a problem that communication time and traffic increase due to communication again. . Further, when the image data displayed once is cached, there is a problem that a memory capacity for caching the image data is further required.
[0017]
In addition, when code data is cached by Flahspix / IIP, since the encoding method is the JPEG method, it is impossible to decode while adjusting the image quality. For this reason, in order to decode low-resolution code data, there is a problem that a decoding time proportional to the number of tiles is fixedly required.
[0018]
Furthermore, if only the code data that has already been received and is compressed and encoded by the JPEG2000 method that is cached on the client is used, the part that is newly required for display is decoded and displayed. It is conceivable that the portion not to be decoded is displayed assuming that the discrete wavelet transform (DWT) coefficient is 0, but in this case, there is a problem that it takes time to decode.
[0019]
Furthermore, as a method of supplementing the code data of the unreceived part, there is a method of using zero length packet. In this case as well, in the same way as described above, it takes time to finally decode the image data with the necessary resolution and SNR. There is a problem that it takes.
[0020]
Here, as described above, the “image editing apparatus” described in Patent Document 1 performs high-speed enlargement / reduction / scroll processing by synchronizing two types of images, an image cut out from a base image and a base image. Is. Therefore, information for managing the cut-out image data portion is held and controlled, and the entire displayed area is scrolled. That is, the display is scrolled at high speed while decoding JPEG2000, and is not performed after scrolling after holding all the image data, and the above problem cannot be solved.
[0021]
In addition, the “still image reproducing device” described in Patent Document 2 described above prevents the bus contention on the local bus for the control signal for the image processor for read / write to the image memory and for image processing. It performs hardware configuration and control. That is, it is not a technology related to a display device (hardware), but a technology related to a configuration combining a JPEG2000 decoder and IIP2k (image communication), and the above-described problems cannot be solved.
[0022]
Further, the “image scroll device” described in Patent Document 3 described above is a technique for decoding run-length code data and scroll-displaying it, and in particular, processing is performed with a line memory instead of the memory for one screen. Is what you do. That is, when the image data larger than the display area is decoded and the screen is scrolled, not all the code data of the resolution currently displayed is decoded and then scrolled but displayed while being decoded. Therefore, the decoding time of the JPEG2000 is controlled to reduce the decoding time, and the above problem cannot be solved.
[0023]
Furthermore, as described above, the “drawing display control device and the drawing display device including the drawing display control device” described in Patent Document 4 can transfer data from the display memory to the display control unit. This is a technique for setting a portion not to be displayed in advance and controlling the data not to be displayed to be read from the memory. However, in the embodiment of the present invention described below, there is no particular area corresponding to the non-displayed portion, and it is possible to display over the entire display area. As in the invention described in Patent Document 4, two screens are provided. There is no need for processing such as superimposing the image data.
[0024]
The present invention has been made in view of such circumstances, and provides a scroll display method that can reduce communication traffic between a server and a client and improve user response during scroll display of image data. The purpose is to do.
[0025]
[Means for Solving the Problems]
  In order to solve the above-described problems, the present invention provides a scroll display method in a client that receives a part of encoded data stored in a server via a network and displays the decoded image data in a scrolling manner. ,
  Caching the fragmented encoded data received by the client;
  A first decoding step of decoding only the encoded data cached in the cache step during a scroll operation;
  A first display step of enlarging and displaying the image data decoded in the first decoding step;
  A second decoding step of decoding the encoded data cached in the cache step at a higher resolution and a higher layer than the first decoding step after the scroll operation is completed;
  A second display step for enlarging and displaying the image data decoded in the second decoding step;
  After the second display step,A determination step of determining whether or not the encoded data can be displayed at the highest resolution and highest layer using the cached encoded data;
  As a result of the judgment, the highest resolution and the highest Layer If the display of the encoded data is not possible,A request step for newly requesting the server to transmit the encoded data that is insufficient;
  A third decoding step for decoding the encoded data cached in the cache step and the encoded data newly transmitted from the server;
  A third display step for displaying the image data decoded in the third decoding step;
  It is characterized by having.
  Further, the present invention is a scroll display device in a client that receives a part of encoded data stored in a server in a fragmented manner via a network and scrolls and displays decoded image data.
  Caching means for caching the fragmented encoded data received by the client;
  First decoding means for decoding only the encoded data cached by the cache means during a scroll operation;
  First display means for enlarging and displaying the image data decoded by the first decoding means;
  A second decoding means for decoding the encoded data cached by the cache means at a higher resolution and higher layer than the first decoding means after the scroll operation is completed;
  Second display means for enlarging and displaying the image data decoded by the second decoding means;
  After display by the second display means,A determination means for determining whether the encoded data can be displayed at the highest resolution and highest layer using the cached encoded data;
  As a result of the judgment, the highest resolution and the highest Layer If the display of the encoded data is not possible,Request means for newly requesting the server to transmit the encoded data that is insufficient;
  Third decoding means for decoding the encoded data cached by the cache means and the encoded data newly transmitted from the server;
  Third display means for displaying the image data decoded by the third decoding means;
  It is characterized by providing.
[0026]
That is, as will be described in the following embodiment, when displaying image data on the client, a certain part of the image data is displayed, and the display area is changed according to a user's request. Here, when scrolling the screen, only the JPEG2000 code data already received at the client is used, and only the SNR portion of the encoded data with a resolution lower than the resolution to be originally displayed is decoded at high speed. indicate.
[0027]
Then, the code data closest to the resolution and SNR that must be finally displayed that can be decoded with the encoded data at hand at the end of scrolling is decoded and displayed. At the same time, if the image data displayed at this time is not all the necessary encoded data finally, the server further requests the encoded data for the missing part and finally receives it after reception. Decodes and displays the correct image data. As a result, even at the end of scrolling, the image closest to the image data that should be finally displayed is once displayed with the encoded data at hand, and is displayed again after all the encoded data is transmitted.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
[0029]
The following embodiment relates to a method of scrolling and displaying image data while receiving fragmented code data via a network. In particular, by using a communication protocol such as the Internet Imaging Protocol (IIP) for JPEG2000 encoded data compliant with ISO / IEC-15444, the client can obtain a fragmented area or fragment of image encoded data from the server. The encoded data is received, and the experience speed of scroll processing at the client is improved.
[0030]
<First Embodiment>
FIG. 1 is a conceptual diagram showing a configuration of a network system including a server and a client according to the first embodiment of the present invention. That is, it is a conceptual diagram showing a state in which a plurality of computers including a server computer and a client computer are connected to a network represented by the Internet.
[0031]
In FIG. 1, reference numeral 100 denotes a network represented by the Internet. Reference numerals 101 and 103 denote server computers, each of which includes software necessary for the WWW server function, including an IIP server for JPEG2000 for transmitting image data. The servers 101 and 103 are connected to large-capacity hard disks 101a and 103a, respectively, and a large amount of image data is stored in the respective hard disks. Reference numerals 102a and 102b denote client computers, each of which includes software necessary on the client side, such as a web (Web) browser and a JPEG2000 decoder.
[0032]
FIG. 2 is a block diagram showing a hardware configuration of each computer such as the server computers 101 and 103 and the client computers 102a and 102b according to the first embodiment of the present invention.
[0033]
In FIG. 2, reference numeral 201 denotes a CPU which controls the entire apparatus such as each computer. A keyboard 202 is used to input various data, commands, and the like to the device together with the mouse 202a. A display unit 203 includes, for example, a CRT, a liquid crystal display, and the like. Reference numeral 204 denotes a ROM, and 205 denotes a RAM, which constitutes a storage unit in the apparatus, and stores programs executed by the apparatus, data used by the apparatus, and the like.
[0034]
A hard disk 206 constitutes an external storage device used in the apparatus. Reference numeral 207 denotes a flexible disk, which is configured to be detachable via an interface of the device as an external storage device used in the device. Reference numeral 208 denotes a printer. Furthermore, reference numeral 209 denotes a network interface (I / F) for controlling the network, from which access to resources on a network such as a server computer connected to the network 100 such as the Internet (in the case of a client), An encoded image or document data can be transmitted in response to a request from a client (in the case of a server).
[0035]
In the present embodiment, already generated JPEG2000 code files are managed by the server computers 101 and 103. On the other hand, the client computers 102a and 102b request only necessary portions in the JPEG2000 code file such as an image display / print application from the server computers 101 and 103 using the IIP for JPEG2000. Then, the encoded data sent in response thereto is received in fragments, and the received fragmented encoded data is cached in the hard disk 206, for example.
[0036]
In the client computers 102a and 102b, the fragmented JPEG2000 code data cached in the hard disk 206 or the like may be directly processed by a JPEG2000 decoder, or the cached code data is temporarily processed according to JPEG2000-compliant JPEG2000 code data. May be transferred to a JPEG2000 decoder. In the present embodiment described below, in order to simplify the description, it is assumed that JPEG 2000 code data that is directly cached is processed by a decoder.
[0037]
The user of the client computer 102a, 102b opens a home page using, for example, a Windows (registered trademark) machine, and clicks a link to a JPEG 2000 image written therein. As a result, JPEG2000 images are acquired as fragmented data necessary for display at an image size and resolution suitable for the use of the clients 102a and 102b, cached, and displayed after decoding necessary portions. Is done.
[0038]
In the following, it is assumed that the displayed part is changed while scrolling in response to a user request.
[0039]
Here, with reference to FIG. 3, general JPEG2000 code data will be described. FIG. 3 is a conceptual diagram showing a configuration of a JPEG2000 code file recorded along the layer-resolution level-component-position progression (SNR Progression) in the first embodiment. When conforming to this SNR Progression, recording is performed in the order of Layer / Resolution (resolution) / Component (component) / Position (position).
[0040]
FIG. 4 is a diagram for explaining the resolution scalability of JPEG 2000 in the first embodiment. That is, the relationship between the resolution (image size) and the Resolution number is shown. In FIG. 4, the resolution number of the image with the smallest resolution is set to “0”, and the width and height of the decoded image are both doubled as the Resolution number increases by one. Each layer (Layer) is assigned to each resolution in ascending order of Layer number. The layer number corresponds to the S / N ratio with respect to the original image of the image to be restored. The smaller the layer number, the worse the S / N ratio.
[0041]
The maximum values of the Resolution number, Layer number, and Component number in one JPEG2000 code file are preset by the encoder, encoded according to the parameters, and the information is stored in the encoded data. Each packet is composed of a packet header section that manages information of a code block (code-block) stored in the packet, and encoded data of each code block. Yes.
[0042]
Using such JPEG2000 code data, the clients 102a and 102b can receive only necessary data from the server 101 without acquiring all the image data stored in the server 101. . In the embodiment described below, the case of the server 101 will be described as an example, but the case of the server 103 can be similarly executed.
[0043]
As a unit of data received by the client, a JPEG2000 code data packet or a code block unit which is an encoding unit smaller than the packet can be considered. In this embodiment, a packet unit is assumed as a data unit received by the client from the server. However, the data unit can be executed in the same manner as in this embodiment in the code block unit.
[0044]
That is, the scroll display method according to the present embodiment is characterized in that the encoded data is transmitted from the server via the network in units of packets or code blocks.
[0045]
FIG. 5 is a conceptual diagram for explaining a data request and a response in a packet unit between the server and the client in the first embodiment.
[0046]
In FIG. 5, the client 102 a requests data by specifying a tile number (Tile), a resolution number (resolution level), a layer (Layer), a component, and a position number of an image stored in the server 101. In the example illustrated in FIG. 5, the tile number (Tile) is “1”, resolution “0”, layer “0”, Component “0”, and Position number “0”. As a result, the server 101 analyzes the code stream of the image 503, extracts packet data corresponding to the specified tile number, resolution level and layer, component, and position number, and sends the packet 0 back to the client 102a. .
[0047]
FIG. 6 is a conceptual diagram of tile division in the first embodiment. The example shown in FIG. 6 is a case where the image data is divided into eight tiles in the X direction and the Y direction, that is, the entire image is divided into 64 tiles. In addition, the number in a figure shows a tile number. In the present embodiment, the maximum resolution image size of the image data is 1024 × 1024 pixels. In this case, the number of pixels of each tile at the maximum resolution is 128 × 128 pixels.
[0048]
FIG. 7 is a diagram illustrating an example of a JPEG2000 code file managed (stored) by the server 101 used in the first embodiment.
[0049]
In FIG. 7, reference numeral 700 denotes a configuration of the entire code data of an example encoded by SNR Progression. The code file shown in FIG. 7 shows the code when divided into the 64 tiles shown in FIG. In FIG. 7, reference numeral 701 denotes an example of code data of each tile. A tile part header 702 and packet data 703 are included in the tile code data.
[0050]
In the example illustrated in FIG. 7, there are three types of Layers “0” to “2”, six types of Resolution levels “0” to “5”, and one type of Component and Position. In order to simplify the description, the image size of the maximum resolution of JPEG 2000 managed by the server 101 is assumed to be 1024 × 1024 pixels as described above. That is, when there are six types of resolution levels, the resolution size 5 is 1024 × 1024, and the image size is 512 × 512, 256 × 256, 128 × 128, and 64 × 64 in the following order. Number 0 is 32 × 32 pixels.
[0051]
FIG. 8 is a flowchart for explaining an outline of processing by an application in the client 102a according to the first embodiment. This flowchart shows a display processing procedure for image data after a JPEG2000 application is activated on the client 102a and a JPEG2000 file to be displayed is designated by another means. Note that the same processing is performed in other clients (for example, the client 102b).
[0052]
First, the thumbnail image of the designated file is displayed by the client application (step S800). Thereafter, the display is enlarged (step S801) and scrolled (step S802).
[0053]
Specifically, if the size of the image display area in the client application is 128 × 128 pixels, the display of thumbnail images in step S800 decodes up to the maximum SNR code data of resolution number 2 of all tiles. indicate. Next, in the enlarged display in step S801, for example, if the image displayed in step S800 is enlarged by two stages, the resolution becomes resolution number 4. In this case, since the size of the display area is 128 × 128 pixels, the entire image cannot be displayed. Therefore, for example, if the center of the image displayed in step S800 is fixed and enlarged, four tiles of tile numbers 27, 28, 35, and 36 indicated by 600 in FIG. 6 are displayed.
[0054]
FIG. 9 is a diagram for explaining a communication protocol for thumbnail display and received code data in step S800 of FIG. That is, FIG. 9 shows the cache state after execution of the process of step S800. In FIG. 9, the same number is attached | subjected to the code data of the same content as FIG. In step S800, since the same request is issued for all tiles, the cache is in the same state.
[0055]
In FIG. 9, reference numeral 901 denotes a cache for each tile. In this embodiment, since all the code data up to resolution number 2 is received, the received code data portion is described by the expression “Layer numerical value / Reso numerical value / Comp numerical value / Pos numerical value”. The portion that has not yet been received uses the expression “ZLP (Zero Length Packet)”.
[0056]
FIG. 10 is a diagram for explaining the communication protocol and the received code data after moving from step S800 of FIG. 8 to step S801. In step S801, six code data indicated by a part “Layer [0-2] / Reso [3-4] / Comp0 / Pos0” in which only four tiles of tile numbers 27, 28, 35, and 36 in FIG. This part is filled with data. The tiles other than the above are in the same state as that shown in FIG.
[0057]
Next, the scroll display process in step S802 will be described. In the scroll display in this embodiment, the resolution number 4 and the maximum SNR of the four tiles of tile numbers 27, 28, 35, and 36 are displayed, and the area indicated by 602 in FIG. A case will be described in which the image is continuously scrolled to the area indicated by 601 (the areas of tile numbers 45, 46, 53, and 54) via the areas 44 and 45). FIG. 11 is a flowchart for explaining in detail the scroll process (step S802) according to the first embodiment.
[0058]
First, a scrolling operation is started by a pointing device such as a user's mouse, and the amount of movement of the mouse when image data at a corresponding position is displayed according to the amount of movement is calculated (step S802a). Next, an area of image data to be displayed is specified based on the movement amount obtained in step S802a (step S802b). Here, the corresponding tile group is calculated. That is, in the scroll display method according to the present embodiment, the area of the image data to be scroll-displayed on the display screen is specified based on the amount of movement of the pointer in the display screen on which the decoded image data is displayed. And
[0059]
Thereafter, the tile group obtained in step S802b is decoded with the lowest resolution and the lowest layer (step S802c). Then, the image data obtained by decoding in step S802c is enlarged and displayed as necessary (step S802d). Thereafter, it is determined whether scroll requests from the user are continuous (step S802e). This determination method is, for example, when the scroll instruction method is a method of moving while pressing the left button of the mouse, determining the state of the button of the mouse and whether or not the button is kept pressed. Judging. As a result, when the button is kept pressed, it is determined that the process is finished and the process returns to step S802a. On the other hand, if the mouse button is released, the process moves to step S802f.
[0060]
In other words, the scroll display method described in the present embodiment is a scroll display method in a client that receives a part of encoded data stored in a server in a fragmented manner via a network and scrolls and displays decoded image data. . In this scroll display method, first, fragmented encoded data received by the client is cached (cache process). Next, only the encoded data cached in the cache process is decoded during the scroll operation (first decoding process), and the decoded image data is enlarged and displayed (first display process). Then, the encoded data cached in the cache process is decoded at a higher resolution and higher layer than the first decode process after the scroll operation is finished (second decode process). The image data decoded in the second decoding step is enlarged and displayed (second display step).
[0061]
In the scroll display method according to the present embodiment, the first display step obtains the enlargement ratio of the image data from the resolution and the layer of the encoded data in the first decoding step, and enlarges and displays the image data. It is characterized by. Furthermore, in the scroll display method according to the present embodiment, the second display step obtains the enlargement ratio of the image data from the resolution and layer of the encoded data in the second decoding step, and enlarges and displays the image data. It is characterized by.
[0062]
A specific example from steps S802a to S802e will be described. For example, assume that the mouse moves to 602 in FIG. 6 in step S802a. In this case, it is determined in steps S802a and S802b that the area to be displayed this time is an area indicated by tile numbers 36, 37, 44, and 45. In step S802c, the JPEG2000 decoder is instructed to decode resolution number 0 and layer number 0 of the four tile groups of tile numbers 36, 37, 44, and 45.
[0063]
Thereafter, in step S802d, the coarse image data obtained by decoding in step S802c is displayed. In the case of the present embodiment, since the resolution of the lowest resolution is being performed, it is necessary to enlarge to the resolution that should be finally displayed, and this time, the display is performed while performing the 16-times enlargement process. In the case of the present embodiment, since scrolling is continued thereafter, the process returns to step S802a if the process is not ended in step S802e.
[0064]
Thereafter, it is further scrolled by the user's designation, and the areas indicated by tile numbers 45, 46, 53, and 54 are displayed. This determination is made in step S802b. In step S802c, the above four tiles (tile numbers 45, 46, 53, 54) are decoded with resolution number 0 and layer number 0 in the same manner as described above, and tile number 45 is determined in step S802d. , 46, 53, and 54, the image data is displayed while being enlarged. Thereafter, in this embodiment, since the user releases the left button of the mouse, in step S802e, the process moves to step S802f because it is finished. That is, this embodiment is characterized in that the first decoding step decodes encoded data with the lowest resolution and the lowest layer.
[0065]
Next, the process in the case of the scroll end after step S802f will be described in detail. First, decoding is performed up to the point closest to the resolution and SNR that must be displayed, which can be reproduced with the cached code data at hand (step S802f). Then, the image data obtained as a result of decoding in step S802f is displayed while being enlarged (step S802g). Further, it is determined whether or not the image data decoded in step S802f has a resolution and SNR that should be finally displayed (step S802).
[0066]
As a result, when the resolution is final and SNR (that is, when it is not a new request) (No), the processing ends. On the other hand, if it is different (ie, if it is a new request) (Yes), it is necessary to reproduce the already received code data currently in hand and the image of the resolution and SNR that must be finally displayed. The server is requested for code data of a difference from the code data (step S802i). Then, the code data requested in step S802i is received (step S802j). Further, decoding for final display is performed (step S802k), and final display is performed (step S802l). In this case, it is not necessary to enlarge the decoded image data.
[0067]
That is, in addition to the above-described procedure, the scroll display method according to the present embodiment further determines whether the encoded data can be displayed at the highest resolution and the highest layer using the cached encoded data. (Determination step). Then, the server is newly requested to transmit the encoded data that is insufficient (request process), and the encoded data cached in the cache process and the encoded data newly transmitted from the server are decoded. (Third decoding step). Then, the image data decoded in the third decoding step is displayed (third display step).
[0068]
Here, the processing after step S802f will be described using specific numerical values. First, in step S802f, it is determined to what extent code data of tile numbers 45, 46, 53, and 54 are filled. In the case of the present embodiment, the resolution and layer displayed before scrolling are resolution number 4 and layer number 2, respectively, and the code data currently owned by tile numbers 45, 46, 53, and 54 are resolution numbers. 2 and SNR number 2. Therefore, the above four tiles are decoded with resolution number 2 and layer number 2. In step S802g, display is performed while enlarging from the resolution number 2 to the resolution number 4, that is, by 4 times. As a result, it is possible to display higher-resolution image data than the image data displayed in step S802d.
[0069]
Next, in steps S802h and S802i, the encoded data of the “Layer [0-2] / Reso [3-4] / Comp0 / Pos0” portion of the four tiles (tile numbers 45, 46, 53, and 54) is stored in the server. To request. In step S802j, the requested code data is received. In step S802k, the four tiles are decoded up to resolution number 4 and layer number 2. In step S802l, the image data obtained as a result of decoding in step S802k is displayed without being enlarged as it is, and the scroll display process is terminated.
[0070]
As described above, according to the scroll display method in the server / client system in the first embodiment, during the scroll process, only the code data that has already been received before entering the scroll process (cached in the client). Code data only). That is, during scrolling, display is repeated at the lowest resolution and lowest SNR, and at the end of scrolling, the image data is displayed at the highest resolution that can be reproduced with the code data at hand, and the final display portion (final display) Only the code data in the display area) is further requested from the server and displayed. Accordingly, it is possible to minimize communication traffic between the server and the client.
[0071]
In addition, even when there is no image data to be finally displayed at the end of scrolling, the image speed closest to the final display that can be reproduced with the code data at hand is temporarily displayed, thereby improving the user's sensation speed. be able to.
[0072]
Furthermore, by dynamically controlling the image quality and the like of the image displayed during scrolling, it is possible to display according to the sensitivity of the user. Furthermore, in each step, since the image data is displayed in small increments, the user can perform an image display operation without feeling stress.
[0073]
In this embodiment, JPEG2000 encoded data is decoded directly from cached code data. However, the application of the present invention is not limited to this, and JPEG2000-compliant encoding is performed from a cache. A method may be used in which a data file is created and decoded.
[0074]
<Second Embodiment>
In the first embodiment described above, decoding is performed with the lowest resolution and the lowest SNR in the middle of scrolling, and enlarged display is performed. In the second embodiment, a case will be described in which the resolution is not limited to the minimum resolution, but is displayed at the resolution closest to the resolution that should be finally displayed in the already received code data.
[0075]
That is, in this case, the process in step S802c in FIG. 11 is different from that in the first embodiment. Specifically, as an example, resolution number 2 is decoded instead of resolution number 0. The SNR is the same as that in the first embodiment.
[0076]
In this embodiment, since the resolution to be decoded in step S802c changes, the enlargement ratio at the time of display in the next step S802d was 16 times in the first embodiment, but in the case of this embodiment, it is 4 times. Doubled. Other processes are the same as those in the first embodiment.
[0077]
<Third Embodiment>
In the first and second embodiments, when decoding is performed in step S802c in FIG. 11, the scalability in the SNR direction is controlled by layer. In such a case, it depends on the code data in the server. That is, it cannot be used unless Layer is specified in advance in the encoding condition.
[0078]
Therefore, in this embodiment, coding-path is used for control in the SNR direction. By using this, it is possible to control the SNR even for code data that is not cut into layers.
[0079]
Specifically, when decoding in step S802c of FIG. 11, a parameter indicating how far the coding-path is processed instead of the layer number is given to the JPEG2000 decoder. For example, if coding-path is specified up to 3, the JPEG2000 decoder decodes only the MSB (most significant bit) when restoring the DWT coefficient. As a result, the SNR can be controlled more finely than the Layer. Other processes are the same as those in the first and second embodiments.
[0080]
<Other embodiments>
In the embodiments so far, the basic processing procedure of scroll display using JPEG2000 and IIP has been described. Here, as another embodiment, a display method according to the speed at which the user moves the mouse when operating the scroll display will be described. For example, when the mouse is moved quickly, an image with a low SNR is displayed. Conversely, when the mouse is moved slowly, an image with a high SNR is displayed. That is, the present embodiment is characterized in that the SNR of the image data to be scroll-displayed on the display screen is specified based on the moving speed of the pointer in the display screen on which the image data is displayed.
[0081]
In order to perform the above operation, the processing from step S802b to S802d in FIG. 11 (hereinafter referred to as step S802m) may be changed as shown in FIG.
[0082]
That is, FIG. 12 is a flowchart for explaining an example of a processing procedure in another embodiment. In the figure, the same reference numerals are given to portions that perform the same operations as those of the above-described embodiment. Below, the newly added part is demonstrated.
[0083]
First, the maximum (Max) coding-path of each tile of the image currently displayed is acquired (step S1200). In step S802a, the moving amount is calculated, and at the same time, the moving speed (moving speed) is calculated (step S1201). This can be calculated by using a timer such as how much has moved per unit time. Next, the tile group to be displayed this time is obtained (step S802b), and the SNR to be decoded from the moving speed calculated in step S1201 and the Max coding-path obtained in step S1200 is calculated with the coding-path (step S1202). .
[0084]
Further, the coding-path and resolution calculated in step S1202 are designated and decoded (step S1203). Then, the image decoded in step S1203 is displayed while being enlarged according to the magnification calculated from the decoded resolution (step S802d).
[0085]
Here, the above-described operation flow will be described in detail using specific numerical values. First, in step S1200, the value of Max coding-path is acquired from the code data for each tile of all the tiles of the current code data. Since this cannot be obtained from the code data cached in the client, it is requested from the server.
[0086]
Thereafter, scroll display starts by the user's operation of the mouse or the like, and the amount of movement of the mouse is calculated in step S802a. Next, in step S1201, the relationship between the movement amount in step S802a and unit time is classified into the types of “high speed”, “medium speed”, and “low speed” as shown in the table of FIG. 13A, for example. To do. FIG. 13 is a diagram showing determination of mouse speed and coding-path corresponding to it. In the classification method shown in FIG. 13, since the image size of the current display area is 128 × 128 pixels, “high speed” is “128 pixels or more” and “medium speed” is “ “64 to 128 pixels” and “low speed” are determined by “64 pixels or less”. Therefore, in this step, the speed is classified into three types.
[0087]
Next, the tile group corresponding to the area to be scrolled and displayed in step S802b is specified. Thereafter, step S1202 is performed for each tile. In step S1202, a coding-path for determining the SNR is calculated from the speed obtained in step S1201. Here, for example, as shown in FIG. 13B, coding-paths to be decoded for three types of speeds are associated.
[0088]
In the example of FIG. 13B, when the Max coding-path acquired in step S1200 is “26”, “high speed” is “4”, “medium speed” is “10”, and “low speed” is “19”. As described above, the SNR is lowered as the speed increases, and the decoding time is shortened. In step S1203, the coding-path and resolution obtained in step S1202 are decoded, for example, resolution number 2.
[0089]
In step S802d, the enlargement ratio is obtained from the resolution decoded in step S1203 and the resolution that should be displayed at present. In the case of this embodiment, it is four times. Then, display is performed while enlarging at this enlargement rate. Next, in step S802e, it is determined whether or not the mouse button has been released. If not released (No), the process returns to step S802a, and if released (Yes), the process proceeds to the next step. The following steps are the same as those in the above-described embodiment.
[0090]
In the present embodiment, the SNR to be displayed is changed depending on the speed at which the user's mouse is moved. In particular, control is performed so that the SNR of the displayed image decreases as the mouse is moved faster. In other words, the JPEG2000 decoder can perform a higher-speed decoding as the SNR is lowered and can perform a scroll display according to the movement of the user's mouse.
[0091]
Note that the speed classification method and the number of classifications relating to mouse movement and the number of coding-paths to be decoded are not limited to those described above. Further, the resolution for decoding is not limited to this.
[0092]
Furthermore, although the description has been made with respect to the decoding method in which the SNR and the resolution are lowered as the decoding method during scrolling, the present invention is not limited to this. Also good.
[0093]
Note that the present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), but a device (for example, a copier, a facsimile machine, etc.) composed of a single device. You may apply to.
[0094]
Also, an object of the present invention is to supply a recording medium (or storage medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or Needless to say, this can also be achieved when the MPU) reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0095]
Further, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.
[0096]
When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.
[0097]
When the present invention is executed by a program, it is not limited to the processing flow described in the above-described embodiment, and it is easily guessed that user response can be further improved by effectively using multi-threading or the like. can do.
[0098]
【The invention's effect】
As described above, according to the present invention, it is possible to shorten the communication traffic between the server and the client and improve the user response during the scroll display of the image data.
[Brief description of the drawings]
FIG. 1 is a conceptual diagram showing a configuration of a network system including a server and a client according to a first embodiment of the present invention.
FIG. 2 is a block diagram showing a hardware configuration of each computer such as server computers 101 and 103 and client computers 102a and 102b according to the first embodiment of the present invention.
FIG. 3 is a conceptual diagram showing a configuration of a JPEG2000 code file recorded along a layer-resolution level-component-position progression (SNR Progression) in the first embodiment.
FIG. 4 is a diagram for explaining resolution scalability of JPEG2000 in the first embodiment.
FIG. 5 is a conceptual diagram for explaining a data request and a response in a packet unit between a server and a client in the first embodiment.
FIG. 6 is a conceptual diagram of tile division in the first embodiment.
FIG. 7 is a diagram showing an example of a JPEG2000 code file managed (stored) by the server 101 used in the first embodiment.
FIG. 8 is a flowchart for explaining an outline of processing by an application in the client 102a according to the first embodiment.
9 is a diagram for explaining a communication protocol for displaying thumbnails and received code data in step S800 of FIG.
10 is a diagram for explaining a communication protocol and received code data after moving from step S800 to step S801 in FIG. 8;
FIG. 11 is a flowchart for explaining in detail scroll processing (step S802) according to the first embodiment;
FIG. 12 is a flowchart for explaining an example of a processing procedure in another embodiment.
FIG. 13 is a diagram showing determination of mouse speed and coding-path corresponding to the determination.

Claims (11)

A scroll display method in a client that receives a part of encoded data stored in a server in a fragmented manner via a network and scrolls and displays decoded image data.
Caching the fragmented encoded data received by the client;
A first decoding step of decoding only the encoded data cached in the cache step during a scroll operation;
A first display step of enlarging and displaying the image data decoded in the first decoding step;
A second decoding step of decoding the encoded data cached in the cache step at a higher resolution and a higher layer than the first decoding step after the scroll operation is completed;
A second display step for enlarging and displaying the image data decoded in the second decoding step;
After the second display step , a determination step of determining whether the encoded data can be displayed at the highest resolution and highest layer using the cached encoded data;
As a result of the determination, if the encoded data cannot be displayed at the highest resolution and highest layer , a request step for newly requesting the server to transmit the encoded data that is insufficient,
A third decoding step for decoding the encoded data cached in the cache step and the encoded data newly transmitted from the server;
And a third display step for displaying the image data decoded in the third decoding step. The area specifying step of specifying an area of image data to be scroll-displayed on the display screen based on a movement amount of a pointer in the display screen on which the image data is displayed. Scroll display method.   The scroll display method according to claim 1 or 2, wherein the encoded data is transmitted from the server in packet units via the network.   The scroll display method according to any one of claims 1 to 3, wherein the encoded data is transmitted from the server in units of code blocks via the network.   5. The scroll display method according to claim 1, wherein the first decoding step decodes the encoded data with a minimum resolution and a minimum layer. 6.   6. The first display step obtains an enlargement ratio of the image data from a resolution and a layer of the encoded data in the first decoding step, and displays the image data in an enlarged manner. The scroll display method according to any one of the above.   The second display step obtains an enlargement ratio of the image data from the resolution and Layer of the encoded data in the second decoding step, and enlarges and displays the image data. The scroll display method according to any one of the above.   8. An SNR specifying step of specifying an SNR of image data to be scroll-displayed on the display screen based on a moving speed of a pointer in the display screen on which the image data is displayed. The scroll display method according to any one of the above. A scroll display device in a client that receives a part of encoded data stored in a server in a fragmented manner via a network and scrolls and displays decoded image data.
Caching means for caching the fragmented encoded data received by the client;
First decoding means for decoding only the encoded data cached by the cache means during a scroll operation;
First display means for enlarging and displaying the image data decoded by the first decoding means;
A second decoding means for decoding the encoded data cached by the cache means at a higher resolution and higher layer than the first decoding means after the scroll operation is completed;
Second display means for enlarging and displaying the image data decoded by the second decoding means;
Determination means for determining whether or not the encoded data can be displayed at the highest resolution and highest layer using the cached encoded data after the display by the second display means ;
As a result of the determination, when the encoded data cannot be displayed at the highest resolution and highest layer , request means for newly requesting the server to transmit the encoded data that is insufficient;
Third decoding means for decoding the encoded data cached by the cache means and the encoded data newly transmitted from the server;
And a third display unit for displaying the image data decoded by the third decoding unit. A computer that receives a part of the encoded data stored in the server in a fragmented manner via a network and scrolls and displays the decoded image data.
A caching procedure for caching the fragmented encoded data received by the client;
A first decoding procedure for decoding only the encoded data cached in the cache procedure during a scroll operation;
A first display procedure for enlarging and displaying the image data decoded in the first decoding procedure;
A second decoding procedure for decoding the encoded data cached in the cache procedure at a higher resolution and higher layer than the first decoding procedure after the scroll operation is completed;
A second display procedure for enlarging and displaying the image data decoded in the second decoding procedure; and the encoded data at the highest resolution and highest layer using the cached encoded data after the second display procedure. A determination procedure for determining whether or not display is possible,
As a result of the determination, if the encoded data cannot be displayed at the highest resolution and the highest layer, a request procedure for newly requesting the server to transmit the encoded data that is insufficient;
A third decoding procedure for decoding the encoded data cached in the cache procedure and the encoded data newly transmitted from the server;
And a third display procedure for displaying the image data decoded by the third decoding procedure.   A computer-readable recording medium storing the program according to claim 10.

Images