JP2004153562A - Image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an image according to the request from a user while preventing overflow of data in a cache by deleting the cache data which may produce less effect on the regenerated image from the encoded image data in the cache. <P>SOLUTION: An image processing method for regenerating the encoded image data fragmentarily input and cached is presented. When an image area of interest is specified, it is determined whether or not the cache data amount obtained by caching the image data exceeds an allowed amount, caused by modifying the area F of interest. If the cache data amount exceeds the allowed amount, the data amount Y to be deleted from the cache data stored in the cache memory is acquired (S11), and the distance from the area F of interest to a unit area to be deleted is measured. Based on the measured distance, a deletion area R of which data are to be deleted from the cache data is determined on a tile-by-tile basis (S14), and the cache data included in the above determined deletion area R are deleted (S16). <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for receiving, for example, fragmentary encoded data from a server at a client, and caching and displaying the fragmented image data.
[0002]
[Prior art]
2. Description of the Related Art Accessing a WWW server from a Web browser on the Internet to browse document data, image data, and the like has been actively performed. In this mechanism, there are a WWW server that publishes information on the Internet and a plurality of clients that browse the information, and each client browses the information of the server using a Web browser. This WWW server has a document in which information to be disclosed called a home page is described in HTML, and the Web browser on the client side fetches the document and displays it on the client computer. The Web browser on the client side can follow the link in the displayed page to obtain necessary information.
[0003]
Further, as a method of downloading a file managed by a Web server, there is a method called File Transfer Protocol (hereinafter abbreviated to FTP). The FTP is a mechanism that transfers a file on a server to a client computer at one time via a network.
[0004]
Also, there is Flashpix / IIP as a protocol for fragmentarily accessing and displaying an image data file. The IIP is a Flashpix, which is an optimal protocol for an image data file format, and a partial access unit of image data is performed in units of Flashpix tiles.
[0005]
On the other hand, if this IIP is applied to JPEG2000 as it is, the coded data of each scalability of JPEG2000 is difference data from the scalability data one level lower than the scalability, so the fragmentary coding received on the client side is performed. Data must be cached.
[0006]
[Problems to be solved by the invention]
However, if all of the fragmentary encoded data is saved on the client side, all data in the server's JPEG2000 bitstream will be cached on the client when the client eventually browses the entire image at the highest resolution and highest SNR. become. This causes a problem that a terminal having a small memory capacity exceeds the memory capacity while browsing an image, and it becomes impossible to perform display according to a user's request. Further, since JPEG2000 is differential data, when the memory capacity is saturated and new received data is overwritten with already cached data, an accurate image display (display desired by the user) occurs. It can happen.
[0007]
SUMMARY OF THE INVENTION The present invention has been made in view of the above-described conventional example, and deletes cache data having little effect on a reproduced image from cached encoded image data to prevent a data overflow in a cache while preventing a user from overflowing. The purpose is to provide a display as requested.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, the image processing method of the present invention includes the following means. That is,
An image processing method for inputting, caching, and reproducing encoded image data in a fragmentary manner,
An attention area specifying step of specifying an attention area of the image;
A determination step of determining whether or not the amount of cache data that caches the image data exceeds an allowable amount of a cache memory by changing the attention area;
A deletion data amount determination step of determining a data amount to be deleted from the cache data stored in the cache memory when it is determined that the data amount exceeds the determination step;
A distance measurement step of measuring a distance from the attention area specified in the attention area identification step to a unit area to be deleted;
A deletion area determination step of determining a deletion area for deleting data from cache data in units of the unit area based on the attention area and the distance measured in the distance measurement step;
A deletion step of deleting cache data included in the deletion area determined in the deletion area determination step,
An image processing method comprising:
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0010]
FIG. 1 is a block diagram illustrating a configuration of a client or a server according to the present embodiment. This device may be configured by, for example, a personal computer or a workstation.
[0011]
The CPU 101 controls the operation of the entire system, and executes various control operations based on a program stored in the primary storage 102. The primary storage 102 is mainly a memory (RAM), and reads and stores programs and the like stored in the secondary storage 103. The secondary storage 103 corresponds to, for example, a hard disk. In general, the capacity of the primary storage 102 is smaller than the capacity of the secondary storage 103, and programs and data that cannot be stored in the primary storage 102 are stored in the secondary storage 103. Data that must be stored for a long time is also stored in the secondary storage 103. In the present embodiment, a program is stored in the secondary storage 103, read and stored in the primary storage 102 when the program is executed, and the CPU 101 performs processing according to the program stored in the primary storage 102. The input device 104 corresponds to, for example, a mouse or a keyboard. The input device 104 is used to generate an interrupt signal in a program or the like and input desired data by operating the input device 104. The output device 105 is, for example, a monitor or a printer. Although various other forms are conceivable for the configuration method of this apparatus, they are not the main subject of the present invention, and thus description thereof will be omitted.
[0012]
FIG. 2 is a diagram illustrating an outline of a network according to the present embodiment.
[0013]
In the figure, reference numerals 201 and 202 denote users (clients), each having the device configuration described in FIG. The user 201 or 202 can communicate with the server 204 via the network 203 irrespective of wired or wireless. The server 204 has the configuration shown in FIG. 1 and includes a large-capacity storage device (hard disk) 205 for storing image files corresponding to the secondary storage 103. The large-capacity storage device 205 corresponds to, for example, a hard disk. This hard disk stores a lot of image data encoded by the JPEG2000 encoding method. The users 201 and 202 receive and store fragmented data from the JPEG2000 image data stored in the server 204.
[0014]
In the present embodiment, a method will be described in which a client that has already received data of a JPEG2000 file that has already been generated and that has cached them deletes the cached data.
[0015]
Each user opens a homepage using a Windows (registered trademark) machine, and clicks (instructs) a link to a JPEG2000 image written therein to convert the JPEG2000 image suitable for the user's use. Get the fragmentary data needed to display at image size and resolution and cache those data. Then, it is assumed that when the limit value of the cache capacity of the user is exceeded, a program having a function such as a cache manager runs and deletes the cache data of the user.
[0016]
FIG. 3 is a view for explaining general JPEG2000 encoded data, and shows a configuration of a JPEG2000 file recorded along a Layer-resolution level-component-position progression (hereinafter, referred to as LRCP). When this LRCP is followed, recording is performed in the order of layer / solution / component / position. Such a data arrangement is called a progression order.
[0017]
FIG. 4 is a diagram showing the relationship between the resolution (image size) in JPEG2000 encoded data and the Resolution number.
[0018]
The resolution number of the image with the smallest resolution is set to “0”, and the width and height of the image are doubled each time the resolution number increases by one. In each layer, data is stored in ascending order of the resolution number. The layer number corresponds to the S / N ratio of the image to be restored to the original image, and the smaller the layer number, the worse the S / N ratio. The maximum value of the resolution number, the layer number, and the component number in one JPEG2000 file is set in advance by an encoder and is encoded according to the parameters, and the information is stored in the encoded data. Each packet (packet) includes a packet header section that manages information on a code block (code-block) stored in the packet, and coded data of each code-block.
[0019]
By using such JPEG2000 encoded data, each user can receive only necessary data from the server 204 without acquiring all image data in the server 204. As a unit of the user's received data, a JPEG2000 packet or a code-block unit which is an encoding unit smaller than the packet can be considered. In the present embodiment, a packet unit is assumed as a data unit received by the user from the server 204.
[0020]
FIG. 5 is a diagram showing a conceptual diagram of a request and a response in a packet unit.
[0021]
The user 201 requests data from the server 204 by specifying a tile number of an image, a resolution level, a layer, a component, and a position number. Thus, the server 204 analyzes the code stream of the image 503, extracts packet data corresponding to the specified tile number, resolution level, layer, component, and position number, and sends the packet data to the user 201.
[0022]
In this embodiment, the original image in the server 204 is divided into 64 tiles (Tiles) each having a maximum resolution size of 2048 × 2048 [pixels] and 8 × 8, and the number of components is “3”. It is assumed that there are resolution levels 0 to 3, that is, four levels in the image size direction, which are divided into two layers.
[0023]
FIG. 6 is a diagram illustrating the data structure of each layer of the original image.
[0024]
As shown in FIG. 6, the tiles constituting the original image are numbered sequentially from the upper left tile. Therefore, the server 204 stores data as indicated by 701 in FIG.
[0025]
FIG. 7 is a diagram for explaining the configuration of JPEG2000 encoded data managed by the server 204. Reference numeral 703 denotes a main header, and reference numeral 702 denotes encoded data of tile 0.
[0026]
[Delete data in tile units]
The client (user) 201 caches the fragmentary JPEG2000 bit stream sent from the server 204. The user 201 knows in advance the memory capacity available for data caching, and erases the bit stream before the fragmentary JPEG2000 bit stream cache data sent from the server 204 overflows the cache memory capacity. .
[0027]
FIG. 8 is a flowchart showing a cache control process in a user (client) according to the present embodiment.
[0028]
First, in step S1, it is grasped which partial area of the image the image data sent from the server this time corresponds to, and the displayed image area is specified as the attention area F. For example, in the image as shown in FIG. 6, when the reception of the current data is for displaying the area 601 indicated by the thick frame of the resolution 3, the tiles 33 to 35, 41 to 43, 49 to 51 Nine tiles are specified as the attention area F.
[0029]
Next, the process proceeds to step S2, where the capacity of the cache memory of the user 201 is obtained, and a threshold value for determining whether or not the cache data needs to be deleted is determined. Here, if 100% of the available memory capacity is used to cache the received JPEG2000 packet data, the threshold value becomes equal to the memory capacity. However, when data other than the JPEG2000 fragmentally received bit stream (for example, meta information of an image) is also stored in the same memory capacity, a part of the available memory capacity is limited to the JPEG2000 code data. Used for caching. Alternatively, when caching a plurality of JPEG2000 encoded data in the same system, the threshold value of the memory capacity available for a certain image is set to a value smaller than the usable memory capacity. For example, for a memory capacity of 3 M [bytes], a value of 2 M [bytes] is set as a threshold value of the cache of this image. When the threshold value is determined, a secondary storage medium such as a hard disk may be used. However, in the present embodiment, the threshold value is determined based on the memory size for simplicity. Next, the process proceeds to step S3, and the number of bytes of the fragmented JPEG2000 code data received from these is acquired.
[0030]
Next, proceeding to step S4, it is determined whether or not the received data for the number of bytes acquired in step S3 exceeds the threshold value determined in step S2 when cached. That is, if the value obtained in step S3 is larger than the value obtained by subtracting the currently cached data amount from the threshold value, it is determined that the threshold value is exceeded. If the value obtained in step S3 is smaller, the threshold value is determined. Is determined not to exceed. When it is determined that the threshold value is exceeded, the process proceeds to step S5, and when it is determined that the threshold value is not exceeded, the process proceeds to step S6. For example, if the threshold value determined in step S2 is 2M [bytes] and the already cached JPEG2000 packet data amount is 1.95M [bytes], the number of bytes of the received data acquired in step S3 is 120K [bytes]. ], It is determined that if the received data is cached as it is, the threshold value will be exceeded, and the process proceeds to step S5.
[0031]
In step S5, even if the number of bytes acquired in step S3 is cached, the encoded data is deleted from the cache data in tile units so that the cache capacity is equal to or smaller than the threshold value. For example, if the number of bytes acquired in step S3 is 120K [bytes] and the threshold value determined in step S2 is 2M [bytes], in step S5, until 2M-120K = 1.88M [bytes] or less. The cache data is deleted. Then, the process proceeds to step S6 to cache the received data. Next, in step S7, it is determined whether the request for this image has been completed. If the request has not ended, the process returns to step S3, and if the request has ended, this cache control process ends.
[0032]
Next, the cache data deletion process in step S5 in FIG. 8 will be described with reference to the flowchart in FIG. FIG. 9 is a flowchart showing the processing in step S5.
[0033]
First, in step S11, the data amount Y to be deleted is acquired from the cache memory, and “0” is substituted as an initial value into a variable Del for calculating the data amount deleted in the cache data deletion processing. In the present embodiment, the user has already displayed the entire image at the maximum SNR of resolution level 1, and the area 602 in FIG. 6 is currently surrounded by the maximum SNR, that is, tiles 16 to 21, 56 to 61. It is assumed that an area of 36 tiles is displayed at resolution level 2, maximum SNR. At this time, in the cache memory, the data 701 of the header portion of the image, the packet data 704 and 705 constituting the resolution level 0 and the resolution level 1 of all the tiles, and the resolution of the 36 tiles in the area 602 shown in FIG. The packet data 706 and 708 forming level 2 are stored. That is, regarding the 36 tiles in the area 602, as indicated by 1002 in FIG. 10B, the header data 707 of each tile, the packet group 704 constituting the resolution level 0 of each tile, and the resolution level of each tile 1 and a packet group 706 forming a resolution level 2 are stored in the cache memory of the user 201. For the other tiles, as indicated by 1001 in FIG. 10A, packet data obtained by removing the packet group 706 constituting the resolution level 2 from the cache data indicated by 1002 in FIG. Cached. When the combined size of these data is 1.95 M [bytes], the threshold value is 2 M [bytes], and the amount of received data is 120 K [bytes], the data amount Y to be deleted is Y = 1.95 + 0. .12-2.0 = 0.07M [byte] = 70K [byte]. That is, it is necessary to delete data of 70K [bytes] or more.
[0034]
Next, the process proceeds to step S12, in which the attention area F specified in step S1 of FIG. 8 is obtained. For example, when the user who is currently displaying the area indicated by the area 602 in FIG. 6 with the resolution level 2 requests the enlarged display centering on the tile 42 and desires to display the area 601 with the resolution level 3 , Nine tiles of tiles 33 to 35, 41 to 43, and 49 to 51 are obtained as the attention area F.
[0035]
Next, the process proceeds to step S13, and an area including a tile group far from the attention area F obtained in step S12 is obtained as a data deletion area R. For example, a region 603 surrounded by a broken line and configured by tiles 0 to 7 in FIG.
[0036]
Next, the process proceeds to step S14, and the order of tiles from which data is to be deleted is determined for the tiles included in the data deletion region R acquired in step S13. For example, in the case of tiles inside the area 603 in FIG. 6, numbers are assigned in order from the right tile as the order of data deletion. That is, the tile T_del (1) having the first data deletion order is the tile 7, and the tile T_del (8) having the eighth data deletion order is the tile 0.
[0037]
Next, the process proceeds to step S15, in which "1" is substituted for a variable X for counting the data deletion order as an initial value. Next, the process proceeds to step S16, in which the cache data of the tile T_del (X) whose deletion order is X is deleted, except for the packet data of the tile header portion 701 and the resolution level 0, layer 1 except for the tile data. . For example, if X = 1 and T_del (1) is tile 7, the cache data of tile 7 is the data indicated by 1001 in FIG. 10A, so that a packet group 705 constituting resolution 1 and resolution 0 , Layer 2 are deleted.
[0038]
Next, the process proceeds to step S17, and the data amount deleted in step S16 is added to the variable Del that accumulates the data amount deleted in the deletion process. Next, the process proceeds to step S18, where the data amount Del deleted in this deletion processing is compared with the data amount Y to be deleted in this deletion processing, and the data amount Del actually deleted exceeds the data amount Y to be deleted. Is determined, and if it is exceeded, this process ends. On the other hand, if it has not exceeded, the process proceeds to step S19, and it is determined whether or not the deletion number X of the tile from which data was deleted last is smaller than the number N of tiles included in the data deletion region R. If the deletion number X is smaller than the tile number N, the process proceeds to step S20. Otherwise, the cache data of all the tiles included in the deletion region R is regarded as deleted, and a new deletion region R is obtained. The process proceeds to step S13. In step S20, the deletion number X is incremented by one to be the next deletion number.
[0039]
For example, when the data amount Y to be deleted is 70K [bytes] and the data size deleted in the tile T_del (1) of the deletion number 1 is 50K [bytes], the value of the data amount Del deleted in step S17 is 50K [bytes]. Byte], so that in step S18, Y (= 70K)> Del (= 50K), and the process proceeds to step S19. In step S19, since X (= 1) <N (= 8), the process proceeds to step S20, in which the deletion number X is incremented by 1 to be changed to "2", and then proceeds to step S16, where T_del (2). The data of tile 6 will be deleted.
[0040]
[Embodiment 1]
Next, the process of acquiring the data deletion area R in step S13 of FIG. 9 will be described with reference to the flowchart of FIG. FIG. 11 is a flowchart showing a process of acquiring the deletion region R in step S13 according to Embodiment 1 of the present invention.
[0041]
First, in step S31, the tile T_cen including the center point of the attention area F specified in step S1 of FIG. 8 is specified. For the attention area F indicated by the area 601 in FIG. 6, the tile 42 is the tile T_cen. When the upper left tile of the image is defined as a 0-row, 0-column tile, and the center tile T_cen is represented by a Cy row and a Cx column, the position of the tile 42 is represented by Cx = 2 and Cy = 5. Next, the process proceeds to step S32, in which the number of tiles from the center tile T_cen specified in step S31 to tiles located at the upper, lower, left, and right ends where the cache data exists is calculated. That is, the row or column located at the end of the image has already been selected as the deletion area R in this cache control process, and if there is no cache data that can be deleted, the number of tiles is calculated excluding the row or column. .
[0042]
This will be described with reference to FIG.
[0043]
In FIG. 12, assuming that the number of tiles from the tile 42 to the upper, lower, left, and right ends is N_up, N_down, N_left, and N_right, if the deletion region R has not been selected even once,
N_up = Cy, N_down = maximum line number of image-Cy
N_left = Cx, N_right = maximum column number of image−Cx
Is calculated as If rows X to X of the image have already been selected as the data deletion area R and there is no data to be deleted, X is subtracted from N_up for adjustment. Similarly, when the line at the bottom of the image is deleted, N_down is deleted, and when the sequence at the left end and right end of the image is deleted, N_left and N_right are already deleted. Adjust by subtracting the number of columns. Therefore, if the deletion region R has not been designated yet, N_up = 5, N_down = 2, N_left = 2, and N_right = 5 for the attention region 601.
[0044]
Next, proceeding to step S33, a variable having the largest value is obtained as N_max from the four values obtained in step S33, N_up, N_down, N_right, and N_left. Here, both N_up and N_right for the attention area 601 are “5”, which is the maximum value.
[0045]
Next, the process proceeds to step S34, and the direction in which the number of tiles from the center tile T_cen to the image edge is the largest is determined from the variable (N_up, N_right) having the maximum value obtained in step S33. If N_up is selected as the variable having the maximum value, this direction is the upward direction of the image, that is, the direction in which the row number decreases. When N_right is selected, this direction is the right direction of the image, that is, the direction in which the column number increases. In addition, as an example, for example, when N_down is selected, a downward direction of the image, that is, a row number is increased, and when N_left is selected, a left direction of the image, that is, a column number is small. Direction. However, as described above, when there are two variables corresponding to N_max, the following three conditions are provided.
[0046]
1. N_up and N_down have priority over N_right and N_left.
[0047]
2. N_up has priority over N_down.
[0048]
3. N_left has priority over N_right.
[0049]
That is, the value in the vertical direction has priority over the value in the horizontal direction, and the direction in which the number is reduced in both the column and the row has priority. Accordingly, in the above-mentioned attention area 601, N_up and N_right correspond to N_max, but N_up in the vertical direction is preferentially selected.
[0050]
Next, proceeding to step S35, the row or column from which cache data is to be deleted is determined as the data deletion area R based on the direction selected in step S34. However, in this case, when N_up is selected in step S34 as described above, the row indicated by the area 1201 shown in FIG. When N_down is selected, the row indicated by the area 1202 is determined as the deletion area R. Further, when N_left is selected, the area 1203 is determined as the deletion area R, and when N_right is selected, the area 1204 is determined as the deletion area R.
[0051]
Therefore, in this step S35, since N_up has been selected in step S34, the top row corresponding to the area 1201 is determined as the deletion area R for the attention area 601.
[0052]
In this way, the process proceeds to step S14 in FIG. 9, and a number is basically assigned to the data deletion region R so that data is deleted from a tile as far as possible from the attention region F as far as possible.
[0053]
FIG. 13 is a flowchart illustrating a method of determining the order of deleting cache data (step S14: FIG. 9).
[0054]
First, in step S41, it is checked which of N_up, N_down, N_right, and N_left has been selected as the value corresponding to N_max in step S33 of FIG. If one of N_up and N_down is selected as N_max, the process proceeds to step S42, and if one of N_right and N_left is selected, the process proceeds to step S45. That is, when a row-direction tile group is selected as the deletion region R, the process proceeds to step S42, and when a column-direction tile group is selected, the process proceeds to step S45. In the first embodiment, since N_up is selected as N_max, the process proceeds to step S42.
[0055]
In step S42, the values of N_right and N_left are compared to determine which value is larger. If the value of N_right is large, the process proceeds to step S43, and if the value of N_left is large, the process proceeds to step S44. That is, if the number of tiles on the right side is large, the process proceeds to step S43, and if the number of tiles on the left side is large, the process proceeds to step S44. When the process proceeds to step S43, a row of tiles arranged in the horizontal direction is selected as the deletion region R, and the attention region F is shifted to the left, so that the tile from the right end of the deletion region R moves leftward. Assign data deletion numbers in order. On the other hand, when the process proceeds to step S44, a row of tiles arranged in the horizontal direction is selected as the deletion region R, and the attention region is shifted to the right side. Assign a data deletion number in order to.
[0056]
In the first embodiment, since N_left = 2 and N_right = 5, the process proceeds from step S42 to step S43. In the region 1201 of FIG. Will be shaken. That is, the tile 7 is assigned a deleted data number “1”, the tile 6 is assigned a deleted data number “2”, and so on, and the tile 0 is assigned a deleted data number “8”.
[0057]
When the process proceeds from step S41 to step S45, the values of N_down and N_up are compared to determine which value is larger. If N_down is larger, the process proceeds to step S46. If N_up is larger, the process proceeds to step S47. That is, if the number of rows of tiles below the attention area F is large, the process proceeds to step S46. If the number of rows of tiles above the attention area F is large, the process proceeds to step S47. When the process proceeds to step S46, a row of tiles arranged in the vertical direction is selected as the deletion region R and the attention region F is shifted upward, so that the tiles at the lower end of the deletion region R are sequentially moved upward from the tiles. Assign a data deletion number. On the other hand, when the process proceeds to step S47, a row of tiles arranged in the vertical direction is selected as the deletion region R, and since the attention region F is shifted downward, the tile from the upper end tile of the deletion region R Data deletion numbers are sequentially assigned in the direction.
[0058]
Therefore, in the case of the first embodiment, the deletion area R is a row including the tiles 0 to 7, the deletion order “1” is the tile 7, the deletion order “2” is the tile 6, and so on. Are sequentially deleted, and the deletion number “8” is the tile 0.
[0059]
FIGS. 14A and 14B are diagrams illustrating a state in which the tile data of the deletion area R is cached. Assuming that each tile is cached with data up to resolution level 2 and layer 2 as indicated by 1402 in FIG. 14A, the tiles shown in FIG. Of the 7, 6, and 5 cache data, a total of 165 K [bytes] of data indicated by 1402 will be deleted. Therefore, the cache data after deletion is in a state as shown by 1601 in FIG.
[0060]
By adopting such a data deleting method, the tile data of the partial area corresponding to the attention area important for the current display is retained, and the attention is considered to be relatively less important for the current display. Cache data is deleted from tiles far from the area.
[0061]
Further, since packet data other than the data of the resolution level 0 and the layer 1 having the smallest data amount is deleted for each tile, a large-capacity empty area can be provided in the cache memory. Is reduced.
[0062]
In addition, by retaining the data of the resolution level 0 and the layer 1 of each tile without deleting them, there is no tile in which no data is cached at all. Thus, even if a higher-resolution image display is requested, the image decoded from the stored data is temporarily enlarged and displayed, and after the image display request, the image data of the area is displayed. Until reception, it is possible to prevent a situation where no image is displayed. In addition, there is an effect that an image which can grasp the entire image can be displayed immediately at any time.
[0063]
The order in which the cache data is deleted is not fixed to the order described in the above embodiment, and it can be easily inferred that another order may be added.
[0064]
Further, in the first embodiment, for simplicity of description, the deletion area is specified in tile units. However, the present invention is not limited to this, and may be in Precinct or code-block units, for example. In the case of Precinct, the unit of data deletion is a packet as in the case of a tile, and in the case of code-block, the unit is code data composed of subbands constituting the data of the code block.
[0065]
[Embodiment 2]
[Delete data of specified tile in packet unit]
In the first embodiment described above, a row or a column composed of tiles far from the attention area is designated as the deletion area R, and a deletion order is assigned to the tiles therein, and the tiles of the deletion area R are assigned All the packet data except the data of resolution level 0 and layer 1 were deleted one by one according to the deletion order.
[0066]
However, the overall SNR and resolution of all the tiles included in the area are not set as a unit but for all the packets except for the resolution level 0 and the resolution 1 included in the cache data of the tile specified as the deletion target. A method of lowering the cache data amount and reducing the cache data amount is also conceivable. In this case, the difference from the first embodiment is the step S16 and subsequent steps in the data deletion processing of FIG.
[0067]
FIG. 15 is a flowchart illustrating the deletion processing according to the second embodiment. Steps that perform the same operations as in FIG. 9 described above are given the same reference numerals as in FIG. 9, and description thereof will be omitted.
[0068]
In the second embodiment, it is assumed that the user has already cached the entire image as data of the maximum SNR of resolution level 2. Accordingly, in the cache memory, as shown by 1002 in FIG. 10B, the header data 707 of each tile and the packet constituting the resolution level 0 of each tile are stored for the data 701 of the header portion of the image and all tiles. A group 704, a packet group 705 (FIG. 10A) configuring a resolution level 1 of each tile, and a packet group 706 configuring a resolution level 2 are stored in the cache memory of the user 201. In addition, the user who has displayed the area 602 in FIG. 6 at the resolution level 2 requests an enlarged display centering on the tile 42, desires to display the area 601 at the resolution level 3, and sets the tiles 33 to It is assumed that nine tiles 35, 41 to 43, and 49 to 51 have been acquired. Steps S11 to S14 are the same as in FIG. 9 of the first embodiment.
[0069]
That is, in step S11, the data amount Y to be deleted from the cache is acquired, and “0” is substituted for the variable Del that accumulates the data amount deleted in the current data deletion process. In step S12, the attention area F specified in step S1 is obtained. In step S13, an area including a group of tiles far from the attention area F obtained in step S12 is obtained as a data deletion area R. In the case of the second embodiment, the area 603 in FIG. In step S14, the order in which data is deleted from the tiles included in the data deletion region R acquired in step S13 is determined. In the case of the second embodiment, the deletion order “1” is tile 7, the deletion order “2” is tile 6, the tiles 5, tile 4,... It has become.
[0070]
In step S51, a tile group TR having the highest cached resolution is specified in the data deletion area R. In the second embodiment, since all tiles included in the deletion area R indicated by the area 603 are cached up to the same resolution level 2, the tile group TR matches the deletion area R. Next, the process proceeds to step S52, in which the tile TR_del having the smaller data deletion order is specified in the tile group TR specified in step S51.
[0071]
In the second embodiment, since the deletion area R is equal to the tile group TR, the tile 7 with the smallest data deletion order is specified as TR_del. Next, the process proceeds to step S53, where data of the highest resolution of the tile TR_del is deleted. Here, in tile 7 which is TR_del, the highest resolution data cached is resolution level 2 706 (FIG. 7), so this data 706 is deleted from the cache. Therefore, the cache data of the tile 7 changes from the state of 1002 in FIG. 10B to the state indicated by 1001 in FIG.
[0072]
Next, the process proceeds to step S17, and the data amount deleted in step S53 is added to the variable Del that accumulates the data amount deleted in the deletion process. Next, the process proceeds to step S18, where the deleted data amount Del is compared with the data amount Y to be deleted, and it is checked whether the actually deleted data amount Del exceeds the amount Y to be deleted. This processing ends.
[0073]
On the other hand, if not exceeded, the process proceeds to step S54, and the tile TR_del from which the highest-resolution cache data has been deleted in step S53 is removed from the tile group TR specified in step S51. That is, the tile 7 is removed from the tile group TR, and seven tiles 0 to 6 remain in the tile group TR. Next, the process proceeds to step S55, and it is determined whether or not the tile group TR has become an empty set. If the tiles included in the tile group TR still remain, the process returns to step S52 to identify the next TR_del and execute the above-described processing.
[0074]
On the other hand, the process proceeds to step S56 where no tiles are included in the tile group TR, and it is determined whether or not only the resolution level 0 and the layer 1 are the cache data of all the tiles included in the deletion area R. Here, if the cache data of all tiles is resolution level 0, layer 1, it is determined that there is no longer any cache data that can be deleted in this deletion area R, and the process returns to step S13 to return to the next data deletion area R. To get. On the other hand, in step S56, if the cache data in the deletion area R is not only the resolution level 0 and the layer 1, it is determined that there is still cache data that can be deleted, and the process returns to step S51 to execute the above-described processing.
[0075]
According to the above-described second embodiment, the deletion region R is a row including tiles 0 to 7, the deletion order “1” is tile 7, the deletion order “2” is tile 6, and so on. The deletion order “8” is the tile 0 in order. Then, as shown by 1402 in FIG. 14A, assuming that each tile has cached data up to resolution level 2 and layer 2, in order to delete 120K [bytes] of data, tiles 7 and 6 are needed. , 5 and 4, the data of resolution level 2 is deleted. That is, data of 30K [bytes] × 4 tiles = 120K [bytes] is deleted, and the cache data is in a state as indicated by 1602 in FIG. 16B.
[0076]
Therefore, according to the second embodiment, data is deleted little by little in the resolution direction from all the tiles included in the deletion area R. Therefore, even if the area included in the deletion area R is browsed, it is included in that area. It is more likely that more encoded data remains for each tile, and it is also possible to reduce the number of packet data requested corresponding to a newly requested image.
[0077]
Further, in the second embodiment, the case where the data of the deletion region R is deleted in the resolution direction has been described. However, the method of deleting the data in the SNR direction can be essentially realized in the same manner. In this case, the tile group TR specified in step S51 is a set of tiles having the highest SNR, that is, the number of cached layers. In step S53, the data with the highest SNR in the tile TR_del is deleted. For example, when there is cache data as indicated by 1002 in FIG. 10B, since the data with the highest SNR in the tile is deleted, the packet data 1006, 1005, and 1004 constituting layer 2 (FIG. B)) will be erased.
[0078]
In addition, the amount of data to be deleted when the resolution level data is deleted is compared with the amount of data to be deleted when the SNR data is deleted. For example, when 18 packets indicated by 1002 in FIG. 10B are stored and when deleting in the resolution level direction, 6 packets indicated by 706 in FIG. 7 are deleted. , And in the layer direction, nine packets 1004, 1005, and 1006 are deleted. Therefore, the data amount of the six packets indicated by 706 in FIG. 7 is compared with the data amount of the nine packets 1004, 1005, and 1006, and the one with the larger data amount is deleted.
[0079]
Further, in the second embodiment, for simplicity of description, the deletion area is specified in tile units. However, the present invention is not limited to this. For example, the same applies to Precinct and code-block units. The cache data can be deleted based on the above concept. For example, when Precinct is used as a unit, the data deletion unit is a packet like a tile, and when code-block is used, code data consisting of subbands constituting data of the code block is used as a unit.
[0080]
[Embodiment 3]
[Delete last tile used for display from the oldest tile data]
In the first and second embodiments, the tile from which the cache data is to be deleted is selected purely based on the distance from the attention area F. However, the time last used for display may be stored for each tile, and the tile with the oldest access time may be determined as the deleted data area. In order to delete based on the displayed time, a time management table for managing the time used for the display is required for each tile.
[0081]
FIG. 17 is a flowchart illustrating a process of creating a management table according to Embodiment 3 of the present invention.
[0082]
In step S61, the main header (main header) 701 of the JPEG2000 image to be browsed is requested to the server 204, and the main header transmitted from the server 204 is received in response to the request. Next, in step S62, the received main header is analyzed, and the number of tiles constituting the JPEG2000 image to be viewed is calculated. This is based on the parameters Xsiz (1801) and Ysiz (1802) indicating the image size of the highest resolution described in the SIZ marker of the main header 701, and the parameters XTsiz (1803) and YTsiz (1804) indicating the tile size at the highest resolution. Can be calculated.
[0083]
In the third embodiment, as shown in FIG. 6, an original image of 2048 × 2048 [pixels] is divided into tiles of 256 × 256 [pixels], so that (2048 ÷ 256 =) 8 tiles are arranged in the horizontal direction. Since (2048２０256 =) 8 tiles are similarly arranged in the vertical direction, a total of (8 × 8 =) 64 tiles are calculated.
[0084]
Next, the process proceeds to step S63, and an access management table for each tile is created based on the number of tiles calculated in step S62. For example, an access management table as indicated by 1901 in FIG. 19A is created. In the initial state, no tile is used for displaying an image yet, so that “Last Access Time (latest access time)” of all tiles contains an initial value, for example, NULL.
[0085]
Next, the process proceeds to step S64, where packet data is requested according to the display request of the user. For example, for all tiles, data up to resolution level 1 and layer 2 is requested. Next, the process proceeds to step S65, and the access management table is updated for the tile requested in step S64. For example, when the data of the resolution level 1 and the layer 1 is requested for all the tiles, as shown by 1902 in FIG. 19B, the item of “Last Access Time” in the table is displayed for all the tiles. Is updated at the time “10:22:34” when the access request is made.
[0086]
Next, proceeding to step S66, it is determined whether it is necessary to issue a request to the server 204 for the packet data requested in step S64. The request in step S64 can be satisfied with the cached packet data. If the data request is not issued to the server 204, the process proceeds to step S69.
[0087]
If the request in step S64 cannot be satisfied with only the cached data, the process proceeds to step S67. For example, at the stage of the first image display, since packet data of any tile has not been received yet, it is necessary to request data from the server 204, so that the process proceeds to step S67. In this step S67, the packet data is requested from the server 204, and the packet data transmitted in response to the request is received. Next, the process proceeds to step S68, and the data received in step S67 is cached. Next, the process proceeds to step S69, where the JPEG2000 bit stream created from the cache data is decoded and displayed. Next, the process proceeds to step S70, in which it is determined whether or not the request for this image has been completed. If the request still continues, the process returns to step S64. If the request for this image has been completed, this processing flow ends.
[0088]
FIG. 20 is a flowchart showing the cache control process in step S68 of FIG. This flowchart differs from the flowchart of FIG. 8 of the first embodiment in that a tile to be deleted is specified based on the value of the access management table. Therefore, those performing the same operations as in FIG. 8 in the steps of FIG. 20 are given the same numbers as in FIG.
[0089]
First, in step S2, the memory capacity of the user 201 is obtained, and a threshold value for determining whether or not the cache data needs to be deleted is determined. Next, in step S3, the number of bytes of the received fragmented JPEG2000 code data is obtained. Next, proceeding to step S4, it is determined whether or not the byte count of the received data acquired in step S3 exceeds the threshold value determined in step S2. That is, if the value obtained in step S3 is larger than the value obtained by subtracting the currently cached data amount from the threshold value, it is determined that the threshold value is exceeded. If the value obtained in step S3 is smaller, Judge not to exceed the threshold value. When it is determined that the threshold value is exceeded, the process proceeds to step S71, and when it is determined that the threshold value is not exceeded, the process proceeds to step S6.
[0090]
In step S71, the data amount Y to be deleted is calculated, and further, "0" is substituted into a variable Del for accumulating the actually deleted data amount, and the initialization is performed. Next, proceeding to step S72, the last used time is compared for each tile with reference to the access management table as shown in FIG. 19 (C), and the tile used is the one farthest from the current time. That is, the tile group X with the oldest access time is specified. For example, if the entire image is displayed at resolution level 1 and then the area 602 of FIG. 6 is enlarged and displayed at resolution level 2, as shown by 1903 in FIG. 19C, tiles 16 to 21 and tiles in the access management table are displayed. The last accessed time is updated for 36 tiles 24 to 29, tiles 32 to 37, tiles 40 to 45, tiles 48 to 53, and tiles 56 to 61. Therefore, the remaining 28 tiles are specified as a tile group X having an old access time.
[0091]
Next, the process proceeds to step S73, in which the tile A having the smallest tile number is specified from the tile group X specified in step S72, and that tile is set as a tile candidate from which cache data is to be deleted. For example, when an area other than the area 602 in FIG. 6 is specified as the tile group X, the tile 0 having the smallest tile number among them is specified as the tile A.
[0092]
Next, proceeding to step S5, as in the first embodiment, even if the number of bytes acquired in step S3 is cached, the resolution is selected from the cache data of tile A so that the cache capacity is equal to or smaller than the threshold value. All cache data except for packets forming level 0 and layer 1 is deleted. Next, proceeding to step S74, the data amount deleted in step S5 is added to the variable Del that accumulates the actually deleted data amount. Next, the process proceeds to step S75, where the data amount Y to be deleted is compared with the deleted data amount Del. If the deleted data amount Del is larger than the data amount Y to be deleted, the process proceeds to step S6, and the next reception is performed. Cache the data you want.
[0093]
On the other hand, if the deleted data amount Del is smaller in step S75, the process proceeds to step S76, the tile A from which cache data has been deleted is removed from the tile group X in step S5, and the process returns to step S73 to specify the tile A. Do it again.
[0094]
In the third embodiment, tables such as those shown in FIGS. 19A to 19C are used as the access management table, but the present invention is not limited to this table.
[0095]
Further, in the third embodiment, for simplicity of description, the deletion area is specified in tile units. However, the present invention is not limited to this. For example, the same concept can be applied to Precinct and code-block. The cache data can be deleted based on the cache data. For example, in the case of Precinct, the data deletion unit is a packet as in the case of a tile, and in the case of code-block, the unit is code data composed of subbands constituting the data of the code block.
[0096]
Further, in the third embodiment, the last access time is stored, and the tile from which the cache data is to be deleted is specified based on the time. However, the tile is used in combination with each of the first to third embodiments. It is also possible. That is, when selecting the tile A from which the cache data is to be deleted in step S72 from the tile group X specified in step S71, the tile farthest from the attention area F is selected as in the first embodiment. It may be specified as tile A.
[0097]
Also, as for the unit for deleting the cache data, only the data with the highest resolution is deleted from the cache data of the tile A specified in step S72, or the data with the highest SNR, as in the second embodiment. It is also possible to delete only.
[0098]
As for the cache method in the first to third embodiments, any method may be used as long as the cached packet data can be individually deleted.
[0099]
[Embodiment 4]
First, the flow of the entire process applied to the first to third embodiments will be described with reference to the flowchart in FIG. This process is executed by issuing a display request from the user 201 (202) to the server 204, and is a prerequisite for the cache process described above.
[0100]
First, in step S81, a display request from a user is received. In the case of the fourth embodiment, displaying the area 601 shown in FIG. 6 at the maximum SNR and the resolution level 3 is a display request by the user. Next, proceeding to step S82, the request accepted in step S81 is compared with the cache data, and it is determined whether or not sufficient data to satisfy the request is stored in the cache. If the cache contains all necessary data, the process proceeds to step S87, and an image is displayed using the cached data. However, if there is insufficient data to display an image corresponding to the display request, the process advances to step S83 to request the server 204 for the missing image data.
[0101]
In the fourth embodiment, the cache memory includes data 701 of a header portion of an image, and 36 tiles 602 in an area 602 surrounded by tiles 16 to 21 and 56 to 61, and forms a resolution level 2 shown in FIG. As for the data group indicated by 1002 in B) and the other tiles, the data group indicated by 1001 in FIG. 10A is cached. Accordingly, in order to satisfy the display request acquired in step S81, the packet data group 708 of the resolution level 3, layer 2 of the nine tiles including the tiles 33 to 35, 41 to 43, and 49 to 51 included in the area 601 is set. Is insufficient, the process proceeds to step S83.
[0102]
In step S83, a request for requesting the server 204 for these missing data is created. In the fourth embodiment, a request for requesting a packet group 708 forming a resolution level 3 and a layer 2 of nine tiles included in the area 601 is created. Next, proceeding to step S84, the request created in step S83 is issued to the server 204. Next, the process proceeds to step S85, where response data from the server 204 is received. Next, the process proceeds to step S86, where the data received in step S85 is cached. Then, the process proceeds to a step S87, in which an image display process is performed using the cached data. As a result, in the fourth embodiment, the area 601 is displayed as the resolution level 3 and the layer 2.
[0103]
Further, as for the cache method in the above-described embodiment, any method may be used as long as it is a method that can individually delete each cached packet data. However, since the purpose of any cache method is to keep the cache capacity within the limit, it is necessary to physically delete the data from the memory.
[0104]
For example, as shown in FIG. 21, one file is formed for each data deletion unit such as a tile, a packet, and a code block, and a management file for managing them is created. For example, a method of deleting a file that caches the corresponding data and updating the management file can be considered. In the case of such a cache creation method, it is possible to easily delete the cache data physically from the cache memory.
[0105]
Alternatively, as indicated by reference numeral 2201 in FIG. 22A, a method may be used in which a plurality of deletion units such as tiles, packets, and code blocks are collectively stored in one file. In this case, since the cache data is managed by one file, the data management is simplified. On the other hand, in order to delete the data to be deleted, the data is deleted as shown by 2203 in FIG. It is necessary to delete the blank part that remains physically after the operation. Therefore, in the example of FIG. 22, it is necessary to move the subsequent data by Y [bytes] corresponding to 2203 in which Tile N and Packet K are stored.
[0106]
FIG. 24 is a flowchart illustrating the flow of the cache data deletion process in this case.
[0107]
First, in step S91, a file name of a cache from which data is to be deleted is stored. For example, if the name of the cache file from which data is to be deleted is “abc.ip2k”, the name is stored. Next, the process proceeds to step S92, where a Temp file is created. This Temp file is finally the file after the cache data is deleted, but in order to avoid the same name, a file given a temporary name such as “temp.ip2k” or “abc_tmp.iip2k” is used. create. Here, it is assumed that “temp.ip2k”.
[0108]
Then, the process proceeds to a step S93, wherein a cache file from which data is to be deleted is opened. Next, the process proceeds to step S94, where the offset value V to the cache data to be deleted and the length L of the deleted data are acquired.
[0109]
In the example of FIG. 22, since it is desired to delete the data of Tile N and Packet K, V = X [bytes] and L = Y [bytes]. Next, the process proceeds to step S95, where data of V [bytes] from the beginning of the cache file is copied from the beginning of the Temp file. Therefore, data including the cache management data 2204 is copied to the Temp file. Next, the process proceeds to step S96, in which a portion from the (V + L) [byte] from the beginning of the cache file to the end of the cache file is copied after the Temp file copied in step S95. Next, the process proceeds to step S97, where both the Temp file and the cache file are closed. Then, the process proceeds to a step S98, where the cache file is deleted. Therefore, at this point, “abc.ip2k” is deleted, and the cache data is stored in the file “temp.ip2k” of the Temp file. Next, the process proceeds to step S99, and the name of the Temp file is changed to the file name saved in step S91, for example, “abc.ip2k” in this example.
[0110]
This makes it possible to physically reduce the file size. However, if this method is adopted, two files of substantially the same size exist temporarily in the cache memory in step S97. Therefore, when determining the threshold value from the cache memory capacity in step S1 in FIG. 8, it is necessary to set the threshold value to about half of the usable memory capacity in consideration of this point. Various other cache creation methods are conceivable, but are omitted because they are not the main application of the present invention.
[0111]
As described above, according to Embodiment 4, cached fragmentary code data can be efficiently deleted, and when browsing the original image, the size of the code data of the same size as the original image is reduced. Does not require memory capacity.
[0112]
After the cached data is deleted, even if the user makes a display request that requires the data again, at least the resolution level 0 and the layer 1 data are retained without being deleted. During the request, data temporarily stored in the cache can be decoded and displayed.
[0113]
[Other embodiments]
As described above, an object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the present embodiment to a system or an apparatus, and to provide a computer (or CPU or MPU) of the system or the apparatus. Is also achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. can be used.
[0114]
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also an OS (Operating System) running on the computer based on the instruction of the program code. Performs some or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.
[0115]
Further, after the program code read from the storage medium is written into the memory provided on the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function is executed based on the instruction of the program code. This also includes the case where the CPU provided in the expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.
[0116]
The above-described image processing apparatus and method according to the present embodiment can be represented by the following embodiments.
[0117]
[Embodiment 1] An image processing method for inputting, caching, and reproducing encoded image data in a fragmentary manner,
An attention area specifying step of specifying an attention area of the image;
A determination step of determining whether or not the amount of cache data that caches the image data exceeds an allowable amount of a cache memory by changing the attention area;
A deletion data amount determination step of determining a data amount to be deleted from the cache data stored in the cache memory when it is determined that the data amount exceeds the determination step;
A distance measurement step of measuring a distance from the attention area specified in the attention area identification step to a unit area to be deleted;
A deletion area determination step of determining a deletion area for deleting data from cache data in units of the unit area based on the attention area and the distance measured in the distance measurement step;
A deletion step of deleting cache data included in the deletion area determined in the deletion area determination step,
An image processing method comprising:
[0118]
[Embodiment 2] An image processing method for inputting, caching and reproducing encoded image data in a fragmentary manner,
An attention area specifying step of specifying an attention area of the image;
A determination step of determining whether or not the amount of cache data that caches the image data exceeds an allowable amount of a cache memory by changing the attention area;
A deletion data amount determination step of determining a data amount to be deleted from the cache data stored in the cache memory when it is determined that the data amount exceeds the determination step;
An access time management step of managing the use status of the unit area constituting the image data,
A deletion area determining step of determining a deletion area for deleting data from cache data in units of the unit area, based on the attention area and the use status;
A deletion step of deleting cache data included in the deletion area determined in the deletion area determination step,
An image processing method comprising:
[0119]
[Embodiment 3] In the determining step, based on a remaining amount of the cache memory for caching the image data and an amount of data for displaying the changed area of interest, whether the amount exceeds the allowable amount of the cache memory is determined. 3. The image processing apparatus according to claim 1, wherein
[0120]
[Embodiment 4] The image processing method according to any one of Embodiments 1 to 3, wherein the image data is JPEG2000 encoded data based on ISO / IEC-15444.
[0121]
[Embodiment 5] The image processing method according to Embodiment 4, wherein the image data is input in units of JPEG2000 fragmented packets.
[0122]
[Sixth Embodiment] The image processing method according to the fourth embodiment, wherein the unit area corresponds to a JPEG2000 tile.
[0123]
[Seventh Embodiment] The image processing method according to the fourth or fifth embodiment, wherein in the deleting step, the packet is deleted in packet units.
[0124]
[Eighth Embodiment] The image processing method according to the fourth embodiment or the fifth embodiment, wherein in the deleting step, data of all packets except for the packets of the resolution level 0 and the layer 1 in the deleted area is deleted.
[0125]
[Embodiment 9] The image according to Embodiment 8, wherein the unit area is a packet excluding the resolution level 0 and the layer 1 and is selected in packet units and is deleted at least once or several times. Processing method.
[0126]
[Embodiment 10] The image processing method according to Embodiment 4 or 5, wherein the unit area is a JPEG2000 precinct.
[0127]
[Embodiment 11] The image processing method according to embodiment 10, wherein the precinct is deleted in packet units in the deletion step.
[0128]
[Embodiment 12] The image processing method according to embodiment 4 or 5, wherein the unit area is a JPEG2000 code-block.
[0129]
[Thirteenth Embodiment] The twelfth embodiment is characterized in that the unit area is code-block data composed of data obtained by removing an LL component from subbands necessary to form the same resolution and the same layer. The image processing method according to 1.
[0130]
[Embodiment 14] An image processing apparatus for inputting, caching, and reproducing encoded image data fragmentarily,
Attention area specifying means for specifying an attention area of an image;
Determining means for determining whether or not the amount of cache data caching the image data exceeds an allowable amount of a cache memory by changing the attention area;
Deletion data amount determination means for determining the data amount to be deleted from the cache data stored in the cache memory when it is determined that the data exceeds the data,
Distance measurement means for measuring the distance from the attention area specified by the attention area identification means to the unit area to be deleted,
A deletion area determining unit that determines a deletion area for deleting data from cache data in units of the unit area based on the attention area and the distance measured by the distance measurement unit;
Deletion means for deleting cache data included in the deletion area determined by the deletion area determination means,
An image processing apparatus comprising:
[0131]
[Embodiment 15] An image processing apparatus for fragmentally inputting, caching, and reproducing encoded image data,
Attention area specifying means for specifying an attention area of an image;
Determining means for determining whether or not the amount of cache data caching the image data exceeds an allowable amount of a cache memory by changing the attention area;
Deletion data amount determination means for determining the data amount to be deleted from the cache data stored in the cache memory when it is determined that the data exceeds the data,
Access time management means for managing the use status of the unit area constituting the image data,
A deletion area determining unit that determines a deletion area for deleting data from cache data in units of the unit area based on the attention area and the usage state;
Deletion means for deleting cache data included in the deletion area determined by the deletion area determination means,
An image processing apparatus comprising:
[0132]
[Embodiment 16] The determination means determines whether or not the capacity of the cache memory is exceeded based on the remaining capacity of the cache memory for caching the image data and the data amount for displaying the changed area of interest. The image processing device according to embodiment 14 or 15, wherein
[0133]
[Embodiment 17] The image processing apparatus according to any one of Embodiments 14 to 16, wherein the image data is JPEG2000 encoded data based on ISO / IEC-15444.
[0134]
Embodiment 18 The image processing apparatus according to embodiment 17, wherein the image data is input in JPEG2000 fragmented packet units.
[0135]
[Embodiment 19] The image processing apparatus according to embodiment 17, wherein the unit area corresponds to a JPEG2000 tile.
[0136]
[Embodiment 20] The image processing apparatus according to embodiment 17 or 18, wherein the deletion unit deletes the packet.
[0137]
[Embodiment 21] The image processing apparatus according to embodiment 17 or 18, wherein the deletion unit deletes data of all packets except for the packets of resolution level 0 and layer 1 in the deletion area.
[0138]
[Embodiment 22] The image according to Embodiment 20, wherein the unit area is a packet excluding the resolution level 0 and the layer 1 and is selected in units of packets and deleted once or several times. Processing equipment.
[0139]
[Embodiment 23] The image processing apparatus according to embodiment 17 or 18, wherein the unit area is a JPEG2000 precinct.
[0140]
[Embodiment 24] The image processing apparatus according to embodiment 23, wherein the precinct is deleted in packet units by the deletion unit.
[0141]
[Embodiment 25] The image processing apparatus according to embodiment 17 or 18, wherein the unit area is a JPEG2000 code-block. [Twenty-sixth Embodiment] The twenty-fifth embodiment is characterized in that the unit area is code-block data composed of data obtained by removing an LL component from subbands necessary to form the same resolution and the same layer. An image processing apparatus according to claim 1.
[0142]
As described above, according to this embodiment, cached fragmentary code data can be efficiently deleted, and when browsing an original image on the server side, the amount of code data of the original image is the same as that of the original image. Does not require large memory capacity. Further, since a tile of interest is specified and a tile that is spatially distant therefrom or a tile whose last time displayed from the current time is temporally distant is to be deleted, no complicated calculation is required.
[0143]
Further, when the cache data is deleted, the data of the resolution level 0 and the layer 1 are always left, so that the user's request has higher resolution and higher SNR than the cached data. Also in the case of downloading from the side, there is an advantage that it is possible to temporarily display a low-resolution and low-SNR image using cached data.
[0144]
【The invention's effect】
As described above, according to the present invention, cached encoded image data is deleted in response to a user request while deleting cache data having little effect on a reproduced image to prevent data overflow in the cache. There is an effect that the display can be performed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a client or a server according to an embodiment.
FIG. 2 is a diagram illustrating an outline of a network according to the present embodiment.
FIG. 3 is a diagram illustrating general JPEG2000 encoded data.
FIG. 4 is a diagram illustrating a relationship between a resolution (image size) and a Resolution number in JPEG2000 encoded data.
FIG. 5 is a diagram showing a conceptual diagram of a request and a response in a packet unit.
FIG. 6 is a diagram illustrating a data structure of each layer of an original image.
FIG. 7 is a diagram illustrating JPEG2000 encoded data stored in a server according to the present embodiment.
FIG. 8 is a flowchart illustrating cache of encoded data according to the first embodiment of the present invention.
FIG. 9 is a flowchart showing a process of deleting cache data in step S5 of FIG. 8;
FIG. 10 is a diagram showing an example of cache data of a tile in a client according to the present embodiment.
FIG. 11 is a flowchart showing a process of acquiring a deletion area R of cache data in step S13 in FIG. 9;
FIG. 12 is a diagram illustrating an algorithm for acquiring a data deletion area R according to the first embodiment.
FIG. 13 is a flowchart illustrating a method (step S14: FIG. 9) for determining an order in which cache data is deleted according to the present embodiment.
FIG. 14 is a diagram illustrating a state in which tile data of a deletion area R is cached according to the embodiment.
FIG. 15 is a flowchart illustrating a deletion process according to the second embodiment of the present invention.
FIG. 16 is a diagram showing an example of cache data after data deletion in a deletion area R according to the second embodiment.
FIG. 17 is a flowchart illustrating a management table creation process according to the third embodiment of the present invention.
FIG. 18 is a diagram illustrating an SIZ marker in a main header of JPEG2000.
FIG. 19 is a diagram illustrating an example of an access management table according to Embodiment 3 of the present invention.
FIG. 20 is a flowchart showing a cache control process in step S68 of FIG. 17 according to the third embodiment.
FIG. 21 is a diagram illustrating a cache management file according to Embodiment 4 of the present invention.
FIG. 22 is a diagram illustrating deletion of cache management data and cache data according to Embodiment 4 of the present invention.
FIG. 23 is a flowchart illustrating an outline of processing according to a display request from a user to a server according to Embodiment 4 of the present invention.
FIG. 24 is a flowchart illustrating a cache data deletion process according to Embodiment 4 of the present invention.

Claims (1)

An image processing method for inputting, caching, and reproducing encoded image data in a fragmentary manner,
An attention area specifying step of specifying an attention area of the image;
A determination step of determining whether or not the amount of cache data that caches the image data exceeds an allowable amount of a cache memory by changing the attention area;
A deletion data amount determination step of determining a data amount to be deleted from the cache data stored in the cache memory when it is determined that the data amount exceeds the determination step;
A distance measurement step of measuring a distance from the attention area specified in the attention area identification step to a unit area to be deleted;
A deletion area determination step of determining a deletion area for deleting data from cache data in units of the unit area based on the attention area and the distance measured in the distance measurement step;
A deletion step of deleting cache data included in the deletion area determined in the deletion area determination step,
An image processing method comprising:

Images