JP2007053593A - Image processing system, method and program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing system with which transposition of data is managed during overflow and transposition unit is set adaptively, based on the utilization conditions of code data, in order to enhance the hit rate of code data of cache data in partial code data access. <P>SOLUTION: In the image processing system, partial code data of hierarchic code data stored in the storage of a server are received by a client, based on a transfer request of partial code data of hierarchic code data from the client, and the received code data are stored partially or entirely in a cache memory and are reused, where when the storage capacity of code data stored in the cache memory causes overflow; and the code data stored in the cache memory is transposed partially or entirely in units of structuring element, transposition unit is set at the code sequence (packet) unit contained at a specified level of a specified hierarchy of the hierarchic code data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing system, an image processing method, a program, and a recording medium. In particular, the present invention relates to an encoding processing apparatus having a function of accessing partial image encoded data using a cache model, and JPIP (JPEG2000 image coding system-Part9). : Interactivity tools, APIs and protocols).

  JPIP (JPEG2000 image coding system-Part9: Interactivity tools, APIs and protocols) is a method for accessing code data encoded based on the JPEG2000 specification in the server. The specification includes a cache function that saves a part of the code data on the client as a mechanism that allows the code data already received by the client side to be accessed at high speed by using it without being transferred. ing.

By the way, since the storage capacity that can be stored as a cache on the client side is limited, if it cannot be stored, a part of it is deleted and new high priority data is saved.
In a code processing system having a function of extracting and referring to partial code data of hierarchical code data stored in a storage device such as a cache as seen in the JPIP specification, hierarchical code data is configured in progression order A In many cases, code data is cut out and used at the highest hierarchical level in the progression order A. This is because it is possible to easily delete the code strings in the hierarchy below the highest hierarchy level.

  For example, when accessing (referring to) the partial code data unit corresponding to the image data corresponding to the partial area, it is easily specified that the highest hierarchical level in the progression order of the corresponding hierarchical code data is the area level. It is possible to refer to or extract the partial code data of the partial area.

(1) Prior application related to cache data replacement strategy:
Regarding the optimization of deletion of cache data at the time of cache overflow, there are the following prior applications.
According to Japanese Patent Laid-Open No. 2004-260688, image reproduction display is performed based on cache data, and the reproduction display is speeded up.
Further, in Patent Document 2, deletion is performed based on the distance from the attention area to the unit area to be deleted.
Further, Patent Document 3 deletes based on a region of interest and a moving direction.
Further, Patent Document 4 deletes based on a region of interest and an image region to which metadata is assigned.
Further, in Patent Document 5, deletion is performed based on determination of a deletion target area in image data.
Patent Document 6 does not erase the low resolution data.

(2) JPIP prior application:
In Patent Document 7, a cache manages code data in tile units. In the case of structured document image data such as image data having a common (standard) format, image data is often referred to in units of structure rather than in units of tiles.
Patent Document 8 converts a JPP-stream into a JPEG2000 code stream.
JP 2004-152172 A JP 2004-153562 A JP 2004-145760 A JP 2004-145759 A JP 2003-298847 A JP 2003-224704 A JP 2005-12686 A JP 2004-274758 A

  In the present invention, based on such a tendency, partial code data is controlled to be accessed based on the usage tendency of the hierarchical code data in the order of progression, and the code data temporarily stored in a high-speed memory such as a cache is reproduced. In use, the problem is to increase the hit rate of the code data of the cache data when accessing the code data, in connection with the process of replacing the code data stored in the cache when the cache overflows.

  Conventionally, a method has been proposed in which image code data of a region of interest or its peripheral region is left on the cache and the other is replaced (Patent Documents 2, 3, etc.). Typically, it is conceivable to make determination based on code data in units of rectangular areas (blocks). However, depending on the method of use, the unit for replacing data is a unit of rectangular areas (blocks) fixed in advance.

  However, when considering the code data to be reduced, the configuration of the code data to be used is not always fixed. For example, since only the code data of a specific frequency component is not important, there may be a case where it is removed. A problem arises in that the unit (range) of the code data corresponding to the region of interest is flexibly changed in accordance with the use situation. In particular, in the case of a JPEG2000 standard code in which code data has a hierarchical structure and can be partially deleted, how to determine the range of partial code data to be replaced becomes a problem.

Another problem is how to determine an appropriate replacement unit (range) and its efficient processing, particularly when the code data stored on the cache is replaced.
In other words, the partial code data to be accessed may be accessed in units of areas by utilizing the hierarchical structure of the code data, or may be accessed for a plurality of purposes, in order to increase the resolution. When the code data has a hierarchical structure as described above, the access unit also depends on the hierarchical structure of the code data, so that a cache data replacement process taking into account the hierarchical structure of the code data is necessary.

In general, an access unit (range) of partial code data of image data is configured by a set of minimum code strings (packets) of code data. For this reason, uploading must be performed (it cannot be hit) unless all the packets required as cache data for accessing partial code data are available.
Therefore, it is necessary to consider that a packet that does not have all the packets to be used is preferentially replaced.

  The main object of the present invention is to increase the hit rate of the code data of the cache data in the partial code data access, and in order to achieve such an object, the management of the data exchange at the time of overflow is managed by the use status of the code data ( The task was to adaptively set the replacement unit based on the estimation of usage tendency.

  In order to solve the above-described problem, the invention of the image processing system according to claim 1 is based on the transfer request of the partial code data of the hierarchical code data of the image processing apparatus (client). An image processing apparatus (client) having a function of receiving partial code data of hierarchical code data stored in a storage device, wherein a part or all of the received code data is stored in a storage device or a storage device (cache memory). When the storage capacity of the code data stored in the storage device (cache memory) has an overflow, the storage device or the storage device (cache memory) is stored in units of structuring elements. When replacing part or all of the stored code data, the replacement unit is the hierarchy specified by the hierarchical code data It characterized in that it is a code string (packet) units contained in the specified level.

According to a second aspect of the present invention, in the image processing system according to the first aspect, the replacement unit is specified based on a use frequency for each layer of code data when the layer code data is used. Features.
According to a third aspect of the present invention, in the image processing system according to the second aspect, the frequency of use of the code data for each layer is the frequency of use of the code data in the order of progression when the layered code data is used. It is estimated based on this.

According to a fourth aspect of the present invention, in the image processing system according to the first aspect, the hierarchy of the partial code data that is the replacement unit is the progression order of the code data when the configuration of the hierarchical code data is used. It is converted based on the frequency of use.
According to a fifth aspect of the present invention, in the image processing system according to the second or third aspect, in the replacement, a hierarchical level code string including a code string not stored is preferentially replaced.

A sixth aspect of the present invention is the image processing system according to any one of the first to fifth aspects, further comprising data code amount calculation means for calculating a data code amount for each hierarchical level, wherein the calculated data code Based on the quantity, it comprises means for exchanging code data in units of hierarchical levels of the hierarchical code data according to claim 1.
According to a seventh aspect of the present invention, in the image processing system according to any one of the first to sixth aspects, the hierarchical code data is data encoded based on JPEG2000 (ISO / IEC 15444-1) standard. It is characterized by.

  The invention according to claim 8 is the image processing system according to any one of claims 1 to 7, wherein the hierarchical code data is based on a JPIP (JPEG2000 image coding system-Part 9: Interactivity tools, APIs and protocols) standard, It has a partial code data access function in which a client receives partial code data that is a part of encoded image data stored in a storage device of a server.

  The invention of the image processing method according to claim 9 is the hierarchical code data stored in the storage device of the image processing apparatus (server) based on the transfer request of the partial code data of the hierarchical code data of the image processing apparatus (client). Image processing apparatus (client) image processing method for receiving the partial code data of the above, when a part or all of the received code data is stored in a storage device or storage device (cache memory) and reused, When the storage capacity of the code data stored in the storage device (cache memory) overflows, part or all of the code data stored in the storage device or storage device (cache memory) in units of structured elements Code sequence (packet) included in the specified level of the specified hierarchy of the hierarchical code data. Characterized in that it is a unit.

A tenth aspect of the present invention is a program for causing a computer to realize the functions of the image processing system according to any one of the first to eighth aspects.
The invention described in claim 11 is a computer-readable recording medium in which the program according to claim 10 is recorded.

According to the present invention, the code data (code string) on the client side cache and the server side original code data can be efficiently collated, and even the partial code data is based on the hierarchical structure. Can be easily verified.
In addition, even fragmented code strings stored in the cache in a non-continuous manner can be efficiently and reliably checked against code data, and the range of data (code strings) to be determined is limited. And can be collated efficiently.

DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the drawings.
First, JPEG2000 and JPIP will be described.

(1) Encoding with JPEG2000:
Encoding according to the JPEG2000 (ISO / IEC 15444-1) standard is performed in the following procedure.
The frame data of the interlaced image is converted into data for each color component of Y, Cr, and Cb, the color data of each color component is subjected to two-dimensional discrete wavelet transform, and the obtained wavelet coefficient is a scalar defined in JPEG2000. A quantization process is performed, and entropy coding processing (arithmetic coding processing by so-called coefficient modeling) defined in JPEG2000 is performed on scalar quantized data, and then a code string defined in JPEG2000 is generated.
The decoding process is the reverse procedure.

  Of course, these processes may be realized by a hardware circuit. The processing speed can be increased. Note that there is already an image processing apparatus that implements all the encoding processing compliant with JPEG2000 with a hardware circuit.

FIG. 1 is a diagram for explaining a hierarchical encoding algorithm that is the basis of JPEG2000. It comprises a color space conversion / inverse conversion unit, a two-dimensional wavelet transform / inverse conversion unit, a quantization / inverse quantization unit, an entropy encoding / decoding unit, and a tag processing unit.
One of the biggest differences compared to the JPEG algorithm is the conversion method. Discrete cosine transform (DCT) is used in JPEG, and discrete wavelet transform (DWT) is used in the hierarchical encoding compression / decompression algorithm.
The advantage that DWT has better image quality in the high compression region than DCT is one of the major reasons adopted by JPEG2000, which is a successor algorithm of JPEG.

Another major difference is that, in the latter case, a function block called a tag processing unit is added in order to perform code formation at the final stage. In this part, compressed data is generated as a code stream during the compression operation, and a code stream necessary for decompression is interpreted during the decompression operation. And with the codestream, JPEG2000 can realize various convenient functions.
For example, as shown in FIG. 2, the compression / decompression operation of a still image can be freely stopped at an arbitrary hierarchy (decomposition level) corresponding to octave division in block-based DWT. .

  A color space conversion unit is often connected to the input / output portion of the original image. For example, the RGB color system composed of R (red) / G (green) / B (blue) components of the primary color system and the Y (yellow) / M (magenta) / C (cyan) components of the complementary color system This corresponds to the part that performs conversion from the YMC color system consisting of the above to the YUV or YCbCr color system or the reverse conversion.

Hereinafter, the JPEG2000 algorithm will be described in detail.
As shown in FIG. 3, in a color image, each component (here, RGB primary color system) of an original image is generally divided by a rectangular area (tile). Each tile, for example, R00, R01, ..., R15 / G00, G01, ..., G15 / B00, B01, ..., B15, is a basic unit for executing the compression / decompression process. Therefore, the compression / decompression operation is performed independently for each component and for each tile.

  At the time of encoding, the data of each tile of each component is input to the color space conversion unit of FIG. 1 and subjected to color space conversion, and then the two-dimensional wavelet conversion unit applies the two-dimensional wavelet conversion (forward conversion). Space is divided into frequency bands.

  FIG. 2 shows subbands at each decomposition level when the number of decomposition levels is three. That is, the tile original image (0LL) (decomposition level 0) obtained by the tile division of the original image is subjected to two-dimensional wavelet transform, and the subbands (1LL, 1HL, 1LH shown in the decomposition level 1) , 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transformation to separate the subbands (2LL, 2HL, 2LH, 2HH) indicated by the decomposition level 2.

  Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate subbands (3LL, 3HL, 3LH, 3HH) shown in the decomposition level 3. Further, in FIG. 2, the subbands to be encoded at each decomposition level are represented in gray. For example, when the number of decomposition levels is 3, the subbands shown in gray (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) are to be encoded, and the 3LL subband is encoded. Not.

Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits in the quantization unit in FIG.
The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each subband. This was introduced to use memory efficiently in implementation. As shown in FIG. 4, one precinct consists of three rectangular regions that are spatially coincident. Furthermore, each precinct is divided into rectangular “code blocks” that do not overlap. This is the basic unit for entropy coding.

  The entropy encoding unit (see FIG. 1) encodes the tile of each component by probability estimation from the context and the target bit. In this way, encoding processing is performed in tile units for all components of the original image.

  The minimum unit of code data formed by the entropy encoding unit is called a packet. The packets are sequenced in progressive order, which is indicated by one of the image header segments. A packet is a piece of progressive data, and is arranged by region, resolution, layer, and color component, respectively. That is, in the JPEG2000 standard, by changing the priority order of the four image elements of image quality (layer (L)), resolution (R), component (C), and position (precinct (P)), the following 5 Street progression is defined.

LRCP progression: Since decoding is performed in the order of precinct, component, resolution level, and layer, the image quality of the entire image is improved each time the layer index is advanced, and the progression of the image quality can be realized. Also called layer progression.

RLCP progression: Since it is decoded in the order of precinct, component, layer, and resolution level, resolution progression can be realized.

RPCL progression: Since decoding is performed in the order of layer, component, precinct, and resolution level, it is a progression of resolution as in RLCP, but the priority of a specific position can be increased.

PCRL progression: Since decoding is performed in the order of layer, resolution level, component, and precinct, decoding of a specific part is prioritized, and progression of spatial position can be realized.

CPRL Progression: Since decoding is performed in the order of layer, resolution level, precinct, and component, for example, when a color image is progressively decoded, a component progression that first reproduces a gray image can be realized.

  As described above, in the JPEG2000 standard, an image is divided into regions (image constituent elements such as tiles or precincts), resolutions, hierarchies (layers), and color components, and each is independently encoded as a packet. These packets are characterized in that they can be identified and extracted from the codestream without decoding.

  Finally, the tag processing unit (code string forming unit) combines all the encoded data from the entropy encoding unit into one code stream and performs processing for adding a tag thereto. FIG. 5 simply shows the structure of the code stream. Tag information called a header is added to the head of the code stream and the head of the partial tiles constituting each tile, and the encoded data of each tile follows. A tag is placed again at the end of the code stream.

  On the other hand, at the time of decoding, contrary to the time of encoding, image data is generated from the code stream of each tile of each component. This will be briefly described with reference to FIG. In this case, the tag processing unit interprets tag information added to the code stream input from the outside, decomposes the code stream into code streams of each tile of each component, and for each code stream of each tile of each component. Decryption processing is performed. The position of the bit to be decoded is determined in the order based on the tag information in the code stream, and the context is determined by the inverse quantization unit from the sequence of the peripheral bits (that has already been decoded) of the target bit position. Is generated.

  The entropy decoding unit performs decoding from this context and code stream by probability estimation, generates a target bit, and writes it in the position of the target bit. Since the data decoded in this way is spatially divided for each frequency band, by performing two-dimensional wavelet inverse transformation on the two-dimensional wavelet inverse transformation unit, each tile of each component of the image data becomes Restored. The restored data is converted into original color system data by the color space inverse conversion unit.

(2) Code data exchange in JPIP:
The international standard JPIP (JPEG2000 image coding system-Part 9: Interactivity tools, APIs and protocols-JPEG2000 coding-JPEG2000 coding-JPEG2000 coding-JPEG2000 coding-P9 Interactivity tools, APIs and protocols "ISO / IEC FDIS 15444-9: 2004"]).

  Such a protocol for partially accessing a hierarchical image can be seen in FlashPix, which is a multi-resolution representation of an image, and IIP (Internet Imaging Protocol), which is a protocol for accessing the protocol. it can. There is Patent Document 1 as a patent document related to JPIP.

In JPIP, in response to a request (request) from a client, the server returns code data having a complete image file, tile part stream (JPT-stream) or precinct stream (JPP-stream) format (response). To do).
The JPT-stream is a method for returning code data of a sub-image as a set of tile parts defined by the JPEG2000 part 1 standard.
On the other hand, the JPP-stream is a method for returning the code data of the sub-image as a precinct set defined by the JPEG2000 part 1 standard.

  When the code data sequentially accessed by the JPT-stream is reproduced, the tile area is completely reproduced for each tile area, whereas the code area sequentially accessed by the JPP-stream is reproduced by the tile area. It is possible to reproduce gradually in small areas (precinct) in a wide area extending over the area. That is, in the reproduction by accessing the JPT-stream, the range to be reproduced at one time is limited to the tile area in the entire image, whereas in the reproduction by accessing the JPP-stream, the range to be reproduced at one time is the tile. It is possible to reproduce the entire image without being limited to the area.

  In this way, a JPIP client can identify portions of the received code data, decode those portions and generate an image, in the form of a tile part stream or precinct stream, Provides the ability to return a subset of the JPEG2000 codestream.

  That is, the JPP-stream is converted into a JPEG2000 code stream and can be decoded by a JPEG2000 decoder. It is sufficient for this conversion to receive all of the main header and all of the tile headers along with the data to be decoded.

  A JPP-stream is composed of a sequence of messages (see FIG. 6). Each message indicates the format of the included data-bin information (main header, metadata, precinct, or tile header). Each message also indicates the starting position in the data-bin and the length of the message. Each message has a header that identifies its type, an index within that type, an offset of the message to a “data-bin” with that type and index, and a message length.

  A message typically does not include all of the data-bins, but includes a portion of the data-bins. The message includes an identifier that indicates which tile, component, position, and resolution data bin the data-bin is for. In the present invention, the tile, component, position, and resolution are hierarchies, and the code sequence (packet) included in the data-bin corresponds to the information of which level (hierarchy level) of each hierarchy corresponds. Yes. For example, it is indicated that a certain code string included in the data-bin is tile A, the component is Cb, and the resolution is 1 level at position 4.

  The precinct data-bin is composed of a series of packet headers (PH) and packet data (PD) (see FIG. 7). The packet header includes information regarding which code block has data in the packet data portion of the packet, information regarding the number of bytes stored in each code block, and the number of coding passes for each code block. In the standard specification of JPIP, the packet header does not have information (hierarchy level information) for identifying which component, position, resolution, tile, or layer to which the packet header belongs.

  As an embodiment of the present invention, the message data shown above may be analyzed and the layer level information in units of packets may be recorded in the packet header. After receiving the JPP-stream on the client side, these pieces of information can also be recorded.

Similarly, the standard specification of JPIP does not have clear information regarding the length of the packet header. The length of the packet data can be determined by adding the data length to all code blocks in the packet.
As an embodiment of the present invention, the packet header can be configured to store the result of calculating these data lengths. Packets can be extracted by the configuration defined in the JPP-stream.

The JPIP protocol has two different types of requirements. There are stateless requests and other requests in the sense that the server does not need to maintain a cache model.
The server cache model records information about what has been sent to the server so far.
Also, the code data received from the server is stored in the client cache, and the server can reuse the code data already transferred without transferring it to the client by using the cache model. Typically, the client appends the JPEG2000 code data received in pieces in the order of reception and stores the data in the cache. Subsequently, a bitstream compliant with the JPEG2000 syntax is created from the cached data as described above, and the bitstream is decoded.

(3) Client, server, and network in JPIP:
FIG. 8 is a configuration example of a client server model having a cache in JPIP, and FIG. 9 is a configuration example of a client server network model in JPIP.

  FIG. 8 shows an embodiment of a network environment in which code data is communicated between a client and a server. As shown in FIG. 8, the server 102 is connected to at least one client such as the client 101 via a certain type of network. The client 101 is often a “program” controlled by a human user, but may be a fully automated system. The client 101 issues a request 110 for a certain image subset, and receives a response 113 including corresponding code data in the manner indicated in the request 110. Due to the bandwidth of the network or the resources of the server 102 or the structure of the file request, the client 101 may receive approximate code data other than the partial code data of the requested image. The client 101 typically uses a cache to issue requests for the output of different parts of the image and receives additional code data relative to previously received code data.

One of the basic features of JPIP is the ability to avoid repeating the code data that the client has already received. There are two methods for avoiding retransmission of this code data.
In the first method (FIG. 9), the client 211 has information on the code data transmitted to the server 203 and already received by the client, so that the client 211 requests the server 203 to distribute redundant data. This is a method in which the server 203 does not have to send redundant code data. When the server 203 operates “stateless”, the server 203 operates without storing the previous interaction. In general, servers often operate stateless because they do not need to maintain a special cache model that stores previous interactions.

  As a second method (FIG. 8), the client 101 and the server 102 establish a “session”, and the server 102 has a cache model for storing the code data sent to the client 101. In other words, information about the transmission to the server 102 is recorded in the server cache model. The code data received from the server 102 is stored in the cache of the client 101, and the server 102 can reuse the code data already transferred without transferring it to the client 101 by using the cache model. Typically, the client 101 appends the JPEG2000 code data received in a fragmented manner in the order of reception and stores the data in a cache. Subsequently, a bitstream compliant with the JPEG2000 syntax is created from the cached data as described above, and the bitstream is decoded.

  The present invention mainly relates to FIG. 8 for processing a stateless request, and manages the cache data of the cache 103 by including the cache management unit 108.

  The client 101 can change the code data corresponding to the portion of the image of interest even before receiving all of the code data requested so far. Some servers can interrupt the response to previous requests to handle these changes.

  FIG. 9 shows a typical network. Referring to FIG. 9, the network includes a plurality of clients (eg, clients 240, 311 and 312) and a plurality of servers. A single server can handle different parts of the same image to millions of different clients. A single client receives images and other data (eg, HTML pages) from several servers. A proxy server may exist in the network to allow a quick response to some requests without communicating with the original server. Effective bandwidth and bandwidth costs vary widely between different clients and the same server or between different servers.

  The most commonly used network is the Internet or the World Wide Web. However, JPIP can be applied to a local area network (LAN) or any common interconnection system. JPIP is most commonly transferred over Hypertext Transfer Protocol (HTTP), but is expected to be used with other transfer protocols including TCP / IP and UDP.

  From the above, by analyzing the code format (message) of the JPIP stream, the number of levels of each layer constituting the hierarchical code data such as the number of components, the number of layers, the precinct size, and the number of decomposition divisions can be found. You can also know the progression order.

  FIG. 10 shows an example of an output display screen when JPIP is used. In FIG. 10A, data composed of two tiles is displayed. For example, image data is displayed by clicking each tile, and four tiles with poor image quality as shown in FIG. 10B are displayed. An image consisting of Further, by clicking on the lower right, the resolution of the partial image (FIG. 10C) of the tile is changed, and the partial image quality level is improved. Thus, it is used when referring to partial image data. Each time the partial image data is referred to, the code data is requested to be transferred from the display client side to the server side and reproduced.

  Thus, in JPIP, it is assumed that the client (output side) requires partial code data instead of all (hierarchical) code data. Therefore, there is a problem that the hierarchical structure of the original code data cannot be known from the partial code data.

(4) JPEG2000 codestream format:
A JPEG 2000 codestream includes all entropy encoded data for an image and data describing the method to be used to decode the encoded data.
The code stream includes information on the wavelet transform used, tile size, precinct size, information on resolution, order of packets in the file, and the like. The code stream must contain all the parameters necessary to decode entropy encoded data into image samples. The code stream may also include information that provides fast access to the encoded portion of data, such as the length of the packet.

  A schematic configuration of a JPEG2000 code stream format is shown in FIG. The “code stream” of JPEG2000 has several marker segments. The marker segment is composed of a marker having a length of 2 bytes and a parameter group associated therewith. Each marker segment represents a function, usage, and data length. The format of the code stream starts with an SOC (Start of Codestream) marker indicating the start of the code data. The SOC marker is followed by a main header describing coding parameters, quantization parameters, etc., followed by actual code data.

  The structure of the main header is shown in FIG. The main header is composed of COD and QCD essential marker segments and COC, QCC, RGN, POC, PPM, TLM, PLM, CRG, and COM optional marker segments. Here, the SIZ marker segment describes image information at the time of non-compression such as the number of components and tile size, and includes information about the width and height of the image component. The COD and COC marker segments contain parameters that indicate how the compressed data should be decoded. The COD marker describes the progression order, the number of layers, the precinct size, and the number of decomposition divisions. The QCD marker describes information relating to quantization. The COM marker is a marker used when adding information such as a comment, and can be used in both the main header and the tile header.

  After the main header is a series of “tile parts”. Each tile part begins with a “SOT (Start of Tile-part)” marker segment that identifies the particular tile and part. The encoded data for each tile part is preceded by an “SOT” marker segment. The “SOT” marker segment includes information on the number of tile parts.

  The actual code data (FIG. 12) starts with an SOT marker, and includes a tile header, an SOD (Start of Data) marker, and tile data (code). After the code data corresponding to the whole image, an EOC (End of Codestream) marker indicating the end of the code is added. The main header is composed of COD and QCD essential marker segments and COC, QCC, RGN, POC, PPM, TLM, PLM, CRG, and COM optional marker segments.

The tile header configuration is shown in FIGS. This figure is a marker segment string added to the head of tile data, and marker segments of COD, COC, QCD, QCC, RGN, POC, PPT, PLT, and COM can be used. On the other hand, FIG. 11D shows a marker segment sequence added to the head of the divided tile part when the tile is divided into a plurality of segments, and marker segments of POC, PPT, PLT, and COM can be used. It is. There is no mandatory marker segment in the tile header, all are optional.
Tile data (code) is composed of continuous packets. The order of packets in the code stream is called the progression order.

  From the above, by analyzing the JPEG2000 code format, the number of levels of each layer constituting the hierarchical code data such as the number of components, the number of layers, the precinct size, the number of decomposition divisions, and the like can be known. You can also know the progression order.

  Based on the above facts, an embodiment of the present invention will be described in detail in accordance with the problem solving concept shown below.

(A) A unit for accessing partial code data with high frequency is estimated based on the usage frequency of the hierarchical code data in the order of progression. The code data is replaced when the cache data overflows in units of hierarchical levels that are accessed with high frequency estimated. In this way, the cache hit rate is improved.

(B) The code data on the cache has a data structure that reflects the hierarchical structure so that the hierarchical structure can be changed flexibly, and has a structure in which the hierarchical structure can be converted adaptively according to the usage situation. And effective cache control.

(C) Further, the management of data replacement when the cache data overflows determines whether or not all code data is prepared in units of layers corresponding to the data structure of the cache data, and not all packets are prepared in units of layers. To improve the cache hit rate. The cache data is exchanged so that all the packets that are access units are available.

(5) Data replacement when cache data overflows is a structural element unit of hierarchical code data:
In the configuration of the present invention, the data (code string) replacement processing unit at the time of cache overflow is the access unit.

  FIG. 13 is a diagram for explaining the concept of data (code string) replacement processing when the cache overflows according to the present invention. In FIG. 13, two memories including a cache are provided. However, a general image processing apparatus includes a cache that can be accessed at high speed, a storage device that requires a normal access time, and an external storage that has a slow access time. A plurality of memories having different processing speeds such as devices are provided. In the present invention, processing at the time of overflow of a cache that can be accessed at high speed will be mainly considered.

The code data on the cache is managed by the structure table, and indicates the storage location of the code data.
When new code data is stored in the cache, when the code data storage area on the cache becomes full, the following processing is performed. In the present invention, code data in the case of deletion or replacement of the code data is performed. We focused on the method of adjusting the unit.

(B) A part of the code data is deleted, or the code data is transferred to another storage device, and NULL is written in the structure table.
(C) Write new code data to the cache.
(D) A new item is added to the structure table, and an address (pointer) to the code data is written.

FIG. 14 is a flowchart showing data exchange processing when cache data overflows.
In the stored data replacement process, part or all of the hierarchical code data is awaited for a storage request (S1), and if the end is specified (S2 / YES), the process ends. On the other hand, if there is a storage request (S2 / NO), the code amount of the code data to be stored is calculated and compared with the remaining code amount (S3).

Next, the hierarchy of the data replacement unit is determined based on the usage status (frequency of use in progression order) (S4, S5, S6).
For this purpose, the usage frequency in the progression order is calculated (S4), the progression order in which the usage frequency is the maximum is selected (S5), and the cache data structure generation process of the hierarchical structure matching the progression order is executed (S6).

Next, stored data replacement processing is performed for each hierarchical level (S7 to S10).
If the previously calculated remaining code amount is insufficient (S7 / YES), the hierarchical level to be replaced is determined (S8), part or all of the hierarchical code data is deleted in units of hierarchical levels (S9), and the process proceeds to step S1. Return.
On the other hand, if the remaining code amount is sufficient (S7 / NO), the data is stored in the storage device (S10), and the process returns to step S1.

FIG. 15 is a basic configuration diagram of a data structure for managing cache data (code string structure). It consists of a management table, a structure table, and a code string storage unit. In this embodiment, the structure table is a table formation, but in another embodiment (FIG. 16) described later, it has a tree structure.
The management table includes a code data name (file name), the number of hierarchies, the number of hierarchy levels for each hierarchy (maximum value of hierarchy levels), and an address (pointer) to the structure data. The number of items of the hierarchy level number is changed depending on the hierarchy number.
This structure table includes all items of combinations of each hierarchy level number. The code string is stored in a pointer portion to code data corresponding to the hierarchical level of each hierarchy to which the code string belongs.

  FIG. 16 or FIG. 17 is an example of a cache management table and a structure table. The structure data is represented by a tree structure, and this tree structure is compatible with the table structure shown in FIG. In other words, the table structure can be converted into a tree structure and vice versa. A structure table having a tree structure is composed of hierarchical nodes. Each hierarchical node includes a hierarchical level No, all pointers to lower hierarchical levels, and information items. In the lowest hierarchy, each hierarchy node has a hierarchy level number and a pointer storage area for a code string. I don't have all pointers to lower levels. When there is no code string, the pointer storage area is NULL.

  By the way, it is easy to refer to a code string belonging to a certain hierarchical level by using the structure data shown in FIGS. This is because it is easy to know the hierarchical level to which the code string belongs and is stored in a format organized by hierarchical level for each hierarchy.

  For example, code data having a hierarchy level of 1 in the highest hierarchy may be searched for a code string having a hierarchy level of 1 corresponding to the code data. However, in the table type structure data of FIG. Since they are lined up, the reference can be made at high speed. On the other hand, in the structure data of FIG. 16, the code string can be referred to by following a pointer from a hierarchical node having a hierarchical level number of 1 corresponding to the hierarchical node to a lower hierarchical node.

  In the structure data of the tree structure, each hierarchical node has an information item, but in FIG. 17, the total value of the code amount of the code string of the code string of the lower hierarchy is recorded. When the code string is stored, the code string is generated by counting up the value of the same item (code amount value) of the hierarchical node corresponding to the hierarchical level to which the code string belongs.

FIG. 18 is a flowchart showing the cache structure generation process of FIG.
In the cache data structure generation process, first, after receiving the partial code data, the partial code data is received (S11).

  Next, the hierarchy level of the code string included in the partial code data is extracted (S12), and the highest hierarchy level number of the code string is written into the cache data management table (the structure is grasped from the fragmented code string (packet) group). (S13).

Next, cache update processing is performed (S14 to S22).
First, the number of levels of each hierarchy on the cache data management table is read (S14), and hierarchical node data having the number of levels is created (S15). Here, the hierarchical node data includes the following items.

(I) Hierarchy level No.
(Ii) a pointer to hierarchical node data of a lower hierarchy,
(Iii) Lower layer data presence / absence flag,
(Iv) The total code amount of the lower-level code string.

  Then, (i) the hierarchical level number of each hierarchical node data is described (S16), and (ii) a pointer to hierarchical node data of the lower hierarchical level of each hierarchical node data is described. (Ii) The initial value of the pointer to the hierarchy node data of the lower hierarchy is NULL, (iii) the lower hierarchy data presence / absence flag is 0, and (iv) the total code amount of the code string of the lower hierarchy is 0. (S17).

Next, S18 to S22 are sequentially performed for each code string to be stored.
A code string to be stored is acquired (S18), and if there is no target code string (S19 / NO), the process returns to step S11.
On the other hand (S18), if there is a target code string (S19 / YES), the code string is saved (S20), and the hierarchical node data is traced from the upper hierarchy to the lower hierarchy, and at each hierarchical level of the code string. (I) Search for the lowest hierarchy node data that matches the hierarchy level No, and (ii) specify the pointer destination to the hierarchy node data of the lower hierarchy of the lowest hierarchy node data as the address where the code string is stored (S21). At this time, the hierarchical level No when the pointer destination from the hierarchical node data of the upper hierarchy to (ii) the hierarchical node data of the lower hierarchy is the hierarchical level of the code.

  (Iii) The lower hierarchy of the traced hierarchy node data when the pointer to the hierarchy node data of (ii) lower hierarchy of the lowest hierarchy node data is written while tracing the hierarchy node data from the upper hierarchy to the lower hierarchy. 1 is added to the data presence / absence flag, and (iv) the code amount of the code string is added to the total code amount of the lower-level code string (S22), and the process returns to step S18.

FIG. 19 is a flowchart of cache management processing, and FIG. 20 is a flowchart of cache update request processing.
The client cache management process of FIG. 19 analyzes a cache management request (S31) and processes it in response to this request (S32).

  In the case of a cache update process request, a cache update request process is performed (S33), and the process ends.

  In the case of the cache structure change process request, the cache structure change process is performed (S34), and the most frequently used hierarchy level is calculated based on the use frequency of the highest hierarchy level of the code data (S35). Accordingly, the structure data of the cache memory or the external storage device is changed (change of the highest hierarchical level of the code data) (S36). Further, the structure data is changed so as to correspond to the allowable progression of the decoding process as required (S37), and the process is terminated.

In the case of a new generation request, new cache generation processing is performed (S38), management data is newly constructed (S39), code data is stored (S40), and the processing is terminated.
In the case of an additional storage request, an additional storage request process is performed (S42), the storage position of the additional code data is specified based on the structure data (S43), the code data is stored (S44), and the process ends. To do.

  In the case of a reference request, reference request processing is performed (S45), reference is made to code data within the range of code data based on the structure data (S46), and the process is terminated.

In the cache update request process of FIG. 20, the partial code data is received after receiving the partial code data. (S51).
The hierarchy level of the code string included in the partial code data is extracted (S52), and the highest hierarchy level of the code string is written to the cache data management table (the structure is grasped from the fragment code string (packet) group) (S53). Then, update the structure data table and write the code data (S54), and return to step S51.

FIG. 21 is a block diagram of a replacement unit selection system, which is a stand-alone type used for hierarchical code data recording management.
The recording management apparatus 201 includes a code data analysis unit 1010, a management table structure table generation processing unit 1011, a management table structure table change processing unit 1012, a management table structure table reference processing unit 1013, a code data storage processing unit 1014, and data A replacement necessity determination unit 1015, a replacement unit setting unit 1016, and a replacement control unit 1017 are provided.

The code data analysis unit 1010 analyzes the code data and extracts information on each code string (the number of layers and the level to which the code string belongs for each layer) necessary for estimating the structure.
The management table structure table generation processing unit 1011 generates a management table and a structure table using information of each code string (hierarchy level of code string included in partial code data). At this time, the code string is stored in the code data storage processing unit 1014. It also has a function of changing the management table and the structure table (management table structure table change processing unit 1012) or referencing (management table structure table reference processing unit 1013).

As described in the data exchange process at the time of cache data overflow (FIG. 14), the data exchange process of the present invention comprises the data exchange necessity determination, the determination of the data exchange unit, and the stored data exchange process for each hierarchical level. Yes.
The data replacement necessity determination unit 1015 executes “data replacement necessity determination”, the replacement unit setting unit 1016 executes “determination of data replacement unit”, and the replacement control unit 1017 executes “stored data replacement processing for each hierarchical level”. To do.

FIG. 22 is another configuration diagram of the replacement unit selection system, which is a client server cache system.
As in FIG. 21, the cache management unit 108 includes a code data analysis unit 1010, a management table structure table generation processing unit 1011, a management table structure table change processing unit 1012, a management table structure table reference processing unit 1013, and a code data storage process. A unit 1014, a data replacement necessity determination unit 1015, a replacement unit setting unit 1016, and a replacement control unit 1017.

(6) Replacement unit learning function:
The structure table shown above has a flexible adjustment function for each replacement unit.
FIG. 23 is a conceptual diagram for explaining a method of changing the progression order of code data on the cache. As the structure table, the table type described in FIG. 15 is shown, but the structure table is the same as that of the tree type shown in FIG. This is because mutual conversion is possible. The replacement of the structure table is realized by changing the order of the column unit according to the value of the row in which the hierarchy level No is described. That is, the data in column units is rearranged in the order of the hierarchy level of the highest hierarchy, the hierarchy level order of the next hierarchy is rearranged so that the level order of the upper hierarchy is not impaired, and the similar rearrangement is performed. Iterates to the lower hierarchy level.

Therefore, by using these structural data, the replacement unit can be selected and controlled based on the hierarchy of the code data.
Thus, the replacement unit of cache data is a hierarchical level unit, so that the data can be replaced in accordance with the usage status. A hierarchy is a resolution, a region, or an image quality, which is a division unit of hierarchical code data. Further, the level of the hierarchy is a division unit for each hierarchy of the hierarchical code data, and is, for example, resolution 1, resolution 2,.

FIG. 24 is a diagram for explaining the relationship between progression and access units. The structure of the code data is shown, and the hatched block indicates the deletion target. In Example 1, the code data at the structure level of L3 is deleted. There is a high possibility that the image quality level of the reproduced image is adjusted and the image quality level is the highest hierarchy.
On the other hand, in Example 2, the code data is deleted in units of regions, and it is estimated that there is a high possibility that the region level is the highest hierarchy. From this point of view, it can be estimated that the progression order of hierarchical code data (arrangement order in units of hierarchies) indicates a high possibility that the code data is deleted in units of hierarchical levels of the hierarchies.

  In the present invention, based on such a concept, the usage status is tabulated using a table for learning the progression as shown in FIG. In the configuration example of FIG. 22, the server-side usage frequency calculation unit 109 compiles data and notifies the client side of the results.

FIG. 26 is a flowchart of a process for recording a typical use situation.
In the stored data replacement processing, the server records the structure of the lossless code data (S61), calculates the progression order of the generated code data (S62), and records the usage status (frequency of use in the progression order) (S63). Then, the usage record is transferred to the client (S64).
The client performs a process for replacing the stored data transferred from the server (S65).

  By the way, the combination pattern of the hierarchy level for each hierarchy shown in FIG. 25 can estimate the number of hierarchy levels from the code string as in the process of grasping the structure of the code data in FIG.

  FIG. 27 is a conceptual diagram of realization in which the replacement unit is determined based on the progression order of the code data (arrangement order in hierarchical units) similar to FIG. In this example, the hierarchy is divided into a resolution level, an area level, and an image quality level. Depending on the frequency of use in the order of progression, the number of times that the highest hierarchy is resolution is 2 times, and the area is 8 times. Since this is three times, it is assumed that there are many code data replacement requests in descending order, and the structure data table is configured to replace the code data in units of the highest hierarchy. The cache hit rate can be improved by replacing the data on the cache in units that are likely to be used together.

(7) Replacement control based on the data code amount:
FIG. 17 is an example of a data structure related to control of the data code amount generated using FIG. 18 as described above. By using the structure table of FIG. 17, it is possible to control the replacement of code strings on the cache based on the code amount.

  FIG. 28 is a flowchart of the exchange control process based on the data code amount. A hierarchy is selected based on the data code amount, and code data (code string) is exchanged and controlled.

In the replacement process based on the data code amount of the client, the necessity of data replacement is determined (S71), and the data replacement unit is determined (S72).
Next, stored data replacement processing is performed for each hierarchical level (S73 to S84).
First, if the remaining code amount is sufficient (remaining code amount ≧ required code amount) (S73 / NO), the process ends.
On the other hand, if it is insufficient (remaining code amount <required code amount) (S73 / YES), the deletion layer is determined as the layer determined in (S72), and the layer level to be replaced is determined (S74 to S83).

  First, the accumulated code amount is set to 0 (S74), and the code amount A for each layer level of the highest layer in the hierarchy level of the deleted layer is extracted (S75), and “integrated code amount = integrated code amount + code amount A” is extracted. Is calculated (S76).

  Here, if the necessary code amount> the accumulated code amount (S77 / YES), the hierarchical level is set as a deletion target (S78), all the hierarchical levels are completed (S79 / YES), and if all the hierarchical levels are finished ( (S80 / End), all code strings of the code data are set as deletion targets (S82), part or all of the hierarchical code data is deleted in units of hierarchical levels (S84), and the process ends.

On the other hand, if all hierarchies have not been completed (S80 / incomplete), the deletion hierarchy is set as the upper hierarchy (S81), and the process returns to step S74.
If the hierarchy level is not completed (S79 / NO), the next hierarchy level is set (S83), and the process returns to step S75.

  If necessary code amount ≦ integrated code amount (S77 / NO), part or all of the hierarchical code data is deleted in units of hierarchical levels (S84), and the process is terminated.

  This provides means for replacing the code data in units of hierarchical levels of the hierarchical code data based on the data code amount.

(8) Cache replacement considering missing code string:
It is also possible to preferentially replace a group including a missing code string.
The client missing code string discrimination process is performed in the following procedure.
Read the number of levels of each hierarchy on the cache data management table and trace the hierarchy node data of the specified hierarchy level S from the upper hierarchy to the lower hierarchy, and (iii) presence of lower hierarchy data of the hierarchy node data of the Mth hierarchy The following determination process is performed for the flag value P.

In the case of P <D (1) × D (2)... × D (M−1), the code string constituting the specified hierarchical level S includes a missing code string, and the number of missing code strings is D (1) × D (2)... × D (M−1) −1. Here, N is the number of hierarchies on the cache data management table, m = 1 to N, and D (m) levels in the mth hierarchy from the bottom.
Further, in the case of P> D (1) × D (2)... × D (M−1), the code string constituting the designated hierarchical level S does not include a missing code string. The number of missing code strings is 0.

FIG. 29 is a flowchart of a process for performing cache replacement control in consideration of a missing code string using the determination result of the missing code string.
In the replacement process based on the data code amount of the client, the necessity for data replacement is determined (S91), and the data replacement unit is determined (S92).
Next, stored data replacement processing is performed for each hierarchical level (S93 to S105).
First, if the remaining code amount is sufficient (remaining code amount ≧ required code amount) (S93 / NO), the process ends.
On the other hand, if it is insufficient (remaining code amount <required code amount) (S93 / YES), the deletion hierarchy is determined as the hierarchy determined in (S92), and the hierarchy level to be replaced is determined (S94 to S104).
Thereafter, the number of missing code strings is calculated for the deletion candidate hierarchy, and the hierarchy level is selected in descending order of the number of missing code strings (S94).
The accumulated code amount is set to 0 (S95), and the code amount A for each hierarchical level of the highest layer in the hierarchy level of the deleted layer is extracted (S96), and “integrated code amount = integrated code amount + code amount A” is set. Calculate (S97).

  Here, if the necessary code amount is greater than the accumulated code amount (S98 / YES), the hierarchical level is targeted for deletion (S99), all the hierarchical levels have been completed (S100 / YES), and if all the hierarchical levels have been completed ( (S101 / End), all code strings of the code data are set as deletion targets (S103), a part or all of the hierarchical code data is deleted in units of hierarchical levels (S105), and the process ends.

On the other hand, if all hierarchies have not been completed (S101 / incomplete), the deletion hierarchy is set as the upper hierarchy (S102), and the process returns to step S95.
If the hierarchy level is not completed (S100 / NO), the next hierarchy level is set (S104), and the process returns to step S96.
If necessary code amount ≦ integrated code amount (S98 / NO), part or all of the hierarchical code data is deleted in units of hierarchical levels (S105), and the process ends.

  As described above, when all the code strings of the code data of the units to be used are not prepared, the use value is low, so that means for preferential replacement can be provided.

It is basic explanatory drawing of a hierarchical coding compression / decompression algorithm. It is explanatory drawing of a decomposition level and a subband. It is a basic explanatory view of tile division. It is explanatory drawing of a precinct and a code block. It is explanatory drawing of the structure of a code stream. It is explanatory drawing which shows the relationship between a precinct data bin and a message. It is an example of a precinct data bin. It is a block diagram of a client-server model with a cache. It is a block diagram of a client server network model. It is an example of an output display. This is a JPEG2000 code stream format. It is an example of a code stream of JPEG2000. It is a conceptual diagram of the replacement | exchange process of the data (code sequence) at the time of the overflow of the cache of this invention. It is a flowchart which shows the data replacement process at the time of cache data overflow. It is a basic composition figure of the data structure for cache data (code sequence structure) management. It is a conceptual diagram concerning structuring of stored data. It is an example of a data structure which concerns on control of the amount of data codes. It is a flowchart which shows the production | generation process of the cache structure of FIG. It is a flowchart of a cache management process. It is a flowchart of a cache update request process. It is a block diagram of a replacement unit selection system. It is another block diagram of the replacement unit selection system. It is a conceptual diagram for demonstrating the change method of the progression order of the code data on a cache. It is explanatory drawing of the relationship between a progression and an access unit. It is an example of a progressive setting of preservation code data. It is a flowchart of the process which records a typical use condition. It is explanatory drawing of the determination of a replacement unit (data collation by the unit used collectively). It is a flowchart of the exchange control processing based on the data code amount. It is a flowchart of the process which performs the cache replacement | exchange control which considered the missing code sequence using the discrimination | determination result of a missing code sequence.

Explanation of symbols

101, 211, 212, 240, 311, 312 ... client, 102, 203 ... server, 103, 203 ... cache, 104 ... J2K decoder, 105 ... output image, 106 ... cache model, 107 ... J2K file, 108 ... cache Management unit 109, 10160 ... Data structure usage frequency calculation unit, 110 ... Request, 113 ... Response, 112, 202 ... Storage device, 210, 310 ... LAN, 250 ... Large-scale network, 1010 ... Code data analysis unit, 1011 ... management table structure table generation processing unit, 1012 ... management table structure table change processing unit, 1013 ... management table structure table reference processing unit, 1014 ... code data storage processing unit, 1015 ... data exchange necessity determination unit, 1016 ... replacement unit Setting part, 1017 ... replacement Control unit, 201 ... recording management apparatus.

Claims (11)

  Image processing apparatus having function of receiving partial code data of hierarchical code data stored in storage device of image processing apparatus (server) based on transfer request of partial code data of hierarchical code data of image processing apparatus (client) A code stored in the storage device (cache memory) and having a function of storing and reusing part or all of the received code data in a storage device or storage device (cache memory) When the data storage capacity overflows, when replacing part or all of the code data stored in the storage device or storage device (cache memory) in the structured element unit, the replacement unit is a hierarchical code. It is a code string (packet) unit included in a specified level of a specified hierarchy of data Image processing system.   The image processing system according to claim 1, wherein the replacement unit is specified based on a use frequency for each layer of code data when layer code data is used.   3. The image processing system according to claim 2, wherein the frequency of use of the code data for each layer is estimated based on the frequency of use of the code data in the order of progression when the layer code data is used. Image processing system.   2. The image processing system according to claim 1, wherein a layer of the partial code data that is the replacement unit is converted based on a use frequency of a code data in a progression order when the structure of the hierarchical code data is used. An image processing system.   4. The image processing system according to claim 2, wherein in the replacement, a hierarchical level code string including a code string that is not stored is preferentially replaced. 5.   6. The image processing system according to claim 1, further comprising: a data code amount calculating unit that calculates a data code amount for each hierarchical level, and based on the calculated data code amount. An image processing system comprising means for replacing code data in units of hierarchical levels of the described hierarchical code data.   7. The image processing system according to claim 1, wherein the hierarchical code data is data encoded based on JPEG2000 (ISO / IEC 15444-1) standard.   8. The image processing system according to claim 1, wherein the hierarchical code data is a code stored in a storage device of a server based on a JPIP (JPEG2000 image coding system-Part9: Interactivity tools, APIs and protocols) standard. An image processing system having a partial code data access function for a client to receive partial code data that is part of the converted image data.   Image processing apparatus (client) that receives partial code data of hierarchical code data stored in a storage device of an image processing apparatus (server) based on a transfer request of partial code data of hierarchical code data of the image processing apparatus (client) The code data stored in the storage device (cache memory) when part or all of the received code data is stored in a storage device or storage device (cache memory) and reused. When a part of or all of the code data stored in the storage device or the storage device (cache memory) is replaced in a structured element unit when the storage capacity of the storage unit overflows, the replacement unit is hierarchical code data An image processing method characterized in that it is a code string (packet) unit included in a specified level of a specified hierarchy of .   A program for causing a computer to realize the functions of the image processing system according to any one of claims 1 to 8.   A computer-readable recording medium on which the program according to claim 10 is recorded.