JP2014165868A - Image processing device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a natural image with a reduced image quality difference at the boundary of tiles generated by different coding methods, when decoding coded image data of each tile unit in which coded data generated by a plurality of coding methods at least including an irreversible coding method are mixed.SOLUTION: When decoding the coded data of an image having the coded data of a tile based on JPEG 2000, mixed with the coded data of a tile based on JPEG-LS, it is determined for each tile unit which coding method the coded data is based on, so that decoding processing for a type corresponding to the determination result is executed. As a result of the decoding, if the tile of interest is based on JPEG 2000 and the adjacent tile is based on JPEG-LS, image processing using a filter is executed so as not to produce unnaturalness at the tile boundary therebetween.

Description

  The present invention relates to image data encoding and decoding techniques.

  Conventionally, there has been a demand for keeping encoded data within a specified capacity while suppressing deterioration of image quality as much as possible. For example, an image forming apparatus such as a page printer converts an input page description language into image data, encodes the image data, and temporarily stores it in a predetermined area. Need to control.

  In response to such a requirement, a method (Patent Documents 1 and 2) has been proposed in which a single image is used as an area for lossless compression and an area for irreversible compression, and code amount control is performed for an image in the area for irreversible compression. ing. In this type of method, the lossy compressed data may be largely deleted in order to realize a target code amount (target code amount) after encoding. As a result, the difference in image quality between the reversible compression area and the irreversible pressure area becomes noticeable, and it becomes dirty when viewed as a whole screen. In order to alleviate this problem, it is conceivable to perform a filtering process called a deblocking filter on the boundary between the lossless compression area and the lossy compression area after decoding the image. A deblocking filter is a filter that reduces block distortion that occurs in a decoded image such as JPEG, and is a type of smoothing filter that is applied to block boundaries (Patent Document 3).

Japanese Unexamined Patent Publication No. 2006-148585 Japanese Patent No.4693603 JP 2010-4555

  In lossy compression, there is a type of method that greatly distorts pixels at a specific boundary between regions. For this reason, when applying a deblocking filter, it is desirable to determine a filter coefficient according to how much each pixel used in the filter may be distorted (reliability of decoded pixel value). However, the conventional deblocking filter does not pay attention to the reliability of each decoded pixel, and determines the filter coefficient based on how much the region is distorted (specifically, compression conditions and the like). That is, when a conventional deblocking filter is used, smoothing can be achieved, but there is a problem that distortion of a decoded pixel value with low reliability is diffused.

In order to solve this problem, for example, an image processing apparatus disclosed in this specification has the following configuration. That is,
The encoded data of the first type tile encoded by the irreversible first encoding means, and the second encoded by the second encoding means different from the first encoding means An image processing apparatus that decodes encoded image data in which encoded data of a type tile is mixed,
Identifying means for identifying which encoded means the encoded data of each tile is encoded data;
A decoding processing means for generating a tile image by executing a corresponding type of decoding processing on the encoded data of each tile according to the identification result of the identification means;
A tile in which the first type tile and the second type tile are adjacent to each other in an image composed of tile images obtained by the decoding processing by the decoding processing unit based on the identification result by the identification unit. Detection means for detecting the boundary;
A value of a pixel that touches the tile boundary in the first type tile that touches the tile boundary detected by the detection means has a size across the tile boundary, and the first encoding means and the second Filter processing means for performing filter processing using a filter having characteristics according to the reliability of the encoding means.

  According to the present specification, when encoded image data in which encoded data in units of tiles generated by a plurality of encoding methods including at least an irreversible encoding method is decoded, tiles of different encoding methods are decoded. It is possible to reduce a difference in image quality at the boundary and generate a natural image.

Configuration diagram and functional block diagram of a computer system on the encoding side in the first embodiment Server / client configuration diagram The figure explaining the image compression processing flow with code amount control in 1st Embodiment The figure explaining the code amount control processing flow in 1st Embodiment The figure explaining the re-encoding process flow of a reversible area | region in 1st Embodiment. The figure explaining the structure of the code sequence in 1st Embodiment The figure explaining distribution of the lossless / lossy compression field before going into code amount control in a 1st embodiment. Configuration diagram and functional block diagram of a computer system on the decoding side in the first embodiment The figure explaining the whole flow of the decoding process in 1st Embodiment The figure explaining the positional relationship of the tile of the deblocking filter object in 1st Embodiment, and the tile to refer. The figure explaining the row | line | column of the tile in 1st Embodiment The figure explaining the processing flow of the deblocking filter in 1st Embodiment. The figure explaining the pixel used by the deblocking filter process in 1st Embodiment The figure explaining the state where the part other than the photograph area is divided into a reversible tile and a non-reversible tile Diagram explaining distortion characteristics of JPEG2000 Illustration explaining reverse DWT of JPEG2000 The figure explaining the processing flow of the deblocking filter in 2nd Embodiment. The figure explaining positional relationship other than the adjacent JPEG tile, JPEG2000 tile, and photograph area | region in 3rd Embodiment. The figure explaining the whole flow of the deblocking filter process in 3rd Embodiment The figure explaining the flow of the deblocking filter process to the JPEG2000 tile in 3rd Embodiment The figure explaining the pixel used by a deblocking filter process The figure explaining the deblocking filter processing flow to the JPEG tile in 3rd Embodiment It is a figure which shows an example of the filter table in 2nd Embodiment.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[First Embodiment]
First, an outline of the encoding method used in the present embodiment and problems with image quality will be briefly described. Next, a conventional technique (deblocking filter) for overcoming the problem of image quality, and its problem and solution will be described.

  For the encoding method, in order to realize high compressibility and code amount control in a document image, encoding (adaptive encoding) is used that adaptively uses lossless encoding and lossy encoding that can control the amount of code. In the present embodiment, JPEG2000 (ITU-T T.800 | ISO / IEC 15444-1), which is an international standard encoding method, is used as lossy encoding (first encoding). As lossless encoding (second encoding), JPEG-LS (ITU-T T.87 | ISO / IEC 14495-1) of the international standard encoding method is used (second encoding). The code amount control is realized by deleting the encoded data of the bit plane in the encoded data by JPEG2000. In JPEG200, encoded data for each bit plane composed of the same bits of each transform coefficient obtained by wavelet transform is generated. It can be said that the lower bits have less influence on image quality. Therefore, the code amount control is realized by determining to which bit position the encoded data of the bit plane from the least significant bit (bit 0) to the higher order is deleted.

  In this adaptive encoding method, after an image such as a document is input, the image is divided into rectangles (tiles). After that, each tile determines whether it is an area having the characteristics of a photograph (photo area) or an area having characteristics other than a photograph, such as halftone dots, line drawings, text, and gradation (other than the photograph area). Then, a lossy encoding method is applied to tiles in the photographic region, and a lossless encoding method is applied to tiles other than the photographic region (hereinafter, the former tile is referred to as an irreversible tile and the latter tile is referred to as a lossless tile). There is one problem in image quality common to such encoding methods. As shown in FIG. 14, a tile including a photographic region in the image and a boundary other than the photographic region is in contact with the photographic region, and therefore, lossy encoding is often applied. As a result, in spite of the fact that lossless encoding is applied to tiles other than adjacent photo areas, lossy encoding may be applied to the boundary portion, resulting in a difference in image quality. In particular, when high compression is used, the problem appears remarkably.

  On the other hand, a method of reducing the image quality difference by applying a deblocking filter to the boundary between the reversible tile and the irreversible tile to enhance the continuity of the pixel value change between the tiles can be considered. However, the conventional deblocking filter does not consider the existence of pixels with low reliability, and has a problem of diffusing distortion. This is simply because the filtering method is determined based on the tile compression condition. As a specific example of the pixel with low reliability, there is a JPEG2000 decoding result. In this regard, as shown in FIG. 15, the pixel at the right end of the tile (right end pixel) and the pixel at the bottom row (lower end pixel) may be greatly distorted (reliability is low). This is due to the inverse wavelet (DWT) transformation in the JPEG2000 decoding process. This will be described by taking DWT conversion called 5 × 3 conversion supported by JPEG2000 as an example.

JPEG2000 inverse DWT performs processing on DWT coefficients generated through entropy decoding and inverse quantization. The DWT coefficients are stored in a two-dimensional array, and the inverse DWT is processed in the order of the horizontal direction and the vertical direction. In the processing in each direction, the inverse conversion is performed using the low frequency component and the high frequency component. The inverse transformation of 5x3 transformation is as follows.
X (2n) = L (n) -Floor ((H (n-1) + H (n) +2) / 4)
X (2n + 1) = H (n) + Floor ((X (2n) + X (2n + 2)) / 2)
However, X (n) is a decoded pixel, L (n) is a low frequency component, and H (n) is a high frequency component. N is an index indicating the position of the decoded pixel, the low frequency component, and the high frequency component. When N is the number of pixels in the target direction of the tile, 0 ≦ n <N / 2. The tile size is often set to an even number, and N is an even number in the following description. FIG. 16 is a diagram showing the conversion process of the above conversion formula when N = 8. In the figure, L (0), H (0), ..., L (3), H (3) are DWT coefficients after inverse quantization, and X (0), ..., X (7 ) Is a decoded pixel. H (-1) and X (8) are values that are virtually set to perform inverse DWT, and H (-1) = H (0), X (8) = X (6). It should be noted that in this conversion process, for even-numbered decoded pixel values such as X (0), X (2), ..., the contribution ratio of the low-frequency component and the high-frequency component is approximately L: H = 2 : 1. On the other hand, for odd-numbered decoded pixel values such as X (1), X (3),..., The contribution ratio between the low-frequency component and the high-frequency component is approximately L: H = 2: 3. In general, it is said that, in encoding, a high-frequency component has a smaller visual impact than a low-frequency component, and information is further reduced. For this reason, a decoded pixel with an odd index is more easily distorted than an even index. Further, when the DWT coefficients are limited to the odd index, the appearance of distortion is different between the decoded pixel at the tile boundary (X (7) in FIG. 16) and other pixels. That is, the decoded pixels other than the tile boundary are generated from the sum of two different even-numbered decoded pixels. On the other hand, the decoded pixels at the tile boundary are generated from the sum of the decoded pixels having the same even index. This means that the error amount is canceled out if the error (difference from the pixel value in the original image) is reversed between the two decoded pixels of even index. However, the latter is not the case, and the error amount is amplified. Therefore, the odd-indexed boundary pixel is likely to generate a highly distorted pixel, that is, the reliability is low. As described above, DWT coefficients are stored in a two-dimensional array, and an index is assigned with the left end set to 0 in the horizontal direction. On the other hand, in the vertical direction, the upper end is set to 0 and an index is assigned. For this reason, the reliability of the rightmost pixel and the lowermost pixel in the tile is lowered.

  Thus, in JPEG2000, the reliability of pixels at a specific boundary is often lowered, and distortion is diffused using a conventional deblocking filter. Therefore, in the present embodiment, a method for setting the coefficient of the deblocking filter according to the reliability of the boundary pixel is proposed as a solution. In addition, since distortion does not arise in a reversible tile, if a deblocking filter is applied to the boundary pixel of a reversible tile, reversibility will be impaired. For this reason, a deblocking filter is not applied to a reversible tile.

  In this embodiment, a system is assumed in which a server that holds uncompressed image data receives a request from a client and returns image data corresponding to the request to the client. And this implementation data demonstrates the encoding process in the server. The request from the client to the server has an image that the client wants to acquire and a data size that can be received. In this embodiment, a process in which the client receives the encoded image data, decodes it, and performs a deblocking filter will also be described.

  FIG. 1 is a block diagram of a server constituting the system according to the present embodiment. The CPU 101 controls the operation of the entire system and executes a program stored in the primary storage 102. The primary storage 102 is a main memory composed of a RAM, and reads and stores programs stored in the secondary storage 103. The secondary storage 103 corresponds to, for example, a storage medium such as a hard disk. Generally, the capacity of the primary storage is smaller than the capacity of the secondary storage, and program codes and data that cannot be stored in the primary storage are stored in the secondary storage. Further, data that must be stored for a long time is also stored in the secondary storage. In the present embodiment, the program is stored in the secondary storage 103, read into the primary storage 102 when the program is executed, and the CPU 101 executes the execution process. Examples of the input device 104 include a mouse and a keyboard. Used to send an interrupt signal to a program or the like. Examples of the output device 105 include a monitor and a printer. Various other configurations of the apparatus configuration method are conceivable, but the description thereof is omitted because it is not the main point of the present invention. The network I / F 106 is an interface for communicating with the network, and may be wired or wireless.

  FIG. 2 is a diagram illustrating an example of an outline of a system that realizes the present embodiment. Reference numerals 201 and 202 in the figure represent clients. A plurality of users, such as the client 201 or the client 202, can communicate with the server 204 via the network 203 regardless of wired or wireless. The clients 201 and 202 are PCs, tablet terminals, portable terminals, printers, and the like, and are devices that have a communication function and can output images. The server 204 has the configuration shown in FIG. 1 and has a large-capacity storage device 205 that stores image files. The large-capacity storage device 205 corresponds to, for example, a hard disk. This hard disk stores a large amount of uncompressed image data including text, line drawings, and natural images. The clients 201 and 202 request image data and the upper limit value of the compressed data size from the server. The upper limit value of the compressed data is a value calculated from the network status, client memory capacity, and the like. The server 204 compresses the image data designated by the clients 201 and 202 so that the file size is equal to or smaller than the designated upper limit value, and returns the compressed data to the clients 201 and 202. The clients 201 and 202 decrypt the received compressed data.

  First, the operation of the server 204 will be described in more detail. The server 204 divides the image data (file name etc.) specified by the clients 201 and 202 into tiles, selects reversible compression or lossy compression according to the characteristics of each tile, Turn into. In the present embodiment, the tile size is 128 × 128 [pixel]. In addition, a JPEG-LS encoding method that is good at compression of characters and line drawings as lossless compression and a JPEG 2000 encoding method that is good at compression of natural images as lossy compression are used. Therefore, the JPEG-LS method is selected if the tile includes many characters and line drawings, and the JPEG 2000 method is selected if the tile includes many natural images. After encoding all tiles, the code amount of tiles compressed with JPEG2000 (JPEG2000 tiles) is the same as that of tiles compressed with JPEG-LS (JPEG-LS tiles) without making a difference between tiles. Remove the image quality difference within the allowable range. As a result of the deletion, if the code amount is equal to or less than the specified code amount, the data is returned to the client. If it is necessary to delete the JPEG2000 tile data so that the image quality difference from the JPEG-LS tile cannot be kept within the allowable range, the excess code amount and the distribution of the lossless compression area and lossy compression area are used. Change the code amount control method.

  Also, the order of encoding the tiles is the raster scan order in units of tiles, that is, when the leftmost tile reaches the rightmost tile, and when reaching the rightmost tile, the leftmost tile in the downward direction is followed by the rightmost tile. It will become.

  The operation of the server 204 will be described in more detail with reference to FIG. This figure shows the procedure of the application program executed by the CPU 201 in the server 204. The following processing is processing when an image request is received from the client 201. The image request from the client includes information for specifying the image (for example, a file name and an index number) and an upper limit value of data.

  In step S301, an image designated by the client 201 (or 202) is acquired from the storage device 205. This image is an uncompressed image. In the present embodiment, it is assumed that the color space is RGB, the bit accuracy of each component is 8 [bit / channel], and the image size is 7040 × 4992 [pixel]. That is, the image before compression is about 100 [MB]. In step S302, the upper limit code amount D_max as a result of compressing the image acquired in step S301 is acquired. This is the upper limit value of the compressed data designated by the client. In this embodiment, it is assumed that 14 [MB] is designated from the memory capacity of the client. In step S303, the image acquired in step S301 is divided into tiles, and the total number of tiles Tsum is calculated. Since the tile size of this embodiment is 128 × 128 [pixel], Tsum = (7040/128) × (4992/128) = 55 × 39 = 2145 tiles. In step S305, the tile number counter T is initialized by substituting zero.

  Thereafter, in steps S305 to S310, encoding is performed by switching between JPEG-LS and JPEG2000 for each tile. In step S305, the encoding method of the target tile T is determined. The encoding method is determined by selecting the JPEG-LS encoding method when the tile T includes many characters and line drawings, and selecting the JPEG 2000 encoding method otherwise. Whether or not the tile T includes many characters and line drawings can be estimated from various feature amounts by analyzing the tile T. For example, when the number of colors is small, it is determined as a character or line drawing, and when the number of colors is large, it is determined that there are many natural images. Alternatively, edge detection is performed, and if there are many edges, it is determined that there are many characters and line drawings, and if not, it is determined that there are many natural images. Various other methods are conceivable, but since they are not the main point of this proposal, a detailed description is omitted. In the present embodiment, edge detection is performed within the tile of interest, and if the number of pixels determined to be an edge is greater than a predetermined threshold, JPEG-LS encoding is performed assuming that many characters / line drawings are included, otherwise , JPEG2000 encoding shall be selected.

  In step S306, based on the determination in step S305, it is determined whether or not the compression method of the tile of interest T is lossless compression encoding. If the encoding method of the target tile T is the JPEG-LS encoding method, that is, the lossless compression method, the process proceeds to step S307. Otherwise, the process proceeds to step S308. In step S307, the tile of interest T is compressed using the JPEG-LS encoding method. In step S308, the tile T is compressed using the JPEG2000 encoding method. The encoded data of the tile of interest generated as a result of the compression process (encoding process) is stored in the primary storage device 102, but the storage destination may be the secondary storage device 103. In step S309, the tile number T is incremented by one. In step S310, the tile number T is compared with the tile total number Tsum calculated in step S303. If T and Tsum are the same, it is determined that all the tiles have been encoded, and the process proceeds to step S311. If the two values are different, it is determined that there is a tile that has not been encoded yet, and the process returns to step S304.

  In step S311, the code amount control is performed so that the total value (file size) of the code amounts generated thus far is equal to or less than the upper limit code amount D_max acquired in step S302. Finally, a code string is generated in step S312. As shown in FIG. 6, the code string is composed of a header, a JPEG2000 code string, and a JPEG-LS code string. In the header, the vertical and horizontal sizes of the image, the tile size, flag information indicating whether each tile is compressed by JPEG2000 / JPEG-LS, each JPEG2000 / JPEG-LS code string length, and encoding parameters (described later) The number of deleted bit planes in JPEG2000 encoded data) is described. The JPEG2000 code string is generated as if no code was generated for tiles for which JPEG-LS was selected. For example, in FIG. 7, it is assumed that areas 801 and 804 are compressed with JPEG 2000, and areas 802 and 803 are compressed with JPEG-LS. In this case, a JPEG2000 code string is generated assuming that no JPEG2000 code is generated for the tiles 802 and 803. The JPEG-LS code string has the same configuration. When the code string is generated in this way, the code string is transmitted to the client 201 or 202.

  Next, the code amount control process in step S311 will be described with reference to FIG. In step S501, the data size D_ls of the lossless compressed data is acquired. This can be easily calculated from the sum of the compressed data of the tiles JPEG-LS compressed in step S307 of FIG. In the present embodiment, it is assumed that D_ls = 1.26 [MB]. In step S502, the maximum number of bit planes C that can be deleted is acquired so that the difference in image quality between the lossless compression tile and the lossy compression tile is within a predetermined allowable range. The maximum number C of bit planes that can be deleted is a value determined by the color space and bit accuracy of an uncompressed image, the compression parameter of JPEG2000 encoding, and the like. The maximum number of bit planes C that can be deleted in each state is determined in advance based on experiments and statistics using a certain sample image so that the image quality difference between the JPEG-LS tile and the JPEG2000 tile is acceptable. In the present embodiment, it is assumed that the difference in image quality between the JPEG-LS tile and the JPEG2000 tile falls within an allowable range even if the maximum 2 bit plane (bit 0, 1 plane) that can be deleted is deleted.

In step S503, the JPEG2000 code amount D_c is calculated when it is assumed that the number C of bit planes acquired in step S502 is deleted from the code data of all JPEG2000 tiles. In this embodiment, it is assumed that D_c = 12.9 [MB]. In order to accurately calculate the JPEG2000 code amount, it is necessary to calculate the JPEG2000 main header, the JPEG 20000 tile header, and the code amount of each packet. The code amount of each packet is the sum of the bit plane (coding path) data included in the packet and the packet header calculated therefrom. Since the calculation method of the JPEG2000 code amount as a result of deleting the designated number of bit planes is a well-known method, a detailed description thereof is omitted here. In step S504, it is determined whether the total code amount (D_ls + D_c) calculated from step S501 and step S503 is smaller than the upper limit code amount D_max acquired in step S302.
D_ls + D_c <D_max
If this is satisfied, it can be said that even if the bit plane of the JPEG2000 tile is deleted until the code amount of the entire image is equal to or less than the upper limit code amount D_max, the difference in image quality between the JPEG-LS tile and the JPEG2000 tile is within an allowable range. In that case, the process proceeds to step S507.

  If it is determined that D_ls + D_c ≧ D_max, if the bit plane of the JPEG2000 tile is deleted until the code amount of the entire image is equal to or less than the upper limit code amount D_max, the JPEG-LS tile and the JPEG2000 The difference in image quality of tiles exceeds the allowable range. That is, the tile boundary distortion becomes conspicuous. Therefore, in this case, the process proceeds to step S505. In the case of the present embodiment, (D_ls + D_c) = 1.26 + 12.9 = 14.16 [MB]> D_max = 14 [MB], so the process proceeds to step S505. In step S505, all JPEG-LS tiles are re-encoded with JPEG2000. This re-encoding method will be described in detail later. In step S506, the data size D_ls of the lossless compressed data is set to zero.

  In step S507, the upper limit code amount D_j2k of the lossy compression area is calculated. This is obtained by subtracting the data amount D_ls of the lossless compression area from the upper limit code amount D_max. In step S508, all JPEG2000 tile data are uniformly deleted so that the JPEG2000 code amount is equal to or less than the code amount D_j2k calculated in step S507. The process of step S508 will be briefly described as follows. For example, when the bit plane is not deleted and the total code amount is equal to or less than D_max, the deletion process is not necessary. Also, when the bit 0 plane of all tiles that have been lossy encoded is deleted and D_ls + D_c <D_max is satisfied for the first time, the encoded data of the bit 0 plane of all tiles that have been lossy encoded will be deleted. To do. Further, when the bit 0 and bit 1 planes of all tiles subjected to lossy encoding are deleted and D_ls + D_c <D_max is satisfied for the first time, bits 0 and 1 of all tiles subjected to lossy encoding are satisfied. The encoded data of the plane is deleted.

  Next, JPEG-LS tile re-encoding in step S505 will be described with reference to FIG. In step S701, the tile number counter T is initialized by substituting zero. In step S702, it is determined whether or not the tile T is losslessly encoded, that is, whether or not it is compressed by JPEG-LS encoding. If it is JPEG-LS compressed, it becomes a re-encoding target, and the process proceeds to step S703. Otherwise, it is determined that the tile does not need re-encoding, and the process proceeds to step S705. In step S703, the JPEG-LS code data of the tile T is expanded (decoded). Since JPEG-LS encoding is lossless compression, the decompressed data matches the image data before encoding. In step S704, the tile T expanded in step S703 is compressed again with JPEG2000 encoding. In step S705, the tile number T is incremented by one. In step S706, the tile number T is compared with the tile total number Tsum calculated in step S303. If it is a different value, there may still be a tile that needs to be re-encoded, and therefore the process returns to step S702. If they are the same, it is determined that re-encoding has been completed for all tiles.

  If the process proceeds to step S508 via step S505, all tiles are lossy-encoded, and there are no losslessly encoded tiles. That is, the problem of distortion at the boundary between the lossless encoded tile and the lossy encoded tile does not occur. For this reason, when all tiles are lossy encoded, the encoded data of bit planes from bit 0 to the upper bits are deleted until the upper limit value D_max or less, regardless of the number C of allowable deletion bit planes. .

  As described above, the server 204 in the embodiment mixes encoded data of tiles that are losslessly encoded (JPEG-LS in the embodiment) and tiles that are lossy encoded (in the embodiment, JPEG2000). Encoded image data can be generated. Moreover, the encoded data of the target code amount (upper limit code amount D_max) or less is generated by deleting the encoded data of the bit plane having the deletion bit plane number C or less indicating the deletion allowable range. However, when a plurality of types of encoded data coexist, distortion at the tile boundary can be reduced, but it is not guaranteed that distortion is completely suppressed, and even if the number of deleted bit planes is “2” depending on the image. Distortion can occur at the tile boundary. Therefore, in the present embodiment, when the client 201 (or 202) receives and decodes the encoded data, the client 201 (or 202) performs a decoding process that further suppresses the occurrence of distortion at the tile boundary.

  Hereinafter, decryption processing in the client 201 (or 202) will be described. This decryption processing is performed by loading an application program stored in the secondary storage device 103 in the client 201 into the primary storage 102 and executing it by the CPU 101. In the decoding process, the encoded data of each tile is decoded according to the encoding method applied thereto. In the process, image processing is applied to the decoded tile. In this processing, as shown in FIG. 10, the horizontal right direction is defined as the X direction, and the vertical downward direction is defined as the Y direction. In addition, after decoding of the tile of interest T (a, b) in the a-direction in the X direction and the b-th in the Y direction is completed, T (a, b-1) directly above is the reversible tile and the irreversible tile. It is determined which one. If the tile T (a, b-1) is an irreversible tile, the tiles adjacent to the right and down from T (a, b-1), that is, T (a + 1, b-1) , T (a, b) is determined. If any of the tiles is a reversible tile, it is determined that there is a non-photograph area near the boundary between the irreversible tile T (a, b-1) and the right or bottom adjacent tile determined to be reversible. To do. Then, a deblocking filter is applied to the corresponding boundary in order to alleviate the image quality difference occurring outside the same photographic area. Note that the reason for excluding the left direction and the upper direction from the deblocking filter is that, as described at the beginning of this embodiment, the leftmost pixel and the uppermost pixel of the tile to which JPEG2000 is applied are the rightmost pixel and the lowermost pixel. This is because the reliability is higher than that of the pixel. When the number of tiles in the X direction is Num_T_x, it is necessary to buffer Num_T_x + 1 decoded tiles in order to apply a deblocking filter.

  FIG. 8A is a block diagram of the client 201 (same for the client 202) in this embodiment. Since the processing blocks 801 to 806 in the figure are substantially the same as those in FIG. 1, the description thereof is omitted.

  FIG. 9 is a flowchart of decryption processing in the client 201 (or 202) in this embodiment. This process is performed when the server 204 has transmitted encoded data within the requested range as a result of transmitting an image request (including information specifying an image and an upper limit value of the data amount) to the server 204. It is also an application program related to the decoding process executed by the CPU 801. It is also executed when decoding the encoded image data once stored in the secondary storage device.

  In step S901, the CPU 801 stores the received code string in the primary storage device 802. In step S902, the header of the code string is analyzed, and information regarding the image and information regarding the encoding are acquired. Specifically, the vertical and horizontal (horizontal and vertical) size of the encoded image, the tile size, the number of tiles in the vertical and horizontal directions, flag information indicating whether each tile was compressed with JPEG2000 / JPEG-LS, JPEG2000 Get information such as / JPEG-LS code string length and the number of deleted bit planes. In step S903, the total number Tsum of tiles is calculated from the vertical and horizontal sizes and tile size information of the encoded image. In step S904, the tile counter T is initialized. Specifically, initialization is performed by substituting zero for the counter T for tile numbers. In step S905, the encoding method of the target tile T is confirmed from the header information obtained in step S902, and in step S906, it is identified whether it is reversible or irreversible. If it is determined that the tile T is encoded by the lossless encoding method as a result of the identification, JPEG-LS decoding is performed in step S907, and the decoding result (tile image) is secured in the primary storage device 102. Stored in the specified buffer. Furthermore, each tile updates an encoding scheme map indicating which encoding scheme is applied. Specifically, a flag “0” indicating that the tile T of interest has been encoded by JPEG-LS is described in the encoding method map. On the other hand, when it is determined that the encoding method of the tile T is JPEG2000, JPEG2000 is decoded in step S908, and the decoding result is stored in the buffer. Further, a flag “1” indicating that the tile of interest has been encoded by JPEG2000 is described in the encoding method map. Since both JPEG-LS and JPEG2000 decoding methods are well-known techniques, a description thereof is omitted here. Next, image processing is performed in step S909. This image processing is targeted for the tile positioned above the tile decoded in step S907 or step S908. Note that if the target tile is a tile that touches the upper side of the image, there is no tile to be processed, and therefore the process of S909 is not performed. As shown in FIG. 10, when the target tile T is the tile T (a, b), the tile subject to image processing is T (a, b-1). In this image processing, the encoding applied to the tile T (a, b-1) to be processed and the adjacent tiles T (a, b) and T (a + 1, b-1), respectively. A deblocking filter is performed based on the method. Details will be described later. In step S910, the tile subjected to the image processing in step S909 is output to the outside. This is because the corresponding tile is unnecessary in the subsequent processing. In step S911, the tile counter T is incremented, and in step S912, the size is compared with Tsum. If T <Tsum, there is an undecoded tile, and the process returns to step S905. If T = Tsum, the process proceeds to step S913.

  At the time of entering the processing of step S913, the image processing of the tiles in the last row shown in FIG. 11 is incomplete. For this reason, in step S913 and subsequent steps, image processing of the tiles in the last row is performed. In step S913, the tile counter T 'is initialized. That is, the number T ′ of the first tile in the last row is calculated as T ′ = Tsum-Num_T_x. Here, Num_T_x is the number of tiles arranged in the horizontal direction.

  In step S914, image processing is performed on the tile T '. This process is the same as step S909, and will be described later. The tile that has undergone image processing in step S915 is output to the outside, and the tile counter T 'is incremented in step S916. In step S917, the size is compared with Tsum. If T ′ <Tsum, there is an undecoded tile, and the process returns to step S914. If T ′ <Tsum is not satisfied, all the processes are finished.

  Next, the image processing in steps S909 and S914 will be described using the flowchart of FIG. In the description so far, the tile T is the target tile, but it should be noted that in the image processing in FIG. 12, the tile immediately above the tile T is an image processing target. That is, in the image processing target, it should be noted that the tile directly above the tile T is described as the target tile. The reason for this is that it is necessary to finish the decoding process for the tile on the right side of the tile of interest and the tile on the lower side.

  In step S1201, with reference to the encoding method map, it is determined whether the target tile is an irreversible tile. If it is not a non-reversible tile, that is, if it is a reversible tile, the deblocking filter is unnecessary, and the processing is terminated. If the target tile is an irreversible tile, the process proceeds to step S1202.

  In step S1202, it is determined whether the number of deleted bit planes is “2” or more. When the lossless encoding tile and the lossy encoding tile coexist, the server 204 of the above embodiment deletes the encoded data of up to two bit planes from the lossy encoded data. This is to suppress the occurrence of distortion at the tile boundary when lossless encoded data and lossy encoded data are mixed. When the number of deleted bit planes is less than 2, that is, when there is no deletion or when the encoded data of the bit 0 plane is deleted, it is assumed that no distortion occurs between the tiles, and this processing ends. However, when the number of deleted bit planes is 2, there is little probability that distortion will occur, but this is not guaranteed. Therefore, when the number of deleted bit planes is 2 or more, the generation of the distortion is suppressed by the decoding process in the processes after step S1203.

  In step S1203, it is determined based on the encoding method map whether or not the encoding method of the tile on the right of the tile of interest is lossless encoding. If it is determined that the tile on the right is lossless encoded, the process advances to step S1204. In this step S1204, in order to correct the pixel located at the boundary with the tile on the right in the tile of interest (in the embodiment, since the size of one tile is 128 × 128 pixels, 128 pixels arranged in the vertical direction) The deblocking filter is performed. This is a filter applied to each boundary pixel of the irreversible tile as shown in FIG. Then, processing is performed using the left and right pixels for the target boundary pixel. As a deblocking filter, FIR is most often used to weight the pixel to be filtered. However, in this embodiment, since the reliability of the target boundary pixel is low, the same weighting as that of the boundary pixel of the right-side reversible tile is performed. P0 '= (1/5) * p1 + () where the boundary pixel of the non-reversible tile (that is, the pixel of interest) is p0, the left adjacent pixel is p1, and the right adjacent pixel (that is, the boundary pixel of the reversible tile) is q0. 2/5) * p0 + (2/5) * q0 is calculated. Then, the pixel value P0 is replaced with the pixel value P0 ′ obtained by this calculation. Note that p1 and q0 are adjacent to the pixel to be filtered, and if the weighting factor of q0 is set to a value larger than that of p1, p1 is a value generated from lossy encoded data. On the other hand, q0 is a value obtained from lossless encoded data and has higher reliability than P1. When step S1204 is completed and the process proceeds to step S1205, it is determined whether or not the encoding method of the tile next to the tile of interest is lossless encoding. A deblocking filter for correcting a pixel (128 pixels in the embodiment) located at the boundary with the lower adjacent tile in the target tile when it is determined that the lower adjacent tile is losslessly encoded. I do. This process is the same as that in step S1204, except that the application direction of the deblocking filter is horizontal or vertical, and the tile to be used is the right direction or the downward direction. For this reason, explanation is omitted. When step S1206 is finished, the image processing is finished.

  In the decoding process of this embodiment, the operation has been described as a program stored in the secondary storage device 803 shown in FIG. However, this may be implemented by hardware. An outline of the configuration in this case is shown in FIG. In the figure, reference numeral 806 denotes a code string input unit, and reference numeral 807 denotes a code string buffer. These blocks are used in the process of step S901 in FIG. Reference numeral 808 denotes a header analysis unit, which is used in step S902. Reference numeral 809 denotes a JPEG2000 decoding unit, which is used in step S908. Reference numeral 810 denotes a JPEG-LS decoding unit, which is used in step S907. Reference numeral 811 denotes a decoding tile buffer, which is used to store the decoding results of steps S907 and S908. An image processing unit 812 is used in steps S909 and S914. An output unit 813 is used in steps S910 and S915.

  As described above, according to the present embodiment, when the second type tile that has been losslessly encoded is adjacent to the first type tile that has been lossy encoded, distortion occurs at the boundary position. Can be further suppressed. In the above embodiment, the deblocking filter process is performed when the number of deleted bitplanes is 2. However, the number of deleted bitplanes may be 0 or 1.

[Second Embodiment]
In the first embodiment, when the losslessly encoded tile and the lossy encoded tile are both 1 or more, the maximum number of bit planes C that can be deleted from the lossy encoded data is a fixed value “2”. As explained. However, an example in which the client 201 (or 202) can set the maximum number of bit planes C will be described below. The client 201 transmits the image request information to be transmitted to the server 204 including the maximum number of bit planes C in addition to the information specifying the image and the allowable data amount D_max. The server performs the encoding process based on these pieces of information. Since the encoding process is clear from the first embodiment, the description thereof will be omitted and the client-side decoding process will be described. .

  Here, consider a case where the image quality difference between the reversible tile and the irreversible tile is encoded under a compression condition that causes more. For example, it is assumed that the request transmitted from the client 201 to the server 204 includes information for setting the maximum number C of deleted bit planes to 4. Then, consider a case where the server 204 returns encoded data in which reversible and lossy tiles are mixed and the 4-bit bit plane is deleted to the client 201. In this case, the difference in image quality between the reversible tile and the irreversible tile becomes larger. For this reason, in the first embodiment, the deblocking filters are applied in the right direction and the lower direction of the target irreversible tile, but deblocking filters in the upper direction and the left direction are also required. However, the reliability is different between the right or lower boundary pixel and the left or upper boundary pixel. For this reason, a deblocking filter is changed according to the boundary of filtering object. In the second embodiment, how to apply such a deblocking filter will be described. Note that the different processing compared to the first embodiment is the image processing in steps 910 and 915 in FIG. Therefore, in the following description, the process will be focused on.

  In FIG. 17, it is determined in step S1701 whether or not the tile of interest has been irreversibly encoded. If it has been losslessly encoded, this process ends. On the other hand, if it is determined that the tile of interest has been lossy encoded, the process advances to step S1702. In this step S1702, the filter information used in the deblocking filter processes A and B is read from the filter table stored in advance in the secondary storage device 803 using the number of deleted bit planes read from the file header to be decoded as a key. FIG. 23 is an example of the filter table. As shown in the figure, filter information used for two types of deblocking filter processes A and B is defined according to the number of deleted bitplanes. The deblocking filter process A is a filter process that corrects the pixels at the right end and the lower end in the tile, as in the first embodiment. The deblocking filter process B is a filter process for correcting the leftmost and uppermost pixels in the tile. In the figure, “-” indicates that the corresponding deblocking filter processing is not performed. For example, when the number of deleted bit planes is “2”, the deblocking filter process A is performed using the filter A0, but the deblocking filter process B is not performed. In the case of the embodiment, since it is a 3-tap filter process, three filter coefficients (weighting coefficients) are stored in one filter.

  In step S1703, it is determined whether or not the tile on the right side of the tile of interest has been reversible. If the tile on the right is reversible, it means that the tile on the right side of the target tile is in contact with the reversible tile. Therefore, the process proceeds to step S1704, the deblocking filter processing A is executed, and the pixel value at the right end in the target tile is set. to correct.

Here, for example, it is assumed that the number of deleted bit planes is 4, and three filter coefficients indicated by {1/5, 2/5, 2/5} are stored in the filter A2 used in the deblocking filter processing A. . In this case, the corrected pixel p0 ′ is calculated by performing the following filter processing on the rightmost pixel value p0 in the tile of interest.
p0 '= (1/5) * p1 + (2/5) * p0 + (2/5) * q0
Here, p1, p0, and q0 have the same meaning as in the first embodiment.

  In step S1705, it is determined whether or not the tile below the target tile is reversible. If the lower adjacent tile is reversible, it means that the lower tile is in contact with the reversible tile. Therefore, the process proceeds to step S1706, the deblocking filter process A is executed, and the lower end pixel value in the target tile is reached. Correct.

  In step S1707, it is determined whether the tile adjacent to the left of the tile of interest has been reversible. When the tile on the left is reversible, it means that the tile on the left side of the target tile is in contact with the reversible tile. Therefore, the process proceeds to step S1708, the deblocking filter process B is executed, and the pixel value at the left end in the target tile is set. to correct.

Here, for example, it is assumed that the number of deleted bit planes is 4, and three filter coefficients indicated by {1/6, 3/6, 2/6} are stored in the filter B0 used in the deblocking filter processing B. . In this case, the corrected pixel p0 ′ is calculated by performing the following filter processing on the left end pixel value p0 in the tile of interest.
p0 '= (1/6) * p1 + (3/6) * p0 + (2/6) * q0.

  As can be seen from the above description, the difference between the filter A and the filter B with respect to the number of deleted bits is that the latter has a large weight before the correction and reduces the influence in the adjacent lossless encoding tile. is there.

  In step S1709, it is determined whether or not the adjacent tile above the target tile is reversible. When the upper adjacent tile is reversible, it means that it is in contact with the reversible tile above the target tile, so the process proceeds to step S1710, and the deblocking filter processing B is executed, and the upper end pixel value in the target tile is set. to correct.

  As described above, when decoding an image in which lossless encoded tile data and lossy encoded tile data are mixed, adaptive decoding is performed according to the number of bit planes deleted for data amount control. A filter used in the blocking filter process is determined. As a result, distortion (noise) at the tile boundary can be suppressed, and a high-quality image can be obtained.

[Third embodiment]
The third embodiment allows not only JPEG2000 but also JPEG (ITU-T T.81 | ISO / IEC 10918-1: 1994) to be used as lossy encoding. That is, the tile is compressed with JPEG-LS, JPEG2000, or JPEG. The aim of introducing JPEG is to shorten the processing time. In other words, JPEG2000 has the advantages of high compressibility and “easy control of code amount”, but it takes time for the encoding process. On the other hand, although JPEG is inferior to JPEG2000 in terms of compression performance and code amount control, the encoding process is simple. For this reason, the processing time of the entire encoding process can be shortened by applying JPEG to some tiles in the photographic area.

  As described in the first embodiment, when attention is paid to the tile of the decoding result of JPEG 2000, there is a characteristic that the reliability of the tile boundary pixel in the right direction and the downward direction in the tile becomes low. JPEG does not have such characteristics, and the reliability of pixels at any boundary does not change. In the third embodiment, attention is paid to this point, and a deblocking filter processing method in the case of using the irreversible method in which the reliability changes depending on the position of the boundary pixel and the irreversible method in which the reliability does not change will be described.

  The encoding method in the third embodiment is different from that in the first embodiment because JPEG is used, but the gist of the present embodiment is only how to use the deblocking filter. Therefore, the description of the encoding method is simplified.

  In encoding, first, whether to apply JPEG2000 or JPEG to tiles in a photographic area is determined according to how many edges of a predetermined size or more are distributed in the tiles. Specifically, if many large edges appear, it is encoded with JPEG, otherwise it is encoded with JPEG2000. The reason for this is that the more high frequency components, the longer the processing time of JPEG2000.

  Next, regarding code amount control, after encoding the tile, the sum of the data of tiles compressed with JPEG-LS (D_ls), the sum of the data of tiles compressed with JPEG2000 (D_j2k), and the data of tiles compressed with JPEG Find the sum of (D_jp). Then, the sum (D_sum) of D_ls, D_j2k, and D_jp is calculated and compared with the upper limit code amount D_max as a result of compressing the image. If D_sum ≦ D_max, the code amount control is unnecessary and is output as it is. On the other hand, when D_sum> D_max, code amount control is required. First, a code amount (D_j2k_c) when a JPEG2000 tile is deleted by a 2-bit plane and a code amount (D_jp_c) when a JPEG tile (JPEG tile) is deleted by a 1-bit plane are obtained. This is the maximum number of bit planes that can be deleted so that the difference in image quality between the reversible tile and the irreversible tile is within an allowable range. Regarding this deletion, JPEG2000 data has a data structure that can be easily deleted, but JPEG data does not have such a structure. Therefore, JPEG-encoded tiles (JPEG tiles) are locally decoded and re-encoded. Specifically, all the JPEG tile data is temporarily copied to the memory. Then, each JPEG tile is Huffman-decoded to generate a quantization index (quantized DCT coefficient). Then, the lower 2-bit plane of each quantization index is deleted and Huffman coding is performed. By this series of processing, bit plane deletion of JPEG data is performed. Then, the sum (D_sum_c) of D_ls, D_j2k_c, and D_jp_c is calculated. If D_sum_c is smaller than D_max, the code amount control is performed on the JPEG2000 tile and the JPEG tile without re-encoding the JPEG-LS tile data. In this code amount control, first, JPEG2000 tiles are tiled one by one, and the lower one bit plane is deleted in raster scan order. If the total data of JPEG-LS, JPEG, and JPEG200 exceeds D_max even if the lower 1 bit plane of all JPEG2000 tiles is deleted, the lower 1 bit plane is deleted in the raster scan order for each JPEG tile. Even if the lower 1 bit plane of all JPEG tiles is deleted, if the data sum of JPEG-LS, JPEG, and JPEG 200 exceeds D_max, 1 bit plane is further deleted from the lower bit plane of the JPEG 2000 tile.

  On the other hand, if D_sum_c is larger than D_max, the JPEG-LS tile is re-encoded to JPEG2000, and the code amount control is performed. In this processing, the reason for re-encoding to JPEG2000 instead of JPEG is to avoid the complexity of the processing in the code amount control of JPEG data. Note that the code amount control after JPEG-LS re-encoding is the same as the process when D_sum_c is smaller than D_max, and thus description thereof is omitted.

  Next, a description will be given of how to apply a deblocking filter in the decoding process.

  In the first embodiment, the method of performing the deblocking filter on the boundary where the reversible tile and the irreversible tile are adjacent to each other has been described. The aim of the method was to alleviate the difference in image quality that occurs outside the photographic area of interest, because the area other than the photographic area is divided into a reversible tile and a non-reversible tile. In this embodiment, the aim is expanded. Specifically, regardless of whether or not there is a non-photograph area inside the irreversible tile, when tiles of different encoding methods are adjacent to each other, the generated distortion differs, so there is a difference in image quality between the two tiles. I think it has occurred. Then, apply a deblocking filter. That is, when the JPEG tile and the JPEG2000 tile are adjacent to each other, there are cases in which these two parts divide other than the photograph area as shown in FIG. 18 (a), or as shown in FIGS. 18 (b), (c), and (d). , Sometimes not. In the third embodiment, in any case, the deblocking filter is applied to the boundary where the JPEG tile and the JPEG2000 tile are adjacent to each other. Hereinafter, how to apply such a deblocking filter will be described. Note that the different processing between the first embodiment and this embodiment is the image processing in steps 910 and 915 in FIG. Therefore, in the following description, the process will be focused on.

  In FIG. 19, it is determined in step 1901 whether the target tile is a JPEG2000 tile. If it is a JPEG2000 tile, the target JPEG2000 tile and adjacent tiles are filtered in step S1902. This process will be described in detail later. If the target tile is not a JPEG2000 tile in step S1902, it is determined in step S1903 whether the target tile is a JPEG tile. If it is not a JPEG tile, the image processing in steps S910 and 915 ends. On the other hand, if the target tile is a JPEG tile, the target JPEG tile and adjacent tiles are filtered in step S1904. This process will be described in detail later. When the process of step S1904 is finished, the image processing of steps S910 and 915 is finished.

  Next, the process of step S1902 will be described with reference to FIG. In step S2001, it is determined whether the tile adjacent to the right of the tile of interest (the right adjacent tile) is a JPEG-LS tile. If it is a JPEG-LS tile, the deblocking filter processing shown in the first embodiment is performed in step S2002. Hereinafter, this deblocking filter process is referred to as filter process A. When this process ends, the process proceeds to step S2006.

  On the other hand, if it is determined in step S2002 that the right adjacent tile is not a JPEG-LS tile, the process advances to step S2003 to determine whether the right adjacent tile is JPEG. If it is JPEG, filtering processing B is performed on the rightmost pixel located on the side in contact with the JPEG tile in the target tile (tile encoded with JPEG2000) in step S2004. The filter used in this filtering process is p0 ′ = (1/3) * p1 + (1/3) * p0 + (1/3) * q0. The reason why the coefficient of the target boundary pixel in this equation is the same as the coefficient of the adjacent pixel is that the target tile is a JPEG2000 tile and the reliability of the target boundary pixel is low. Further, in step S2005, since the right adjacent tile is an irreversible tile, the deblocking filter C is also applied to the boundary pixel located at the left end in the right adjacent tile. The filter is q0 ′ = (1/6) * p0 + (3/6) * q0 + (2/6) * q1. Here, the description of the pixels p0, q0, and q1 is as shown in FIG. In the filter processing C, the reason why the coefficients are different between the adjacent pixels p0 and q1 adjacent to the boundary pixel is that the reliability of p0 is low. When step S2005 is completed, the process proceeds to step S2006.

  In step S2006, it is determined whether the upward adjacent tile (upper adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process proceeds to step S2008. On the other hand, if it is a JPEG-LS tile, the filter B in the second embodiment is performed, and the process proceeds to step S2008. This filter is hereinafter referred to as filter D. In step S2008, it is determined whether the adjacent tile in the left direction (left adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process proceeds to step S2010. On the other hand, if it is a JPEG-LS tile, filter D is performed. In step S2010, it is determined whether the downward adjacent tile (lower adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process proceeds to step S2012. On the other hand, if it is a JPEG-LS tile, filter A is performed in step S2011. In step S2012, it is determined whether the lower adjacent tile is a JPEG tile. If it is not a JPEG tile, the process ends. On the other hand, if it is a JPEG tile, the filter B is applied to the boundary pixel of the target tile in step S2013, and the filter C is applied to the boundary pixel of the lower adjacent tile in step S2014. Then, the process ends.

  Next, the process of step S1904 will be described with reference to FIG. In step S2201, it is determined whether the tile adjacent to the right of the tile of interest (right adjacent tile) is a JPEG-LS tile. If it is a JPEG-LS tile, filtering is performed on the boundary pixel of the target tile in step S2202. This filtering is p0 '= (1/6) * p1 + (3/6) * p0 + (2/6) * q0. In this equation, the reason why the coefficients are different between adjacent pixels p1 and q0 adjacent to the boundary pixel is that q0 is a pixel of a reversible tile. This deblocking filter is referred to as filter E. After performing filter E, the process proceeds to step S2206. On the other hand, if it is determined in step S2201 that the right adjacent tile is not JPEG-LS, it is determined in step S2203 whether the right adjacent tile is a JPEG2000 tile. If it is determined that the tile is a JPEG2000 tile, filtering is performed on the rightmost pixel of the tile of interest in step S2204. This filtering is called p0 ′ = (1/5) * p1 + (3/5) * p0 + (2/5) * q0 and is called filter F. In step S2205, since the right adjacent tile is an irreversible tile, a deblocking filter is also applied to the left end pixel in the right adjacent tile. The filter equation is q0 ′ = (1/5) * p0 + (3/5) * q0 + (2/5) * q1 and is called filter G. Note that the filter F and the filter G have the same coefficients, although the pixels used for filtering are different. When step S2205 is completed, the process proceeds to step S2206.

  In step S2206, it is determined whether the upward adjacent tile (upper adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process advances to step S2208. On the other hand, if it is a JPEG-LS tile, filter E is performed and the process proceeds to step S2208. In step S2208, it is determined whether the adjacent tile in the left direction (left adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process advances to step S2210. On the other hand, if it is a JPEG-LS tile, filter E is performed in step S2209. In step S2210, it is determined whether the adjacent tile in the downward direction (lower adjacent tile) is a JPEG-LS tile. If it is not a JPEG-LS tile, the process advances to step S2212. On the other hand, if it is a JPEG-LS tile, filter E is performed in step S2211. In step S2212, it is determined whether the lower adjacent tile is a JPEG tile. If it is not a JPEG tile, the process ends. On the other hand, if it is a JPEG tile, the filter F is applied to the boundary pixel of the target tile in step S2213, and the filter G is applied to the boundary pixel of the lower adjacent tile in step S2214. Then, the process ends.

(Other examples)
In the above-described embodiment, an example in which the server and the client communicate with each other has been described. However, the present invention is limited by the above-described embodiment because a single apparatus may perform both encoding and decoding. It is not something.

  In the above embodiment, a method using JPEG2000, JPEG-LS, or JPEG as an encoding method has been described. However, the present invention is not limited to this. Moreover, although two cases and three cases have been shown for combinations of encoding methods, the present invention is not limited to this.

  Although the number of taps of the deblocking filter has been described as three, the present invention is not limited to this. Further, the filter coefficient may be appropriately changed according to the mounting form. For example, it is also within the scope of the present invention to change according to compression conditions such as quantization applied to tiles and bit plane deletion.

  A pixel with low reliability has been described as a tile boundary pixel. However, a filtering method that reduces the weighting of a pixel other than the tile boundary, for example, when the reliability decreases in the vicinity of the tile boundary, is also within the scope of the present invention. .

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

The encoded data of the first type tile encoded by the irreversible first encoding means, and the second encoded by the second encoding means different from the first encoding means An image processing apparatus that decodes encoded image data in which encoded data of a type tile is mixed,
Identifying means for identifying which encoded means the encoded data of each tile is encoded data;
A decoding processing means for generating a tile image by executing a corresponding type of decoding processing on the encoded data of each tile according to the identification result of the identification means;
A tile in which the first type tile and the second type tile are adjacent to each other in an image composed of tile images obtained by the decoding processing by the decoding processing unit based on the identification result by the identification unit. Detection means for detecting the boundary;
A value of a pixel that touches the tile boundary in the first type tile that touches the tile boundary detected by the detection means has a size across the tile boundary, and the first encoding means and the second Filter processing means for performing filter processing using a filter having characteristics according to the reliability of the encoding means;
An image processing apparatus comprising:   The filter processing means executes filter processing using a filter having a plurality of weighting factors based on high reliability of the first encoding means and the second encoding means. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein the first encoding unit is a JPEG2000 encoding unit, and the second encoding unit is a reversible encoding unit.   The image processing apparatus according to claim 3, wherein the second encoding unit is a JPEG-LS encoding unit.   Among the right end pixel, the lower end pixel, the left end pixel, and the upper end pixel in the first type tile, the reliability of the right end pixel and the lower end pixel is lower than the reliability of the left end pixel and the upper end pixel, and the tile boundary 5. The storage device according to claim 3, further comprising: a storage unit configured to store a filter when the tile boundary is located on the right side or the lower side and a filter coefficient of each filter when the tile boundary is located on the left side or the upper side. Image processing apparatus.   3. The image processing apparatus according to claim 1, wherein the first encoding unit is a JPEG2000 encoding unit, and the second encoding unit is an irreversible JPEG encoding unit. .   When the second type tile is adjacent to the right side or the lower side of the first type tile, the filtering means touches the second type tile in the first type tile. The image processing apparatus according to claim 6, wherein the pixel on the side is filtered, and the pixel that is in contact with the first type tile in the second type tile is also filtered.   The plurality of coefficients of the filter used by the filter processing unit are further determined based on an encoding parameter related to code amount control by the first encoding unit. The image processing apparatus according to item.   The image processing apparatus according to claim 8, wherein the encoding parameter includes the number of bit planes deleted when performing lossy encoding. The encoded data of the first type tile encoded by the irreversible first encoding means, and the second encoded by the second encoding means different from the first encoding means A method for controlling an image processing apparatus that decodes encoded image data in which encoded data of a tile of a type is mixed,
An identification step in which the identification means identifies which encoded data of each tile is encoded data;
A decoding processing step in which a decoding processing unit generates a tile image by executing a corresponding type of decoding processing on the encoded data of each tile according to the identification result of the identification step;
Based on the identification result in the identification step, the detection means includes the first type tile and the second type tile in the image composed of tile images obtained by the decoding process in the decoding process step. Detecting a tile boundary adjacent to each other;
The filter processing means has a size of the pixel that touches the tile boundary in the first type of tile that touches the tile boundary detected by the detection step, and has a size across the tile boundary and the first encoding. And a filter processing step of performing a filter process using a filter having characteristics according to the reliability of the means and the second encoding means;
A control method for an image processing apparatus, comprising:   The program for making the said computer perform each process of Claim 9 by making a computer read and run.   A computer-readable storage medium storing the program according to claim 11.

Images