JP2005184086A - Image processing apparatus, image processing method, program and information recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a code stream from which the contents of an image can be grasped at an early stage of progressive decoding. <P>SOLUTION: A region determining processor 34 for determining whether a region is character region or non-character region per tile is provided on an encoder for performing coding processing for an image, and a result of the determination is inputted into a code stream generation processor 37. The processor 37 generates a progressive code stream of a specific progressive order, and controls the hierarchy configuration of the code stream in units of tile according to the result of region determination. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to the field of image processing, and more particularly, to generation and progressive decoding of a progressive code stream of an image.

  In recent years, in addition to the popularization of digital cameras and scanners, the speed of the Internet has increased, and high-definition multi-valued still images can be easily handled by home users. Usually, these multilevel still images have a very large amount of information, and therefore some kind of compression encoding processing is performed.

  Currently, the most commonly used compression coding method for color still images is the JPEG baseline system recommended by the Joint Photographic Experts Group IS (International Standard) 10918-1 (ITU-T T.81). In addition to the baseline system, JPEG has also standardized an extended system (IS 10918-3: ITU-T T.84) with expanded functions such as progressive. As a next-generation compression method, standardization (part 1) of JPEG2000, which has higher image quality and more functionality than JPEG, was completed in December 2000 (IS 15444-1).

  Usually, when transferring image data, the image data compressed and encoded by the compression method as described above is transferred via a network, decoded on the receiving side, displayed on a display device such as a monitor, or on paper. The image is recorded on a recording medium such as For example, in a client / server environment, when the client side receives image data and displays it on a monitor or the like, the data is transferred under conditions where the bandwidth of the transfer path is narrow, such as the Internet, and it takes time to receive the image data. A progressive encoding method is employed in which a coarse image is displayed on the screen at the initial stage of image data transfer and a fine image is displayed as image data transfer proceeds.

  JPEG defines a progressive encoding system whose main purpose is to display an image via a transfer path in its extended system, but JPEG2000 defines a progressive encoding system from the basic system. .

  In the embodiment of the present invention described later, since JPEG2000 is used as a compression encoding method, an outline of the JPEG2000 basic method and progressive order will be described here.

  FIG. 26 is a basic block diagram of compression encoding processing in JPEG2000part1 (hereinafter JPEG2000). Here, description will be made assuming that color Red, Green, Blue (hereinafter referred to as RGB) image data is handled as input image data.

  The input RGB image data is divided into rectangular blocks (called tiles) by the tiling processing unit 222 and input to the color conversion unit 223 in tile units. When raster format image data is input, the tiling processing unit 222 performs raster / block conversion.

  In JPEG2000, encoding or decoding can be performed independently for each tile. This contributes to the multi-functionality of JPEG2000, such as reducing the amount of hardware when encoding / decoding by hardware, and decoding and displaying only the necessary tiles. Yes. In JPEG2000, tiling is an option, and tiling can be omitted.

  Next, the image data is converted into a luminance / color difference signal by the color conversion processing unit 123. In JPEG2000, two types of color conversion are defined corresponding to the types of filters (5x3 and 9x7) used for discrete wavelet transform (hereinafter DWT). For example, when using a 5x3 filter capable of reversible conversion Uses reversible color conversion according to the following equation (1).

なお、上記の色変換に先立ち、RGBの各信号毎にＤＣレベルシフトが行われる。ＤＣレベルシフトは、例えば入力RGB信号が８bitの場合は、下記の式（２）により表される。

Prior to the above color conversion, a DC level shift is performed for each RGB signal. The DC level shift is expressed by the following equation (2) when the input RGB signal is 8 bits, for example.

色変換後の信号はＤＷＴ処理部224において各コンポーネント毎に離散ウェーブレット変換（ＤＷＴ）を施される。ＤＷＴは２次元にて行われるが、通常は、リフティング演算と呼ばれる演算方法により、１次元フィルタ演算のコンボリューションにて実施される。１次元の変換式を式（３）に示す。

The signal after color conversion is subjected to discrete wavelet transform (DWT) for each component in the DWT processing unit 224. Although DWT is performed in two dimensions, it is usually performed in a convolution of a one-dimensional filter operation by an operation method called lifting operation. A one-dimensional conversion formula is shown in Formula (3).

ＤＷＴはダウンサンプリングを伴うため、上記L(k)、H(k)は入力画像と比較し１／２の解像度となる。

Since DWT involves downsampling, the above L (k) and H (k) have half the resolution compared to the input image.

  FIG. 27 is a diagram illustrating wavelet coefficients that have been divided into octaves. DWT outputs directional components called LL, HL, LH, and HH for each decomposition (decomposition) level. By recursively performing DWT on LL, lower resolution is achieved. Raise the decomposition level. Decomposition level 1 coefficients with the highest resolution are represented as 1HL, 1LH, 1HH, decomposition level 2 coefficients are represented as 2HL, 2LH, 2HH, and so on. FIG. 27 shows subband division at 3 decomposition levels. Also, the numbers written on the right shoulder of each subband in FIG. 27 indicate the resolution level.

  It is possible to divide each decomposition level subband into rectangular areas called precincts to create a set of codes. Each precinct is divided into predetermined rectangular blocks called code blocks, and encoding is performed in units of code blocks.

  Now, scalar quantization is performed on the wavelet coefficients output from the DWT processing unit 224 by the quantization processing unit 225. When reversible DWT is performed, scalar quantization is not performed. Quantization with a size of 1 is performed. Also, post quantization by the post quantization unit 227 in the subsequent stage can provide substantially the same effect as scalar quantization. Scalar quantization parameters can be changed on a tile basis.

  The entropy coding unit 226 performs entropy coding on the quantized wavelet coefficients output from the quantization processing unit 225. In the entropy encoding method in JPEG2000, a subband is divided into rectangular areas called code blocks (however, when the size of the subband area is equal to or smaller than the code block size, it is not divided), and encoding is performed in units of code blocks.

  In this encoding, as schematically shown in FIG. 28, the wavelet coefficients in the code block are decomposed into bit planes, and the bit planes are divided into three encoding propagation paths (Significance propagation paths) according to the state representing the degree of influence on image quality. , Magnitude refinement pass, Clean up pass), and is encoded by an arithmetic coding method called MQ coder. The bit plane has a higher importance (contribution to image quality) in the order of the MSB side and the encoding pass in the order of significance propagation, magnitude refinement, and cleanup. Also, the end of each path is also called a truncation point, and is a unit that can be used for truncating the code in post-quantization at the later stage.

  The post-quantization unit 227 performs code truncation (truncation) on the code data generated by entropy coding as necessary. However, post-quantization is not performed when it is necessary to output a reversible code. In this way, the code amount can be controlled by truncating the code after encoding, and the feedback control is not required for the code amount control (one-pass encoding). This is one of the features of JPEG2000.

  The code stream generation processing unit 228 generates a code stream by rearranging codes and adding a header to the post-quantized code data according to a progressive order described later.

  FIG. 29 is a schematic diagram of a code stream for layer progression in JPEG2000. The code stream includes a main header and one or more tile data. Each tile data is composed of a tile header and a plurality of layers obtained by dividing the code in the tile into code units (to be described later) called layers. Layer 0, layer 1... Are lined up. A layer is composed of a tile header for the layer and a plurality of packets, and the packet is composed of a packet header and code data.

  The packet is a minimum unit of code data, and is composed of code data of one layer in one precinct at one resolution level (decomposition level) in one tile component.

  Next, the progressive order in JPEG2000 will be described. In JPEG2000, by changing the priority order of the four image elements of image quality (layer (L)), resolution (R), component (C), and position (precinct (P)), the following five progressions (Progressive order) is defined.

  LRCP progression: Since decoding is performed in the order of precinct, component, resolution level, and layer, the image quality of the entire image is improved each time the layer index is advanced, and image quality progression can be realized. Also called layer progression.

  RLCP progression: Decoding is performed in the order of precinct, component, layer, and resolution level, so that resolution progression can be realized.

  RPCL progression: Since decoding is performed in the order of layer, component, precinct, and resolution level, as with RLCP progression, although it is resolution progression, the priority of a specific position (precinct) can be increased.

  PCRL progression: Since decoding is performed in the order of layer, resolution level, component, and precinct, decoding of a specific part is prioritized and spatial position progression can be realized.

  CPRL progression: Since decoding is performed in the order of layer, resolution level, precinct, and component, for example, in the case of progressive decoding of a color image, component progression that first reproduces a gray image can be realized.

  FIG. 30A schematically shows the progressive order of LRCP progression (hereinafter referred to as layer progression), and FIG. 30B schematically shows the progressive order of RLCP progression or RPCL progression (hereinafter referred to as resolution progression). The horizontal axis is the decomposition level (the higher the level, the lower the resolution), and the vertical axis is the layer number (the higher the number, the lower the layer, by decoding by adding the lower layer code to the upper layer, Higher quality playback is possible). In the figure, a filled rectangle represents a code in each decomposition level and layer, and its size schematically indicates a code amount ratio. Dotted arrows in the figure represent the code order.

  The broken line in (A) represents the decoding order (code order) in layer progression. Decoding of all resolutions of the highest layer is performed, and decoding of the next (lower) layer is performed. At the wavelet coefficient level, decoding is performed from the higher bits of the coefficient, and progressive decoding with gradually improving image quality is possible.

  The broken line in (B) represents the decoding order (code order) in resolution progression. After decoding all the layers having the highest decomposition level, decoding of the next decomposition level is performed, and progressive decoding with gradually improving resolution is possible.

  In FIG. 30, the concept of components and precincts is omitted. For understanding, the arrangement of packets in LRCP progression and RLCP progression is schematically illustrated in FIGS. 31 and 32.

  In recent years, copying machines (multifunction printers; MFPs) that integrate functions such as scanners, printers, and copiers have become very popular. Such an MFP includes a large-capacity storage device such as a hard disk device (hereinafter referred to as HDD), reads a plurality of originals only once, and stores (accumulates) the image data in the HDD or the like. Even if multiple print errors occur due to a function called “electronic sort” that outputs multiple copies of a document in page order, or an error such as paper misfeed (paper jam), the document is automatically read without re-reading. A function to perform reprinting is provided. In addition to these stand-alone functions, scanned image data can be transferred to a PC, sent as an e-mail attachment, or scanned image data can be temporarily stored in the HDD. A document distribution function via a network is provided to transfer files to a personal computer as necessary.

  However, conventionally, even if a document image is progressively encoded from such an MFP and transferred to a personal computer or the like, and the progressive decoding is performed by the personal computer or the like, the overall layout of the document can be grasped at an early stage of transfer, but the character There is a problem that the contents of the document cannot often be grasped because of being unreadable.

  In order to solve such a problem, for example, use of the technique described in Patent Document 1 can be considered. The technique described in Patent Document 1 performs layout analysis that classifies image data according to its attributes, divides each of the areas segmented by layout analysis into tiles, and attributes of image data included in the tiles obtained by the division Are associated as tile attributes, and the progressive order is switched according to the associated tile attributes for encoding. More specifically, tiles with character attributes select SNR progression (layer progression), and tiles other than characters select resolution progression.

  However, there is another problem with the method of switching the progressive order. For example, since the progressive order is switched at the boundary between a tile including characters and a tile not including characters, it tends to appear uncomfortable on the image. In addition, characters on a pattern often seen on a map, a flyer, or the like may be erroneously determined as a “photo” attribute.

  In the present invention, an area determination technique for determining the image type is used. There are a large number of publicly-known documents related to area determination, and examples thereof include Patent Documents 2, 3, and 4.

JP 2003-23544 A JP 2000-175032 A JP 2003-101770 A JP 11-220622 A

  In view of the above problems, the present invention is intended to provide a novel image processing apparatus and image processing method for enabling grasping of image contents at an early stage of progressive decoding of a code stream.

  According to the first aspect of the present invention, there is provided encoding means for performing an image encoding process and outputting a progressive code stream of an image in a predetermined progressive order, and an area determination means for determining an image type of the image in a predetermined block unit. Code stream generation means for generating the progressive code stream in the encoding means, and means for controlling the hierarchical division structure of the progressive code stream for each block according to the determination result by the area determination means Is an image processing apparatus including:

  The invention of claim 2 takes in the code stream stored in the storage means for storing the code stream of the image, the area determination means for determining the image type of the image in a predetermined block unit, and the code stored in the storage means. Code stream generation means for generating a progressive code stream in a predetermined progressive order from the stream, wherein the code stream generation means determines the hierarchical division configuration of the progressive code stream for each block by the area determination means An image processing apparatus including means for controlling according to a result.

  The invention according to claim 3 is characterized in that the control means included in the code stream generation means controls the code amount of the progressive code stream in a hierarchical unit to a predetermined set code amount. Alternatively, the image processing apparatus according to the second invention is provided.

  The invention according to claim 4 is characterized in that the control means included in the code stream generation means switches the set code amount in units of layers of the progressive code stream according to the determination result by the area determination means. An image processing apparatus according to the third aspect of the invention.

  According to a fifth aspect of the present invention, at least one of a character, a black character, a color character, and a halftone dot is included in the image type determined by the region determining means. An image processing apparatus according to the invention.

  The invention according to claim 6 is the invention according to claim 1 or 2, characterized in that the region determination means determines an image type based on a frequency conversion coefficient of an image generated in the course of an image encoding process. An image processing apparatus.

  The invention according to claim 7 is the image processing apparatus according to claim 2, wherein the area determination unit determines an image type based on a code stream fetched by the code stream generation unit.

  According to an eighth aspect of the present invention, there is provided encoding means for performing an image encoding process and outputting an image progressive code stream in a predetermined progressive order, and feature amount detection for detecting image feature amounts in predetermined block units. And a code stream generation means for generating the progressive code stream in the encoding means, wherein the hierarchical division structure of the progressive code stream is configured for each block according to the feature amount detected by the feature amount detection means. An image processing apparatus including means for controlling in stages.

  The invention of claim 9 is a storage means for storing a code stream of an image, a feature quantity detection means for detecting a feature quantity of an image in a predetermined block unit, and a code stream stored in the storage means. Code stream generation means for generating a progressive code stream in a predetermined progressive order from the code stream, wherein the code stream generation means has a hierarchical division configuration of the progressive code stream for each block, the feature amount detection means The image processing apparatus includes means for controlling in multiple stages in accordance with the feature amount detected by the method.

  The invention of claim 10 is characterized in that the control means included in the code stream generating means controls the code amount of the progressive code stream in a hierarchical unit to a predetermined set code amount. Or an image processing apparatus according to the invention of (9).

  In the invention of claim 11, the control means included in the code stream generation means switches the set code quantity of the progressive code stream in units of layers according to the feature quantity detected by the feature quantity detection means. An image processing apparatus according to the invention of claim 10.

  According to a twelfth aspect of the invention, there is provided the image processing apparatus according to the eighth, ninth, tenth or eleventh aspect of the invention, wherein the feature quantity of the image detected by the feature quantity detecting means is an edge quantity.

  According to a thirteenth aspect of the present invention, the feature quantity detecting means detects a feature quantity based on a frequency conversion coefficient of an image generated in the course of an image encoding process. Is an image processing apparatus.

  The invention according to claim 14 is the image processing device according to claim 9, wherein the feature quantity detection means detects a feature quantity based on the code stream fetched by the code stream generation means.

  According to a fifteenth aspect of the present invention, there is provided an image processing apparatus that receives a progressive code stream of an image and performs progressive decoding, an area determination unit that determines an image type of an image in a predetermined block unit, and a received progressive code stream An image processing apparatus comprising: means for controlling a hierarchical division configuration of progressive decoding for each block according to a determination result by the area determination means.

  According to a sixteenth aspect of the present invention, there is provided an image processing apparatus that receives a progressive code stream of an image and performs progressive decoding, the feature amount detecting means for detecting a feature amount of the image in a predetermined block unit, and the received progressive code stream And a means for controlling the hierarchical division structure of progressive decoding for each block in multiple stages according to the feature quantity detected by the feature quantity detection means for each block.

  The invention of claim 17 includes an area determination step of determining an image type of an image in a predetermined block unit, and a code stream generation step of generating a progressive code stream of the image according to a predetermined progressive order, the code stream The generation step is an image processing method characterized by including a step of controlling the hierarchical division configuration of the progressive code stream to be generated for each block according to the determination result by the area determination unit.

  The invention according to claim 18 includes a feature amount detection step of detecting a multi-value feature amount of an image in a predetermined block unit, and a code stream generation step of generating a progressive code stream of the image in a predetermined progressive order. The code stream generation step includes a step of controlling the hierarchical division configuration of the progressive code stream to be generated for each block in multiple stages according to the feature amount detected by the feature amount detection step. Is the method.

  The invention of claim 19 is an image processing method for performing progressive decoding by receiving a progressive code stream of an image, comprising: an area determination step for determining an image type of an image in a predetermined block unit; and the received progressive code stream An image processing method comprising: controlling a hierarchical division configuration of progressive decoding for each block according to a determination result of the region determination step.

  The invention of claim 20 is an image processing method for performing progressive decoding by receiving a progressive code stream of an image, the feature amount detecting step for detecting a feature amount of the image in a predetermined block unit, and the received progressive code stream And a step of controlling the hierarchical division configuration of progressive decoding for each block in multiple stages according to the feature amount detected by the feature amount detection step for each block.

  A twenty-first aspect of the invention is a program that causes a computer to function as each unit of the image processing apparatus according to any one of the first to sixteenth aspects of the invention.

  A twenty-second aspect of the invention is a computer-readable information recording medium on which a program according to the twenty-first aspect of the invention is recorded.

  According to the first to seventh and seventeenth aspects of the present invention, a progressive code stream having a hierarchical division configuration suitable for the image type can be generated by controlling the hierarchical division configuration according to the region determination result, and is transferred to an external device. In the case of progressive decoding, it is possible to grasp the image contents at an early stage of transfer or decoding. In addition, since the progressive order of the progressive code stream is not switched depending on the region determination result, there is no sense of incongruity as seen in the method of switching the progressive order at the boundary between regions of different image types. According to the second aspect of the present invention, the progressive order and the hierarchical division configuration of the code stream stored in the storage means are arbitrary, and need not be progressive, so that they are suitable for use inside the image processing apparatus. Can be stored in the storage means. On the other hand, when requested by an external device, the progressive code stream of the hierarchical division configuration suitable for the image type in the progressive order suitable for the external device Can be generated. Further, according to the third and fourth aspects of the invention, the code amount of the progressive code stream in units of layers can be guaranteed, which is advantageous when transferring to an external device via a low-speed transfer path. Furthermore, according to the invention of claim 4, for example, by increasing the set code amount of the hierarchical unit in the character area as compared with the non-character area, the character readability at the early stage of the transfer and decoding of the progressive code stream is improved. be able to. According to the sixth and seventh aspects of the present invention, the area determination process can be completed within the hardware or software for the encoding process or the code stream generation process, and the area determination process is relatively simple. Since the processing can be performed, it is possible to obtain various effects that are generally advantageous for simplification of hardware and software related to area determination.

  According to the eighth to fourteenth and eighteenth aspects of the present invention, a progressive code stream having a layer division configuration suitable for the image type can be generated by controlling the layer division configuration according to the feature quantity, and therefore, the progressive code stream can be transferred to an external device. Thus, in the case of progressive decoding, it is possible to grasp the image content at an early stage of transfer or decoding. Further, in the configuration in which the hierarchical division configuration is controlled according to the region determination result, the decoded image is likely to feel uncomfortable as a result of the hierarchical division configuration corresponding to the image type different from the original image type in the erroneous determination region. Such a sense of incongruity is less likely to occur by controlling the structure in stages according to the feature amount. In addition, since the progressive order of the progressive code stream is constant regardless of the feature amount, there is no sense of incongruity as seen in the method of switching the progressive order at the boundary between regions having different feature amounts. According to the ninth aspect of the present invention, the progressive order and the hierarchical division configuration of the code stream stored in the storage means are arbitrary, and need not be progressive, and are therefore suitable for use inside the image processing apparatus. Can be stored in the storage means. On the other hand, when requested by an external device, the progressive code stream of the hierarchical division configuration suitable for the image type in the progressive order suitable for the external device Can be generated. According to the inventions of claims 10 and 11, the code amount of the progressive code stream can be guaranteed in units of layers, which is advantageous when transferring to an external device through a low-speed transfer path. Furthermore, according to the invention of claim 11, for example, by increasing the set code amount of the hierarchical unit in the character area larger than the non-character area, the character readable property at an early stage of the transfer and decoding of the progressive code stream is improved. Can be made. Further, although the edge amount is detected as the feature amount in the invention of claim 12, since the edge amount has a good correspondence with the code amount of the entropy coding and the character-likeness, the character readability is improved. Hierarchical division configuration control can be performed accurately. According to the inventions of claims 13 and 14, the feature amount detection process can be completed within the hardware or software for the encoding process or code stream generation process, and the feature amount detection process is relatively simple. Therefore, it is possible to obtain advantageous effects that are generally advantageous for simplification of hardware and software related to feature amount detection.

  According to the fifteenth and nineteenth aspects of the present invention, since the hierarchical division configuration of the progressive decoding of the received code stream is controlled according to the region determination result, it is possible to grasp the image content at an early stage of decoding. In addition, since the progressive order of decoding cannot be switched according to the region determination result, it is possible to obtain an effect such as not causing a sense of incongruity as seen in the method of switching the progressive order at the boundary between regions of different image types.

  According to the inventions of claims 16 and 20, since the hierarchical division structure of progressive decoding of the received code stream is controlled stepwise according to the feature amount, it is possible to grasp the image content at an early stage of decoding. . In addition, in the configuration in which the hierarchical division configuration is controlled according to the region determination result, the decoded image is likely to be uncomfortable as a result of the hierarchical division configuration suitable for the image type different from the original image type in the erroneous determination region. Since the structure is controlled stepwise according to the feature amount, such a sense of incongruity is unlikely to occur. In addition, since the progressive order is constant regardless of the feature amount, it is possible to obtain an effect such as not causing a sense of incongruity as seen in the method of switching the progressive order at the boundary between regions having different feature amounts.

  According to the inventions of claims 21 and 22, the inventions of claims 1 to 20 can be easily implemented using a computer.

  As an example of an embodiment of the present invention, an image processing apparatus used in a server / client environment as shown in FIG. 1 can be cited.

  In FIG. 1, a server 1 is an image processing apparatus such as an MFP, and a client 3 is an image processing apparatus capable of progressive decoding processing, a personal computer, or the like. A transfer path 2 between the server 1 and the client 3 is a network line such as a LAN, an intranet, or the Internet. The server 1 receives a command from the client 3 and transmits a progressive code stream of an image. The client 3 receives the progressive code stream and performs progressive decoding, thereby displaying the image on the monitor device 4 so that the image quality is improved step by step.

  The inventions of claims 1 to 14, 17 and 18 relate to an image processing apparatus such as an MFP used as the server 1. The inventions of claims 15, 16, 19, and 20 relate to an image processing apparatus used as the client 3. However, the present invention can also be applied to a stand-alone image processing apparatus having a function of generating a progressive code stream of an image and progressively decoding it.

  Hereinafter, embodiments of the present invention will be described in detail with reference to several examples. In addition, in order to reduce duplication of description, the same reference numerals are given to the same or corresponding parts in a plurality of drawings referred to in the description, and some drawings are used for the description of a plurality of embodiments. . Further, the JPEG 2000 basic method, which has been outlined above, is assumed as the compression encoding method in each embodiment.

  FIG. 2 is a block diagram of the image processing apparatus according to the present embodiment. This image processing apparatus is an MFP that can be used as the server 1, and is a scanner unit 9 that reads a document, and a scanner that performs image processing such as known γ correction processing and spatial filter processing on image data input from the scanner unit 9. An image processing unit 11; an image region separation processing unit 10 that performs image region separation processing based on image data input from the scanner 9; an encoder 12 that performs image data encoding processing and outputs a code stream; CPU 13 that performs data processing in the controller unit, volatile memory 14 that is used as a work area for temporarily storing image data and compression-encoded code data, and a hard disk device for storing and accumulating code data and the like (HDD) 15, an operation with a start button, an operation mode setting button, etc., which is an interface with the operator 16, an external transfer path interface (I / F) 17 that transmits / receives data to / from an external device such as the client 3 via the external transfer path 2, a decoder 18 that decodes the code data and outputs image data, and the image data A printer image processing unit 19 that performs known color correction processing, γ conversion processing, pseudo gradation processing, and the like, a bus 20 that connects the constituent elements 12 to 18, and image data input from the printer image processing unit 19 is recorded on recording paper, etc. Printer unit 21 for outputting to the recording medium.

  First, a basic copying operation in this image processing apparatus will be described. When a start button (not shown) on the operation unit 16 is pressed by the user, the CPU 13 that has received a signal from the operation unit 16 via the bus 20 performs setting of necessary parameters prior to the copying operation, and the like. Control for a predetermined copying operation is performed.

  The scanner unit 9 scans an original, performs photoelectric conversion by a CCD (not shown), converts the digital image into red (R), Green (G), and Blue (B) color image data and outputs the color image data. Known image processing is performed on the image data output from the scanner by the scanner image processing unit 11, and image area separation processing is performed by the image area separation processing unit 10 based on the image data. The content of the image processing in the scanner image processing unit 11 is not particularly limited in the present invention. For example, γ conversion for converting a reflectance attribute signal read from a CCD into a density attribute, log conversion, MTF degradation correction of scanner optical system, spatial filter processing for moire suppression, background removal processing for removing original background, color conversion processing for converting scanner color space to standard color space such as sRGB Etc. are considered. In such a process, the separation signal output from the image area separation processing unit 10 may be used to perform different processing for the character part and the non-character part.

  The image area separation processing unit 10 separates an image into characters and non-characters in units of pixels and outputs a 1-bit separation signal. A known image area separation processing technique may be used for the image area separation processing here, and the processing method is not particularly limited, but the image area separation method described in Patent Document 2 is an example of a suitable method. is there. The outline is as follows.

  In the case of RGB image data, for the G data, the determination of the plain character area and the determination of the halftone dot character area are performed. In the determination of the plain text region, the G data is applied to the edge enhancement filter, and then ternarized into white, black, and others using two threshold values, and matching with the black edge pattern and the white edge pattern is performed. If there are one or more pixels matching the white edge pattern and one pixel matching the black edge pattern in the 5 × 5 pixel block, the block (or the central pixel) is determined as a non-ground character area. In the determination of the plain text area, G data is applied to a smoothing filter and then converted into white, black, and others using another two threshold values, and matching is performed with the black edge pattern and the white edge pattern. If one or more pixels that match the white edge pattern and one that matches the black edge pattern exist in the 5 × 5 pixel block, that block (or the center pixel) is determined as a halftone dot character area. To do. Then, a pixel determined to be a plain character area or a halftone dot character area is finally set as a character area, and other pixels are finally set as non-character areas.

  The image area separation method based on such image data can perform image area separation with high accuracy in units of images, and therefore, the tile area determination can be performed accurately by using the image area separation result. . Since color conversion is performed in the encoder 12 as will be described later, similar image area separation may be performed using the Y signal after color conversion, and this aspect is also included in this embodiment.

  The image data processed by the scanner image processing unit 11 and the separation signal output from the image area separation processing unit 10 are input to the encoder 12. In the encoder 12, the image data is compression-coded according to the separated signal, and a code stream is generated. The encoder 12 is directly related to the present invention, and details thereof will be described later.

  The code stream output from the encoder 12 to the bus 20 is temporarily stored in the memory 14, and then read from the memory 14 and stored in the HDD 15. In parallel with the storage in the HDD 15, the code stream is input to the decoder 18 to be decoded. The bus control as described above is performed by the CPU 13.

  Normally, data write access to the memory connected to the bus in this way is performed by each processing unit connected to the bus in a certain data unit of several tens of bytes to several kilobytes by DMA (Direct Memory Access). In general, the bus connection section is composed of a buffer and a DMA controller, but the DMA controller is omitted for the sake of simplicity. In addition, since reading of data from the memory is similarly performed by DMA, an input buffer memory is necessary, but this is also omitted.

  Further, when the code data is stored in the HDD 15, the writing to the memory 26 is interposed, for the following reason. HDDs may change read / write speeds or cause read / write errors on the side closer to and far from the center of the disk, and may not be suitable for reading / writing synchronization signals compared to semiconductor memories. It is a device. For this reason, the code data once stored in the memory 14 is asynchronously written and used stably without directly writing the output data of the encoder 12 as a synchronization signal. The same applies to reading. However, the HDD operates in synchronism when viewed in a macro for each page.

  The code data stored in the HDD 15 is used for backup when a paper jam or the like occurs during copying, an electronic sort function that outputs a multi-page document in multiple copies in page order, data distribution to an external device, etc. The

  When the code stream is input from the memory 14, the decoder 18 performs a predetermined decoding process and outputs image data to the printer image processing unit 19. Image processing in the printer image processing unit 19 is not particularly limited in the present invention, as in the scanner image processing unit 11, but for example, RGB signals are converted into printer color material signals such as Cyan (C). , Magenta (M), Yellow (Y), Black (K), color matching for color matching, black generation, gamma correction to match the γ characteristics of the image data with the γ characteristics of the printer 21, half dither and error diffusion Pseudo gradation processing for converting to a tone can be considered. The image data that has been subjected to such image processing is printed on a recording sheet or the like by the printer 21 and output, and the copying operation is completed.

  After the copying operation is completed, the image data in the middle of the copying operation is stored in the HDD 15 in a compressed and encoded state. By storing the image data in the HDD 15 in this way, the image data can be transferred to an external device or reprinted when necessary. If it is only necessary to save the document as electronic data in the HDD 15, the processing after the decoder 18 is not performed.

  Next, the encoder 12 will be described. FIG. 3 is a block diagram of the encoder 12 according to the present embodiment. The encoder 12 is an encoding unit compliant with the JPEG2000 basic system, and includes an input buffer 30, a color conversion processing unit 31 directly related to the encoding process, a discrete wavelet transform (DWT) processing unit 32, an entropy encoding unit 35, The post quantization processing unit 36, the code stream generation processing unit 37, and the region determination processing unit 34 are included.

  The RGB image data and separation data output from the scanner image processing unit 11 and the image area separation processing unit 10 are temporarily stored in the input buffer 30 and read out in tile units. Since the output data from the scanner 9 is in the form of raster data, the input buffer 30 undergoes raster / block conversion.

  The image data read from the input buffer 30 is converted into a luminance / color difference signal (YUV) by the color conversion processing unit 31. The signal after color conversion is subjected to DWT for each component in the DWT processing unit 32. The DWT coefficient output from the DWT processing unit 32 is entropy encoded by the entropy encoding unit 35. The entropy-encoded code data is subjected to post-quantization processing by code truncation as necessary in the post-quantization processing unit. Normally, image data input by a scanner often contains a noise component, and the compression rate can be improved by performing post-quantization to such an extent that the image quality is not affected. However, post-quantization is not performed when a reversible code is output. The code data after the post-quantization processing is input to the code stream generation processing unit 37. Instead of post-quantization, scalar quantization may be performed on the DWT coefficient output from the DWT processing unit 32.

  On the other hand, the area determination processing unit 34 performs area determination as to whether a tile is a character tile or a non-character tile by ORing a separation signal (a 1-bit signal that is “1” when a character is used) in tile units. An area determination signal representing the result is output. This region determination signal is input to the code stream generation processing unit 37. Note that the region determination signal may be input to the post-quantization processing unit 36 to change the contents of the post-quantization processing between the character tile and the non-character tile, and this aspect is also included in this embodiment.

  The code stream generation processing unit 37 generates a progressive code in a predetermined progressive order (in this embodiment, a layer (LRCP) progression) by rearranging the codes of the code data of each tile after the post-quantization processing, A progressive code stream for tiles is formed, and header data is added to the stream and output to the memory 14 via the bus 20. When the code stream of each tile is formed, the code stream has a hierarchical division configuration (here, a layer division configuration, in other words, depending on whether the region determination signal from the region determination processing unit 34 indicates a character tile or a non-character tile. If so, the priority order of the codes is changed.

  In the present embodiment, layer (LRCP) progression is used as the progressive order of the code stream. In the resolution progression, the progressive order of decoding the low resolution and then the high resolution is performed, so that the layer progression is generally suitable for an image having high energy in high frequency components such as characters. On the other hand, in layer progression, although there is a restriction that the decoding order from the upper bit to the lower bit must be observed, the layer boundary can be set freely in units of code blocks, and the degree of freedom of the hierarchical division configuration is high. It has become. This is the main reason for using layer progression.

  As is apparent from the above description, the code stream generation processing unit 37 corresponds to the code stream generation means in the invention of claim 1, and the region determination processing unit 34 together with the image area separation processing unit 10 in the invention of claim 1. It constitutes an area determination means. A tile corresponds to a block in the invention of claim 1.

  This embodiment is also an embodiment of the invention of claim 17. That is, the processing by the code stream generation processing unit 37 corresponds to the code stream generation step in the invention of claim 17, and the processing by the region determination processing unit 34 corresponds to the region determination step in the invention of claim 17.

  Here, the control of the layer division configuration (hierarchical division configuration) by character tile / non-character tile will be described.

  4A and 4B are schematic diagrams for explaining the layer division configuration. FIG. 4A shows a layer division configuration for character tiles, and FIG. 4B shows a layer division configuration for non-character tiles. Here, in order to simplify the description, the wavelet transform with two decomposition levels is used. Further, the concepts of component, precinct, and code block are omitted.

  In FIG. 4, the horizontal axis represents the name of each subband, and the vertical axis represents the layer number. Although the number of layers is four, this is only an example. As in FIG. 30A, the filled rectangles in the figure indicate codes in each subband and each layer, and the size schematically represents the ratio of the code amount.

  The difference between the character tile and non-character tile layered configurations will be described. In particular, layer 0 is characterized.

  In the character tile, as shown in FIG. 4A, the reproducibility of the character image having a high value in the high frequency component is enhanced by including the codes of the highest layer 0 to 1HL and 1LH subbands. . However, the code amount of the 1HH subband, which is an oblique component, is set to 0, and the code amount of the 2HH subband is also 2HL and 2LH subbands, in order to avoid as much as possible the code amount from the higher layer (decrease in compression ratio) Compared to smaller.

  On the other hand, in the non-character tile, as shown in FIG. 4B, in layer 0, the code amount of the component of decomposition level 1 is all set to 0 to prevent the compression rate from being lowered. Layer 1 includes the upper bits of the high-frequency component. Even if it is a component with a high decomposition level (low resolution), the lower bits are likely to be noise components, so by giving priority to the upper bits of the higher frequency over such components, photos including edges etc. Therefore, the layer resolution is intended to improve the resolution of the image from the early stage of progressive. In a scanner or a digital camera that is an image input device that acquires an image using a photoelectric conversion element such as a CCD, a noise component is often included in the image. Therefore, such a layer division configuration is particularly effective.

  If the layer progression code stream in which the layer division configuration is controlled in this way is transferred from the external I / F 17 to an external device such as the client 3 via the external transfer path 2 and is progressively decoded, it is transferred or decoded at an early stage. Since the characters can be read, it is easy to grasp the image contents. In addition, the reproduction of the edge portion such as the photo portion of the image can be reproduced with high image quality at an early stage. Further, since the progressive order of the code stream is not switched between the character tile and the non-character tile, there is no sense of incongruity due to the difference in the progressive order at the boundary between the character tile and the non-character tile.

  FIG. 5 is a block diagram of the code stream generation processing unit 37 according to the present embodiment. As shown in the figure, the code stream generation processing unit 37 of this embodiment includes an input code buffer 41, a code amount counter 42, a pattern table 43, a selector 44, a packet generation processing unit 45, a tile header generation processing unit 46, and an output. It consists of a code buffer 47. The input code buffer 41 and the output code buffer 47 may not be divided and the same memory may be used.

  The code stream generation processing operation is as follows. The code data output from the post-quantization processing unit 36 is stored in the input code buffer 41 in units of subband code blocks. This is schematically shown in the input code buffer 41, and CD1 to CD4... Indicate code blocks and numbers for convenience. Note that the concept of components is omitted for the sake of simplicity. The code of each code block is arranged by being divided into sub bit planes as shown in FIG.

  The code data is also input to the code amount counter 42, and the code amount is counted by the code amount counter 42 for each sub-bit plane of the code block. The code amount data is input to the packet generation processing unit 45 and the tile header generation unit 46, and is used for the code amount information of the packet header and the tile header.

  When all the code data of the tile is input to the input code buffer 41, first, the tile header generation processing unit 46 generates a tile header and a tile header for the partial sequence, and writes them to the output code buffer 47. Next, the packet generation processing unit 45 generates packet data and a packet header in units of the same layer, decomposition level, component, and precinct.

  The pattern table 43 is means for storing information (hereinafter referred to as a layer pattern) for controlling the layer division configuration. The pattern table 43 has a capacity capable of storing two types of layer patterns for character tiles and non-character tiles, and two types of layer patterns are set by the CPU 13 via the bus 20 prior to processing. Each layer pattern in the present embodiment is information on “up to which sub-bit plane is included in each layer” for each subband. For example, in layer 0, from the 2LL MSB to the upper 4 bits + 5 bit Significance propagation pass, from the 2HL MSB to the upper 3 bits (up to the 3rd clean up pass) Stores the set layer boundary information.

  The selector 44 selects a character layer pattern or a non-character layer pattern for each tile according to the region determination data output from the region determination processing unit 34, and inputs it to the packet generation processing unit 45. The packet generation processing unit 45 reads necessary code data from the input code buffer 41 according to the layer pattern input from the selector 44, forms a code stream for each layer, and sequentially outputs the code stream to the output code buffer 47. When a code stream of one layer is completed, a tile header for a partial sequence is generated by the tile header generation processing unit 46 and written to the output code buffer 47, and the generation process of the code stream of the next layer is started. When a code stream having a layer division configuration according to the character tile layer pattern is completed for one tile in this way, the code stream stored in the output code buffer 47 is output to the memory 14. Similar processing is repeated for all tiles.

  When the code stream for all tiles of the image data is completed on the memory 14, the CPU 13 adds a main header to the code stream on the memory and starts transferring the code stream from the memory 14 to the HDD 15.

  As is apparent from the above description, the pattern table 43, the selector 44 that selects the layer pattern according to the region determination data and passes it to the packet generation processing unit 45, and the operation thereof are the code stream in the inventions of claims 1 and 17. This corresponds to means or process for controlling the hierarchical division structure of each block according to the area determination result.

  In this embodiment, the JPEG 2000 basic method is used as the compression encoding method, but it does not necessarily conform to the standard compression method. This is the same not only in this embodiment but also in each embodiment described later. However, from the viewpoint of versatility of the code stream, it is generally advantageous to comply with the standard compression method. In addition, a compression encoding method that uses frequency transforms other than wavelet transform, such as discrete cosine transform, or a compression encoding method that directly compresses and encodes real space image data without performing frequency transform can be used. The same is true.

<Modification of Example 1>
In the first embodiment, the code stream of the layer progression is generated. However, by changing the contents of the pattern table 43, in the same device configuration, the hierarchical division configuration according to the character tile and the non-character tile It is also possible to generate a resolution progression code stream in which the above is controlled.

  FIG. 6 is a schematic diagram similar to FIG. 4, illustrating an example of a hierarchical division configuration of a resolution progression code stream. 6A shows an example of a hierarchical division configuration of character tiles, and FIG. 6B shows an example of a hierarchical division configuration of non-character tiles.

  In the case of resolution progression, since the high resolution (low decomposition level) code cannot be output before the low resolution (high decomposition level) code, the progressive order is (2LL) → (2HL, 2LH, 2HH) → (1HL, 1LH, 1HH) in this order, and decoding is performed from the upper layer code at each resolution level.

  In the character tile, as shown in FIG. 6A, many codes are output at the upper layer stage. With such a hierarchical division configuration, it is possible to improve the legibility of a character image that requires many codes for reproduction. On the other hand, in the non-character tile, as seen in (B), it is preferable to reduce the code amount in the upper layer and output many codes in the lower layer. By adopting such a hierarchical division configuration, it is possible to suppress an increase in the code amount in the upper layer at each resolution level.

  In (A), there is an empty code amount layer, but there is no problem in the code configuration. However, since the image does not change in the empty layer portion when progressive display is performed, there is a possibility that a sense of incongruity may occur. Therefore, in practice, it is preferable to set some codes for all layers.

  Here, the advantage of the resolution progression when compared with the layer progression will be described. For example, when the client side device is a device with a very small display unit such as a mobile terminal, the resolution (image size) of the client side display unit is compared with the resolution (image size) of the server side image data. If it is small, even if the data of the high resolution part is sent by layer progression, it becomes meaningless, and transfer is wasted. In such a case, the resolution-progress code stream has an advantage that the transfer is stopped at the necessary resolution stage to prevent useless transfer and reduce the traffic on the transfer path.

  Therefore, it is desirable that one of layer progression and resolution progression can be selected as the progressive order of the code stream on the server side according to the resolution of the display unit of the client that is the transfer destination of the code stream. For example, the CPU 13 selects a layer progression or a resolution progression according to the resolution of the display unit of the client that is the code stream transfer destination, sets the layer pattern for the selected progression in the pattern table 43, and also selects the selected progression. A configuration in which the packet generation processing unit 45 is set can be adopted. A mode in which such a progressive order can be selected is also included in the present invention.

  In the first embodiment, the progressive code stream in which the hierarchical division configuration is controlled according to the region determination result is generated in the encoding process. In the present embodiment, the progressive code stream is transferred to the external device. In addition, the progressive code stream is generated from the code stream stored in the HDD 15.

  FIG. 7 is a block diagram of the image processing apparatus according to the present embodiment. This image processing apparatus is an MFP that can be used as the server 1. The main difference from the first embodiment is that a code stream change processing unit 50 is added.

  In the first embodiment, the code stream output from the encoder 12 is a layer progression code stream in which the hierarchical division configuration is controlled according to the region determination result. However, in this embodiment, for the code stream generated by the encoder 12 and stored in the HDD 15, the hierarchical division configuration control based on the region determination result is not performed (however, as described later, the region determination result is Retained in the code stream). The progressive order of the code stream output from the encoder 12 is basically arbitrary, and does not necessarily need to be progressive, and the hierarchical structure is also arbitrary. For example, the entire page may have a uniform hierarchical structure. Good. This is an advantage of this embodiment. That is, in the copying operation, it is not necessary to use a progressive code stream. Rather, if the image of one page has a uniform hierarchical structure, the load of the decompression process by the decoder 16 for the copying operation is reduced. It is. Also, depending on the type of client 3, resolution progression such as RPCL progression may be more convenient. For example, in the case of a client with a small monitor display size such as a mobile terminal, it is only necessary to output a code of a necessary resolution. Therefore, the type of progressive order of the code stream stored in the HDD 15 can be easily selected. Sometimes a good resolution progression is better. That is, it is preferable to receive a request such as the type of progression from the client as a command, and generate and transmit a code stream of the progression selected according to the request. The present embodiment has a configuration capable of generating and transmitting a progression code stream selected according to such a client, which is one of the advantages of the present embodiment.

  Hereinafter, this embodiment will be described in detail. The encoder 12 according to the present embodiment has the same configuration as that shown in FIG. 3 as in the first embodiment, and performs a coding process to generate a code stream. The code stream generation processing unit 37 in the encoder 12 does not control the hierarchical division configuration according to the area determination result in tile units by the area determination processing unit 34, but retains the area determination result in the generated code stream. . In the present embodiment, as a method for retaining the region determination result in the code stream, a method of recording as a flag in the comment portion of the tile header of the code stream is used, but the method is not limited to this. For example, it is possible to record the region determination result in the code stream using a technique such as digital watermarking. The flag data of the area determination result can be stored in the memory 14 or the HDD 15 separately from the code stream. The code stream output from the encoder 12 is temporarily stored in the memory 14 and then stored in the HDD 15.

  When the code stream transmission request is received from the client 3 via the external transfer path 2, the CPU 13 sets the necessary parameters, reads the code stream from the HDD 15, and writes it to the memory 14 via the bus 20. The code stream recorded in the memory 14 is read out in units of tiles and input to the code stream change processing unit 50.

  FIG. 8 is a block diagram of the code stream change processing unit 50. The code stream change processing unit 50 has a configuration very similar to the code stream generation processing unit 37 (FIG. 5) according to the first embodiment, except that a header analysis processing unit 51 and a packet decomposition processing unit 52 are added.

  First, the CPU 13 sets, in the pattern table 43, two types of layer patterns for character tiles and non-character tiles for the progressive order (layer progression or resolution progression) selected according to the client's request. The selected progression type is also set in the packet generation processing unit 45.

  Next, code data is input in units of tiles from the memory 14 via the bus 20, and the header analysis processing unit 51 analyzes the tile header, reads out the area determination result described in the tile header, and outputs an area determination signal. Is output to the selector 44.

  Next, when a packet is input, the header analysis processing unit 51 analyzes the packet header and outputs a control signal to the packet decomposition processing unit 52. The packet decomposition processing unit 52 decomposes the code portion of the packet into code data for each sub-bit plane in accordance with this control signal, and records it in the input code buffer 41. At this time, the code amount counter 42 counts the code amount for each sub-bit plane of the code block. When all the code data of the tile is input to the input code buffer 41, first, the tile header generation processing unit 46 generates a tile header and a tile header for the partial sequence, and writes them to the output code buffer 47. Next, the packet generation processing unit 45 reads necessary code data from the input code buffer 41 according to the layer pattern selected by the selector 44 in accordance with the region selection signal from the header analysis processing unit 51, and forms a code stream in units of layers. And sequentially output to the output code buffer 47. The code stream is output from the output code buffer 47 to the memory 14 when the code stream having a hierarchical division structure according to the layer pattern is completed for one tile. Similar processing is repeated for each tile.

  When the code stream for all tiles of the image data is completed on the memory 14 in this way, the CPU 13 adds the main header to the code stream and transmits it to the client 3 from the external I / F 17 via the external transfer path 2. Let The client 3 progressively decodes the received code stream and displays the decoded image on the monitor 4 in a progressive manner.

  As is apparent from the above description, the code stream change processing unit 50 and its processing correspond to the code stream generating means or the code stream generating step in the inventions of claims 2 and 17. The selector 44 that selects a layer pattern according to the pattern determination signal from the pattern table 43 and the area analysis signal from the header analysis processing unit 51 and passes it to the packet generation processing unit 45, and the operation thereof, have the hierarchical structure of the code stream according to claims 2 and 17 This corresponds to a means or process for controlling each block according to the region determination result. Further, from the viewpoint of the code stream change processing unit 50, the function of extracting the region determination signal of the header analysis processing unit 51 corresponds to the region determination means or process in claims 2 and 17. However, from the viewpoint of the entire image processing apparatus, the image region separation processing unit 10 and the region determination processing unit 34 in the encoder 12 correspond to a region determination unit or process.

  As described above, according to the present embodiment, the read image data can be stored in the HDD 15 as a code stream having a configuration suitable for print output processing in the MFP. On the other hand, when a transmission request is received from the client, a progressive order suitable for the client is selected, and a progressive code stream in which the hierarchical division configuration is controlled according to the region determination result is generated from the stored code stream and transmitted to the client. can do. Further, the generation of the progressive code stream is performed by the operation of the code level of the stored code stream, that is, the rearrangement of codes, and does not include the decoding / re-encoding process. The process can be performed in a short time without deteriorating the quality.

  In the first and second embodiments, a layer pattern that is layer boundary information that can be set in units of sub-bit planes set for each subband is used when generating a code stream. According to the control of the hierarchical division configuration using such a layer pattern, the image quality level in units of layers can be made constant. By stopping decoding at an intermediate layer or transmitting up to a predetermined layer, the image quality level can be easily controlled, and a system that guarantees image quality can be constructed. However, since the code amount in units of layers (hierarchical units) cannot be controlled, it is not possible to guarantee the code amount during progressive transfer.

  Here, an embodiment for guaranteeing the code amount in units of layers will be described.

  As a method of controlling the code amount in units of layers, the following two methods can be used. One of them is a method of calculating the code amount per layer (set code amount) from the code amount of the entire tile or image and controlling the code amount of each layer to be the set code amount. In other words, this is a method of adaptively controlling the set code amount for each layer according to the image (hereinafter referred to as method A).

  The other is to determine the set code amount for each layer in advance and control the code amount of each layer to be the set code amount. If there is a remaining code at the time of the last layer, This is a method of configuring the final layer. That is, this is a method for non-adaptively controlling the set code amount in units of layers (hereinafter, method B).

  The A method and the B method have advantages and disadvantages, and neither method is generally good. For example, in the method A, the code amount for each layer can be made substantially equal, and the number of layers can be guaranteed. On the other hand, since the code amount for each layer varies depending on the image, the transfer time for each layer of code data from the server to the client cannot be guaranteed.

  On the other hand, in the method B, the code amount of the layers other than the final layer can be set to a known value, so that the transfer time for each layer can be guaranteed. On the other hand, there is a possibility that only the final layer has an extremely large code amount, or that all code data has already been output before the set final layer and the number of layers changes.

  Accordingly, whether to select the A method or the B method should be determined according to the system conditions. Here, an embodiment using the B method will be described. The apparatus configuration for this purpose may be the same as that of the first embodiment or the same as that of the second embodiment. That is, the code amount control for each layer is performed in the code stream generation processing unit 37 shown in FIG. 5 according to the first embodiment, or in the code stream change processing unit 50 shown in FIG. 8 according to the second embodiment. However, for the sake of convenience, a case where code amount control is performed in the code stream generation processing unit 37 of FIG. 5 will be described here.

  In the embodiment described here, in the code stream generation processing unit 37 shown in FIG. 5, the pattern table 43 stores information indicating the priority order of codes for character tiles and non-character tiles regardless of subbands. . This information is: 2LL (from MSB to upper 3 bits + 4th Magnitude refinement path) → 2HL (from MSB to upper 3 bits) →. . . . It is such information. There is no distinction between layers (hierarchies) in this priority order, and the order of contribution to image quality is simply in subband units.

  In one embodiment, the packet generation processing unit 45 sequentially generates a code stream for each packet according to the code order information of the pattern table 43 selected by the selector 44, and reaches a preset code amount for each layer. At this point, the code stream of the layer is completed. However, for the final (lowest) layer, a code stream is generated using all the remaining codes. That is, the set code amount in tile (hierarchy) units is the same for both character tiles and non-character tiles. However, what kind of code is included in each tile (hierarchical division configuration) differs between the character tile and the non-character tile.

  In another embodiment, the area determination data is also input to the packet generation processing unit 45. Then, the packet generation processing unit 45 generates a code stream of the layer so that the set code amount of each layer set for the character tile is set for the character tile, and the layer set for the non-character tile is set for the non-character tile. A layer code stream is generated so as to have a set code amount per unit. That is, the set code amount for each layer is switched depending on whether the tile is a character tile or a non-character tile. It is generally preferable to set the set code amount for each layer for character tiles to a value larger than the set code amount for each layer for non-character tiles. This is because the character area generally has a larger amount of information than the non-character area, and the character area requires a larger amount of code in order to reproduce the same image quality.

  As is apparent from the above description, this embodiment is an embodiment of the invention of claims 3 and 4.

  As described above, since this embodiment can guarantee the code amount in units of layers, it is particularly suitable for a system having a low transfer rate on the transfer path. In addition, by switching the set code amount for each layer according to the region determination result and increasing the code amount for each layer of character tiles compared to non-character tiles, the character readability at the initial stage during progressive transfer is improved. Can do.

  As described above, it is clear from the above description that the same code amount control can be performed also in the code stream change processing unit 50 of FIG. 8 according to the second embodiment. Such embodiments are also encompassed by the present invention.

  It will be apparent that the code amount control in the intermediate stage (hierarchy) of progressive transfer can be similarly performed in resolution progression or the like.

  In the first, second, and third embodiments, the image types determined by the region determination are two types of characters and non-characters. However, the region determination is performed with more image types, and the hierarchy according to the region determination result It is also possible to control the division configuration. Such an embodiment will now be described.

  The apparatus configuration may be the same as in the first and second embodiments, but here, for convenience, the apparatus configuration is the same as in the first embodiment, and FIGS. 2, 3 and 5 related to the first embodiment are used. To explain.

  In the embodiment described here, the image area separation processing unit 10 (FIG. 2) includes an edge determination processing unit 61, a halftone determination processing unit 62, a color determination processing unit 63, and an overall determination processing unit 64 as shown in FIG. It is supposed to be configured. The image area separation processing unit 10 performs black character, color character, halftone dot, and other region separation processing based on the RGB image signal output from the scanner unit 9 (FIG. 2).

  That is, the edge determination processing unit 61 determines whether the edge is an edge based on the G signal, and the dot determination processing unit 62 determines whether the image is a halftone area based on the G signal. The processing unit 63 determines whether the color is chromatic or achromatic, and outputs Edge, AMI, and IRO signals (each 1-bit signal), which are input to the general determination unit 64.

  For such edge determination, halftone dot determination, and color determination, a known technique may be used, and the method thereof is not limited. For example, the method described in Patent Document 3 can be used for edge determination and color determination. . For the halftone dot determination, for example, the method described in Patent Document 2 can be used. Details of these determination methods are given in Patent Documents 2 and 3, and the outline thereof will be described next.

  First, in the edge determination, the G signal is ternarized into white, black, and others using two threshold values, and matching with the black edge pattern and the white edge pattern is performed. For example, the white signal is contained in a 5 × 5 pixel block. If there are one or more pixels that match the edge pattern and one pixel that matches the black edge pattern, the block is determined to be an edge.

  In the halftone dot determination, for example, in a 3 × 3 pixel block, the peak pixel is detected by determining whether the value of the central pixel is higher than a certain level from the surrounding pixels. Next, for example, in a 4 × 4 pixel block, when there are two or more blocks including peak pixels, the target block is set as a halftone dot region candidate. Finally, in a 3 × 3 block having 4 × 4 pixels as one block and the target block as a central block, when the four or more blocks are halftone dot area candidates, the target block is set as a halftone dot area.

  For color determination, the maximum value max (| R−G |, | GB− ||| B−R |) of the difference values of the R, G, and B signals is calculated for each pixel. This value is compared with a predetermined threshold, and pixels that are equal to or higher than the threshold are determined as chromatic colors, and pixels that are lower than the threshold are determined as achromatic colors. Next, the number of pixels determined to be a chromatic color in a 5 × 5 pixel block centered on the target pixel is counted, and if the coefficient value is equal to or greater than a predetermined threshold, the target pixel is finally determined to be a chromatic color. If it is less than the threshold, the target pixel is finally determined to be an achromatic color.

  Refer to FIG. 9 again. The overall determination unit 64 determines black characters, color characters, and halftone dot areas from the signals of Edge, AMI, and IRO, for example, according to the rules shown in Table 1 below.

第１表は、Edge、IRO、AMI各信号の状態と総合判定の結果を対応させたものである。総合判定の結果は、黒文字、色文字、網点、その他の４種類である。各信号の状態は、１がオン(例えば、Edge信号がオンの場合その画素はエッジであることを表す。同様にIRO信号がオンの場合は有彩色、AMI信号がオンの場合は網点となる。)、０がオフ、＊はその信号の状態によらずに総合判定が行われることをそれぞれ示す。

Table 1 associates the state of each signal of Edge, IRO, and AMI with the result of comprehensive judgment. There are four types of comprehensive determination results: black characters, color characters, halftone dots, and others. The state of each signal is 1 (for example, when the Edge signal is on, the pixel is an edge. Similarly, when the IRO signal is on, chromatic colors are used, and when the AMI signal is on, halftone dots are displayed. ), 0 is off, and * indicates that comprehensive judgment is performed regardless of the state of the signal.

  As can be seen from the table, the Edge signal is given the highest priority, and when the Edge signal is on, the character pixel is determined regardless of the AMI signal, and the black character and the color character are determined by the IRO signal. When the Edge signal is off and the AMI signal is on, it is determined as a halftone dot, and when the Edge signal and the AMI signal are both off, other determinations are made.

  This comprehensive determination method is not limited to this. For example, as shown in Table 2 below, the AMI signal may be given the highest priority, and when the AMI signal is on, the halftone dot region may be determined regardless of the state of the Edge signal.

第１表と第２表の違いは、網点地の上の文字を文字として判定するか網点として判定するかである。本実施例の場合は、文字の判読性向上を優先すため、網点地の上の文字を文字として判定することとしている。

The difference between Table 1 and Table 2 is whether a character on a halftone dot is determined as a character or a halftone dot. In the case of the present embodiment, in order to give priority to improving the legibility of the character, the character on the halftone dot is determined as the character.

  Next, the operation of the encoder 12 (FIG. 3) in the present embodiment will be described. Since the basic flow of the encoding process is the same as that in the first embodiment, only the differences will be described.

  First, the operation of the area determination unit 34 will be described. Based on the black character, color character, and halftone dot separation signals in pixel units from the image area separation processing unit 10, the region determination unit 34 determines regions in tile units. In this determination method, OR is performed on each separation signal in units of pixels in the entire tile, and it is determined whether each pixel of black characters, color characters, and halftone dots exists in the tile. If no pixel is present in the tile, it is determined as another area. Further, a tile having only black characters is determined as a black character region, a tile having only color characters is determined as a color character region, and a tile having only halftone dots is determined as a halftone dot region.

  If there are a plurality of image areas in the tile, the area is determined as one area according to the priority of each area regarding the area determination. The priority order may be set as appropriate according to the system environment. As one preferable example, an image type to which a code should be preferentially assigned at the initial stage of a layer, in other words, a case where an image is progressively transferred. In the initial stage, a larger number of codes are transmitted compared to other image types, and the priority of the region of the image type that should improve the reproducibility of details is set higher.

  Here, the priority order is set in the order of color characters, black characters, halftone dots, and the like. In other words, if there is at least one color character pixel in the tile, the color character region is used regardless of the presence of other separation results. If there is no color character pixel, the black character region and the color character pixel are present. If there is a halftone dot pixel and no black character pixel, a halftone dot region is assumed.

  The region determination data regarding these four types of regions is input to the code stream generation processing unit 37 (FIG. 5), and the code stream generation processing unit 37 uses the code stream layer division configuration (hierarchy division configuration) corresponding to the region determination data. Control is performed. For this control, four types of layer patterns corresponding to each image type are set in the pattern table 43, and any one of the layer patterns is selected by the selector 44 according to the area determination data.

  Here, the characteristics of each image type region will be briefly described. In the color character region, a high DWT coefficient value appears in every subband of luminance and color difference compared to a natural image. A black character region has a feature that a high DWT coefficient value appears in every subband of luminance, and a high DWT coefficient value appears in a specific subband in a halftone dot region. Considering these, the layer division configuration in each region is determined, and the layer pattern corresponding to it is set in the pattern table 43.

  The hierarchical division configuration of each region can be determined based on the following concept. First, in the black character area, the priority order and code amount of the U and V components are lowered. In the color character area, all the Y, U, and V components are handled in the same manner (because a high frequency component is necessary). In the halftone dot region, a higher frequency component than a specific high frequency component is not sent or the priority is lowered. In other regions, the high frequency component may not be included.

  Note that the components of the specific subband of the halftone dot are not particularly necessary components for recognizing the natural image because they are the line number components of the halftone dot. For this reason, in the halftone dot region, the upper layer (the initial stage of transfer) has a layer division configuration that does not include a halftone line number component, and the amount of codes can be reduced.

  As described above, in the present embodiment, the image area is determined to be a black character, a color character, a halftone dot, and other areas in tile units, and in order to control the hierarchical division configuration of the code stream according to the determination result, Compared to the first and second embodiments, it is possible to generate a progressive code stream adapted to the image type in more detail.

  In the second embodiment as well, it is clear from the above description that the same region determination and the code stream hierarchy division configuration according to the determination result can be performed, and such an aspect is also included in the present invention. Is done.

  Note that the number of image types for region determination can be further increased, and the hierarchical division configuration can be controlled in accordance with higher-definition image types. For example, a silver halide photograph or a white background may be determined. In addition, even for the same image type, characteristics such as size and color may be determined to change the hierarchical division configuration (reproduction priority). For example, priority may be given to reproduction of a large character such as a title, the priority may be changed depending on the character color, or the priority may be changed depending on the number of lines of halftone dots. In addition, it is possible to perform control so that priority is given to reproduction of a highly saturated color region such as a human face or flower in an image composed of natural images.

  In the first to fourth embodiments, the region determination is performed based on the image region separation signal of the real space image data. That is, area determination is performed based on real space image data.

  Here, a description will be given of an embodiment in which region determination is performed based on DWT coefficients (frequency conversion coefficients in a broad sense) generated in the process of encoding image data.

  FIG. 10 is a block diagram of the encoder 12 of the image processing apparatus according to the present embodiment. The overall configuration of the apparatus may be the same as that of the first embodiment, but the image area separation processing unit 10 is unnecessary.

  In the encoder 12 shown in FIG. 10, the region determination processing unit 34 performs character / non-character region determination in tile units based on the DWT coefficient output from the DWT processing unit 32, and does not input an image region separation signal. The configuration of the encoder 12 other than the area determination unit 34 is the same as that in each of the above embodiments.

  FIG. 11 shows the DWT coefficient at the 3 decomposition level. In the figure, a thick solid line represents a subband boundary, and a thin line represents a coefficient boundary. That is, a lattice surrounded by a thin line is one coefficient. The DWT coefficient is arranged at a position corresponding to the image input to the DWT processing unit 32. In other words, it can be said that the DWT process preserves the position information of the input image. As shown in FIG. 11, a coefficient corresponding to the input image at a specific position exists in each subband. The shaded coefficients in the figure represent the coefficients having the above correspondence.

  In the DWT coefficient, it is known that the absolute values of the high frequency components other than the LL subband have a correlation between subbands in the same direction. The term “between subbands in the same direction” refers to between subbands of different resolutions having the same direction component such as 3HL, 2HL, and 1HL.

  For example, if the absolute value of the coefficient of the subband having a high decomposition level (low resolution) is 0, the absolute value of the coefficient of the subband having a high resolution of the same direction component is often 0. On the contrary, if the absolute value of the coefficient of the subband having a high decomposition level (low resolution) is a large value, the coefficient of the subband having a high resolution of the same direction component is often a large value. Further, the absolute value of the DWT coefficient increases if the input image is an edge portion, and decreases if the input image is a flat portion. In the present embodiment, the region determination process is performed using such a property of the DWT coefficient.

  FIG. 12 is a block diagram of the area determination processing unit 34 according to the present embodiment. The area determination unit 34 receives the DWT coefficient of the Y signal (of the YUV signal) that has been converted to an absolute value. Note that the encoding of the DWT coefficient in JPEG2000 is performed on the bit plane of the plus / minus sign bit and the DWT coefficient converted to an absolute value. For this reason, it is more convenient that the DWT coefficient is converted to an absolute value at the time when it is output from the DWT processing unit 32, and this embodiment is treated as having such a configuration.

  The region determination processing unit 34 includes a subband determination processing unit 71 and a component determination processing unit 72 for HL subbands, a subband determination processing unit 73 and a component determination processing unit 74 for LH subbands, and a subband determination processing unit 74 for HH subbands. A subband determination processing unit 75, a component determination processing unit 76, a comprehensive determination processing unit 77, and a latch 78 are included.

  As shown in the figure, the subband determination processing unit 71 for the HL subband includes 16 binarizing means corresponding to the 1HL subband coefficient, a means for ORing the binarized result, and the 2HL subband coefficient. 4 binarizing means corresponding to the above, means for taking the OR of the binarized result, and one binarizing means corresponding to the 3HL subband coefficient. Although simplified in FIG. 12, the other subband determination processing units 73 and 75 have the same configuration.

  Next, processing is described for the subband determination processing unit 71 and the component determination processing unit 72, but similar processing is performed for other subbands.

  The absolute values of the 1HL, 2HL, and 3HL subband coefficients input to the region determination processing unit 34 are converted into predetermined threshold values (TL) predetermined for each subband by the binarization means in the subband determination processing unit 71. ) And is binarized to 1 (on) if it is greater than or equal to the threshold, and 0 (off) if it is less than the threshold. Here, the number of coefficients used for one determination is the number of shaded coefficients in FIG. That is, 16 coefficients of 1HL, 4 coefficients of 2HL, and 1 coefficient of 3HL are binarized at a time, and 1HL and 2HL are subjected to OR processing after binarization and output as a binary signal. In other words, 1HL and 2HL are equivalent to binarizing the maximum value of the input coefficient. The three binarized signals are input to the component determination unit 72 in this way. The component determination unit 72 outputs a 1 (ON) signal as a determination result if two or more of the three binarized signals are 1 (ON), and a 0 (OFF) signal otherwise. . When the determination result signal is on, it indicates that the image is a character. This is because an image having a strong edge such as a character has a large value in the DWT coefficient and the above-described characteristics of “correlation between subbands in the same direction”. The reason why only one coefficient is ON is not determined as a character is to eliminate the influence of noise. Similar determination result signals are also output from the component determination processing units 74 and 76.

  In the comprehensive determination processing unit 77, the OR of the determination result signal of the component determination processing units 72, 74, and 76 and the comprehensive determination result (output of the comprehensive determination processing unit 77) by the coefficient group processed immediately before in the same tile is performed. If it is on, it is determined to be a character. This process is performed for all the coefficients in the tile, and the overall determination result at the time when the determination of all the areas in the tile is completed is latched in the latch 78 and becomes the area determination data of the tile. That is, if there is one ON in the tile, the tile is determined as a character, and if there is no ON, it is determined as a non-character area. At the start of determination of the next tile, the output of the comprehensive determination processing unit 77 is cleared to zero.

  Note that a plurality of the circuits in FIG. 12 may be provided in parallel, and an OR operation may be performed on the output signal of the comprehensive determination processing unit 77 for all coefficient sets in the tile. Such a configuration increases the amount of hardware, but the determination process per tile can be performed in a very short time.

  As is apparent from the above description, the area determination processing unit 34 of this embodiment corresponds to the area determination means in the invention of claim 6.

  As described above, in this embodiment, the region determination process is completed only within the encoder 34 because the region determination is performed based on the discrete wavelet transform coefficients generated in the course of the encoding process. Also, the area determination process is relatively simple. This is advantageous for simplification of hardware related to area determination, reduction of hardware amount, or simplification and speeding up of software related to area determination.

  Although an example in which the DWT coefficient of 3 decomposition levels is used is shown here, the same region determination can be performed using more or less decomposition levels. In addition, it is not necessary to use all the decomposition level DWT coefficients generated by the encoding process, and the area determination may be performed by selectively using a coefficient of a necessary resolution. In general, in the DWT, the decomposition level is increased with octave division, so that the resolution decreases by 1/2 as the decomposition level increases. Therefore, when an edge of a character or the like is determined, the determination of the edge becomes inaccurate at a character dense portion or the like at a very low resolution. Therefore, it is preferable to use a DWT having a necessary resolution for the region determination in consideration of this.

  In the second embodiment, the same region determination using DWT may be performed during the encoding process by the encoder. If the result is stored in the code stream by a method such as describing it in the header of the code stream, the code stream change processing unit 50 can recognize the result of area determination by header analysis. Such embodiments are also encompassed by the present invention.

<Modification>
Here, as a modification of the present embodiment, a method for determining black characters, color characters, halftone dots, and other regions from DWT coefficients will be briefly described. In the determination of only the character area / non-character area, the DWT coefficient of only the Y component is used, but in this modification, all the components of Y, U, and V are used for the area determination.

  That is, for each of the Y, U, and V components, the character / non-character region determination is performed by the region determination processing unit configured as shown in FIG. When the determination result of the U component and / or the V component is a character area, it is determined as a color character area. When the determination result of both the U and V components is a non-character area and the determination result of the Y component is a character area, it is determined as a black character area.

  As described above, a high DWT coefficient appears in a specific subband in the halftone dot region. Based on this property, for example, the component determination processing unit 72 can perform determination such as a character region when two or more coefficients are equal to or greater than a threshold value, and a halftone dot region when there is one. However, such a halftone dot area determination method is prone to misjudgment. Therefore, a more preferable method is to include a counter that counts the number of halftone dots determined by the overall determination processing unit 77, and includes a tile area. If the method of determining a halftone dot region only when the number determined as a halftone dot exceeds a predetermined number, the determination accuracy of the halftone dot region is improved.

  In addition, you may utilize the halftone dot determination method described in the patent document 4 which concerns on the patent application of this applicant. This Patent Document 4 describes several methods for determining a halftone dot region using DWT coefficients. The outline of one method will be described below. The DWT coefficient used is that of the Y component.

  The absolute values of the 1HL, 1LH, and 1HH subband coefficients, which are the highest frequency components, are compared with thresholds corresponding to the subbands in a predetermined block unit. If the absolute value of any subband coefficient is not greater than the threshold value, the block is set as a non-halftone area. When any of the subbands is larger than the threshold, a subband coefficient that is an intermediate frequency component, for example, 3HL, 3LH, and 3HH subband coefficients in the case of 4 decomposition levels, is selected, and the absolute value is set to the threshold corresponding to the subband. Compare with each. Then, when the absolute values of all the subband coefficients are smaller than the threshold value, the block is determined as a halftone dot region.

  In the first to fifth embodiments, tiles are used as blocks which are units for area determination and hierarchical division configuration control. However, tiling in JPEG2000 is an option, and tiling is not necessarily performed. If tiling is not performed, it is not appropriate to use tiles as blocks. JPEG2000 code blocks and precincts can be used as applicable blocks regardless of tiling.

  Here, an embodiment for controlling the layer division configuration in units of code blocks will be described. The basic configuration of the image processing apparatus according to the present embodiment is the same as that of the fifth embodiment, and only differences will be described. The encoder 12 of the image processing apparatus according to the present embodiment has the same configuration as shown in FIG. 10 as in the fifth embodiment, but the area determination processing unit 34 has the configuration as shown in FIG.

  First, the relationship between subbands and code blocks will be described with reference to FIG. In the description of the fifth embodiment, the lattice delimited by the thin lines in FIG. 11 is treated as one wavelet coefficient. Here, each lattice is treated as one code block.

  In JPEG2000, the code block size must be equal in each subband. This means that if the decomposition level is different, the area (influence range) of the actual image reproduced by one code block is different. Therefore, even if it is determined as a character area at the composition level 3, it may be determined as a non-character area at the composition level 2 or 1. For example, even if the shaded 3HL code block in FIG. 11 is determined to be a character area, the 16 code blocks of 1HL corresponding thereto may contain a character area and a photo area. However, even in such a case, according to the region determination method of the present embodiment, appropriate layer division and progressive reproduction of an image can be realized without hindrance.

  The region determination processing unit 34 of the encoder 12 according to the present embodiment will be described with reference to FIG. As shown, a subband determination processing unit 71 and component determination processing unit 72 for HL subbands, a subband determination processing unit 73 and component determination processing unit 74 for LH subbands, and a subband for HH subbands The determination processing unit 75, the component determination processing unit 76, and the comprehensive determination processing unit 77 are configured. As shown in the figure, the HL subband subband determination processing unit 71 includes three sets of binarization means and buffers corresponding to the composition level. Although simplified in FIG. 13, the other subband determination processing units 73 and 75 have the same configuration.

  Next, processing is described for the subband determination processing unit 71 and the component determination processing unit 72, but similar processing is performed for other subbands.

  The area determination processing unit 34 sequentially receives the absolute value of the DWT coefficient of each code block of each subband. In the subband determination processing unit 71, the binarization means corresponding to the decomposition level sequentially compares one coefficient at a time with a predetermined threshold value (TL) determined in advance for each subband. If it is less than the threshold, it is binarized to 0 (off). The binarization result is stored in a buffer in the subband determination processing unit 71.

  The reason for adopting such a configuration will be described below. The code block size in JPEG2000 can take a free value of 4096 or less, which is a power of 2 for both main and sub-scanning. However, in consideration of coding efficiency and convenience, a size of 32 × 32 or 64 × 64 is often used, and if all DWT coefficients in a code block are to be binarized at the same time, binarization means Since the amount of hardware increases, the binarization means prepares one by one corresponding to the decomposition level, sequentially performs binarization processing, and stores the result in the buffer. Of course, if the increase in the amount of hardware is not an issue, all the coefficients can be binarized in parallel.

  Similar processing is performed in the other subband determination processing units 73 and 75.

  As shown in FIG. 14, the component determination processing unit 72 includes three sets of OR processing units 81 and buffers 82 corresponding to the composition level, and a determination processing unit 83. The other component determination processing units 74 and 76 have the same configuration.

  The process of the component determination processing unit 72 will be described with reference to FIG. The binarized HLb signal for each subband read from the buffer in the subband determination processing unit 71 is ORed by the OR processing unit 81 for each code block, and the binary signal (1HLcb, 2HLcb, 3HLcb ) Is obtained. The binarized signal after the OR process is 1 (on) if there is a coefficient whose binarization result is 1 (on) in the code block (meaning that the code block is a character candidate). ). The binarized signal in units of code blocks output from the OR processing unit 81 is temporarily stored in the buffer 82.

  The determination processing unit 83 reads a binary signal in units of code blocks from the buffer 82, and outputs a determination result (J_1HL, J_2HL, J_3HL) for each subband. Since this determination result is output in units of code blocks for each subband, four J_2HLs and 16 J_1HLs are output for one J_3HL. The determination in the determination processing unit 83 is performed as follows. In J_3HL determination, if 2 or more of the 3HLcb signal, the corresponding 4HL signal of 2HLcb signal, and the corresponding 16HL signal of 3HLcb signal are 1 (ON), 1 (character) is output. Otherwise, 0 (non-character) is output. In J_2HL determination, if 2 or more of 2HLcb signal, corresponding 3HLcb, and corresponding OR signals of 1HLcb signal are 1, 1 (character) is output, otherwise 0 (non-character) ) Is output. In the determination of J_1HL, if two or more of the 1HLcb signal, the corresponding 2HLcb signal, and the corresponding 3HLcb signal are 1, 1 (character) is output, otherwise 0 (non-character) is output.

  The processing of the other component determination processing units 74 and 76 is the same.

  The overall determination processing unit 77 performs OR processing on the determination output signals of the LH, HL, and HH components output from the component determination processing units 72, 74, and 76 for each corresponding code block for each decomposition level, thereby performing decomposition. Outputs area judgment data J1, J2, and J3 for each code block by level. As is apparent from the description so far, four J2s and 16 J1s are output for one J3.

  When tiling is performed, the size of a code block having a high composition level may be smaller than that of a code block having a composition level 1 due to the number of DWT coefficients included in the subband (code code). The block size is the smaller of the default value and the number of coefficients in the subband). In this case, the number of area determination data to be output and the number of binary data to be input to the OR process vary from the above-mentioned number, but this is not a problem.

  Since the configuration of the code stream generation processing unit 37 in the present embodiment is basically the same as that of the first embodiment, description will be made with reference to FIG. In the case of the code stream generation processing unit 37 according to the present embodiment, a pattern table 43 and a selector 44 are provided for each composition level, and region determination data corresponding to the composition level is input to each selector 44. Information of the pattern table 43 for each composition level selected by each selector 44 is input to the packet generation processing unit 45, and a code stream in which the layer division configuration is controlled for each code block is generated.

  In addition, by inputting the region determination data of each decomposition level to the packet generation processing unit 45, the code amount of each layer as described in the fifth embodiment can be switched between the character block and the non-character block. Is clear.

  This embodiment can be applied regardless of the presence or absence of tiling. Further, according to the present embodiment, it is possible to perform area determination with higher accuracy in units of blocks smaller than the tile size, and to control the code stream hierarchy division configuration and further control the code amount in units of hierarchies.

  Although this embodiment is an example in which region determination is performed based on the DWT coefficient, the same code is used when performing region determination based on an image region separation signal of image data as in the first to fourth embodiments. It is also possible to perform block area determination. In this case, the region determination data at each decomposition level may be generated by performing OR processing on the image area separation signal of the pixel (area) corresponding to each decomposition level. Then, according to the area determination data, code stream layer division control similar to that of the above embodiment, and code amount control for each layer can be performed.

  It is also possible to perform area determination in units of precincts by a method similar to the method shown in the present embodiment. The precinct in JPEG2000 can be selected to a certain degree of freedom for each decomposition level, so that the determination area can be selected flexibly.

  Here, an embodiment in which region determination is performed based on a code stream will be described.

In the case of JPEG2000, as a method for performing region determination based on code data, using information recorded in the packet header is a simple and high-speed method. The packet header includes the following information regarding the code block included in the packet.
Zero length packet: Whether the packet length is 0 (whether it is an empty packet) or not.
Code block inclusion: whether or not the current code block was included in the previous layer packet.
Zero bit plane information: The number of bit planes (no information) in code block units (number of MSBs until a code appears).
The number of coding passes: the number of coding passes included in the current packet in code block units. The code length (code amount) of the code block: the amount of code included in the current code block.
That is, these pieces of information can be acquired only from the packet header. Basically, a variable length code has many values when the code amount is large (the code is long). In the case of JPEG2000, since it is bit plane encoding, when the code amount of the upper bit plane is larger than a specific value (greater than 0), the corresponding DWT coefficient is represented by that bit plane. It is clear that it has a value greater than the numerical value.

  By using such a property, it is scanned whether there is a coefficient having a large value (character candidate) in all code blocks in each subband, and similar to the method described in connection with the fifth embodiment. It is possible to perform tile area determination by the above method (method of making a character area when a large DWT coefficient value (character candidate) exists in two or more subbands).

  The simplest method is to use the above 0 bit plane information to determine from the information on the bit position in the DWT coefficient of the non-zero sub bit plane. For example, when the dynamic range of a DWT coefficient included in a certain code block is 10 bits and the number of 0-bit planes is 3, for example, the maximum absolute value of the DWT coefficient included in the code block is 64 to 127. It is a range. Note that the value of the DWT coefficient represents the value of the DWT coefficient immediately after the DWT process in which quantization and post-quantization are not performed. An embodiment in which region determination is performed by this method will be described.

  The overall configuration of the image processing apparatus according to the present embodiment will be described on the assumption that the configuration shown in FIG. In this embodiment, the area determination process is performed inside the code stream change processing unit 50 of FIG.

  The configuration of the code stream change processing unit 50 according to the present embodiment is shown in FIG. The difference between the present embodiment and the second embodiment is that an area determination processing section 53 that performs area determination in units of tiles based on the header analysis information output from the header analysis processing section 51 is added. In the case of the second embodiment, the area determination is performed by the encoder 12 during the encoding process, and the area determination result is held in the tile header portion of the code stream. However, in the present embodiment, this is not necessary. . Therefore, the area determination processing unit 34 of the encoder 12 need not be provided. In addition, the progressive order and the hierarchical division configuration of the code stream generated by the encoder 12 are arbitrary, and may not be progressive.

  The header analysis processing unit 51 analyzes the packet header information of the input packet and outputs the number of 0-bit planes in the code block unit related to the Y signal (of the YUV signal). Here, for example, in FIG. 11 showing a DWT coefficient of 3 decomposition levels, it is assumed that each grid divided by thin lines is one code block, and the number of 0 bit planes for each code block is calculated from the header analysis processing unit 51. This is output and input to the area determination processing unit 53.

  FIG. 16 is a block diagram of the area determination processing unit 53. As illustrated, the region determination processing unit 53 includes a subband determination processing unit 201 and a component determination processing unit 202 for HL subbands, a subband determination processing unit 203 and a component determination processing unit 204 for LH subbands, The sub-band determination processing unit 205 and the component determination processing unit 206 for the HH sub-band, an overall determination processing unit 207, and a latch 208 are included.

  As shown in the figure, the subband determination processing unit 201 for the HL subband includes 16 binarizing units corresponding to the 1HL subband coefficient, a unit for ORing the binarized signal, and the 2HL subband coefficient. 4 binarizing means, a means for ORing the binarized signal, and one binarizing means corresponding to the 3HL subband coefficients. Although simplified in FIG. 16, the other subband determination processing units 203 and 205 have the same configuration.

  The number of 0-bit planes in units of code blocks output from the header analysis processing unit 51 is once recorded in the buffer 200. When the number of 0-bit planes of all code blocks in the tile is accumulated in the buffer 200, the subband determination processing unit 201 uses the binarization unit to calculate the number of 0-bit planes for 1HLz, 2HLz, and 3HLz for each subband. Compared with a predetermined threshold value (TL_1HLz, TL_2HLz, TL_3HLz) determined in advance, it is 0 (off) if it is larger than the threshold value, and 1 (on) if it is less than the threshold value. Turn into. As described above, this binarization process has an output level opposite to that of the normal binarization process. This matches the relationship that when the number of 0-bit planes is small, there is a large DWT coefficient in the code block, and when the number is large, it indicates that there is no large DWT coefficient in the code block. Because.

  The number of 0-bit planes used for one determination here is the number of shaded coefficients in FIG. 11 as in the fifth embodiment. That is, 16 coefficients of 1HLz, 4 coefficients of 2HLz, and 1 coefficient of 3HLz are binarized at a time, and 16 binaries and 4 binarized signals are ORed by OR means for 1HLz and 2HLz, The binary signal after the OR process is output from the subband determination processing unit 201.

  The component determination processing unit 202 is 1 (on), that is, a character region if two or more of the three binarized signals output from the subband determination processing unit 201 are 1 (on), and 0 otherwise. (Off), that is, a non-character area determination result signal is output.

  Although the subband determination processing unit 201 and the component determination processing unit 202 have been described, the subband determination processing unit 203 and the component determination processing unit 204 corresponding to the LH subband, the subband determination processing unit 205 and the component corresponding to the HH subband The same applies to the processing of the determination processing unit 208.

  The comprehensive determination processing unit 207 takes an OR of the determination result signals of the component determination processing units 202, 204, and 206 and the previous comprehensive determination result signal in the same tile, and outputs the OR signal as a comprehensive determination result signal. This total processing is performed for all coefficients in the tile, and the total determination result signal at the time when all the total determination processing in the tile is completed is latched in the latch 208, which becomes the area determination data of the tile. . That is, if there is one ON in the tile, it is determined as a character tile, and if there is no ON, it is determined as a non-character tile. Note that the output of the comprehensive determination processing unit 207 is cleared to 0 at the start of determination of the next tile.

  As is apparent from the above description, this embodiment is an embodiment of the invention of claim 7.

  As described above, according to the present embodiment, since area determination can be performed from code data, it is not necessary to perform area determination at the time of encoding processing. Therefore, an area can be stored in a storage medium such as a memory or an HDD or a code stream. There is no need to hold the determination result. Further, the area determination process based only on the packet header information in this embodiment has an advantage that the process is simple.

  In this embodiment, the area determination is performed in units of tiles. However, it is possible to easily perform the area determination in units of code blocks or the area determination in units of precinct as in the sixth embodiment by the same method. it is obvious.

  In the first to seventh embodiments, region determination is performed, and the hierarchical division configuration is controlled according to the region, and further, the code amount is controlled in units of layers. However, it is not easy to realize accurate region determination for various image data read from the scanner, and some misjudgment is inevitable. In particular, there is a high possibility that erroneous determination will occur in low-contrast characters such as pencil writing or characters on the color ground. And when a misjudgment arises and the process according to it is made, a sense of incongruity may arise on a decoded image. In order to alleviate such a sense of incongruity, it is effective to detect the feature amount of the image and control the layer division configuration (hierarchy division configuration) of the code stream according to the feature amount in “multistage”. Here, an embodiment that performs multi-stage control according to such feature amounts will be described.

  FIG. 17 is a block diagram illustrating the overall configuration of the image processing apparatus according to the present embodiment. This image processing apparatus is an MFP that can be used as the server 1, and is different from the configuration shown in FIG. 2 in that the image area separation processing unit 10 is replaced with a feature amount calculation unit 111. Other overall configurations and operations as the MFP are the same as described in connection with the first embodiment.

  Note that image data output from the scanner 9 is input to the image area separation processing unit 10, but image data input to the encoder processing unit 12 is input to the feature amount calculation unit 111. This is to improve the calculation accuracy of the feature amount by using the image data itself that is the object of the encoding process for the calculation of the feature amount.

  FIG. 18 is a block diagram of the feature amount calculation unit 111. As illustrated, the feature amount calculation unit 111 includes an edge amount calculation filter 121, an absolute value processing unit 122, a maximum value selection unit 123, and a lookup table (LUT) 124.

  The G signal of the image data is input to the feature amount calculation unit 111. The input G signal is subjected to filter calculation by two types of filters by the edge amount calculation filter 121, and the respective filter calculation results are input to the absolute value processing unit 122. The absolute value processing unit 122 converts each filter calculation result to an absolute value and outputs two data. The maximum value selection unit 123 selects and outputs the maximum value of the two data. This data is converted by the LUT 124 into an Edge signal that represents a predetermined number of feature quantities (edge quantities). In this embodiment, the Edge signal has eight values of 0 to 7, and the larger the value, the larger the edge amount. Note that the conversion by the LUT 124 is performed in order to bring the relationship between the Edge signal and the input image close to linear.

  In the edge amount calculation filter 121, for example, two types of first-order differential filters with different directivities as shown in FIGS. 19A and 19B are used.

  Note that the feature amount calculation processing unit 111 may calculate the edge amount from each of the R, G, and B signals of the image data, and select the maximum absolute value thereof. Alternatively, the edge amount may be calculated from one signal generated by a predetermined calculation from the RGB signal. Furthermore, it is also possible to input the Y signal generated by the color conversion process in the encoder 12 to the feature amount calculation unit 111 and perform the same edge amount detection. These embodiments are also encompassed by the present invention.

  FIG. 20 is a block diagram of the encoder processing unit 12 according to the present embodiment. Compared to the configuration shown in FIG. 3 according to the first embodiment or the like, the area determination processing unit 34 is replaced with a maximum value selection unit 131. The maximum value selection unit 131 selects the maximum value of the Edge signal in the tile for each tile, and outputs the value as a feature amount (edge amount) in the tile. This feature amount is input to the code stream generation processing unit 37.

  The configuration of the code stream generation processing unit 37 is the same as the configuration shown in FIG. 5 according to the first embodiment, but 8 types of layer patterns corresponding to the number of feature amount stages are set in the pattern table 43. Is done. For example, the layer pattern corresponding to the feature amount 0 (minimum edge amount) is the layer pattern for the non-character area in the first embodiment, and the layer pattern corresponding to the feature amount 7 (maximum edge amount) is the character in the first embodiment. As the layer pattern for the region, an intermediate layer pattern between the non-character region layer pattern and the character region layer pattern is used as the layer pattern corresponding to the intermediate feature amounts 1 to 6.

  The selector 44 selects a layer pattern corresponding to the feature amount and inputs it to the packet generation processing unit 45. Accordingly, the packet generation processing unit 45 generates a code stream having a hierarchical division configuration suitable for a non-character region for a tile having a feature amount of 0, and generates a code stream having a hierarchical division configuration suitable for a character region for a tile having a feature amount 7. In the tiles having the feature amounts 1 to 6, a code stream having an intermediate hierarchical division structure corresponding to the value is generated. In this way, the hierarchical division configuration (layer division configuration) of the code stream is controlled in multiple stages according to the multi-value feature quantity.

  As is apparent from the above description, this embodiment is an embodiment of the inventions of claims 8 and 18. The feature amount calculation unit 111 and the maximum value selection unit 131 and their operations correspond to the feature amount detection means and the process in claims 8 and 18. The pattern table 43 and the selector 44 and their operations correspond to the means and process for controlling the hierarchical division structure in claims 8 and 18.

  As described above, according to the present embodiment, since the layer division configuration of the code stream is controlled in multiple stages according to the feature amount (edge amount), the decoded image in the configuration in which the layer division configuration is controlled according to the region determination result. The problem of uncomfortable feeling due to erroneous determination is solved.

  In the code stream generation processing unit 37, only the layer pattern corresponding to the feature amount 0 and the layer pattern corresponding to the feature amount 7 are set in the pattern table 43, and the feature amounts 1 to 6 are set using these two types of layer patterns. An intermediate layer pattern corresponding to the above may be calculated, and such an aspect is also included in the present invention.

  Also, image area separation is performed, and the image area separation signal is input to the LUT 124. When the image area separation signal indicates a character area, an edge signal of 7 is output from the LUT 124, and the image area separation signal indicates a non-character area. In this case, it is also possible to combine image area separation with feature amount detection so that the LUT 124 outputs 0 to 6 Edge signals corresponding to the edge amount selected by the maximum value selection unit 123. Such embodiments are also encompassed by the present invention.

  Further, the edge amount is very suitable as a feature amount because it corresponds well to the code amount at the time of entropy encoding and to the character quality. However, it is also possible to use a feature amount other than the edge amount. For example, it is possible to perform pattern recognition and use the degree of similarity as a feature amount, and such an aspect is also included in the present invention.

  It is also possible to detect feature quantities in units of code blocks or precincts. For this purpose, for example, the maximum value in the code block or precinct may be selected by the maximum value selection unit 131.

  It is also possible to control the code amount in units of layers (hierarchical units) as described in connection with the third embodiment to the set code amount. Furthermore, the set code amount for each layer can be performed according to the feature amount. For example, in the code stream generation processing unit 37, by inputting the feature amount also to the packet generation processing unit 45, the packet generation processing unit 45 can control the code amount in units of layers according to the feature amount in multiple stages. This aspect corresponds to an embodiment of the inventions of claims 10 and 11.

  Further, in the image processing apparatus according to the second embodiment (FIG. 7), the feature amount detection as described above is performed, and the information of the detected feature amount is held in the code stream generated by the encoder 12, and the code The stream change processing unit 50 (FIG. 8) can also control the hierarchical division configuration (layer division configuration) according to the feature amount, as in the present embodiment. This aspect corresponds to an embodiment of the ninth aspect. Furthermore, it is possible to control the code amount of each layer in multiple stages according to the feature amount.

  In the eighth embodiment, the feature amount is detected from the real space image signal. Next, an embodiment in which the feature amount is calculated from the DWT coefficient will be described.

  The overall configuration of the image processing apparatus according to the present embodiment is a configuration in which the feature amount calculation unit 111 is omitted from the image processing apparatus according to the eighth embodiment (FIG. 17). Since the overall operation is the same as that of the eighth embodiment, description thereof will be omitted, and differences from the eighth embodiment will be described below.

  FIG. 21 is a block diagram of the encoder 12 according to the present embodiment. The encoder 12 has a configuration in which the area determination processing unit 34 of the encoder according to the fifth embodiment (FIG. 10) is replaced with a feature amount calculation processing unit 141.

  FIG. 22 is a block diagram of the feature amount calculation processing unit 141 according to the present embodiment. As illustrated, the feature amount calculation unit 141 includes an addition processing unit 151 corresponding to the HL subband, an addition processing unit 152 corresponding to the LH subband, an addition processing unit 153 corresponding to the HH subband, and a maximum value. A selection processing unit 154, a lookup table (LUT) 155, a maximum value selection processing unit 156, and a latch 157 are included.

  The DWT coefficient (absolute value) of the shaded portion in FIG. 11 is input to the feature amount calculation processing unit 141 at the same time as in the fifth embodiment (here, the fine line grid in FIG. 11 is the DWT). To indicate the coefficient value). The input WT, LH, and HH DWT coefficients are added by the addition processing units 151, 152, and 153, respectively, and the total value is output. The maximum value selection processing unit 154 selects the maximum value among the three total values. The selected maximum value is converted into an 8-value feature quantity by the LUT 155 as in the ninth embodiment. This feature value is compared with the previous feature value by the maximum value selection processing unit 156, and the larger one of them is output from the maximum value selection processing unit 156. Similar processing is repeated for all the coefficients in the tile, and the output value of the maximum value selection processing unit 157 at the time when the processing of all the coefficients is completed is latched in the latch 157 and output as the feature amount of the tile. Note that the output of the maximum value selection processing unit 156 is cleared to 0 before the start of the feature value calculation process for the next tile. That is, the feature value output from the feature value calculation processing unit 141 is the maximum value of the feature value in the tile.

  Then, the code stream generation processing unit 37 generates a code stream in which the hierarchical division configuration is controlled in multiple stages according to the feature amount as in the eighth embodiment.

  As is apparent from the above description, this embodiment corresponds to an embodiment of the invention of claim 13.

  It is obvious that the feature amount may be detected in units of code blocks or precincts by a similar method. Such an aspect is also included in the invention of claim 13.

  Further, the method for calculating the feature amount using the DWT coefficient is not limited to the above-described method.

  In the image processing apparatus according to the second embodiment (FIG. 7), the same feature amount detection is performed in the encoder 12, the detected feature amount is held in the code stream, and the code stream change processing unit 50 (FIG. 8) is detected. ), It is also possible to control the hierarchical division configuration (layer division configuration) according to the feature amount as in the present embodiment. This aspect corresponds to an embodiment of the ninth aspect. Furthermore, it is possible to control the code amount of the hierarchical unit (layer) in multiple stages according to the feature amount, and this aspect corresponds to an embodiment of claims 10 and 11.

  Next, an embodiment for calculating a feature amount from code data will be described.

  The overall configuration of the image processing apparatus according to the present embodiment is a configuration in which the image area separation processing unit 10 of the image processing apparatus according to the seventh embodiment (FIG. 7) is omitted. The overall operation is the same as that of the seventh embodiment.

  The code stream change processing unit 50 according to the present embodiment replaces the area determination processing unit 53 in the code stream generation processing unit (FIG. 15) according to the seventh embodiment with a feature amount calculation processing unit having the configuration illustrated in FIG. It is a configuration.

  As shown in the figure, the feature amount calculation processing unit includes a 0-bit plane analysis processing unit 161, addition processing units 162, 163, 164, a maximum value selection processing unit 165, a lookup table (LUT) 166, a maximum value selection processing unit 167, and a latch 168. Consists of.

  When a packet header is input from the header analysis processing unit 51, the 0-bit plane analysis processing unit 161 analyzes the 0-bit plane information of the packet header and obtains 0-bit plane information from the MSB of each subband code block unit. It is done. In other words, a sub bit plane including the most significant bit of the maximum value of each code block unit is obtained. The 0-bit plane analysis processing unit 161 gives weight values corresponding to this information and simultaneously outputs the shaded portion weights (W_1HL, W_2HL, W_3HL) in FIG. Indicates a code block). The weight here is 2 to the power of the “most significant bit”, for example, 128 if the most significant bit is the seventh bit from the LSB. In this case, the weight may be changed in units of sub bit planes. For example, when the most significant bit is the 7th bit significance propagation path from the LSB, 128 + 128 × 3/4 (128 is the 7th power of the 2 and 3/4 is the weight of the significance propagation path) 224, Magnitude refinement In the case of the path, the calculation is 128 + 128 × 2/4, 192, and in the case of the clean up path, the calculation is 128 + 128 × 1/4, 160.

  These weights are added by the addition processing unit 162 by the number of shaded code blocks shown in FIG. 11, that is, 16 W_1HL, 4 W_2HL, and 1 W_3HL. Similarly, addition processing units 163 and 164 perform addition processing on the weights of the LH and HH components. The maximum value among the three added values is selected by the maximum value selection processing unit 165. The subsequent processing is the same as in the ninth embodiment, and the output value of the maximum value selection processing unit 165 is converted to a value of 0 to 7 by the LUT 166, and the output value of the maximum value selection processing unit 167 just before that value is large. One value is selected by the maximum value selection processing unit 167, and the value finally latched by the latch 168 is output as the feature amount of the tile. As in the seventh embodiment, the packet generation processing unit 45 generates a code stream in which the hierarchical division configuration is controlled in multiple stages according to the feature amount.

  As is apparent from the above description, this embodiment corresponds to an embodiment of the invention of claim 10.

  It should be noted that the feature quantity calculation unit area is not limited to the tile, and it is obvious that the feature quantity in the code block unit or the precinct unit can be calculated by the same method, and this aspect is also included in the invention of claim 10. .

  Each of the above embodiments relates to the image processing apparatus on the server side. However, when the transfer speed of the transfer path is sufficiently high but the decoding speed of the client is low, the client side described the above embodiments. It is effective to control the hierarchical division configuration according to the image type for each region.

  The embodiment described here relates to an image processing apparatus on the client side that controls the hierarchical division configuration of progressive decoding according to the image type for each region.

  FIG. 24 is a block diagram illustrating a configuration related to the decoding process of the image processing apparatus according to the present embodiment. The image processing apparatus according to the present embodiment receives a code stream from a server via the external transfer path 2 and progressively decodes the code stream, and includes an external interface (I / F) 91, a code buffer 92, and a code stream dividing process. This includes a unit 93, an entropy decoding processing unit 94, a coefficient buffer 95, an inverse discrete wavelet transform (IDWT) processing unit 96, an inverse color conversion processing unit 97, and an image buffer 98. That is, the present embodiment has a configuration in which a code stream division processing unit 93 is added to the basic configuration for JPEG2000 decompression processing. The code stream division processing unit 93 includes means for performing character / non-character area determination in units of tiles, and in accordance with the determination result, a code data layer division configuration (hierarchy division configuration) is performed before the entropy decoding process. It is a means for controlling the hierarchical division structure of progressive decoding by controlling.

  The overall processing flow is as follows. The code stream transferred from the server 1 via the external transfer path 2 is accumulated in the code buffer 92 via the external I / F 91. When code data of one whole image is accumulated in the code buffer 92, the code stream decomposition processing unit 93 reads the code data from the code buffer 92 in units of tiles, selects code data of the corresponding progressive layer (layer), The data is output to the entropy decoding processing unit 94. The entropy decoding processing unit 94 decodes code data that is arithmetically coded in units of code blocks and in units of sub-bitplanes, and stores the decoded bitplane data in the coefficient buffer 95 in the form of DWT coefficients. . The IDWT processing unit 96 reads the DWT coefficient from the coefficient buffer 95, performs inverse discrete wavelet transform, which is the inverse transform of Equation (3), and outputs YUV image data. In the inverse color conversion processing unit 97, the YUV data is converted back to RGB image data by the inverse conversion of the above equations (1) and (2) and stored in the image buffer. Similar processing is performed for all tiles in the image, and the decoding of the layer (hierarchy) is completed. The image data in the image buffer 98 is sequentially displayed on the monitor 4.

  When the decoding process for one layer is completed, decoding of the next layer is started. The decoding process is performed in the same flow when decoding the next layer, but since the processed upper layer entropy-decoded bitplane data is stored in the coefficient buffer 95, the entropy decoding is performed again. Not done. That is, code data for only the next layer is entropy-decoded and newly stored in the coefficient buffer 95, and the subsequent processing from IDWT processing is performed again on the data, and the image data in the image buffer is stored. By updating the data, progressive decoding in which the hierarchical division configuration is controlled according to the region determination result is performed.

  FIG. 25 is a block diagram of the code stream decomposition processing unit 93. The code stream decomposition processing unit 93 has substantially the same configuration as the code stream change processing unit 50 (FIG. 15) according to the seventh embodiment, but includes a packet generation processing unit 45 and tile header generation processing related to code stream formation. The unit 46 is replaced with a code selection processing unit 101. Further, the area determination processing unit 53 in the code stream division processing unit 93 has the same configuration as that shown in FIG. 16 according to the seventh embodiment.

  The processing of the code stream division processing unit 93 is almost the same as that of the code stream change processing unit 50 of the seventh embodiment. The packet header of the input packet is analyzed by the header analysis processing unit 51, and the packet decomposition processing unit 52 In accordance with the control signal, the packet decomposition processing unit 52 decomposes the code portion of the packet into code data for each sub-bit plane and records it in the input code buffer 41. At this time, the header analysis processing unit 51 outputs the number of 0-bit planes in the code block unit related to the Y signal (of the YUV signal) to the region determination processing unit 53. The same tile / character character / non-character area determination as described above is performed. Depending on the result of this area determination, the character selection layer pattern or the non-character tile layer pattern set in the pattern table 43 is read into the code selection processing unit 101 via the selector 44.

  The processing up to this point is the same as in the seventh embodiment, but in this embodiment, the code selection processing unit 101 receives the code data of the sub bit plane corresponding to the layer to be decoded according to the layer pattern from the input code buffer 41. The difference is that the data is read and output to the entropy decoding processing unit 94 as it is. Such code stream decomposition processing is sequentially repeated for all tiles in the image from the upper layer to the lower layer, thereby enabling progressive decoding with a hierarchical division configuration according to the region determination result as described above. .

  As is apparent from the above description, this embodiment corresponds to an embodiment of the inventions of claims 15 and 19. The code stream decomposition processing unit 93 and its operation correspond to the means and process for controlling the hierarchical division structure of progressive decoding in claims 15 and 19, and the region determination processing unit 53 and its operation are described in claim 15. , 19 corresponds to the area determination means and process.

  In the present embodiment, the area determination is performed in units of tiles. However, it is obvious that the same area determination can be performed in units of code blocks or precincts, and this aspect is also the invention of claims 15 and 19. Is included.

  Further, as described in connection with the second embodiment, if region determination is performed on the server side and the region determination data is described in the comment marker portion of the tile header of the code stream, the code stream division is performed. In the processing unit 93 (FIG. 25), the header analysis processing unit 51 can extract the region determination data result from the tile header and give the region determination data to the selector 44. In this case, since the header analysis processing unit 51 serves as a region determination unit, the region determination processing unit 53 can be omitted. Such an embodiment is also included in the inventions of claims 15 and 19.

  The present embodiment relates to an image processing apparatus on the client side as in the eleventh embodiment, and the configuration related to the decoding process is the same as in the eleventh embodiment (FIG. 24).

  In addition, the configuration of the code stream decomposition processing unit 93 of the image processing apparatus according to the present embodiment is the configuration illustrated in FIG. 25 according to the eleventh embodiment, but the area determination processing unit 53 is a feature according to the tenth embodiment. It is replaced with an amount calculation processing unit (FIG. 23). A layer pattern corresponding to the feature value (8 levels of 0 to 7) in units of tiles calculated by the feature value calculation processing unit is input from the pattern table 43 to the code selection processing unit 101 via the selector 44, and the code selection processing is performed. The unit 101 performs code selection according to the feature amount according to the layer pattern. Thus, the hierarchical division configuration of progressive decoding can be controlled in multiple stages according to the feature amount in units of tiles.

  That is, this embodiment is an embodiment of the inventions of claims 16 and 20. The code stream decomposition processing unit 93 and its operation correspond to the means and process for controlling the hierarchical division structure of progressive decoding in claims 16 and 20, and the feature quantity calculation processing unit 3 and its operation are described in claim 16. , 20 corresponding to the feature amount calculating means and process.

  In this embodiment, the feature amount is calculated in units of tiles. However, it is obvious that the same feature amount can be calculated in units of code blocks or precincts. Included in the invention.

  If the feature amount calculation is performed on the server side and the feature amount data is described in the comment marker portion of the tile header of the code stream, the header analysis processing unit 51 performs the tile header in the code stream division processing unit 93. More feature amount data can be extracted and given to the selector 44, and a feature amount calculation processing unit can be omitted. Such an aspect is also included in the inventions of claims 16 and 20.

  It is obvious that the means for configuring the image processing apparatus of the present invention or the process for configuring the image processing method of the present invention can be realized using a computer such as a general-purpose computer or a microcomputer. A program for that purpose and various information recording (storage) media such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor storage element that can be read by a computer on which the program is recorded are also included in the present invention.

It is explanatory drawing of a client / server environment. 1 is a block diagram illustrating an overall configuration of an image processing apparatus according to a first embodiment of the present invention. 1 is a block diagram of an encoder according to Embodiment 1 of the present invention. FIG. It is explanatory drawing of the hierarchy division | segmentation structure of the layer progression in a character tile and a non-character tile. It is a block diagram of the code stream production | generation process part which concerns on the Example of this invention. It is explanatory drawing of the hierarchy division | segmentation structure of the resolution progression in a character tile and a non-character tile. It is a block diagram which shows the whole structure of the image processing apparatus which concerns on Example 2 etc. of this invention. It is a block diagram of the code stream change process part which concerns on Example 2 of this invention. FIG. 10 is a block diagram of a gift area separation processing unit according to a fourth embodiment of the present invention. It is a block diagram of the encoder which concerns on Example 5 etc. of this invention. It is explanatory drawing of the wavelet coefficient or code block of each subband. It is a block diagram of the area | region determination process part which concerns on Example 5 of this invention. It is a block diagram of the area | region determination process part which concerns on Example 6 of this invention. It is a block diagram for demonstrating the component determination processing part in FIG. It is a block diagram of the code stream change process part which concerns on Example 7 etc. of this invention. It is a block diagram of the area | region determination process part in FIG. It is a block diagram which shows the whole structure of the image processing apparatus which concerns on Example 8 etc. of this invention. FIG. 18 is a block diagram of a feature amount calculation unit in FIG. 17. It is a figure which shows an edge amount calculation filter. It is a block diagram of the encoder which concerns on Example 8 of this invention. It is a block diagram of the encoder which concerns on Example 9 of this invention. It is a block diagram of the feature-value calculation process part in FIG. It is a block diagram of the feature-value calculation process part which concerns on Example 10 of this invention. It is a block diagram of the image processing apparatus which concerns on Example 11 etc. of this invention. FIG. 25 is a block diagram of a code stream decomposition processing unit in FIG. 24. It is a block diagram for demonstrating the compression encoding process of JPEG2000. It is explanatory drawing of the wavelet coefficient by which the octave division was carried out. It is a schematic diagram of bit plane decomposition and sub-bit plane decomposition. It is a schematic diagram of the code stream of layer progression. It is a schematic diagram which shows the progressive order of LRCP progression of JPEG2000 and RPCL progression. It is a figure which shows the example of the code sequence of LRCP progression. It is a figure which shows the example of the code arrangement | sequence of RLCP progression.

Explanation of symbols

1 Server 2 Client 10 Image Area Separation Processing Unit 12 Encoder 13 CPU
14 Memory 15 HDD
17 External I / F
31 Color Conversion Processing Unit 32 Discrete Wavelet Transform (DWT) Processing Unit 34 Region Determination Processing Unit 35 Entropy Coding Processing Unit 36 Post Quantization Processing Unit 37 Code Stream Generation Processing Unit 41 Input Code Buffer 43 Pattern Table 44 Selector 45 Packet Generation Processing Unit 46 tile header generation processing unit 47 output code buffer 50 code stream change processing unit 51 header analysis processing unit 52 packet decomposition processing unit 53 area determination processing unit 91 external I / F
92 Code buffer 93 Code stream decomposition processing unit 94 Entropy decoding processing unit 95 Coefficient buffer 96 Inverse discrete wavelet transform (IDWT) processing unit 97 Inverse color conversion processing unit 98 Image buffer 101 Code selection processing unit 111 Feature quantity calculation unit 141 Feature quantity Calculation processing unit

Claims (22)

Encoding means for performing an image encoding process and outputting a progressive code stream of the image in a predetermined progressive order;
Area determining means for determining the image type of the image in predetermined block units,
The code stream generation means for generating the progressive code stream in the encoding means includes means for controlling the hierarchical division structure of the progressive code stream for each block according to the determination result by the area determination means. An image processing apparatus. Storage means for storing a code stream of the image;
Area determination means for determining the image type of the image in a predetermined block unit;
Code stream generation means for taking in the code stream stored in the storage means and generating a progressive code stream in a predetermined progressive order from the code stream;
The image processing apparatus according to claim 1, wherein the code stream generation means includes means for controlling the hierarchical division configuration of the progressive code stream for each block according to a determination result by the area determination means.   3. The image processing according to claim 1, wherein the control unit included in the code stream generation unit controls a code amount in a hierarchical unit of the progressive code stream to a predetermined set code amount. 4. apparatus.   The image processing apparatus according to claim 3, wherein the control unit included in the code stream generation unit switches a set code amount in a hierarchical unit of the progressive code stream according to a determination result by the region determination unit. .   The image processing apparatus according to claim 1, 2, 3, or 4, wherein the image type determined by the area determination unit includes at least one of a character, a black character, a color character, and a halftone dot.   The image processing apparatus according to claim 1, wherein the area determination unit determines an image type based on a frequency conversion coefficient of an image generated in the course of an image encoding process.   The image processing apparatus according to claim 2, wherein the area determination unit determines an image type based on a code stream captured by the code stream generation unit. Encoding means for performing an image encoding process and outputting a progressive code stream of the image in a predetermined progressive order;
Feature amount detection means for detecting feature amounts of an image in predetermined block units;
The code stream generation means for generating the progressive code stream in the encoding means controls the hierarchical division configuration of the progressive code stream in a multi-stage manner for each block according to the feature quantity detected by the feature quantity detection means. An image processing apparatus comprising: Storage means for storing a code stream of the image;
Feature amount detection means for detecting feature amounts of an image in predetermined block units;
Code stream generation means for taking in the code stream stored in the storage means and generating a progressive code stream in a predetermined progressive order from the code stream;
The image processing apparatus according to claim 1, wherein the code stream generation means includes means for controlling the hierarchical division configuration of the progressive code stream in multiple stages for each block according to the feature quantity detected by the feature quantity detection means.   10. The image processing according to claim 8, wherein the control unit included in the code stream generation unit controls a code amount of a hierarchical unit of the progressive code stream to a predetermined set code amount. apparatus.   The control means included in the code stream generation means switches the set code amount in a hierarchical unit of the progressive code stream in multiple stages according to the feature amount detected by the feature amount detection means. The image processing apparatus according to 10.   The image processing apparatus according to claim 8, 9, 10, or 11, wherein the feature amount of the image detected by the feature amount detection means is an edge amount.   The image processing apparatus according to claim 8, wherein the feature amount detection unit detects a feature amount based on a frequency conversion coefficient of an image generated in the course of an image encoding process.   The image processing apparatus according to claim 9, wherein the feature amount detection unit detects a feature amount based on a code stream captured by the code stream generation unit. An image processing apparatus that receives a progressive code stream of an image and performs progressive decoding,
Area determination means for determining the image type of the image in a predetermined block unit;
An image processing apparatus comprising: means for controlling a hierarchical division configuration of progressive decoding for a received progressive code stream for each block according to a determination result by the area determination unit. An image processing apparatus that receives a progressive code stream of an image and performs progressive decoding,
Feature amount detection means for detecting feature amounts of an image in predetermined block units;
An image processing apparatus comprising: means for controlling a hierarchical division configuration of progressive decoding for a received progressive code stream in a multistage manner for each block according to the feature quantity detected by the feature quantity detection means. An area determination step for determining an image type of an image in units of a predetermined block;
A code stream generating step for generating a progressive code stream of an image according to a predetermined progressive order,
The code stream generation step includes a step of controlling a hierarchical division configuration of the progressive code stream for each block according to a determination result by the region determination unit. A feature amount detection step of detecting multi-value feature amounts of an image in a predetermined block unit;
A code stream generating step for generating a progressive code stream of an image according to a predetermined progressive order,
The code stream generation step includes a step of controlling the hierarchical division configuration of the progressive code stream in multiple stages according to the feature amount detected by the feature amount detection step for each block. An image processing method for receiving a progressive code stream of an image and performing progressive decoding,
An area determination step for determining an image type of an image in units of a predetermined block;
And a step of controlling, for each block, a hierarchical division configuration of progressive decoding for the received progressive code stream according to a determination result of the region determination step. An image processing method for receiving a progressive code stream of an image and performing progressive decoding,
A feature amount detection step of detecting a feature amount of an image in a predetermined block unit;
And a step of controlling the hierarchical division structure of progressive decoding for the received progressive code stream for each block according to the feature amount detected by the feature amount detection step.   A program that causes a computer to function as each unit of the image processing apparatus according to claim 1.   A computer-readable information recording medium on which the program according to claim 21 is recorded.

Images