CN101072354A - Image coding apparatus, image decoding apparatus, image display apparatus and image processing apparatus - Google Patents

Image coding apparatus, image decoding apparatus, image display apparatus and image processing apparatus Download PDF

Info

Publication number
CN101072354A
CN101072354A CNA2007101088560A CN200710108856A CN101072354A CN 101072354 A CN101072354 A CN 101072354A CN A2007101088560 A CNA2007101088560 A CN A2007101088560A CN 200710108856 A CN200710108856 A CN 200710108856A CN 101072354 A CN101072354 A CN 101072354A
Authority
CN
China
Prior art keywords
image
zone
gazing
conversion coefficient
roi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CNA2007101088560A
Other languages
Chinese (zh)
Other versions
CN101072354B (en
Inventor
武田和彦
冈田茂之
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Corp of America
Original Assignee
Sanyo Electric Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2004032394A external-priority patent/JP4514464B2/en
Priority claimed from JP2004094449A external-priority patent/JP4241463B2/en
Application filed by Sanyo Electric Co Ltd filed Critical Sanyo Electric Co Ltd
Publication of CN101072354A publication Critical patent/CN101072354A/en
Application granted granted Critical
Publication of CN101072354B publication Critical patent/CN101072354B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The invention provides a picture coding device comprises a selective unit for selecting observation zone on dynamic image, a encode unit for encoding the dynamic image, and a generation unit for generating coding stream comprising encoded dynamic image and information used in particular observation zone.

Description

Picture coding device and method for encoding images
The application is that application number is " 200510007933.4 ", and the applying date is on February 5th, 2005, and denomination of invention is divided an application for the application of " picture coding device, image decoder, image display device ".
Technical field
The present invention relates to image coding technique and image decoding technology, particularly with a part of regional priority encoding of image or the device and method of decoding.And, the invention still further relates to the device and the image processing apparatus of image display device and method for displaying image, reproduced picture stream.
Background technology
In ISO/ITU-T, follow-up as the standard technique JPEG (JointPhotgraphic Expert Group) of the compressed encoding of rest image adopted the standardization of the JPEG2000 of wavelet transform (DWT).In JPEG2000, postpone to be encoded to lossless compress from low level, can encode to the picture quality of wide scope with high-performance, also easy realization slowly improves scalability (scalability) function of picture quality.And, in JPEG2000, prepare to have the multiple function that does not have in the existing Joint Photographic Experts Group.
As one of function of JPEG2000, image gazed at zone (Region of Interest; ROI) than other regional priority encodings, the ROI of transmission encodes by standardization.According to ROI coding, when in coding delay, existing in limited time, except that can preferentially make the reproduced picture quality of gazing at the zone become high-quality, when successively with the decoding of coded image stream, can gaze at the zone with high-quality regeneration ahead of time.
Encode as ROI, have: will amplify maximum displacement method corresponding to bit plane the yardstick that image is gazed at the wavelet conversion coefficient (hereinafter referred to as the ROI conversion coefficient) in zone corresponding to the maximum number of digits of the bit plane of the non-wavelet conversion coefficient (hereinafter referred to as non-ROI conversion coefficient) of gazing at the zone, according to this method, all bit planes of ROI conversion coefficient are all encoded earlier than the bit plane of any one non-ROI conversion coefficient.And, also known: as, thereby to make the high-order position of a part of ROI conversion coefficient also preferentially carry out Methods for Coding than non-ROI conversion coefficient by a bit plane yardstick multiplying gauge location number with the ROI conversion coefficient.In patent documentation 1, the improving technology of these ROI coding methods has been proposed.
Fig. 1 is the pie graph of existing picture coding device 100.This picture coding device 100 utilizes the maximum displacement method to carry out the ROI coding.Wavelet transformation portion 10 carries out wavelet transformation with the original image of input, the output wavelet conversion coefficient.Quantization portion 12 is with the wavelet conversion coefficient quantization of original image.Wavelet conversion coefficient 50 after Fig. 2 (a) expression quantization comprises from highest order (Most Significant Bit; MSB) to lowest order (Least Signifcant Bit; LSB) each bit plane of 5.
ROI selection portion 18 is selected the zone of gazing on the original image, and it is the ROI mask of ROI conversion coefficient corresponding to the selecteed wavelet conversion coefficient of gazing at the zone that 20 generations of ROI mask generating unit are used for specific.The ROI conversion coefficient is represented with oblique line in the wavelet conversion coefficient 50 of Fig. 2 (a).
The S position will be amplified by a quantized ROI conversion coefficient yardstick with reference to the ROI mask in ROI yardstick enlarging section 22.That is the value of ROI conversion coefficient is only moved to left S position.At this, yardstick amplification quantity S be than corresponding to non-gaze at the zone wavelet conversion coefficient be the also big natural number of maximum figure place of the quantization value of ROI conversion coefficient.The wavelet conversion coefficient 52 of the state after Fig. 2 (b) expression ROI conversion coefficient is only amplified 5 by yardstick.In the conversion coefficient 52 after yardstick amplifies, to amplifying and new position value of zeroizing that produces by yardstick.
Entropy coding portion 14 is shown in the arrow of Fig. 2 (c), and the quantization value of the wavelet conversion coefficient 52 after yardstick amplified averages the amount of information coding when high-order bit plane scans successively.ROI positional information and yardstick amplification quantity that coded data generating unit 16 is gazed at regional location with the data behind the entropy coding with coding parameters such as quantization amplitude, expression are carried out fluidisation, to export as coded image.
Fig. 3 is the pie graph of existing image decoder 110.Image after this image decoder 110 utilizes the maximum displacement method to the ROI coding is deciphered.Coded data extraction unit 30 is extracted coded data, various coding parameter, ROI positional information and yardstick amplification quantity from the coded image of input.Average information decoding part 32 is deciphered coded data by each bit plane, and the quantization value of the wavelet conversion coefficient that obtains is stored in the memory.Wavelet conversion coefficient 54 after the decoding of Fig. 4 (a) expression average information.Corresponding to the example of Fig. 2 (a)～(c), yardstick amplification quantity S is 5, and the wavelet conversion coefficient 54 after the average information decoding comprises 10 bit planes altogether.
ROI mask generating unit 38 generates the mask of specific ROI conversion coefficient based on the ROI positional information.The ROI yardstick dwindles portion 40 with reference to the ROI mask, will be dwindled the S position by the ROI conversion coefficient yardstick in the quantized wavelet conversion coefficient.Fig. 4 (b) expression ROI conversion coefficient is only dwindled the wavelet conversion coefficient 56 of volume state after 5 by yardstick.Dwindle and 5 of the high positions of remaining non-ROI conversion coefficient are rejected by yardstick, obtain 5 bit planes altogether by the wavelet conversion coefficient after the quantization 56.
Inverse guantization (IQ) portion 34 will comprise the quantization value of the wavelet conversion coefficient of the ROI conversion coefficient that is dwindled by yardstick and carry out inverse guantization (IQ), and 36 pairs in wavelet inverse transformation portion is carried out wavelet transformation by the wavelet conversion coefficient after the inverse guantization (IQ), and the decoding image that obtains of output.
Moreover, Motion-JPEG2000 compares with MPEG moving picture encoding specifications such as (Motion Picture Coding ExpertsGroup), because do not use frame relevant, although so compression rate variance, but have with the frame is the editting function of unit, the advantage of reversible encoding function aspects, we can say and has hidden bigger possibility.
Technology example as this JPEG2000 characteristic of flexible Application discloses in patent documentation 2: in each stage of image decoding, compare elapsed time and binding hours, and simplify the technology that decoding is handled as required.In view of the above, can constitute with smaller scale, the image that contrasts higher picture quality is deciphered.
[patent documentation 1]
The spy opens the 2001-45484 communique
[patent documentation 2]
The spy opens the 2002-325257 communique
In above existing ROI coding,, carry out the coding of bit plane, so in order to store the bit plane after yardstick amplifies, memory span needs surplus because after when coding, will amplifying corresponding to the wavelet conversion coefficient yardstick of gazing at the zone.And, because must only the bit plane part after the yardstick amplification partly be encoded as surplus, so in encoding process, want spended time.
In the time will being deciphered, for coded image being deciphered, according to the unnecessary memory span of yardstick amplification quantity needs by each bit plane by the coded image behind the ROI coding.And, need carry out: obtain the ROI positional information by coded image data, and specific corresponding to the wavelet conversion coefficient of gazing at the zone, with the specific wavelet conversion coefficient of gazing at the zone carry out yardstick dwindle, processing.
Like this, existing ROI coding is owing to follow the yardstick processing and amplifying, thus need memory span that surplus is arranged, the problem that exists operand also to increase.
In above-mentioned existing ROI coding, when gazing at the zone, exist in these and gaze at the interregional problem that the difference of picture quality can not be set as a plurality of.
From now on, expectation is extensively carried out the dispensing of the picture material of utilizing JPEG2000 and is effectively used.By in the image of providing and delivering, for example monitor camera, have only its part to need high image quality sometimes, but do not need high picture quality like that in zone in addition.And in the time of will be by the regional high image quality of user's appointment, the increase of its treating capacity sometimes can surpass the ability of image processing apparatus, and produces coma landing etc.
Summary of the invention
The present invention carries out in view of such problem, and its purpose is to provide a kind of unnecessary cost that do not increase, and can be with the regional priority encoding of a part, with the image encoding and the decoding technique of encoded image data decoding.
Another purpose is to provide a kind of can will comprise a plurality of area image codings, the decoding of different images quality and image coding technique, image decoding technology and the image display technology that shows.
Moreover another purpose is to provide a kind of technology of adjusting picture quality in the scope that is no more than the decoding disposal ability.
A kind of method for encoding images of the present invention is characterized in that, is used for the zone of gazing at that defines on dynamic image is carried out information specific and recorded and narrated encoding stream in the coded data that comprises described dynamic image.
Another kind of method for encoding images of the present invention is characterized in that, is used for a plurality of zones of gazing at that define on image are carried out information specific and recorded and narrated encoding stream in the coded data that comprises described image.
A kind of picture coding device of the present invention comprises: selected cell, select the zone of gazing on the dynamic image; Coding unit is encoded to described dynamic image; And generation unit, generating encoding stream, this encoding stream comprises the coding dynamic image and is used for specific described information of gazing at the zone.
Another kind of picture coding device of the present invention comprises: selected cell, select a plurality of zones of gazing on the image; Coding unit is encoded to described image; And generation unit, generating encoding stream, this encoding stream comprises coded image and is used for specific described a plurality of information of gazing at the zone.
Certain form of the present invention relates to picture coding device.This device comprises: with image transform is spatial frequency zone and the transformation component that generates conversion coefficient; Corresponding in the ranking of the above-mentioned conversion coefficient in a part of zone on the above-mentioned image, only the regulation figure place of low level position is replaced into the replacement section of null value; With by after the displacement of above-mentioned replacement section, with the encoding section of carrying out compressed encoding from high-order position successively of ranking of the above-mentioned conversion coefficient of above-mentioned integral image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned conversion coefficient corresponding to the part zone on the above-mentioned image.
At this, the low level position that is replaced into null value also can be that the lowest order figure place from above-mentioned conversion coefficient begins number, the continuous bit position row of regulation figure place.
Part zone on the image also can be the non-zone of being located on the image of gazing at.The zone also can be gazed at and setting relatively by being provided with in this non-zone of gazing on image.Gaze at zone and non-gazing at regional the existence: as a zone is defined as gazing at, then another zone becomes non-relativeness of gazing at the zone, as an example, also can be certain different relatively zone of relative importance value, particularly picture quality be required certain different relatively zone of level.Gazing at the zone is to compare the demanding zone of relative importance value height, particularly picture quality with other zones, and the non-zone of gazing at also can be to compare with other zones that relative importance value is low, particularly picture quality requires low zone.
Constitute according to this, preferentially encoded than other zones in the zone at image place, when decoding regeneration, can improve this regional picture quality relatively.And, reduce by the low level information that other are regional, thereby can encode to image with high compression rate.
Another form of the present invention also relates to picture coding device.This device comprises: with image transform is the spatial frequency zone, to generate the transformation component of conversion coefficient; In the ranking of the above-mentioned conversion coefficient in a plurality of zones on corresponding respectively to above-mentioned image, only the low level position is replaced into the replacement section of null value corresponding to each regional relative importance value figure place; After the displacement of above-mentioned replacement section, with the encoding section of carrying out compressed encoding from high-order position successively of ranking of the above-mentioned conversion coefficient of above-mentioned integral image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned conversion coefficient corresponding to a plurality of zones on the above-mentioned image.
According to above-mentioned formation, can make in the picture quality in a plurality of interregional whens regeneration variant.Above-mentioned replacement section when above-mentioned a plurality of interregional when repeat region is arranged, in above-mentioned repeat region, also can decide the low level position that is replaced into null value according to the relative importance value in the high zone of relative importance value.
Another form of the present invention also relates to picture coding device.This device comprises: image is carried out wavelet transformation and generate the transformation component of wavelet conversion coefficient; Corresponding in the ranking of the above-mentioned wavelet conversion coefficient in a part of zone on the above-mentioned image, only the regulation figure place of low level position is replaced into the replacement section of null value; After the displacement of above-mentioned replacement section, with the encoding section of carrying out compressed encoding from high-order bit plane successively of ranking of the above-mentioned wavelet conversion coefficient of above-mentioned integral image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned wavelet conversion coefficient corresponding to the part zone on the above-mentioned image.
Another form of the present invention relates to image decoder.This device comprises: the image behind the compressed encoding is deciphered, to obtain the decoding part of the conversion coefficient in the spatial frequency zone; Corresponding in the ranking of the above-mentioned conversion coefficient in a part of zone on the above-mentioned image, only the regulation figure place of low level position is replaced into the replacement section of null value; After above-mentioned replacement section displacement, the ranking of above-mentioned conversion coefficient of above-mentioned integral image carried out inverse transformation, to go back the inverse transformation portion of original image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned conversion coefficient corresponding to the part zone on the above-mentioned image.
According to this formation, when the decoding coded image, the zone at image place is preferentially deciphered than other zones, can be with this zone of high-quality regeneration.In other words, reduce quality by the low level position information that other are regional, thereby can improve this regional picture quality and reproduced picture relatively.
Another form of the present invention also relates to image decoder.This device comprises: the image that compressed encoding is crossed is deciphered, with the decoding part of the conversion coefficient of obtaining the spatial frequency zone; Corresponding in the ranking of the above-mentioned conversion coefficient in a plurality of zones on the above-mentioned image, only the figure place corresponding to each regional relative importance value of low level position is replaced into the replacement section of null value; After above-mentioned replacement section displacement, the ranking of above-mentioned conversion coefficient of above-mentioned integral image carried out inverse transformation, to go back the inverse transformation portion of original image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned conversion coefficient corresponding to a plurality of zones on the above-mentioned image.
According to this formation, interregionally also can in the reproduced picture quality, there are differences a plurality of.
Another form of the present invention also relates to image decoder.This device comprises: the image behind the compressed encoding is deciphered, to obtain the decoding part of wavelet conversion coefficient; Corresponding in the ranking of the above-mentioned wavelet conversion coefficient in a part of zone on the above-mentioned image, only the regulation figure place of low level position is replaced into the replacement section of null value; After above-mentioned replacement section displacement, the ranking of above-mentioned wavelet conversion coefficient of above-mentioned integral image carried out inverse transformation, to go back the inverse transformation portion of original image.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned wavelet conversion coefficient corresponding to the part zone on the above-mentioned image.
Another form of the present invention relates to method for encoding images.This method is: in the conversion coefficient in the spatial frequency zone of image, and specific above-mentioned conversion coefficient corresponding to the part zone on the above-mentioned image; Corresponding in the ranking of the above-mentioned conversion coefficient in specific above-mentioned part zone, only the regulation figure place with the low level position is replaced into null value; On this basis, gather the ranking of above-mentioned conversion coefficient of above-mentioned integral image, and carry out compressed encoding successively from high-order position.
Another form of the present invention relates to image decoding method.This method is: decipher and in the conversion coefficient in the spatial frequency zone of the above-mentioned image obtained specific above-mentioned conversion coefficient corresponding to the part zone on the above-mentioned image at the image that compressed encoding is crossed; Corresponding in the ranking of the above-mentioned conversion coefficient in specific above-mentioned part zone, only the regulation figure place with the low level position is replaced into null value, on this basis, the ranking of above-mentioned conversion coefficient of above-mentioned integral image is carried out inverse transformation and is gone back original image.
Another form of the present invention also relates to method for encoding images.This method is: in the conversion coefficient in the spatial frequency zone of image, and specific above-mentioned conversion coefficient corresponding to a plurality of zones on the above-mentioned image; Corresponding in the ranking of the above-mentioned conversion coefficient in specific above-mentioned a plurality of zones, only the figure place corresponding to each regional relative importance value with the low level position is replaced into null value, the ranking and carry out compressed encoding successively of above-mentioned conversion coefficient of gathering on this basis, above-mentioned integral image from high-order position.
The present invention also has a kind of form also to relate to image decoding method.This method is: decipher and in the conversion coefficient in the spatial frequency zone of the above-mentioned image obtained specific above-mentioned conversion coefficient corresponding to a plurality of zones on the above-mentioned image at the image that compressed encoding is crossed; Corresponding in the ranking of the above-mentioned conversion coefficient in specific above-mentioned a plurality of zones, only the figure place corresponding to each regional relative importance value with the low level position is replaced into null value, on this basis, the ranking of above-mentioned conversion coefficient of above-mentioned integral image carried out inverse transformation and gone back original image.
The present invention also has another form to relate to image display device.This device comprises display part, and by a plurality of zones that picture is set, the dynamic images that picture quality is different are presented on the described picture.
May further include above-mentioned relatively picture appointment and gaze at zone and non-specifying part of gazing at least one side in zone, gaze at zone and the above-mentioned non-appointment of gazing at least one side in zone by above-mentioned, thereby above-mentioned relatively picture is set a plurality of zones.For example, specify one by relative picture and gaze at the zone, thereby picture is set 2 zones gazing at zone and zone in addition relatively.And, specify the non-zone of gazing at by relative picture, thereby picture is set non-2 zones gazing at zone and zone in addition relatively.Gaze at the zone or the non-zone of gazing at also can be designated a plurality of, and, can by combination gaze at the zone and the non-zone of gazing at set, set and gaze at zone, non-a plurality of zones of gazing at zone, zone in addition.
At this, so-called " making the display part that is presented on the picture " is meant: comprise picture such as display at the same time and supply to outside the form of display control unit of output control of picture signal of picture, also comprise and have the interface that is connected with display, do not contain display and have only the form of display control unit.This any form wherein all is included in this technical scope.
The present invention also has another form also to relate to image display device.This device comprises display part, has the different a plurality of zones of picture quality, will have overlapping dynamic image to be presented on the picture with these zones.
The present invention also has another form also to relate to image display device.This device comprises display part, and the dynamic image that will have the different a plurality of zones of maintenance visuality and picture quality is presented on the picture.At this, so-called " keeping visuality and picture quality difference " is to adjust picture quality in can be used as the reference image identification of object, is included in the scope that can hold object, reduces picture quality, perhaps opposite, make and gaze at the zone than other zone raising picture quality.
Another form of the present invention also relates to image display device.This device comprises: show the picture of dynamic image relatively, select to gaze at zone and non-selection portion of gazing at least one side in zone; Above-mentionedly gaze at the zone and above-mentionedly non-ly gaze at regional at least one side by selecting, thereby make different above-mentioned dynamic image in a plurality of zones that picture quality sets at each above-mentioned relatively picture be presented at display part on the above-mentioned picture.Constitute according to this, can change the picture quality in a plurality of zones and make its demonstration by wish.
At this, the user can select a plurality of zones by relative picture appointed area.Moreover a plurality of zones can also be selected by the zone that exists with certain method extraction special object thing.For example, by image-recognizing method, can be with the zone of literal, with personage's zone and other are regional as a plurality of extracted region.Moreover the central part and the ad-hoc locations such as periphery, peripheral part thereof that can preestablish picture are preestablished, and this position that is set is selected as a plurality of zones.
Above-mentioned selection portion is set relative importance value to above-mentioned each zone, and above-mentioned display part also can show above-mentioned a plurality of zones with the picture quality corresponding to relative importance value.Above-mentioned a plurality of interregional when repetition is arranged, also can in the part that repeats, determine picture quality according to the high side of relative importance value.Relative importance value can be specified during to the picture appointed area the user.And, improve the relative importance value of picture central part or improve with the relative importance value in the zone of literal etc., can pre-determine relative importance value.
The present invention also has another form also to relate to image display device.This device comprises: the picture and selecting that shows dynamic image is relatively gazed at regional selection portion; With do not selecting under the above-mentioned state of gazing at the zone, show whole image with the picture quality of regulation, selecting under the above-mentioned state of gazing at the zone, make this picture quality of gazing at the zone than other regional picture qualities display part of showing of highland also.According to this formation, when the dynamic image regular regeneration, for example carry out simple and easy regeneration with low image quality, under the situation of having selected to gaze at the zone, can improve the picture quality that this gazes at the zone, to regenerate.
Above-mentioned display part is being selected a plurality of above-mentionedly when gazing at the zone, can make each above-mentioned gazing in the zone of picture quality differently to show.Above-mentioned selection portion can be selected the zone that life period changes as the above-mentioned zone of gazing at.When the zone that life period changes has when a plurality of, display part can change picture quality according to the degree that the time changes.
Another form of the present invention relates to method for displaying image.This method is: by a plurality of zones that picture is set, allow the picture quality difference, and dynamic image is presented on the described picture.This dynamic image can be read from storage device, also can import by network.
Other forms of the present invention relate to picture coding device.This device comprises: with image transform is spatial frequency zone and the transformation component that generates conversion coefficient; The above-mentioned conversion coefficient that corresponds respectively to above-mentioned a plurality of zones is ranked, and a yardstick amplifies the yardstick enlarging section corresponding to the figure place of each regional relative importance value; Undertaken after yardstick amplifies by above-mentioned yardstick enlarging section, the above-mentioned conversion coefficient of above-mentioned integral image is carried out successively the encoding section of compressed encoding from high-order position.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned conversion coefficient corresponding to a plurality of zones on the above-mentioned image.
Another form of the present invention also relates to picture coding device.This device comprises: image is carried out wavelet transformation and generate the transformation component of wavelet conversion coefficient; To correspond respectively to the ranking of above-mentioned wavelet conversion coefficient in above-mentioned a plurality of zones, a yardstick amplifies the yardstick enlarging section corresponding to the figure place of each regional relative importance value; Undertaken after yardstick amplifies by above-mentioned yardstick enlarging section, the above-mentioned wavelet conversion coefficient of above-mentioned integral image has been carried out successively the encoding section of compressed encoding from high-order bit plane.Can also comprise: the mask generating unit that generates the mask be used for specific above-mentioned wavelet conversion coefficient corresponding to a plurality of zones on the above-mentioned image.
A form more of the present invention relates to method for encoding images.This method is: in the wavelet conversion coefficient of image, specific above-mentioned wavelet conversion coefficient corresponding to a plurality of zones on the above-mentioned image, will ranking corresponding to the above-mentioned wavelet conversion coefficient in specific above-mentioned a plurality of zones, yardstick amplifies the figure place corresponding to each regional relative importance value, on this basis, the above-mentioned wavelet conversion coefficient that gathers above-mentioned integral image ranks and carries out compressed encoding successively from high-order bit plane.
Another form of the present invention also relates to picture coding device.This device comprises: the regional selection portion of selecting the different a plurality of zones of relative importance value on image; Relative importance value separately according to above-mentioned a plurality of zones carries out compressed encoding to above-mentioned image, comprises the encoding section of the coded image in the different a plurality of zones of picture quality with generation.Also comprise: with image transform is the spatial frequency zone, to generate the transformation component of conversion coefficient; Generation is used for the mask generating unit of the mask of specific above-mentioned conversion coefficient corresponding to above-mentioned a plurality of zones of being selected by the above-mentioned zone selection portion, and above-mentioned encoding section can be carried out compressed encoding to the above-mentioned image that is provided with above-mentioned a plurality of zones with reference to above-mentioned mask.
Other forms of the present invention relate to image decoder.This device comprises: the regional selection portion of selecting the different a plurality of zones of relative importance value on image; Relative importance value separately with according to above-mentioned a plurality of zones carries out inverse transformation to the image behind the above-mentioned compressed encoding, comprises the inverse transformation portion of the different a plurality of area images of picture quality with reduction.Also comprise: the image behind the compressed encoding is deciphered, with the decoding part of the conversion coefficient of obtaining the spatial frequency zone; Be used for the mask generating unit of the mask of specific above-mentioned conversion coefficient corresponding to above-mentioned a plurality of zones of selecting by the above-mentioned zone selection portion with generation, above-mentioned inverse transformation portion can carry out inverse transformation to the image behind the above-mentioned compressed encoding that is provided with above-mentioned a plurality of zones with reference to above-mentioned mask.
Another form of the present invention relates to image processing apparatus.This device is deciphered dynamic image data, shows so that the picture quality in a plurality of zones on the picture has difference ground, adjusts the picture quality in above-mentioned a plurality of zones respectively, so that the treating capacity that whole decoding is handled is no more than setting.According to this form, can make the picture quality difference in a plurality of zones on the picture, and can the treating capacity when showing dynamic image be suppressed at below the steady state value.
Another form of the present invention also relates to image processing apparatus.This device has: picture is specified the regional specifying part of gazing at the zone relatively; And on the picture quality of gazing between zone and the common zone beyond it, have difference ground decoding image, with the decoding part of output dynamic image data.At this, so-called " gazing at the zone " is meant that the user wants to use indicating device input units such as (pointing device) to specify by improving the zone that resolution is regenerated in image.According to this form, can be with desirable picture quality only to regenerating by the zone of gazing at of user's appointment.
Decoding part can be adjusted above-mentioned picture quality of gazing at zone and above-mentioned common zone respectively, so that the treating capacity that whole decoding is handled is no more than the threshold value of regulation.Perhaps, decoding part improves above-mentioned picture quality of gazing at the zone, and the picture quality in above-mentioned common zone is reduced.In view of the above, do not increase the treating capacity of image processing apparatus, promptly can high image quality the regeneration user be concerned about certain gaze at the zone.
The zone specifying part also can comprise: when above-mentioned picture quality of gazing at the zone is improved, judge to decipher and handle the judging part whether whole treating capacity surpasses defined threshold; With whether decision allows to improve above-mentioned picture quality instruction unit of gazing at the picture quality in zone according to judged result.In view of the above, increase the decoding treating capacity in desire, when improving the disposal ability of image processing apparatus,, can avoid treating capacity to produce because of the coma that the upper limit disposal ability that improves image processing apparatus causes owing to do not allow to improve the picture quality of gazing at the zone.
Replace it, regional specifying part also can comprise: make the above-mentioned picture quality instruction unit that the area image quality improves of gazing at; With by improving above-mentioned picture quality of gazing at the zone, judge with the relevant treating capacity of decoding whether above the detection unit of defined threshold.In this case, above-mentioned picture quality instruction unit is in the result of determination according to detection unit, distinguishes when surpassing threshold value for above-mentioned treating capacity, and the picture quality in common zone is reduced.In view of the above, along with increase is gazed at the encoding process amount in zone and improved picture quality, by reducing the encoding process amount in zone usually, be below the upper limit disposal ability like this thereby can make the treating capacity of image processing apparatus integral body.
The picture quality instruction unit also can be sent the warning demonstration of the picture quality reduction in common zone to the user when distinguishing that above-mentioned treating capacity surpasses threshold value.So the user can understand the situation that common zone becomes low image quality in advance.
The picture quality instruction unit also can send whether carry out the above-mentioned inquiry demonstration that the area image quality improves of gazing to the user when distinguishing that above-mentioned treating capacity surpasses threshold value.In view of the above, the user can judge whether specially to carry out the picture quality of gazing at the zone and improve according to significance level.
And above-mentioned coded image data can be re-used according to resolution.In an example, coded image data is according to the data that transmit continuously by the Motion-JPEG2000 of the compressed image of every frame.In view of the above, because coded image data can be re-used according to resolution, thus be easy to extract the intermediate image that generates in the decode procedure, with image as low resolution.
In addition, the form of the combination in any of above inscape, conversion performance of the present invention between method, device, system, computer program, recording medium etc. also is effective as mode of the present invention.
Description of drawings
Fig. 1 is the pie graph of existing picture coding device 100.
Fig. 2 is the figure of the wavelet conversion coefficient of the explanation original image situation of being amplified by yardstick.
Fig. 3 is the pie graph of existing image decoder.
Fig. 4 is the figure of the situation of being dwindled by yardstick of the wavelet conversion coefficient after the explanation coded image decoding.
Fig. 5 is the pie graph of the picture coding device that relates to of execution mode 1.
Fig. 6 is the figure of mask that explanation is used for the specific wavelet conversion coefficient of gazing at the zone corresponding to original image.
Fig. 7 is that the low level position of wavelet conversion coefficient of explanation original image is by the figure of the situation of zero displacement.
Fig. 8 be explanation do not exist on the original image gaze under the regional situation, the low level position of wavelet conversion coefficient is by the figure of the situation of zero displacement.
Fig. 9 is the pie graph of the image decoder that relates to of execution mode 1.
Figure 10 is the pie graph of the image decoder that relates to of execution mode 2.
Figure 11 is the figure of the zero situation of replacing of low level position quilt of the wavelet conversion coefficient after the decoding of explanation coded image.
Figure 12 is the pie graph of the picture coding device that relates to of execution mode 3.
Figure 13 is that the figure that a plurality of relative importance values when gazing at the zone are set example is set in explanation in original image.
Figure 14 is that the low level position of wavelet conversion coefficient of explanation original image is by the figure of the situation of zero displacement.
Figure 15 is a plurality of figure that have the example that relative importance value is set when repeating in the zone that gaze at that explanation is provided with on original image.
Figure 16 is that explanation a plurality of on being set in original image gaze at and exist overlappingly in the zone, comprises the figure that relative importance value under the non-situation of gazing at the zone is set example in gazing at the zone.
Figure 17 be explanation in the example of Figure 16, the low level position of the wavelet conversion coefficient of original image is by the figure of the situation of zero displacement.
Figure 18 is the pie graph of the image decoder that relates to of execution mode 4.
Figure 19 is the figure of the zero situation of replacing of low level position quilt of the wavelet conversion coefficient after the decoding of explanation coded image.
Figure 20 is the pie graph of the picture coding device that relates to of execution mode 5.
Figure 21 is the figure of the wavelet conversion coefficient of the explanation original image situation of being amplified by yardstick.
Figure 22 is the pie graph of the image decoder that relates to of execution mode 5.
Figure 23 is the figure of the situation of being amplified by yardstick of the wavelet conversion coefficient after the explanation coded image decoding.
Figure 24 is the pie graph of the image display device that relates to of execution mode 6.
Figure 25 is the figure of presentation video encoding process order.
Figure 26 is the pie graph of the image processing apparatus that relates to of execution mode 7.
Figure 27 is the figure of presentation video decoding processing sequence.
Figure 28 is the figure that the frame processing of carrying out according to image processing apparatus is described.
Figure 29 is the flow chart of the processing of explanation memory controller.
Figure 30 is the pie graph of the image display device that relates to of execution mode 8.
Figure 31 is the pie graph of the image processing apparatus that relates to of execution mode 9.
Figure 32 (a)～(c) is the figure of mask that explanation is used for the specific wavelet conversion coefficient of gazing at the zone corresponding to original image.
Figure 33 (a) and (b) are figure of the zero situation of replacing of low level position quilt of explanation wavelet conversion coefficient.
Figure 34 (a)～(c) is the wavelet conversion coefficient when gazing at the zone is specified in explanation in original image figure.
Figure 35 is the flow chart of the processing of explanation detection unit.
Figure 36 (a) and (b) are figure that expression improves the situation of the picture quality of gazing at the zone.
Figure 37 (a)～(c) is that explanation is specified in original image and gazed under the big situation of zone and treating capacity, and the low level position of wavelet conversion coefficient is by the figure of the situation of zero displacement.
Figure 38 is the flow chart of another embodiment of explanation detection unit processing.
Figure 39 (a) and (b) are that expression improves the figure that gazes at the area image quality and reduce the situation of common area image quality.
Figure 40 (a) and (b) are that the figure that gazes at the area image mass conservation and reduce the situation of common area image quality is kept in expression.
Figure 41 is the pie graph of the image display device that relates to of execution mode 10.
Figure 42 is the pie graph of the image display system that relates to of execution mode 11.
Embodiment
(execution mode 1)
Fig. 5 is the pie graph of the picture coding device 200 that relates to of execution mode 1.The formation of picture coding device 200, on hardware, can realize by CPU, memory, other LSI of any computer, on software, can wait and realize, and, describe the functional block that realizes by above-mentioned cooperation at this by the program in the memory of being loaded in encoding function.Therefore, these functional blocks can be only by hardware, only realize that with various forms this point those skilled in the art is understandable by software or these combination.
Picture coding device 200 with the input original image as an example, carry out compressed encoding by the JPEG2000 mode.The original image of input picture code device 200 also can be the frame of dynamic image.Picture coding device 200 can be encoded to each frame of dynamic image continuously in the JPEG2000 mode, generates the encoding stream of dynamic image.
The original image of 10 pairs of inputs of wavelet transformation portion carries out subband to be cut apart, and calculates the wavelet conversion coefficient of each several part wave band (subband) image, generates by the wavelet conversion coefficient after the classification.
Wavelet transformation portion 10 is divided into 4 frequency subbands and carries out wavelet transformation at x, y all directions application of low-pass filters and the high pass filter of original image.These subbands are: the LL subband that has low-frequency component at x, y both direction; In any one direction of x, y the low frequency composition is arranged, and on the opposing party's direction, have the HL and the LH subband of radio-frequency component; With the HH subband that on x, y both direction, has radio-frequency component.The pixel count in length and breadth of each several part wave band is respectively to handle 1/2 of preceding image, and by a filtering, resolution is that picture size obtains 1/4 subband image.
LL subband in 10 pairs of subbands that so obtain of wavelet transformation portion is carried out Filtering Processing once more, and it further is divided into LL, HL, LH, a HH4 subband and carries out wavelet transformation.Wavelet transformation portion 10 number of times in accordance with regulations carries out this filtering, and original image is classified as the subband image, the wavelet conversion coefficient of output each several part wave band.Quantization portion 12 quantization amplitude is according to the rules carried out quantization to the wavelet conversion coefficient from 10 outputs of wavelet transformation portion.
ROI selection portion 18 is selected the zone of gazing on the original image, and the ROI positional information of expression being gazed at regional location offers ROI mask generating unit 20.When gazing at the zone with rectangular selection, the ROI positional information is provided by the pixel coordinate value in the upper left corner of rectangular area and the pixel in length and breadth of rectangular area.
Gazing at the zone can specify the specific region on the original image to select by the user, also can select predetermined zones such as original image central area.Moreover, the important areas such as zone that show personage or literal can be extracted automatically as gazing at the zone.When the frame of dynamic image is input to picture coding device 200 continuously, also can passes through the action of the specific region on the tracing figure picture frame, and automatically select to gaze at the zone.
ROI mask generating unit 20 is based on the ROI positional information, and it is the ROI mask of ROI conversion coefficient corresponding to the wavelet conversion coefficient of gazing at the zone that generation is used for specific.
Fig. 6 (a)～(c) is the figure of explanation by the ROI mask of ROI mask generating unit 20 generations.Shown in Fig. 6 (a), on original image 80, select to gaze at zone 90 by ROI selection portion 18.ROI mask generating unit 20 is selected on the specific reduction original image 80 in the each several part wave band gazes at the required wavelet conversion coefficients in zone 90.
Fig. 6 (b) expression is by carrying out the 1st grade of changing image 82 that 1 wavelet transformation obtains to 80 of original images.The 1st grade of changing image 82 is made of 4 subband LL1, HL1, LH1, HH1 of the 1st level.ROI mask generating unit 20 wavelet transformation of gazing at zone 90 necessary the 1st grade of changing images 82 of specific reduction original image 80 in the each several part wave band LL1 of the 1st level, HL1, LH1, HH1 is a ROI conversion coefficient 91～94.
Fig. 6 (c) expression is further carried out the 2nd grade of changing image 84 that wavelet transformation obtains by the subband LL1 to the low-limit frequency composition of the changing image 82 of Fig. 6 (b).The 2nd grade of changing image 84 be as shown in the figure, except that 3 subband HL1, LH1 of the 1st level, HH1, also comprises 4 subband LL2, HL2, LH2, the HH2 of the 2nd level.ROI mask generating unit 20 is a ROI conversion coefficient 95～98 at the wavelet conversion coefficient of ROI conversion coefficient 91 necessary the 2nd grade of changing images 84 of the subband LL1 of the 2nd layer each several part wave band LL2, HL2, LH2, the 1st grade of changing image 82 of the specific reduction of HH2.
Equally, by only the number of times with wavelet transformation is specific for the ROI conversion coefficient of gazing at zone 90 at different levels, thereby in the changing image of final level, can whole specific reduction gaze at regional 90 required ROI conversion coefficients.ROI mask generating unit 20 is created on the ROI mask that specific finally specific ROI conversion coefficient position is used on this changing image of final grade.For example, when only carrying out 2 wavelet transformations, but the ROI mask of the position of 7 ROI conversion coefficients 92～98 shown in the dotted line in the generation specific pattern 6 (c).
The low level position of Fig. 5 zero replacement section, 24 bases are with respect to the non-relative priority degree of gazing at the zone of gazing at the zone, in corresponding to non-the ranking of above-mentioned wavelet conversion coefficient of gazing at the zone, adjustment is replaced into the lower bit number of null value, with reference to the ROI mask that generates by ROI mask generating unit 20, carry out in the ranking of non-ROI conversion coefficient of mask can't help the ROI mask, only will be changed to zero from the S position that least significant bits begin number.At this, zero displacement figure place S is with suitable with respect to the non-relative priority degree of gazing at the zone of gazing at the zone, is to be any natural number of the upper limit with the non-maximum number of digits of gazing at the quantization value in zone.Should zeroly replace figure place S by changing, thereby can adjust the non-degradation of gazing at the reproduced picture quality in zone that to gaze at the zone continuously.
Fig. 7 (a)～(c) is the figure of explanation by the situation of the low level position of the wavelet conversion coefficient 60 of low level position zero replacement section, 24 zero displacement original images.Fig. 7 (a) expression comprises 5 bit planes by the wavelet conversion coefficient 60 after quantization portion 12 quantizations, and the ROI conversion coefficient is represented with oblique line.
Shown in Fig. 7 (b), low level position zero replacement section 24 will be can't help the S position that the ROI mask carries out the non-ROI conversion coefficient LSB side of mask and will be changed to zero.In this embodiment, S=2, as shown in symbol 64,2 that can obtain non-ROI conversion coefficient LSB side are replaced into zero wavelet conversion coefficient 62.
The entropy coding portion 14 of Fig. 5 is shown in the arrow of Fig. 7 (c), and one side scans comprising by zero wavelet conversion coefficient 62 that is replaced into the non-ROI conversion coefficient of ROI conversion coefficient successively from high-order bit plane, and one side averages the amount of information coding.
Fig. 8 (a)～(c) be explanation do not exist on the original image gaze under the regional situation, the low level position of wavelet conversion coefficient is by the figure of the situation of zero displacement.Fig. 8 (a) expression owing to do not set on original image gazes at the zone, so the wavelet conversion coefficient 70 of 5 bit planes that form by non-ROI conversion coefficient only.Low level position zero replacement section 24 is 2 o'clock at zero displacement figure place S, and shown in Fig. 8 (b), low level 2 bit planes that generate the LSB side in 5 bit planes are replaced into zero wavelet conversion coefficient 72.
Entropy coding portion 14 averages the amount of information coding to a high position 3 bit planes of the wavelet conversion coefficient 72 after zero displacement successively from last beginning shown in Fig. 8 (c).At this moment, do not encoded by low level 2 bit planes after zero displacement.And, replace low level 2 bit planes are carried out zero displacement, also can simply abandon low level 2 bit planes.
Coded data generating unit 16 will be carried out fluidisation with coding parameters such as quantization amplitudes by the data behind the entropy coding, to export as coded image.
Generally, because the restriction of memory capacity or transfer rate etc., and the size of data of final coded image is set with under the situation of the upper limit, entropy coding portion 14 from high-order bit plane successively when being encoded by quantized wavelet conversion coefficient, finish coding by bit plane should observing in the process of the size of data upper limit sometimes.Perhaps, when coded data generating unit 16 output when high bit plane carries out the coded data of fluidisation successively, in the process that should observe the transfer rate restriction, finish stream output sometimes by bit plane.
Like this, even the size of data at coded image exists under the situation of restriction, in the present embodiment, because in the low level bit plane, by zero displacement, only become coded object as significant information, corresponding to the non-wavelet conversion coefficient of gazing at the zone so the compression ratio height of low level bit plane corresponding to the wavelet conversion coefficient of gazing at the zone, even be encoded to the least significant bits plane, size of data can not increase yet.
Fig. 9 is the pie graph of the image decoder 210 that relates to of execution mode 1.The picture coding device 200 that is related to by execution mode 1 and to make ROI be not carry out the common coded image that the yardstick of ROI conversion coefficient amplifies by the image of priority encoding is so image decoder 210 can be deciphered coded image simply by the decoded mode of common JPEG2000.
Coded data extraction unit 30 is extracted coded data from the coded image of input.As an example, shown in Fig. 7 (c), to 5 bit planes from the last coded data that obtains successively.At this, should note: coded data extraction unit 30 does not need to provide the ROI that gazes at regional location positional information especially.Average information decoding part 32 is to each bit plane decoding coded data.In the example of Fig. 7 (c), 5 bit planes are decoded, and the quantization data are stored in the memory.
The quantization data of 34 pairs of decodings of inverse guantization (IQ) portion are carried out inverse guantization (IQ), and the wavelet conversion coefficient after 36 pairs of inverse guantization (IQ) of small echo inverse guantization (IQ) portion carries out inverse transformation, and export resulting decoding image.
As above illustrated, the picture coding device 200 of present embodiment is replaced by non-ROI conversion coefficient being carried out zero, thereby improves the relative importance value of ROI conversion coefficient relatively, and preferentially encodes to gazing at the zone.Because do not carry out the yardstick processing and amplifying of ROI conversion coefficient, so the computing of can efficiently encoding.And, because the number of bit-planes that should encode does not increase,, can reduce hardware cost so do not need to be provided with the storage area surplus.
Moreover, because the yardstick when not needing to decipher dwindles processing, so in coded data, do not need additional ROI positional information and yardstick amplification quantity.And, as broad as long on form by the picture coding device 200 of present embodiment by ROI image encoded and common coded image, so can decipher to handle identical processing, and can keep deciphering the interchangeability of processing with the decoding of common coded image.
(execution mode 2)
Figure 10 is the pie graph of the image decoder 220 that relates to of execution mode 2.In the present embodiment, the coded image that is input to image decoder 220 is the common coded image that does not carry out the ROI coding.The image decoder 220 of present embodiment is specified when decoding and is gazed at the zone, preferentially deciphers gazing at the zone.
The coded image of input picture code translator 220 also can be the coded frame of dynamic image.Each coded frame of the dynamic image of importing as encoding stream by continuous decoding, thus can regenerate dynamic image.
Coded data extraction unit 30 is extracted coded data from the coded image of input, average information decoding part 32 is in each bit plane decoding coded data, and the quantized wavelet conversion coefficient that decode results is obtained stores in the memory.
ROI selection portion 18 is selected the zone of gazing on the image, and the ROI positional information of expression being gazed at regional location offers ROI mask generating unit.Gazing at the zone can select by the picture positions such as central area of specify image, also can show the important area such as zone of personage and literal or user by automatic extraction and specify and select.When the coded frame of dynamic image is input to image decoder 220 continuously, also can picture frame after the decoding of image decoder 220 output, specify or extract and gaze at the zone, the zone of gazing at of specifying or extract is selected as the zone of gazing at of coded frame.
ROI mask generating unit 20 is based on the ROI positional information, and it is the ROI mask of ROI conversion coefficient corresponding to the wavelet conversion coefficient of gazing at the zone that generation is used for specific.Low level position zero replacement section, 24 bases are with respect to the non-relative priority degree of gazing at the zone of gazing at the zone, adjustment is replaced into the lower bit number of null value in corresponding to non-the ranking of above-mentioned wavelet conversion coefficient of gazing at the zone, with reference to the ROI mask, carry out in wavelet conversion coefficient, the LSB side of non-ROI conversion coefficient is replaced into regulation figure place part zero processing by average information decoding part 32 decoding.
The situation of low level position zero displacement of wavelet conversion coefficient after the decoding of coded image is carried out in Figure 11 (a)～(c) expression by low level position zero replacement section 24.Figure 11 (a) is the wavelet conversion coefficient 74 that averages the image after amount of information is deciphered, and comprises 5 bit planes.In Figure 11 (b), represent corresponding to the ROI conversion coefficient of gazing at the zone by 18 appointments of ROI selection portion by oblique line.Low level position zero replacement section 24 is shown in Figure 11 (c), and the low order position that generates non-ROI conversion coefficient is changed to zero wavelet conversion coefficient 76.
The wavelet conversion coefficient that inverse guantization (IQ) portion 34 will comprise the non-ROI conversion coefficient after ROI conversion coefficient and zero displacement of low level position quilt carries out inverse guantization (IQ), the wavelet conversion coefficient of wavelet inverse transformation portion 36 inverse transformations after by inverse guantization (IQ), and export resulting decoding image.
And ROI selection portion 18 replaces selecting to gaze at the zone, also can select the non-zone of gazing at.For example, in the time will importing iridescence, this zone is chosen as gazes at the zone in the zone of the personal information such as number slip that show the facial and car of personage.At this moment, ROI mask generating unit 20 can generate the mask of specific non-ROI conversion coefficient with the mask counter-rotating of specific non-ROI conversion coefficient.Perhaps, ROI mask generating unit 20 also can offer the mask of specific non-ROI conversion coefficient low level position zero replacement section 24.
When the coded frame of dynamic image is input to image decoder 220 continuously, can make image decoder 220 carry out following action.Generally in order to reduce to handle load, suitably the low level bit plane of discarded wavelet conversion coefficient is to carry out simple and easy regeneration for image decoder 220.In view of the above, even exist under the situation of restriction at the handling property of image decoder 220, because discarded low level position, so can be for example to carry out simple and easy regeneration 30 frame/seconds.
In simple and easy regeneration, when gazing on the image regional when selected, the wavelet conversion coefficient of zero state of replacing has been carried out in the low level position that 220 pairs of image decoders will non-ly be gazed at the zone by low level position zero replacement section 24, deciphers the bit plane of lowest order, with reproduced picture.At this moment, uprise because handle load, so though also sometimes coma reduce to state or the delay reproduced state of 15 frame/seconds etc., also can regenerate and gaze at the zone with high image quality.
Like this, when selecting to gaze at the zone, the non-zone of gazing at can keep and the quality of simple and easy regeneration with degree, only regenerates to gazing at the zone with higher quality.As monitoring picture, do not require high-qualityly usually, have only in the time of need gazing at the position with high-quality regeneration when unusual useful.In addition, with the portable terminal reproduced picture time, also can use following method: under energy-saving mode,, only regenerate with high-quality as required to gazing at the zone with low-quality regeneration dynamic image from the battery life angle.
Image decoder 220 according to present embodiment, for the common coded image that does not carry out the ROI coding, by carrying out zero displacement corresponding to the non-low level position of gazing at the wavelet conversion coefficient in zone, thereby can relatively the picture quality of gazing at the zone be brought up to than the non-also height of zone of gazing at, to decipher.Owing to carry out integral image regeneration operand increased with high-quality, but only preferentially decipher in the present embodiment, so can suppress the increase of operand to gazing at the zone.
(execution mode 3)
Figure 12 is picture coding device 200 pie graphs that execution mode 3 relates to.In the picture coding device 200 that execution mode 1 relates to, only select one to gaze at the zone, but the picture coding device 200 of present embodiment is selected the different a plurality of zones of relative importance value in image, according to each regional relative importance value image is carried out compressed encoding, generate the coded image that comprises the different a plurality of zones of picture quality.Give same-sign for the formation identical, formation and the action inequality with execution mode 1 are described with execution mode 1.
A plurality of zones of gazing at that ROI selection portion 18 is selected on the original image, the ROI positional information of expression respectively being gazed at regional location offers ROI mask generating unit 20.Both can a plurality of gaze at the zone exist overlapping, also can gaze at the zone inside comprise the non-zone of gazing at.ROI mask generating unit 20 is based on the ROI positional information, and it is the ROI mask of ROI conversion coefficient corresponding to the wavelet conversion coefficient of respectively gazing at the zone that generation is used for specific.
Specific processing at the ROI conversion coefficient described in the execution mode 1 is carried out in 20 pairs of each a plurality of zones of gazing at of being selected by ROI selection portion 18 of ROI mask generating unit, relatively respectively gazes at the zone and generates the ROI mask.
Interregional setting relative importance value is gazed at a plurality of in ROI relative importance value configuration part 19.For example, as a plurality of zones of gazing at, select picture centre portion and central part periphery, outer peripheral portion in addition is as non-when gazing at the zone, the relative importance value of the central part of image is set at height, so that regenerate with high image quality; The relative importance value of the periphery of central part is set at low so that regenerate with the standard picture quality.As another example, as a plurality of zones of gazing at, when selecting to show the zone of literal and show personage's face regional, character area is set at relative importance value the highest in the mode that reaches high image quality, facial zone is set at relative importance value in the mode that reaches high image quality and is only second to it, and zone in addition is made as the non-zone of gazing in the mode that reaches the standard picture quality.With the purpose of protection privacy, the zone that shows personage's face can be set at low relative importance value in the mode that reaches low image quality, or is set at the non-zone of gazing at.
Figure 13 is that the figure that a plurality of relative importance values when gazing at the zone are set example is set in explanation in original image 80.As shown in the drawing, in original image 80, set 2 and gazed at the zone at 81,83 o'clock, ROI relative importance value configuration part 19 for example according to: the 1st gazes at that zone 83 (below be made as ROI2) is gazed in zone 81 (below be made as ROI1), the 2nd, in addition non-order of gazing at zone (hereinafter referred to as non-ROI) is set priority, so that reduce relative importance value.
ROI relative importance value configuration part 19 determines according to the relative importance value of setting: be the low level position S0 that carries out zero displacement in the ranking of non-ROI conversion coefficient corresponding to the non-small echo variation coefficient of gazing at the zone; With corresponding to a plurality of each wavelet conversion coefficients of gazing at the zone be the lower bit number Si that carries out zero displacement in the ranking of ROI conversion coefficient (i=1 ..., N; N gazes at number of regions).
In the example of Figure 13, ROI relative importance value configuration part 19 at the 1st preferential region R OI1 that gazes at, is set at 0 with zero displacement figure place S1 for example when the wavelet conversion coefficient of original image is formed by 7 bit planes; At the 2nd preferential region R OI that gazes at, 2 are set at 2 with zero displacement figure place S2; At the non-zone of gazing at, zero displacement figure place S0 is set at 4.Be that relative importance value is low more, it is big more that zero displacement figure place is got.
Zero replacement section, 24 references of the low level position of Figure 12 are with respect to the ROI mask of respectively gazing at the zone that is generated by ROI mask generating unit 20, carry out in the ranking of non-ROI conversion coefficient of mask can't help the ROI mask, only will carry out zero displacement from the S0 position that least significant bits begins to count, simultaneously in the ranking of the ROI conversion coefficient that carries out mask by the ROI mask, only will carry out zero displacement from the Si position that least significant bits begins number.
At this, zero permutated bits Si in the non-ROI conversion coefficient zero displacement figure place S0, the ROI conversion coefficient regional and is a plurality ofly gazed at interregional relative priority degree and decides according to non-gazing at by ROI relative importance value configuration part 19 respectively, be that number of bit-planes with wavelet conversion coefficient is any natural number of the upper limit, satisfy S0＞Si.ROI relative importance value configuration part 19 should zero permutated bits S0, Si by changing, thereby can adjust continuously with respect to the non-degradation of gazing at the reproduced picture quality in zone of gazing at the zone, and a plurality of qualities of gazing at interregional reproduced picture quality.A plurality ofly gaze at the picture quality ground that the zone can keep object can discern degree and regenerate.
Figure 14 (a)～(c) is the situation of zero displacement is carried out in the low level position of the wavelet conversion coefficient 60 of explanation original image by low level position zero replacement section 24 figure.Figure 14 (a) expression comprises 7 bit planes by the wavelet conversion coefficient 60 after quantization portion 12 quantizations, represents the ROI conversion coefficient with oblique line.This figure is in the example of 2 that comprise Fig. 13 original images 80 of gazing at region R OI1, ROI2, and diagram is ranked corresponding to the wavelet conversion coefficient of pixel on the P1-P2 line.
Shown in Figure 14 (b), low level position zero replacement section 24 will be can't help the S0 position of LSB side that the ROI mask carries out the non-ROI conversion coefficient of mask and will be changed to zero.In this embodiment, S0=4, as shown in symbol 64,4 of the LSB side of non-ROI conversion coefficient are replaced into zero.And low level position zero replacement section 24 will be changed to zero by the Si position of LSB side that the ROI mask has carried out the ROI conversion coefficient of mask.In this embodiment, be set with 2 and gaze at region R OI1, ROI2, separately zero displacement figure place S1, S2 are: S1=0, S2=2 shown in symbol 66, are replaced into zero corresponding to 2 of the ROI conversion coefficient LSB side of ROI2.Like this, can obtain carrying out wavelet conversion coefficient 62 after zero displacement by low level position zero replacement section 24.
The wavelet conversion coefficient 62 that comprises the non-ROI conversion coefficient after ROI conversion coefficient and zero is replaced is scanned successively from high-order bit plane in the entropy coding portion 14 of Figure 12 shown in the arrow of Figure 14 (c), average the amount of information coding simultaneously.
And, not existing on the original image under the situation of gazing at the zone, the processing of the low level position integral body of wavelet conversion coefficient being carried out zero displacement and coding is identical with execution mode 1.
Coded data generating unit 16 will be carried out the data of entropy coding and be carried out fluidisation with coding parameters such as quantization amplitudes, to export as coded image.This coded image comprises the different a plurality of zones of when regeneration picture quality, offers storage device or network etc. by efferent, is deciphered and is regenerated on picture by the image display device that comprises decoding mechanism.
As it is illustrated to implement mode 1, according to restriction of memory capacity and transfer rate etc. and final coded image data size is set with under the situation of the upper limit, entropy coding portion 14 is when high-order bit plane is encoded successively, sometimes end coding at bit plane, or in coded data generating unit 16 output when high bit plane carries out the coded data of fluidisation successively, fluid stopping output in the bit plane centre.
Like this, even the size of data at coded image exists under the situation of restriction, in the present embodiment, also because in the low level bit plane, can only become coded object as meaningful information, so the compression ratio height of low level bit plane with gazing at zero displacement of the low wavelet conversion coefficient of gazing at the zone of zone and relative importance value corresponding to non-corresponding to the high wavelet conversion coefficient of gazing at the zone of relative importance value, even be encoded to the least significant bits plane, size of data can not increase yet.
Figure 15 is that a plurality of the gazing at that explanation is provided with on original image 80 exists the example that relative importance value is set when repeating in the zone.As shown in the figure, gaze at the zone 81,83 the time when on original image 80, being provided with 2, identical with the example of Figure 13, ROI relative importance value configuration part 19 is set: the 1st zero displacement figure place S1, the 2nd that gazes at zone 81 (ROI1) gazes at the zero displacement figure place S2 of zone 83 (ROI2), the non-zero displacement figure place S0 that gazes at zone (non-ROI) in addition.
Low level position zero replacement section 24 is changed to 0 from the LSB side with the S0 position in non-ROI conversion coefficient, in the conversion coefficient of ROI separately of gazing at region R OI1, ROI2 corresponding to 2, the Si position is changed to zero simultaneously from the LSB side, but for 2 repeat regions of gazing at region R OI1, ROI2, the side's that the priority application relative importance value the is high zero displacement figure place Si that gazes at the zone.
Figure 16 is that explanation a plurality of on being set in original image gaze at and exist overlappingly in the zone, comprises the figure that relative importance value under the non-situation of gazing at the zone is set example in gazing at the zone.As shown in the figure, in original image 80, set: the 1st gazes at zone 81 (ROI1), the 2nd gazes at zone 83 (ROI2), the 3rd and gazes at zone 85 (ROI3), descends according to this order relative importance value.Form: ROI1 in ROI2, the nested structure of ROI2 in ROI3, in the part of ROI3, include the non-zone (non-ROI) of gazing at.And the outside of ROI3 is right and wrong ROI also.
At this moment, low level position zero replacement section 24 is identical with the situation of Figure 15, for repeat region, because of being suitable for the high side's of relative importance value zero displacement figure place, so zero displacement figure place S1 for ROI1 prior applicability ROI1, for the zero displacement figure place S2 of ROI2 (but removing ROI1) prior applicability ROI2, for the zero displacement figure place S3 of ROI3 (but removing ROI2) prior applicability ROI3.But, be suitable for the zero permutated bits S0 of non-ROI exceptionally for the non-ROI of ROI3 inside.This be because: in gazing at the zone, set non-ly when gazing at the zone, comprising reasons such as personal information, the cause of wanting to make the purpose that this zone blurs also arranged in the zone even gaze in the non-zone of gazing at.
Figure 17 (a) and (b) be explanation in the example of Figure 16, replace the figure of situation of low level position of the wavelet conversion coefficient 60 of original images by low level position zero replacement section 24 0.Figure 17 (a) expression is carried out wavelet conversion coefficient 60 after the quantization by quantization portion 12, comprises 7 bit planes, represents the ROI conversion coefficient with oblique line.This figure is in 3 that comprise Figure 16 examples of gazing at region R OI1-3 original image 80, and diagram is ranked corresponding to the wavelet conversion coefficient of pixel on the P1-P2 line.
Figure 17 (b) is by the wavelet conversion coefficient 62 after zero replacement section, 24 0 displacements of low level position.The S0 position of the non-ROI conversion coefficient LSB side corresponding with the non-ROI of outside that is positioned at ROI3 and inside is replaced into zero.In this embodiment, S0=6.And, be replaced into zero corresponding to the Si position of the ROI conversion coefficient LSB side of R0I1～3.In this embodiment, S1=0, S2=2, S3=4.
In when decoding,, will decipher according to the decoded mode of common JPEG2000 by picture coding device 200 image encoded of present embodiment with the image decoder of the image decoder 210 identical formations of execution mode 1.Image after the decoding comprises the different a plurality of zones of picture quality, offers image display device etc. by efferent, regenerates on picture.
As above illustrated, the picture coding device 200 of present embodiment is when carrying out non-ROI conversion coefficient zero displacement, by make corresponding to a plurality of ROI conversion coefficients of gazing at the zone also only zero displacement corresponding to the figure place of relative importance value, thereby make a plurality of zones of gazing at have relative importance value and encode.And, owing to gaze at different relative importance values of interregional setting and coding a plurality of, so gaze at the interregional difference that can have the reproduced picture quality a plurality of.
(execution mode 4)
Figure 18 is the pie graph of the image decoder 220 that relates to of execution mode 4.In the present embodiment, the coded image of input picture code translator 220 is the common coded images that do not carry out the ROI coding.In the image decoder 220 of execution mode 2, when decoding, only specify one to gaze at the zone, but the image decoder 220 of present embodiment is specified a plurality of zones of gazing at when decoding, according to relative importance value a plurality of zones of gazing at are deciphered.Corresponding add same-sign, formation and the action different with execution mode 2 are described with execution mode 2 identical formations.
A plurality of zones of gazing at that ROI selection portion 18 is selected on the image, the ROI positional information of expression respectively being gazed at the zone offers ROI mask generating unit 20.
When the coded frame of dynamic image is input to image decoder 220 continuously, from the picture frame after the decoding of image decoder 220 outputs, specifying or extracting and gaze at the zone, also the zone of gazing at of specifying or extract can selected as the zone of gazing at of next coded frame.In this case, also can select time dependent a plurality of zone according to the degree that changes.For example, with change maximum zone, change time big zone, in addition zone gazes at preferentially as the 1st respectively that the zone is preferentially gazed in zone, the 2nd, the non-zone of gazing at is selected.
ROI mask generating unit 20 is based on the ROI positional information, and it is the ROI mask of ROI conversion coefficient corresponding to the wavelet conversion coefficient of respectively gazing at the zone that generation is used for specific.The relative importance value of respectively gazing at the zone is set in ROI relative importance value configuration part 19, according to the non-zero displacement figure place Si that gazes at the zero displacement figure place S0 in zone and respectively gaze at the zone of this relative importance value decision.Low level position zero replacement section 24 is with reference to the ROI mask, in the wavelet conversion coefficient after decipher by average information decoding part 32, the LSB side of non-ROI conversion coefficient begins S0 position part is replaced into zero, the LSB side from the ROI conversion coefficient begins Si position part is replaced into zero simultaneously.
The situation that Figure 19 (a)～(c) expression is carried out zero displacement by low level position zero replacement section 24 with the low level position of coded image decoding back wavelet conversion coefficient.Figure 19 (a) is the image wavelet transform coefficient 74 that carried out average information decoding, comprises 7 bit planes.In Figure 19 (b), represent corresponding to a plurality of ROI conversion coefficients of gazing at the zone by 18 appointments of ROI selection portion with oblique line.In this embodiment, be provided with 2 and gaze at region R OI1, ROI2, the relative importance value height of ROI1.Figure 19 (c) is by the wavelet conversion coefficient 76 after zero replacement section, 24 0 displacements of low level position.In this embodiment, 4 of the low levels of non-ROI conversion coefficient are replaced into zero, and the ROI conversion coefficient of ROI1 remains untouched, and 2 of the low levels of the ROI conversion coefficient of ROI2 are replaced into zero.
In simple and easy regeneration, during a plurality of zone on selecting image, image decoder 220 is at replaced non-low level position and a plurality of wavelet conversion coefficient of gazing at the state of at least 1 regional low level position of gazing at the zone by low level position zero replacement section 24 0, decipher the bit plane of lowest order, with reproduced picture.At this moment, uprise because of handling load, thus also make coma be reduced to state or the delay reproduced state of 15 frame/seconds etc. sometimes, but can regenerate to a plurality of zones of gazing at high image quality according to relative importance value.
Like this, selecting a plurality ofly when gazing at the zone, the non-zone of gazing at keeps and the quality of simple and easy regeneration with degree, can be corresponding to the high-quality of relative importance value regenerated in a plurality of zones of gazing at.
Image decoder 220 according to present embodiment, to the common coded image of not encoded by ROI, when will carrying out zero displacement corresponding to the non-low level position of gazing at the wavelet conversion coefficient in zone, by replacing carrying out zero corresponding to a plurality of low level positions of gazing at the wavelet conversion coefficient in zone according to relative importance value, gaze at also height of zone thereby can a plurality of picture qualities of gazing at the zone be improved De Bifei relatively, and encode according to relative importance value.Though,, increase so can suppress operand because of according to relative importance value a plurality of zones of gazing at being deciphered in the present embodiment for operand is increased.
(execution mode 5)
Figure 20 is the pie graph of the picture coding device 300 that relates to of execution mode 5.In the present embodiment, amplify carrying out yardstick according to relative importance value corresponding to a plurality of ROI conversion coefficients of gazing at the zone.Describe for formation and the action different with execution mode 3.
A plurality of relative importance values of gazing at the zone of being selected by ROI selection portion 18 are set in ROI relative importance value configuration part 19, determine the yardstick amplification quantity Si that gaze at each ROI conversion coefficient in zone corresponding to a plurality of according to this relative importance value.Be set at: relative importance value is high more gazes at the zone, and the yardstick amplification quantity Si of ROI conversion coefficient is big more.
ROI yardstick enlarging section 22 is according to the yardstick amplification quantity Si that is determined by ROI relative importance value configuration part 19, and yardstick amplifies corresponding to the ROI conversion coefficient of respectively gazing at the zone.
Figure 21 (a) and (b) are explanation figure by the situation of the wavelet conversion coefficient 320 of ROI yardstick enlarging section 22 yardsticks amplification original image.Figure 21 (a) expression comprises 7 bit planes by the wavelet conversion coefficient 320 after quantization portion 12 quantizations, represents the ROI conversion coefficient by oblique line.In this embodiment, be provided with 3 and gaze at region R OI1, ROI2, ROI3, reduce according to this order relative importance value.ROI relative importance value configuration part 19 is according to this relative importance value, and yardstick amplification quantity S1, S2, the S3 of ROI1, ROI2, ROI3 is set at S1=7, S2=5, S3=3 respectively.
Figure 21 (b) expression is carried out wavelet conversion coefficient 322 after yardstick amplifies by ROI yardstick enlarging section 22.ROI yardstick enlarging section 22 is amplified 7 with the ROI conversion coefficient of a ROI1 yardstick, and the ROI conversion coefficient yardstick of ROI2 is amplified 5, and the ROI conversion coefficient yardstick of ROI3 is amplified 3.In the wavelet conversion coefficient 322 after yardstick amplifies, to amplify the figure place value of zeroizing that regenerates by yardstick.Like this, obtain the wavelet conversion coefficient 322 of all 14 bit planes.
Figure 22 is the pie graph of the image decoder 310 that relates to of execution mode 5.310 pairs of images that undertaken after ROI encodes by picture coding device 300 of the image decoder of present embodiment are deciphered.
Coded data extraction unit 30 is extracted coded data, various coding parameter, ROI positional information and yardstick amplification quantity from the coded image of input.Average information decoding part 32 is deciphered coded data at each bit plane, and the quantization value of the wavelet conversion coefficient that obtains is stored in the memory.
Wavelet conversion coefficient 324 after the decoding of Figure 23 (a) expression average information.Corresponding to the example of Figure 21 (b), the wavelet conversion coefficient 324 after the average information decoding comprises 14 bit planes altogether.The ROI yardstick dwindles portion 40 with reference to the ROI mask of respectively gazing at the zone, and the yardstick amplification quantity Si that respectively gazes at the zone according to being extracted by coded data extraction unit 30 dwindles the Si position with a ROI conversion coefficient yardstick of respectively gazing at the zone.In this embodiment, the yardstick amplification quantity of gazing at region R OI1, ROI2, ROI3 is respectively S1=7, S2=5, S3=3, shown in Figure 23 (b), the ROI conversion coefficient of ROI1, ROI2, a ROI3 yardstick respectively dwindles 7,5,3, dwindle and remaining position goes out of use by yardstick, totally obtain the quantization wavelet conversion coefficient 326 of 7 bit planes.
(execution mode 6)
Figure 24 is the pie graph of the image display device 400 that relates to of execution mode 6.Image display device 400 is that dynamic image is presented at device on the display, as an example, is display control units such as DVD (digitalvideo disk) player, television receiver, monitor camera.
Encoding block 420 is encoded the original image of input, and the image after will encoding is stored in storage part 440.The original image that is input to encoding block 420 is the dynamic image frame, continuous programming code dynamic image frame, and store storage part 440 into.
Encoding block 420 has the formation of any picture coding device 200 of execution mode 1,3,5, accept to be set in the information of gazing at the zone on the picture from regional selection portion 450, according to the relative importance value of gazing at the zone, original image is carried out compressed encoding, generate the different coded image of picture quality of gazing at the zone.
Decode block 430 is read coded image from storage part 440, deciphers and offer display part 410.The coded image of reading from storage part 440 is the coded frame of dynamic image, and coded frame is deciphered continuously, and offers display part 410.And decode block 430 can obtain coded image via the communication interface of wireless or cable network, also can obtain coded image via the reception piece that receives broadcasting wave.
Decode block 430 has the formation of any image decoder 220 in the execution mode 1～5, and the coded image that is stored in storage part 440 is deciphered.At this, when having selected to gaze at the zone in coding stage and having adjusted the picture quality of gazing at the zone, directly decipher, but when coding stage is not selected to gaze at the zone, decode block 430 receives the information of gazing at the zone that is set on the picture from regional selection portion 450, the zone is gazed in preferential decoding, gazes at the different decoding image of picture quality in zone with generation.
Display part 410 receives the decoding image from decode block 430, and outputs to display.On display picture, show picture frame continuously, the regeneration dynamic image by decode block 430 decodings.
Zone selection portion 450 selects to gaze at the zone on the picture of display.Gazing at the zone can freely be specified by the user, also can be pre-set in the central authorities of picture and top etc.In regional selection portion 450, the user can be set on picture, specify the Notified body of gazing at regional usefulness.For example, use indicating device such as mouse and come the appointed area, or the display unit of using touch manners such as touch panel is come the appointed area.Zone selection portion 450 receives the appointed information of gazing at the zone from the user, or receives predefined appointed information of gazing at the zone from display part 410, this is gazed at area information offer encoding block 420, decode block 430.
To offer the ROI selection portion 18 of each piece by the positional information of gazing at the zone that regional selection portion 450 supply to encoding block 420, decode block 430, and afterwards, in each piece, gaze at the zone and preferentially encoded or decipher.And, also can select the non-zone of gazing at replace gazing at the zone by regional selection portion 450, in this case, in encoding block 420 and decode block 430, the non-zone of gazing at beyond the zone preferentially is encoded as gazing at the zone or deciphers.
Above Shuo Ming embodiments of the present invention 1～6th, example it will be appreciated by those skilled in the art that: the combination of these each inscapes or variety of processes can form various variation, and such variation also belongs to scope of the present invention.This variation below is shown.
In the above-described embodiment, only zero displacement is carried out in the low level position of non-ROI conversion coefficient, the yardstick that does not all carry out the ROI conversion coefficient amplifies, and the yardstick of implementing the ROI conversion coefficient amplifies and zero displacement of the low level position of non-ROI conversion coefficient but also can make up.
In any execution mode, the space filtering as being used for image encoding has illustrated wavelet transformation, but also can use other spatial frequency transforms.For example, even under the situation of the discrete cosine transform of using with Joint Photographic Experts Group, also can be by in kind zero displacement being carried out in the non-low level position of gazing at regional conversion coefficient, thereby sacrifice the compression ratio that non-picture quality of gazing at the zone improves integral image, improve the picture quality of gazing at the zone simultaneously relatively.
The signal processing part that is assembled into monitor camera by picture coding device and image decoder with above-mentioned execution mode, thus preferentially can be encoded or decipher in the zone of gazing at that should monitor.Also can be sent via network by monitor camera shooting and encoded image data, at this moment, the image decoder that is connected in network is to deciphering from the coded image data of network reception and regenerating.
In the above description, image is divided into non-zone and a plurality of zones of gazing at gazed at, also can do to gaze at the zone, non-ly gaze at regional this difference, image is divided into a plurality of zones corresponding to relative importance value.In the above-described embodiment, because of non-gaze at the zone and a plurality of gazing at be provided with priority in the zone, so in fact also can catch and non-ly gaze at the zone and gaze at regional the different of relative importance value that only exist, eliminate the non-difference of gazing at the zone and gazing at the zone, even outside relative importance value, during the zoning, also can carry out same processing.
Moreover the present invention relates to be generated by coded image data stream the technology of the different dynamic image of resolution or picture quality.Below, preferred execution mode of the present invention is described.In execution mode,, be that example is studied according to the image processing apparatus of the coded image data of Motion-JPEG2000 stream with decoding as image processing.
At first, with reference to Figure 25, the method for the dynamic image of encoding according to the Motion-JPEG2000 mode is carried out simple declaration.To be unit with the frame carry out continuous programming code to each frame of dynamic image to not shown picture coding device, to generate the encoded data stream of dynamic image.When encoding process began, the original image OI (Original Image) 1102 that is equivalent to 1 frame of dynamic image was read in the frame buffer.The original image OI that is read into frame buffer carries out classification by wavelet transformer.
Wavelet transformer among the JPEG2000 is utilized the Daubechies filter.This filter is brought into play high-pass filtering and low-pass filtering effect simultaneously in x, the y direction of image, and an image segmentation is become 4 frequency subbands.These subbands are: the LL subband that has low-frequency component on x, y both direction; Have low-frequency component on any one direction in x, y and on another direction, have the HL subband and the LH subband of radio-frequency component; With the HH subband that on x, y both direction, has radio-frequency component.And this filter also has simultaneously on x, y both direction and pixel count is reduced to 1/2 effect.That is to say that the pixel count of the longitudinal axis of each several part wave band is respectively to handle 1/2 of preceding image, by a filtering, resolution is that the image size obtains 1/4 subband image.In this manual, at original image OI, the image of accepting 1 wavelet transformation is called the 1st grade of image WI 1, below be called n level image WI according to the number of times of accepting wavelet transformation n
Anticipate as shown in Figure 25, at the 1st grade of image WI 1In 1104, generate 4 subband LL 1, HL 1, LH 1, HH 1At the 1st grade of image WI 1Implement wavelet transformation in 1104, and generate the 2nd grade of image WI 21106.At this, the 2nd later wavelet transformation only implemented LL subband composition in the level image at the moment.Therefore, at the 2nd grade of image WI 2In 1106, the 1st grade of image WI 11104 LL 1Subband is broken down into 4 subband LL 2, HL 2, LH 2, HH 2Put rules into practice this filtering of number of times of wavelet transformer, the wavelet conversion coefficient of output each several part wave band.Implement other processing such as quantization after the picture coding device, final output encoder view data CI (Coded Image).
Simple in order to illustrate, in this embodiment, picture coding device is implemented 3 wavelet transformations to original image OI.Therefore, for example, if original image OI1102 is 1440 * 960 pixels, the 1st grade of image WI then 11104 LL 1The subband size is 720 * 480, the 2nd grades of image WI 21106 LL 2The subband size is 360 * 240,3rd level image WI 31108 LL 3The subband size is 180 * 120.
It should be noted for the image after the classification: the low-frequency component of original image OI in Figure 25 in the most upper left appearance.Under the situation of Figure 25,3rd level image WI 3The LL in the upper left corner 3Subband is a low-limit frequency, otherwise, as long as can obtain this LL 3Subband just can be reproduced the basic properties of original image OI.This discovery is utilized in the following embodiments.
Encoded data stream except that Motion-JPEG2000, for example can be to have the SVC (Scalable VideoCodec) that the SD of the HD stream of high image quality and low image quality flows in 1 stream simultaneously, also can be Motion-JPEG.In JPEG, owing to the low order item of each frame from Fuli's leaf system number is transmitted, so can be by using and deciphering any number of times item and select picture quality.
(execution mode 7)
Execution mode 7 is: the image processing apparatus of accepting also can be provided to a plurality of display unit by the coded image data stream of resolution classification the dynamic image of different resolution.
Figure 26 represents the formation of the image processing apparatus 1100 that execution mode 7 relates to.This is formed on the hardware and can be realized by CPU, memory, other LSI of any computer, can be waited by the program with decoding function in the memory of being loaded on software and realize, but describe the functional block that is realized by working in coordination with of these at this.Therefore, it will be appreciated by those skilled in the art that: these functional blocks can be only by hardware, only by software or by these combination, realize with various forms.
In the decoding unit 1150 of image processing apparatus 1100, the stream of input coding view data CI.Decoding unit 1150 comprises: accept coded image data CI and analyze the flow analysis portion 1010 of this data flow; To the data rows that should decipher that analysis result is distinguished, the arithmetically decoding portion 1012 that implements arithmetically decoding; The bit plane decoding part 1014 that the data that this result is obtained are deciphered with the form of bit plane according to each color component; Quantization data after the decoding are carried out the inverse guantization (IQ) portion 1018 of inverse guantization (IQ); Wavelet inverse transformation portion 1020 with the n level image Win enforcement wavelet inverse transformation that the inverse guantization (IQ) result is obtained.Implement wavelet inverse transformation by 1020 couples of coded image data CI of wavelet inverse transformation portion at every turn, can obtain the more image of high bit levels, finally can obtain deciphering view data DI (DecodedImage).
Present embodiment has following feature: in wavelet inverse transformation portion 1020, will output to low resolution frame buffer 1030 obtaining deciphering the n level image that produces in the wavelet inverse transformation process of image DI.Image processing apparatus 1100 moves, and has the view data of resolution separately so that the high-definition display device 1046 of the low resolution display unit 1036 that shows low resolution and display of high resolution provided.Therefore, memory controller 1022 is obtained the resolution information by low resolution display unit 1036 and high-definition display device 1046 dynamic images displayed, judgement is implemented small echo several times to coded image CI and is become just suitable with separately resolution, and this result is sent to wavelet inverse transformation portion 1020.Wavelet inverse transformation portion 1020 is according to this information, is written in low resolution frame buffer 1030 or the high-resolution frame buffer 1040 with the LL subband of the n level image WIn in the wavelet inverse transformation processing procedure or as the decoding view data DI of decode results fully.About this action, narrate in the back with reference to Figure 29.And low resolution frame buffer 1030 and high-resolution frame buffer 1040 are distinguished simply, not necessarily need to have the buffer of different scales.
The view data that writes low resolution frame buffer 1030 generates shows signal by low resolution display circuit 1032, and is presented on the low resolution display unit 1036.Equally, the view data that writes high-resolution frame buffer 1040 generates shows signal by high resolving power display circuit 1042, and is presented on the high-definition display device 1046.Like this,, use coded image data stream, can be simultaneously displayed on a plurality of display unit by the dynamic image that resolution is different according to image processing apparatus 1100.
Any one of low resolution display circuit 1032 or high resolving power display circuit 1042, or these two also can have resolution conversion portion 1034,1044.In view of the above, when the resolution of handling the resolution that obtains and each display unit 1036,1046 dynamic images displayed by the wavelet inverse transformation of decoding unit 1150 not simultaneously, decipher image with nearest stage resolution ratio, afterwards, can be transformed to desirable resolution by resolution conversion portion 1034,1044.These resolution conversion portions the 1034, the 1044th, optional, if do not wish to show the dynamic image with the resolution resolution in addition that is obtained by wavelet transform process, then low resolution display circuit 1032 or high resolving power display circuit 1042 also can not have resolution conversion portion 1034,1044 separately.
Figure 27 represents the processing sequence of decoding unit 1150.In this explanation: as mentioned above, the coded image data stream of original image OI being implemented 3 wavelet transformations is provided for image processing apparatus 110.
At first, the coded image data CI of input image processing unit 1100 recovers 3rd level image WI through flow analysis portion 1010, arithmetically decoding portion 1012, bit plane decoding part 1014 and inverse guantization (IQ) portion 1018 31122 state.Then, obtain the 2nd grade of image WI by the 1st wavelet inverse transformation according to wavelet inverse transformation portion 1020 21124; Obtain and the 1st grade of suitable image WI by 2 wavelet inverse transformations 11126; At last, obtain deciphering image DI1128 by 3 wavelet inverse transformations.
But as mentioned above, the low-frequency component that LL subband at different levels is extracted in this grade image forms, and size is 1/4 of an earlier stages image, and therefore, these are compared with original image OI, can be described as the image of low resolution.Therefore, as an example, the 1st grade of image WI that will obtain by 2 wavelet inverse transformations 11126 LL 1Subband (720 * 480) outputs to low resolution frame buffer 1030 as the view data of low resolution; The decoding image DI (1140 * 960) that obtains by 3 wavelet inverse transformations can be outputed to high-resolution frame buffer 1040 as high resolution image data.Because by implementing wavelet transformation, the x of image, y direction size become 1/2 respectively, so the wavelet transformation number of times of being implemented by the wavelet transformation of picture coding device is many more, just more can be corresponding to the demonstration of dynamic image with more resolution.
Figure 28 is explanation is generated resolution Different Dynamic image by each frame a concept map.According to instruction from memory controller 1022,1020 pairs of each encoded image frames of wavelet inverse transformation portion are implemented necessary decoding and are handled, and the image of low resolution outputed to low resolution frame buffer 1030, high-resolution image is outputed to high-resolution frame buffer 1040.So,, form low resolution or high-resolution dynamic image thereby can flow by same coded image data by low resolution or high-resolution image are exported continuously with desired frame rate.
Figure 29 is the flow chart of the action of explanation memory controller 1022.At first, memory controller 1022 is obtained the resolution information (S10) by low resolution display unit 1036 and high-definition display device 1046 dynamic images displayed.Replace it, also can be by user input with the resolution information of display unit dynamic images displayed separately.Secondly, memory controller 1022 is judged the low-resolution image that is shown by low resolution display unit 1036, and relative coding image C I is equivalent to the LL subband (S12) of which level.Then, memory controller 1022 is judged the high-definition picture that is shown by high-definition display device 1046, the decoding image DI (S14) after the LL subband that is equivalent to which grade image still is equivalent to decipher fully.And, memory controller 1022, send instruction to wavelet inverse transformation portion 1020, so that when the wavelet inverse transformation processing of the image of the level that obtains judging respectively finishes, subband image LL or decoding image DI are write low resolution frame buffer 1030 or high-resolution frame buffer 1040 (S16) in S12 and S14.Certainly, should have only from the display unit of image processing apparatus output image data under 1 the situation, can adopt either party of low resolution frame buffer 1030 or high-resolution frame buffer 1040.
As mentioned above, in JPEG2000, because of the relative original image of the resolution of LL subband vertically reduces 1/2, laterally reduces 1/2, so also occur to obtain the situation of the image in different resolution that be fit to consistent sometimes with the resolution of desirable display unit.Therefore, when memory controller 1022 is judged the resolution that can not obtain being fit in S12 or S14, can send instruction to the resolution conversion portion 1034,1044 that is arranged at low resolution display circuit 1032 or high resolving power display circuit 1042 respectively, carry out the interpolation that is fit to resolution and handle.
Moreover image processing apparatus 1100 also can pass through to increase frame buffer quantity, thereby shows the dynamic image of different separately resolution in the display unit more than three.For example, if use above-mentioned example, can be respectively with the 2nd grade of image WI that obtains by 1 wavelet inverse transformation 21124 LL2 subband (360 * 240) outputs to the low resolution frame buffer; The 1st grade of image WI that will obtain by 2 wavelet inverse transformations 11126 LL1 subband (720 * 480) outputs to the intermediate-resolution frame buffer; To output to the high-resolution frame buffer by the decoding image DI1128 (1440 * 960) that 3 wavelet inverse transformations obtain.Display circuit by correspondence can be presented at low resolution, intermediate-resolution, high-resolution dynamic image on the display unit.
As described above,, use single coded image data stream, can simultaneously that resolution is different dynamic images be presented on the display unit more than 2 according to present embodiment 7.With respect to existing situation about deciphering at every turn according to purpose resolution, can be by the classification image in the decoding treatment process be outputed to frame buffer, thus generate the view data of a plurality of resolution with single decoding unit, improved efficient.
(execution mode 8)
Figure 30 represents the formation of the image display device 1200 that execution mode 8 is related.Image display device 1200 has the 1st display unit 1222 of display of high resolution dynamic images such as display, projecting apparatus and shows the 2nd display unit 1224, these 2 display unit of low resolution dynamic image.
The picture decoder 1212 of processing block 1210 is collaborative with CPU1214 and memory 1216, and the coded image data stream of input is deciphered continuously.Picture decoder 1212 has the formation of the image processing apparatus 1100 of execution mode 7.And high-resolution view data is output to the 1st display unit 1222 by display circuit 1218, and the view data of low resolution is output to the 2nd display unit 1224 by display circuit 1220.View data by picture decoder 1212 decodings on each picture is shown continuously according to given frame rate, and the regeneration dynamic image.Processing block 1210 can be obtained coded image data stream via wireless or wired network communication interface, also can obtain coded image data stream via the reception piece of accepting broadcasting wave.
For example can realize following action by this image display device 1200.
1. projection in the cabin on the aircraft
In the cabin of aircraft, when the place ahead in the cabin has the screen of big picture, when having personal small-sized LCD at the Ah back side at each seat, only prepares single coded image data stream, the dynamic image of can regenerating on screen and LCD two sides.
2. show (presentation)
When showing, as long as prepare single coded image data stream, the dynamic image of just regenerating on by the big picture screen of projecting apparatus projection and PC picture two sides.
3. two picture mobile phones
By in mobile phone, being assembled into above-mentioned picture display device with basic display unit and slave display, flow thereby can receive single coded image data, and the animated content of can on basic display unit and slave display both sides, regenerating.
And image display device 1200 can also have the display unit of distinguishing the different dynamic image of display resolution more than 3 certainly according to purpose.
(execution mode 9)
Execution mode 9 is: will be according to resolution and picture quality by the decoding of the image stream of classification and be presented in the image processing apparatus on the display unit, when sending the instruction of a part of regional picture quality raising that makes image by the user, control, so that treating capacity is no more than the image processing apparatus of the maximum capacity of image processing apparatus.
Figure 13 is the pie graph of the image processing apparatus 1300 that relates to of expression execution mode 9.Image processing apparatus 1300 comprises: the stream of input coding view data CI, the decoding unit 1310 that image is deciphered; With carry out by the regional designating unit 1320 of gazing at the relevant processing in zone in user's specify image.Be contained in the flow analysis portion 1010, arithmetically decoding portion 1012, bit plane decoding part 1014, inverse guantization (IQ) portion 1018, wavelet inverse transformation portion 1020 of decoding unit 1310 identical with described in the execution mode 7.
View data by decoding unit 1310 decodings is presented on the display unit 1062 by display circuit.Input unit such as unshowned indicating device in user's application drawing is specified the zone of wanting to improve picture quality and regeneration in image (below be referred to as " gazing at region R OI (Region of Interest) ").So the positional information formation portion 1050 in the regional designating unit 1320 generates the ROI positional information that the position of region R OI is gazed in expression.This ROI positional information when gazing at region R OI and be designated as the square type, is provided by the coordinates of pixels value in the upper left corner, rectangular area and the longitudinal axis pixel count of rectangular area.Waiting with circle when carrying out the gazing at the zone and specify of user, also its bounding rectangle can be set at and gaze at the zone.Gaze at the predetermined zones such as central area that the zone also can be set to original image usually.
Detection unit 1052 is according to the ROI positional information that generates, calculate to improve the required data processing recruitment of picture quality of gazing at the zone, and judge that treating capacity that the whole decoding of the treating capacity of current time point and increase handles is whether in the maximum processing capability of image processing apparatus 1300.Picture quality instruction unit 1054 is according to this result of determination, whether decision allows to improve the picture quality of gazing at the zone, or reduce the picture quality in the zone gaze at beyond the zone (below be referred to as " zone usually "), and this indication is outputed to ROI mask generating unit 1056.The details of its processing will be in the back narration with reference to Figure 35 or 38.
ROI positional information from position information formation portion 1050 since the ROI mask generating unit 1056 is the basis, generates to be used for the specific ROI mask of gazing at area part corresponding to wavelet conversion coefficient.The ROI mask that generates is by low level zero replacement section 1058, is used to adjust the lower bit number that is replaced into null value in the ranking of above-mentioned wavelet conversion coefficient.By it is carried out inverse wavelet transform, gaze at image after the area image quality thereby can be improved.To narrate in the back about this.
At this, with reference to Figure 32 (a)～(c), to serving as that the method that the basis generates the ROI mask describes with the ROI positional information by ROI mask generating unit 1056.Shown in Figure 32 (a), on image 1080, gaze at zone 1090 by user's appointment by image processing apparatus 1300 decodings and demonstration.ROI mask generating unit 1056 is the specific regional 1090 necessary wavelet conversion coefficients of selecting on the original image 1080 of gazing at of going back in the each several part wave band.
Figure 32 (b) expression is by carrying out the 1st grade of changing image 1082 that 1 wavelet transformation obtains to 1080 of images.The 1st grade of changing image 1082 is made of 4 subband LL1, HL1, LH1, HH1 of the 1st level.ROI mask generating unit 1056 in the each several part wave band LL1 of the 1st level, HL1, LH1, HH1 specific go back original image 1080 gaze at wavelet conversion coefficient on 1090 necessary the 1st grade of changing images 1082 of zone (below be referred to as " ROI conversion coefficient ") 1091～1094.
Figure 32 (c) expression is further carried out the 2nd grade of changing image 1084 that wavelet transformation obtains by the subband LL1 to the changing image 82 of Figure 32 (b).The 2nd grade of changing image 1084 except that 3 subband HL1, LH1 of the 1st level, HH1, also comprises 4 subband LL2, HL2, LH2, the HH2 of the 2nd level as shown in the figure.The wavelet transformation of ROI mask generating unit 1056 ROI conversion coefficient 1091 necessary the 2nd grade of changing images 1084 of the subband LL1 of the 1st grade of changing image 1082 of specific reduction in the 2nd layer each several part wave band LL2, HL2, LH2, HH2, be ROI conversion coefficient 1095～1098.
Equally, by only the number of times with wavelet transformation is specific for the ROI conversion coefficient of gazing at zone 1090 in recurrence at different levels, thereby in the changing image of final level, can specific all reduction gaze at zone 1090 necessary ROI conversion coefficients.ROI mask generating unit 1056 generates the ROI mask that is used for specific this final specific ROI conversion coefficient position on the changing image of final level.For example, when only carrying out 2 wavelet transformations, generation can specific Figure 32 (c) in the ROI mask of position of 7 ROI conversion coefficients 1092～1098 shown in the dotted line.
Secondly, with reference to Figure 33,34, describe making the method for gazing at regional high image qualityization.And at this, shown in Figure 33 (a), coded image data CI is by 5 bit planes of (Least Significan Bit) constitute from MSB (Most SignifcantBit) to LSB.
Image processing apparatus 1300 is not being gazed at the zone generally by user's appointment, handles load in order to alleviate, and carries out the simple and easy regeneration of the low level bit plane of suitable abandoned well regeneration wavelet conversion coefficient.The picture quality of this moment is called " middle picture quality ".At this moment, low level zero replacement section 1058 as Figure 33 (b) shown in, will bit plane by 1014 decodings of bit plane decoding part in, for example low level 2 bit positions are replaced into zeroly, and only 3 bit planes are deciphered.Gaze at the area image quality in order only to improve by this state, can be only to gazing at the more bit plane of zone decoding.
Figure 34 (a)～(c) expression improves an example of the processing of the picture quality of gazing at the zone.Shown in Figure 34 (a), when simple and easy regeneration, begin zero displacement of 2 quilts of low level from LSB side card by low level zero replacement section 1058.ROI mask generating unit 1056 if gaze at the zone by user's appointment, then generates the ROI mask of gazing at the zone corresponding to this.Its situation is represented by the oblique line of Figure 34 (b).So as shown in Figure 34 (c), low level zero replacement section 1058 with reference to the ROI mask, and only generates and will be changed to zero wavelet conversion coefficient by low level 2 positions of the non-ROI part of ROI mask.
Inverse guantization (IQ) portion 1018 carries out inverse guantization (IQ) with the wavelet conversion coefficient that generates, and the wavelet conversion coefficient after 1020 pairs of inverse guantization (IQ) of wavelet inverse transformation portion carries out inverse transformation.In view of the above, only gazed at the view data that region R OI has been enhanced picture quality.
Then, with reference to the flow process of Figure 35, the processing of detection unit 1052 is described.As prerequisite, there is not appointment to gaze at the zone generally the user, show dynamic image with above-mentioned middle picture quality.
At first, detection unit 1052 receives the ROI positional information (S30) of gazing at the zone from positional information generating unit 1050.Secondly.Go out to gaze at regional area (or pixel count) according to the ROI positional information calculation, and the integral body of computed image processing unit 1300 decoding treating capacity P (S32).
At this, decoding treating capacity P can by: the summation of (treating capacity of each picture quality) * (area of each picture quality) is represented.If the per unit area treating capacity during with low image quality is made as l L, the per unit area treating capacity during with middle picture quality is made as l M, the per unit area treating capacity during with high image quality is made as l H, the integral image area is made as S, the decoding treating capacity when then common is:
P＝l M·S (1)
To be made as s by the area of gazing at the zone of user's appointment H, the decoding treating capacity P in the time of can calculate improving this and gaze at the area image quality by following formula:
P＝l H·s H+l M(S-s H) (2)
Detection unit 1052 judges whether the decoding treating capacity P that is calculated by formula (2) surpasses image processing apparatus 1300 manageable upper limit disposal ability P in 1 image duration Max(S34).If P is at upper limit disposal ability P for the decoding treating capacity MaxBelow (S34 not), then picture quality instruction unit 1054 allows to gaze at the zone and improves picture quality (S36).If decoding treating capacity P surpasses upper limit disposal ability P Max(S34 is), then owing to do not remain a disposal ability of high image quality decoding in image processing apparatus 1300, therefore, picture quality instruction unit 1054 does not allow to gaze at the zone and improves picture quality (S38).
Figure 36 (a) and (b) are in the S34 of the flow process of Figure 35, are illustrated schematically in to judge that decoding treating capacity P is upper limit disposal ability P MaxThe situation of the picture when following.In the drawings, the zone of the zone of picture quality, " H " expression high image quality in the zone of " L " expression low image quality, " M " expression.Shown in Figure 36 (a), when general image being deciphered, if specifying, the user gazes at the zone in picture, then shown in Figure 36 (b) with middle picture quality, only gaze at the zone and be enhanced picture quality (H), picture quality (M) during common zone in addition intactly keeps.
As described above, according to the image processing apparatus of present embodiment, in the image that decoding shows, if user's appointment will be with the zone of high image quality regeneration, then when there is surplus in the decoding disposal ability of image processing apparatus, gazes at the zone and be enhanced picture quality; When there is not surplus in the decoding disposal ability, gazes at the zone and be not enhanced picture quality.
Like this, gaze at the zone if specify, then usually the zone directly with the quality of simple and easy regeneration with degree, can only regenerate with more high-quality to gazing at the zone.This shown in monitoring picture, usually the time, do not require high-quality, only when unusual, want to gaze under the situation of position with high-quality regeneration particularly useful.
Secondly, if improve the picture quality of gazing at the zone, another example of the processing when then illustrating that with reference to Figure 37 (a)～(c) disposal ability of image processing apparatus 1300 is not enough.
Shown in Figure 37 (a), when simple and easy regeneration, begin zero displacement of 2 quilts of low level from the LSB side by low level zero replacement section 1058.ROI mask generating unit 1056 is gazed at the zone if the user specifies, and then generates the ROI mask of gazing at the zone corresponding to this.Oblique line by Figure 37 (b) is represented this situation.This is gazed at more than the situation of area Figure 34 (b) in zone, gazes at regional quality, then the disposal ability deficiency of image processing apparatus 1300 if improve.At this moment, shown in Figure 37 (c), low level zero replacement section 1058, with reference to the ROI mask, for the non-ROI part of ROI mask not, generating is not to be changed to zero wavelet conversion coefficient with 2 of low levels but with low level 3 positions.
So inverse guantization (IQ) portion 1018 carries out inverse guantization (IQ) with the wavelet conversion coefficient that generates, the wavelet conversion coefficient after 1020 pairs of inverse guantization (IQ) of wavelet inverse transformation portion carries out inverse transformation.In view of the above, can obtain: gaze at that region R OI is enhanced picture quality, usually the zone is lowered the view data of picture quality.Like this, improving by the picture quality of gazing at the zone of ROI mask, promptly increasing under the situation of number of bit-planes, when the disposal ability deficiency of image processing apparatus, by reducing, thereby obtain the balance of the treating capacity of image processing apparatus integral body not by the number of bit-planes in the common zone of ROI mask.
Then, with reference to the flow process of Figure 38, illustrate that holder improves the processing of gazing at area image quality, the detection unit 1052 when then the disposal ability of image processing apparatus 1300 is not enough.Identical with the situation of Figure 35, do not specify the user and to gaze at the zone generally, show that with above-mentioned middle picture quality dynamic image is a prerequisite.
It is identical with S30, the S32 of Figure 35 that the calculating (S52) of integral body decoding treating capacity P of zone (S50) and image processing apparatus 1300 is gazed in reception.Detection unit 1052 judges whether the decoding treating capacity P that calculates surpasses image processing apparatus 1300 manageable upper limit disposal ability Pmax (S54) in 1 image duration in S52.If decoding treating capacity P is (S54 denys) below upper limit disposal ability Pmax, then picture quality instruction unit 1054 allows to gaze at zone raising picture quality (S64).
When decoding treating capacity P surpassed upper limit disposal ability Pmax, detection unit 1052 calculated the treating capacity that satisfies following formula, and the picture quality (S56) in the common zone of decision.
P＝l H·s H+l L(S-s H) (3)
So picture quality instruction unit 1054 shows the warning picture (S58) that whether allows to reduce the picture quality of gazing at common zone in addition, zone at the user on display unit.When the user expresses the unallowed meaning by input unit not shown in the figures (S60 denys), picture quality instruction unit 1054 does not allow to improve the picture quality (S66) of gazing at the zone.When the meaning that the user represents to allow (S60 is), picture quality instruction unit 1054 is sent the indication (S62) that reduces common area image quality when the picture quality in zone is gazed in raising.In view of the above, decoding treating capacity P becomes below the upper limit disposal ability Pmax.
Figure 39 (a) and (b) be schematically show in the S60 of Figure 38 flow process, situation that common area image quality beyond the zone picture when reducing is gazed in user tolerance.Shown in Figure 39 (a), when entire image is deciphered,, the user gazing at the zone with middle picture quality (M) in picture, then shown in Figure 39 (b) if specifying, when gazing at the zone and being enhanced picture quality (H), common zone in addition is lowered picture quality (L).
In this embodiment, if the user specifies the gaze at zone of hope with high image quality regeneration, when then it is gazed at the decoding treating capacity in zone and improves picture quality in increase, gaze at the decoding treating capacity in common zone in addition, zone by reduction, thereby the treating capacity of image processing apparatus integral body can be reduced to below the upper limit disposal ability.In view of the above, do not increase the treating capacity of image processing apparatus, promptly can high image quality be regenerated by certain zone that the user is concerned about.And, can avoid surpassing the generation of the coma landing that the ability of image processing apparatus produces because of the decoding treating capacity.
As additional embodiments, when the user has specified when gazing at the zone, the picture quality of gazing at the zone also can directly be kept middle picture quality and make the common zone of gazing at beyond the zone reduce to low image quality.In this case, low level position zero replacement section 1058 is by carrying out zero displacement to the low level position corresponding to the wavelet conversion coefficient in non-ROI zone, thereby the picture quality of gazing at the zone can be improved relatively than the zone is also high usually, to decipher.This situation is shown among Figure 40.Shown in Figure 40 (a), when whole image being deciphered,, then shown in Figure 40 (b), gaze at picture quality in the regional untouched maintenance in picture if the user specifies and gazes at the zone with middle picture quality (M), the zone is lowered picture quality (L) usually.Only gazed at zone display frame clearly thus, so user's subjective attribute increases.
In the above description, picture quality is made as high, normal, basic 3 grades is illustrated, when the image quality level that also can have more than it according to the lower bit number of zero displacement.
The zone of gazing at of user's appointment also can be a plurality of.When specifying 2 to gaze at the zone, picture quality instruction unit 1054 can improve a picture quality of gazing at the zone according to the decoding treating capacity, and another regional picture quality of untouched maintenance is constant.Replacement is specified by the user, and gazing at the zone can form portion 1050 by positional information, and the important areas such as zone that show personage and literal by automatic extraction are set.
Gaze at the judgement of zone when the high image quality according to making, when the decoding treating capacity surpassed upper limit disposal ability Pmax, picture quality instruction unit 1054 also can be sent instruction to decoding unit 1310, so that the frame rate of the dynamic image of output reduces.In view of the above, reduce the decoding treating capacity of the time per unit of image processing apparatus integral body, so, also can form the high image quality of gazing at the zone although temporal resolution reduces.
(execution mode 10)
Figure 41 is the pie graph of the image processing apparatus 1400 that relates to of execution mode 10.Image processing apparatus 1400 is presented at dynamic image in the display unit such as display, as an example, is equivalent to display control units such as television receiver, monitor camera.
Picture decoder 1412 in the processing block 1410 is collaborative with CPU1414 and memory 1416, and the coded image data stream of input is deciphered continuously.Picture decoder 1412 has the formation of the image processing apparatus 1300 of execution mode 9.And processing block 1410 can to obtain coded image data stream, can also be obtained coded image data stream via the reception piece that receives broadcasting wave via wireless or wired network communication interface.
Display circuit 1418 is obtained the decoding image from processing block 1410, and outputs to display unit 1420.Display unit 1420 shows the picture frame of decoding odd even continuously, with the regeneration dynamic image.
Input units 1424 such as user's application indicators specify in the display unit 1420 zone of gazing in the image that shows, or utilize contact display device such as touch panel to specify and gaze at the zone.The information of gazing at the zone is imported in the processing block 1410 by interface 1422.Processing block 1410 is obtained the information of gazing at the zone and is generated to make and gazes at the different decoding image of area image quality.
According to this image display device 1410, for image from film and monitor camera, the zone of can high-quality display user selecting.
(execution mode 11)
Embodiments of the present invention 11 are a kind of image display devices, its acceptance is flowed by the coded image data of classification according to resolution, continuously each coded frame is deciphered with decoding unit, provide dynamic image data to the display unit of demonstration low resolution dynamic image and the display unit both sides of display of high resolution dynamic image, wherein in either party display unit, when the user sends the instruction of the picture quality raising that makes image part zone, in the dynamic image of low resolution and the raising of high-resolution dynamic image both sides carries out image quality.
Figure 42 represents the formation of the image display system 1500 that execution mode 11 relates to.About display circuit the 1218,1220, the 1st display unit 1222 and the 2nd display unit 1224, because of identical, so give identical symbol with execution mode 8.Decoding unit 1512 and regional designating unit 1514 are taked the formation same with the decoding unit 1310 of execution mode shown in Figure 31 9 and regional designating unit 1320.
The coded image data stream decoding continuously of 1512 pairs of inputs of decoding unit of image processing apparatus 1510.And, high resolution image data is output to the 1st display unit 1222 of display of high resolution dynamic image by frame buffer 1516, display circuit 1218, and low resolution image data is output to the 2nd display unit 1224 that shows the low resolution dynamic image by frame buffer 1518, display circuit 1220.This processing is carried out according to the order shown in the execution mode 7.And in the 1st display unit 1222 and the 2nd display unit 1224, the view data of decoding is shown continuously with the frame rate of regulation, with the regeneration dynamic image.Image processing apparatus 1510 can be obtained coded image data stream via wired or wireless network communication interface, also can obtain coded image data stream via the reception piece that receives broadcasting wave.
The user uses input units 1524 such as indicating device, specifies on the 1st display unit 1222 or the 2nd display unit 1224 zone of gazing in the image that shows, or utilizes contact display device such as touch-screen to specify and gaze at the zone.The information of gazing at the zone is imported in the image processing apparatus 1510 by interface 1522.The information that zone designating unit 1514 accepts to gaze at the zone, and judge and whether should improve the picture quality of gazing at the zone, and its result is sent to decoding unit 1512.Decoding unit 1512 respectively to the view data of high-resolution view data and low resolution, is created on the view data of gazing at the zone and forming difference beyond it usually between the zone according to result of determination.This processing is carried out according to the order shown in the execution mode 9.Below, same as described above, the dynamic image of in the 1st display unit the 1222, the 2nd display unit 1224, regenerating.
According to this execution mode,, then in other devices, also can improve the picture quality of gazing at the zone when on a plurality of display unit, simultaneously during the different dynamic image of display resolution, gazing at the zone if in any one display unit, specify.For example, in displaying, when regeneration dynamic image on by the big picture screen of projecting apparatus projection and PC picture both sides, the image that can carry out emphasizing is told the performance that the participant is such.In addition, in monitoring photographic system, when on the display that same monitoring picture stream is presented at a plurality of monitoring rooms, the also image-region that can remind other surveillants to note.
And image display system 1500 can have the display unit of distinguishing the different dynamic image of display resolution more than 3 certainly according to purpose.
Embodiment of the present invention 7 to 11 discussed above is examples, it will be appreciated by those skilled in the art that for: the combination of these each inscapes or processing procedure can be carried out various variation, and such variation also belongs to scope of the present invention.
Even in any execution mode, as the space filtering that is used for image encoding, wavelet transformation has been described all, but also can have adopted other spatial frequency transforms.For example, even in Joint Photographic Experts Group under the situation of applied discrete cosine transform, by in kind zero displacement being carried out in the low level position of the conversion coefficient in common zone, thereby can improve the picture quality of gazing at the zone relatively, on the other hand, by sacrificing the picture quality in common zone, thereby can reduce the treating capacity of image processing apparatus integral body.

Claims (16)

1, a kind of method for encoding images is characterized in that,
Be used for the zone of gazing at that defines on dynamic image is carried out information specific and recorded and narrated encoding stream in the coded data that comprises described dynamic image.
2, method for encoding images according to claim 1 is characterized in that,
When definition on described dynamic image has a plurality ofly when gazing at the zone, describedly be used for specific described information of gazing at the zone and comprise relative importance value.
3, method for encoding images according to claim 1 is characterized in that,
Being different from other regional modes according to the described picture quality of gazing at the zone that defines on described dynamic image encodes to described dynamic image.
4, method for encoding images according to claim 3 is characterized in that,
When definition on described dynamic image has a plurality ofly when gazing at the zone, describedly be used for specific described information of gazing at the zone and comprise separately relative importance value;
According to the described a plurality of picture qualities of gazing at the zone that on described dynamic image, define according to separately relative importance value and mutually different modes described dynamic image is encoded.
5, a kind of method for encoding images is characterized in that,
Be used for a plurality of zones of gazing at that define on image are carried out information specific and recorded and narrated encoding stream in the coded data that comprises described image.
6, method for encoding images according to claim 5 is characterized in that,
Described be used for specific described a plurality of gaze at the zone information comprise each described gaze at the zone relative importance value.
7, method for encoding images according to claim 5 is characterized in that,
According to described a plurality of mutual different modes of picture quality of gazing at the zone described image is encoded.
8, method for encoding images according to claim 6 is characterized in that,
According to described a plurality of picture qualities of gazing at the zone according to separately relative importance value and mutually different modes described image is encoded.
9, a kind of picture coding device is characterized in that, comprising:
Selected cell is selected the zone of gazing on the dynamic image;
Coding unit is encoded to described dynamic image; With
Generation unit generates encoding stream, and this encoding stream comprises the coding dynamic image and is used for specific described information of gazing at the zone.
10, picture coding device according to claim 9 is characterized in that,
When definition on described dynamic image has a plurality ofly when gazing at the zone, be used for specific described information of gazing at the zone and comprise separately relative importance value described.
11, picture coding device according to claim 9 is characterized in that,
Described coding unit is encoded to described dynamic image according to the described picture quality mode different with other zones of gazing at the zone that defines on described dynamic image.
12, picture coding device according to claim 11 is characterized in that,
When definition on described dynamic image has a plurality ofly when gazing at the zone, describedly be used for specific described information of gazing at the zone and comprise separately relative importance value;
Described coding unit is encoded to described dynamic image according to the described a plurality of mutual different modes of picture quality of gazing at the zone that define on described dynamic image.
13, a kind of picture coding device is characterized in that, comprising:
Selected cell is selected a plurality of zones of gazing on the image;
Coding unit is encoded to described image; With
Generation unit generates encoding stream, and this encoding stream comprises coded image and is used for specific described a plurality of information of gazing at the zone.
14, picture coding device according to claim 13 is characterized in that,
Described generation unit is according to comprising that each described mode of gazing at the relative importance value in zone generates described encoding stream.
15, picture coding device according to claim 13 is characterized in that,
Described coding unit is encoded to described image according to the described a plurality of mutual different modes of picture quality of gazing at the zone that define on described image.
16, picture coding device according to claim 14 is characterized in that,
Described coding unit, according to the described a plurality of picture qualities of gazing at the zone that on described image, define according to separately relative importance value and mutually different modes described image is encoded.
CN2007101088560A 2004-02-09 2005-02-05 Image coding apparatus and method Expired - Fee Related CN101072354B (en)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
JP2004032394A JP4514464B2 (en) 2004-02-09 2004-02-09 Image encoding apparatus and image decoding apparatus, and image display apparatus and method using them
JP2004-032394 2004-02-09
JP2004032394 2004-02-09
JP2004-032383 2004-02-09
JP2004032383A JP2005223852A (en) 2004-02-09 2004-02-09 Image-encoding device and method, and image-decoding device and method
JP2004032383 2004-02-09
JP2004094449 2004-03-29
JP2004-094449 2004-03-29
JP2004094449A JP4241463B2 (en) 2004-03-29 2004-03-29 Image processing device

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN200510007933.4A Division CN1655620B (en) 2004-02-09 2005-02-05 Image display apparatus

Publications (2)

Publication Number Publication Date
CN101072354A true CN101072354A (en) 2007-11-14
CN101072354B CN101072354B (en) 2013-06-05

Family

ID=34999119

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2007101088560A Expired - Fee Related CN101072354B (en) 2004-02-09 2005-02-05 Image coding apparatus and method

Country Status (2)

Country Link
JP (1) JP2005223852A (en)
CN (1) CN101072354B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007173940A (en) * 2005-12-19 2007-07-05 Fuji Xerox Co Ltd Image processing apparatus and image processing system
CN102238376B (en) * 2010-04-28 2014-04-23 鸿富锦精密工业（深圳）有限公司 Image processing system and method
JP5590390B2 (en) * 2010-07-15 2014-09-17 ソニー株式会社 Image processing apparatus and method
JP2015222470A (en) * 2014-05-22 2015-12-10 ソニー株式会社 Video image display device, information processing device, and video image display system
JP7390788B2 (en) * 2018-12-20 2023-12-04 キヤノン株式会社 Image encoding device and its control method and program
CN114095728B (en) * 2022-01-21 2022-07-15 浙江大华技术股份有限公司 End-to-end video compression method, device and computer readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07288806A (en) * 1994-04-20 1995-10-31 Hitachi Ltd Moving image communication system
CN1275226A (en) * 1997-10-10 2000-11-29 艾利森电话股份有限公司 Lossless region of interest coding
CN1305684A (en) * 1998-06-18 2001-07-25 艾利森电话股份有限公司 Method and apparatus in transmission of images
JP2002044551A (en) * 2000-07-26 2002-02-08 Toshiba Corp Broadcast reception display device and broadcast reception display method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07288806A (en) * 1994-04-20 1995-10-31 Hitachi Ltd Moving image communication system
CN1275226A (en) * 1997-10-10 2000-11-29 艾利森电话股份有限公司 Lossless region of interest coding
CN1305684A (en) * 1998-06-18 2001-07-25 艾利森电话股份有限公司 Method and apparatus in transmission of images
JP2002044551A (en) * 2000-07-26 2002-02-08 Toshiba Corp Broadcast reception display device and broadcast reception display method

Also Published As

Publication number Publication date
CN101072354B (en) 2013-06-05
JP2005223852A (en) 2005-08-18

Similar Documents

Publication Publication Date Title
CN1655620B (en) Image display apparatus
CN101175208B (en) Image coding method and apparatus, and image decoding method and apparatus
US7916960B2 (en) Compression encoder, compression encoding method and program
CN100442854C (en) Method and apparatus for coding images and method and apparatus for decoding the images
US7330596B2 (en) Image decoding technique for suppressing tile boundary distortion
CN101106719B (en) Wavelet transformation device, wavelet inverse transformation device and method, program, and recording medium
JP3743384B2 (en) Image encoding apparatus and method, and image decoding apparatus and method
CN101072354B (en) Image coding apparatus and method
JP4356033B2 (en) Image data processing apparatus and method
CN1678070A (en) Image processing device and method for displaying images on multiple display devices
JPWO2007066710A1 (en) Encoding apparatus, encoding method and encoding program, and decoding apparatus, decoding method and decoding program
JP2007142614A (en) Image processing apparatus and method, program, and information recording medium
US20040126020A1 (en) Apparatus and method for processing image data based on object movement speed within a frame
US8351723B2 (en) Methods and apparatus for generating and using visual error weights
CN104780383B (en) A kind of 3D HEVC multi-resolution video coding methods
CN101309415B (en) Information processing apparatus and method
JP4229323B2 (en) Encoding apparatus, encoding method, and program
JP4449400B2 (en) Image encoding apparatus and method, program, and recording medium
JP2004186871A (en) Image processing apparatus, imaging apparatus, program, and storage medium
JP2011160075A (en) Image processing device and method
EP2353294B1 (en) Encoding method, encoding device, and encoding program for encoding interlaced image
JP2006074130A (en) Image decoding method, image decoding apparatus, and imaging apparatus
JP4636547B2 (en) Encoding device, program, and information recording medium
JP4241463B2 (en) Image processing device
JP4219303B2 (en) Encoding apparatus, encoding control method, program, and recording medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: MATSUSHITA ELECTRIC (AMERICA) INTELLECTUAL PROPERT

Free format text: FORMER OWNER: SANYO ELECTRIC CO., LTD.

Effective date: 20140820

C41 Transfer of patent application or patent right or utility model
TR01 Transfer of patent right

Effective date of registration: 20140820

Address after: California, USA

Patentee after: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA

Address before: Japan Osaka

Patentee before: Sanyo Electric Co.,Ltd.

CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130605

Termination date: 20210205

CF01 Termination of patent right due to non-payment of annual fee