CN1691766A - Image processing apparatus and method - Google Patents
Image processing apparatus and method Download PDFInfo
- Publication number
- CN1691766A CN1691766A CN 200510072229 CN200510072229A CN1691766A CN 1691766 A CN1691766 A CN 1691766A CN 200510072229 CN200510072229 CN 200510072229 CN 200510072229 A CN200510072229 A CN 200510072229A CN 1691766 A CN1691766 A CN 1691766A
- Authority
- CN
- China
- Prior art keywords
- image
- mentioned
- data
- band
- view data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
There is disclosed an image processing apparatus which implements an image process suitable for an effect for compositing or switching images, and an image process that allows smooth high-speed playback of even an image in motion. The apparatus has band segmentation means for segmenting an image signal into a plurality of frequency band components, and image composition means for, after the band segmentation means segments input first and second image signals, outputting a third image signal by replacing image data for respective segmented band components. When image data, which are recorded while being segmented into a plurality of frequency band components, are composited for respective band, and the composite image data is output, image data obtained by compositing a plurality of image frequency components is decoded and played back in a high-speed playback mode.
Description
The present invention is that application number is 200310113275.8, the applying date is on November 10th, 2003, denomination of invention is the division of the patent application of " image processing apparatus and method ".
Technical field
The present invention relates to be used to carry out the image processing method and the device of high-speed retrieval of synthetic, switching, the image of a plurality of picture signals.
Background technology
In the past, as the image processing method that a plurality of images of being imported are synthesized or switch, the method that is called as " cross compound turbine (cross-fade) " is arranged: as shown in Figure 13, with predetermined ratio the baseband signal of two input pictures is carried out addition, along with the process of time makes the addition rate of change, switch to image B from image A thus.In addition, be called as the image processing method of " wiping (wipe) " in addition: as shown in Figure 14, along with switch to the left in order on the right side through the picture of associating of time.The image processing of these " cross compound turbines " and " wiping " and so on as the effect in when photography, is used to (for example, the opening flat 2-301268 communique with reference to the Japanese Patent Application Publication spy) such as switchings of the photography scene in the video camera.
In addition, other image processing methods as synthetic a plurality of images, also considering has following processing method: extract low-frequency component and radio-frequency component respectively from two images being imported, after the radio-frequency component to two images has carried out synthesizing, synthesize with the low-frequency component of an image, obtain the 3rd image.Consider owing in such the 3rd image that generates, comprising the radio-frequency component of two images, so can access high-precision thin composograph.(for example, opening flat 11-313205 communique) with reference to the Japanese Patent Application Publication spy.
But, because in the image processing of in the past " cross compound turbine " and " wiping " and so on, be to carry out under the state of such base band (baseband) signal such as luminance signal and color difference signal in the picture signal of synthesizing or replacing, so on visual effect, there is restriction.
In addition, because behind the radio-frequency component of synthetic two images, synthesize in such image processing in the past with the low-frequency component of a side image, the profile that is 1 image synthesizing and generate to emphasize is the structure of purpose, so the effect when not being suitable for switching as the like that seamless image of activity diagram.
In addition, in the past, people required: when for example reproducing view data continuous in time as moving image data, to search desirable content at short notice, perhaps roughly holding whole all flow processs etc. is purpose, carries out the reproduction than taller speed of physical record time.
And the image processing apparatus with this high-speed retrieval function also is achieved.Generally, because live image is the set of image that is called as 1 picture amount of frame, thus can be by the frame that is write down being reproduced etc. every the number frames, get (downsample) reproduction to realize the high-speed retrieval function.
But, carrying out under this situation of getting reproduction, along with reproduction speed raises (that is, along with the frame of getting between carrying out increases), it is not steady that the live image that is reproduced just becomes, and visual characteristic is not talkative good.
As the high-speed retrieval method that has improved visual characteristic, for example open in the coded image data that is disclosed in intraframe coding (intra-encoded) frame and the mutual appearance of a plurality of interframe encode (inter-encoded) frame in the flat 7-162851 communique, only the high-speed retrieval method of rendering frame intra coded frame the Japanese Patent Application Publication spy.In addition, open in the flat 7-162851 communique the Japanese Patent Application Publication spy and to record, carried out the parts of images that is called sheet (slice) cut apart in the horizontal direction, carry out the reproduction of 1/frame, and then carry out high-speed retrieval by intracoded frame is resolved into.
In addition, open in the 2001-352524 communique the Japanese Patent Application Publication spy, motion goes out following method: when high-speed retrieval has been carried out the live image of coding with the MPEG form, generate the composograph of 1 frame from the parts of images of continuous a plurality of frames, this composograph reproduced.
The high-speed retrieval method that institute's motion goes out in these documents is got with simple interframe and to be compared, even also can the mobile of image be shown when high-speed retrieval.But since in these motions all be with a plurality of two field pictures be cut into (cutting apart) rectangular-shaped, the picture that makes up each image is to constitute the composograph of 1 frame amount, so be mingled in the image that different time is photographed in composograph.
For this reason, when when having mobile 4 two field pictures to generate composograph in interframe, just becoming has the composograph that departs from (deviation) as shown in Figure 24 as shown in Figure 23, and it is bad that the image that is reproduced visually just becomes.
In addition, in order to carry out high-speed retrieval while generating this composograph, need read in the time and synthetic a plurality of images in the reproduction processes of 1 common frame amount, reproduce, therefore the processing of device load just increases sometimes, perhaps needs resources such as memory in a large number.For example open in the method for 2001-352524 communique motion the Japanese Patent Application Publication spy, just need be when 2 times of speed be reproduced with 2 times speed sense data.
Summary of the invention
One of main purpose of the present invention is exactly to address the above problem, and the image processing method and the device that provide the effect when can be used as the live image switching to utilize rightly.
In addition, other main purposes of the present invention are, the image processing method and the device that irrespectively all can carry out level and smooth demonstration when the high-speed retrieval with the size that moves are provided by simple processing.
In order to achieve the above object, the 1st technical scheme of the present invention provides a kind of image processing apparatus that a plurality of picture signals are handled of importing, and comprising: the band segmentation device that picture signal is divided into different band component; And after a plurality of picture signals having been carried out cutting apart with above-mentioned band segmentation device, synthesize above-mentioned a plurality of picture signal by part or all the view data of replacing in the band component public between above-mentioned a plurality of picture signals, and export the image synthesizer of a picture signal.
In addition, the 2nd technical scheme of the present invention provides a kind of image processing apparatus that a plurality of picture signals are handled of importing, comprise: band shared a plurality of picture signal has been carried out in input, by replacing between above-mentioned a plurality of picture signals, the view data of part or all in the predetermined band component is synthesized the image synthesizer of above-mentioned a plurality of picture signals; And after the above-mentioned predetermined frequency band that the process along with the time is handled above-mentioned image synthesizer has carried out change, the image output device that will export as live image by the image that above-mentioned image synthesizer synthesized.
In addition, the 3rd technical scheme of the present invention provides a kind of image processing method that a plurality of picture signals are handled of importing, and comprising: the band segmentation step that a plurality of picture signals is divided into different band component; And after in above-mentioned band segmentation step, a plurality of picture signals having been carried out cutting apart, synthesize above-mentioned a plurality of picture signal by part or all the view data of replacing in the band component public between above-mentioned a plurality of picture signals, and export the image synthesis step of a picture signal.
In addition, the 4th technical scheme of the present invention provides a kind of image processing method that a plurality of picture signals are handled of importing, comprise: band shared a plurality of picture signal has been carried out in input, by replacing between above-mentioned a plurality of picture signals, the view data of part or all in the predetermined band component is synthesized the image synthesis step of above-mentioned a plurality of picture signals; And after the above-mentioned predetermined frequency band that the process along with the time is handled above-mentioned image synthesis step has carried out change, will export step by the image that the image that above-mentioned image synthesis step is synthesized is exported as live image.
In addition, the 5th technical scheme of the present invention provides a kind of and carries out the image processing apparatus of the synthetic and output of frequency band to being divided into view data that a plurality of band component write down, comprising: the data acquisition device of obtaining above-mentioned view data; Above-mentioned obtained view data is carried out the reproduction output device that frequency band is synthetic and export; And the control device of controlling above-mentioned data acquisition device and above-mentioned reproduction output device according to reproducing condition; Above-mentioned control device, from the view data that constitutes continuous m (m is the integer more than or equal to 2) image 1 image is being carried out under the situation that frequency band is synthetic and reproduction is exported, control so that by above-mentioned data acquisition device, respectively from constituting the data that each view data that (m-1) open image obtains a part of band component above-mentioned a plurality of band component, obtain the data of band component of at least a portion of above-mentioned a plurality of band component from the view data that constitutes remaining 1 image; Make above-mentioned reproduction output device based on these data that obtain, above-mentioned 1 image is carried out the synthetic and output of frequency band.
In addition, the 6th technical scheme of the present invention provides a kind of and carries out the image processing method of the synthetic and output of frequency band to being divided into view data that a plurality of band component write down, and comprising: the data that obtain above-mentioned view data obtain step; Above-mentioned obtained view data is carried out frequency band reproduction synthetic and output export step; And control the controlled step that above-mentioned data obtain step and above-mentioned reproduction output step according to reproducing condition; Above-mentioned controlled step, from the view data that constitutes continuous m (m is the integer more than or equal to 2) image 1 image is being carried out under the situation that frequency band is synthetic and reproduction is exported, control so that obtain step by above-mentioned data, respectively from constituting the data that each view data that (m-1) open image obtains a part of band component above-mentioned a plurality of band component, obtain the data of band component of at least a portion of above-mentioned a plurality of band component from the view data that constitutes remaining 1 image; Make above-mentioned reproduction output step based on these data that obtain, above-mentioned 1 image is carried out the synthetic and output of frequency band.
In addition, above-mentioned purpose also can be reached by the recording medium of embodied on computer readable, and described recording medium is preserved the program of computer execution according to image processing method of the present invention that make.
Description of drawings
Accompanying drawing is included in the specification, constitutes its part, represents form of implementation of the present invention, and is used from explanation principle of the present invention with the record one of specification.
Fig. 1 is the block diagram that comprises the system of the image processing apparatus in the 1st form of implementation of the present invention.
Fig. 2 A, B are the action of explanation discrete wavelet transform unit and the figure of sub-band (subband).
Fig. 3 is the input value of expression in the quantifying unit and the figure of the relation of output valve.
Fig. 4 is the figure of the relation of expression sub-band and code character (code block).
Fig. 5 A, B are the figure of the sub-band of expression two images that will synthesize.
Fig. 6 A～F is the figure of the replacement method of explanation sub-band composition.
Fig. 7 is the block diagram that comprises the system of the image processing apparatus in the 2nd form of implementation.
Fig. 8 is the figure that is used to illustrate the action of anti-discrete wavelet transform unit.
Fig. 9 is two image examples of carrying out cross compound turbine of the present invention.
Figure 10 is the figure of explanation cross compound turbine of the present invention.
Figure 11 A～E is the figure of the additive method when the sub-band composition of replacing two images is described.
Figure 12 A～E is the figure of the additive method when the sub-band composition of replacing two images is described.
Figure 13 is the figure that is used to illustrate the action of special-effect in the past.
Figure 14 is the figure that is used to illustrate the demonstration of special-effect in the past.
Figure 15 is the block diagram of register system configuration example of the record processing unit of expression the 3rd form of implementation of the present invention.
Figure 16 is the block diagram of playback system configuration example of the record processing unit of expression the 3rd form of implementation of the present invention.
The figure of the relation of the sub-band conversion coefficient of each frame when Figure 17 A, B are the common reproduction of explanation in the 3rd form of implementation of the present invention and the image of decoded reproduction.
Figure 18 A～C is in explanation the 3rd form of implementation, the figure of the relation of the sub-band conversion coefficient of reading from each frame when 2 times of speed are reproduced and the image of decoded reproduction.
Figure 19 A～C is in explanation the 3rd form of implementation, the figure of the relation of the sub-band conversion coefficient of reading from each frame when 3 times of speed are reproduced and the image of decoded reproduction.
Figure 20 A, B are in the variation of explanation the 3rd form of implementation, the figure of the relation of the sub-band conversion coefficient of reading from each frame when 2 times of speed are reproduced and the image of decoded reproduction.
Figure 21 A～C is in explanation the 4th form of implementation, the figure of the relation of the sub-band conversion coefficient of reading from each frame when 2 times of speed are reproduced and the image of decoded reproduction.
Figure 22 A～C is in explanation the 4th form of implementation, the figure of the relation of the sub-band conversion coefficient of reading from each frame when 3 times of speed are reproduced and the image of decoded reproduction.
Figure 23 is illustrated in the figure that interframe has the example of mobile image.
Figure 24 schematically shows the figure that the image of Figure 23 has been carried out the state of high-speed retrieval by method in the past.
Embodiment
With reference to the accompanying drawings the preferred embodiment of the present invention is elaborated.
＜the 1 form of implementation 〉
Fig. 1 is the block diagram of configuration example that the image processing system of image processing apparatus, tape deck and display unit in the 1st form of implementation of the present invention has been made up in expression.
In image processing apparatus of the present invention, compression coding technology as image uses the mode that standardized so-called JPEG2000 is discussed of following, the band segmentation of carrying out image is handled and is syntheticly handled, compression is handled and extension process, is at length illustrated in ISO/IEC 15444 grades about the basic technical descriptioon of JPEG2000.Only enumerate the part that is associated with this form of implementation here, and associate with the action of image processing apparatus and to describe.
In Fig. 1,101 is the 1st image input units, and 102 is the 1st discrete wavelet transform unit, and 103 is the 1st memory cells.The 104th, data synthesis unit, the 105th, quantifying unit, the 106th, entropy coding unit, the 107th, code output unit, the 108th, tape deck.And then 201 is the 2nd image input units, and 202 is the 2nd discrete wavelet transform unit, and 203 is the 2nd memory cells.The 204th, outside input unit.The 205th, anti-discrete wavelet transform unit, the 206th, show and use buffer storage location, the 207th, image output unit, the 208th, display unit.
The 1st image input unit 101 is connected to camera and outside video signal output apparatus such as camera.At first, press luminance signal and the carrier chrominance signal of raster scan order input formation, and carry out conversion by the 1st discrete wavelet transform unit 102 as the 1st picture signal of coded object from the 1st image input unit 101.
Fig. 2 A is the structure of band shared the 1st discrete wavelet transform unit 102 is carried out in expression to input signal figure.In the figure, H0 and H1 are the FIR filters, and H0 has low pass (Low Pass) characteristic, and H1 has high pass (High Pass) characteristic.In addition, the circular portion with downward arrow is got between representing.The multivalue image signal of being imported by filters H 0 with after H1 handles the signal that is divided into different frequency bands, be lowered by sampling (downsample) and become 2: 1.
In Fig. 2 A, the multivalue image signal of being imported is handled the conversion process of horizontal direction and vertical direction as one group.In addition, after finishing in one group initial processing, and then to the minimum identical repeatedly processing of signal of frequency band, final just output belongs to a series of data of 7 different frequency bands.
Fig. 2 B represents that the result who has carried out conversion process by the 1st discrete wavelet transform unit of being made up of the structure shown in Fig. 2 A 102 is, the multivalue image signal of being imported is divided into the situation of different frequency bands, is the example that each frequency band has been added mark as HH1, HL1, LH1, HH2, HL2, HL1, LL.In the explanation afterwards, 1 grade that will be considered as decomposing to one group of conversion process of horizontal direction and vertical direction, with each frequency band HH1, HL1, LH1 ..., LL is called sub-band.
Carried out band shared picture signal by the 1st discrete wavelet transform unit 102, be saved in the 1st memory cell 103 as the 1st data set.
In addition, similarly from the picture signal of the 2nd image output unit 201 input, carry out band segmentation by the 2nd discrete wavelet transform unit 202, generate each frequency band sub-band HH1, HL1, LH1 ..., LL, and be saved in the 2nd memory cell 203 as the 2nd data set.
Here under the situation of the synthetic and hand-off process of two images that do not carry out being imported, for example the 1st data set passes through data synthesis unit 104 from memory cell 103 with intact form, and imports to next quantifying unit 105 or anti-discrete wavelet transform unit 205.
105 pairs of quantifying unit from the carrying out of data synthesis unit 104 outputs the data set of discrete wavelet quantize.Fig. 3 is the input value of expression in the quantifying unit 105 and the figure of the relation of output valve.Like this, quantifying unit 105 has become to carry out the coefficient value (abbreviating coefficient value later on as) that quantizes with the wavelet transform coefficients value transform, and outputs to follow-up entropy coding unit 106.
Entropy coding unit 106 is divided into as shown in Figure 4 a plurality of code characters (part of with dashed lines mark in Fig. 4) with the coefficient region of being imported.Then, each code character is resolved into bit plane (bit plane) with coefficient value, carry out exporting as code stream behind the arithmetic coding.
Code stream is by code output unit 107 additional header information etc., and after being generated as the document form of JPEG2000, is output to tape deck 108.Tape deck 108 is by the write section of the recording medium of the storage card that comprises solid-state memory, CD or disk etc. and recording medium and read formation such as portion, with the JPEG2000 file logging imported to recording medium.Recording medium 108 also can be used as with image processing apparatus of the present invention and has carried out incorporate device in addition, perhaps as the available wired or wireless device that is connected with image processing apparatus, perhaps is connected to computer or server on the network.
Also be imported into anti-discrete wavelet transform unit 205 from the data set of data synthesis unit 104 output, can contravariant change the picture signal of original base band into.The picture signal of having carried out inverse transformation here is imported into and shows with buffer storage location 206.Image output unit 207 can be with from showing picture signal with buffer storage location 206 as visual image, and the display unit 208 that outputs to LCD monitor (LCD) etc. shows.
Then, under the situation of using image processing apparatus of the present invention to carry out the synthetic of a plurality of images and switching, particularly switching with the frame is that the action of the data synthesis unit 104 under the situation of the continuous live image of unit describes.
The 1st live image from the input of the 1st image input unit 101 carries out band segmentation like that by the 1st discrete wavelet transform unit 102 shown in Fig. 5 A, Yi Bian and on one side every frame is upgraded as the 1st data set and to be saved in the 1st memory cell 103.
From the 2nd live image of the 2nd image input unit 201 input similarly, shown in Fig. 5 B, carry out band segmentation like that, Yi Bian and every frame is upgraded as the 2nd data set and to be saved in the 2nd memory cell 203 on one side by the 2nd discrete wavelet transform unit 202.
From the output of the 1st and the 2nd memory cell, be imported into data synthesis unit 104.Data synthesis unit 104 is also connected to outside input unit 204 in addition, can carry out band shared data set to live image from device or system's input of outside, except that above-mentioned the 1st, the 2nd live image, also can distinguish freely synthetic live image from outside input unit 204 inputs.
Here as an example, at first to synthetic view data from the 1st memory cell 103 and the 2nd memory cell 203, and the processing under the situation about switching describes.In addition, will describe in this later form of implementation, " synthetic a plurality of live images, and along with the process of time is carried out switching processing " is called " cross compound turbine ".
By using data synthesis unit 104 to carry out the cross compound turbine of this form of implementation, for example at the 1st live image of wanting to show from by 101 inputs of the 1st image input unit, to the 2nd live image, effective during the such visual effect of handoff scenario gradually by the input of the 2nd image input unit 201.Its structure is as follows: as switching to from the 1st live image under the situation of the 2nd live image, the explanation of the processing of data synthesis unit 104 is such shown in Fig. 6 A～F, on one side according to the process (transition of frame) of time to being present in the data of each sub-band that constitutes the every frame of live image, replace one side from the composition that constitutes the 1st live image successively to the composition that constitutes the 2nd live image and constitute new data set, and it is exported from data synthesis unit 104 as the 3rd live image.
To using Fig. 6 cross compound turbine that A～F data synthesis unit 104 carries out to be described in detail.Here, the data set of establishing the 1st live image is the structure as Fig. 5 A, and the data set of establishing the 2nd live image in addition is the structure as Fig. 5 B.
From the data of data synthesis unit 104 output, at first cross compound turbine begins before during in, such as shown in Figure 6A all is the state of the data set of the 1st live image only.
Then, when cross compound turbine begins, such shown in Fig. 6 B, during the video image that begins from cross compound turbine till 30 frames (about 1 second the NTSC mode), with the sub-band composition HH1a in the data set of the 1st live image, the sub-band composition HH1b corresponding with the data set of the 2nd live image replaces, and sub-band in addition makes the 1st data set former state constant, exports the 3rd data set as its result.
Then, such shown in Fig. 6 C, during the video image in the execution of cross compound turbine till from 31 to 60 frames, with sub-band composition HH1a, HL1a, the LH1a in the data set of the 1st live image, sub-band composition HH1b, HL1b, the LH1b corresponding respectively with the data set of the 2nd live image replace, in addition sub-band former state adopts the 1st data set, exports the 3rd data set as its result.
Then, such shown in Fig. 6 D, during the video image in the execution of cross compound turbine till from 61 to 90 frames, with sub-band composition HH1a, HL1a, LH1a, the HH2a in the data set of the 1st live image, sub-band composition HH1b, HL1b, LH1b, the HH2b corresponding respectively with the data set of the 2nd live image replace, in addition sub-band former state adopts the 1st data set, exports the 3rd data set as its result.
Then, such shown in Fig. 6 E, during the video image in the execution of cross compound turbine till from 91 to 120 frames, only remain the sub-band composition LLa in the data set of the 1st live image, to each sub-band respectively with other sub-bands of the 1st data set, replace with the corresponding sub-band in the data set of the 2nd live image, export the 3rd data set as its result.
Beginning the back after 120 frames from cross compound turbine then, all is the state of only exporting the data set of the 2nd live image shown in Fig. 6 F like that, and (from the finishing switching of the 1st live image to the 2nd live image) finished in cross compound turbine.
In addition, although the timing of this switching for example above-mentionedly set like that as one, when carrying out cross compound turbine, as long as on the visual effect in suitable scope, then the frame number or the time interval can be arbitrarily.
After this 3rd data set that is generated respectively in each stage of Fig. 6 A～F after having passed through processing as described above, the actual demonstration on display unit 208, perhaps becomes the moving image data that is write down by tape deck 108.
As by the image processing apparatus that is as above constituted, display format when having carried out cross compound turbine, just can be implemented in the scenario A that the 1st live image is made as Fig. 9, the 2nd live image is set as under the situation of scenario B of Fig. 9, as shown in Figure 10, along with the time through switching such display format from scenario A to scenario B gradually.
If Figure 10 is remarked additionally, then represent following situation: when the cross compound turbine that switches to scenario B from scenario A begins, low-frequency component also reduces gradually along with the radio-frequency component of effluxion scenario A reduces then gradually, and finally the composition with the end scenario A of cross compound turbine becomes zero.Otherwise to scenario B, when cross compound turbine began, low-frequency component also increased gradually along with the effluxion radio-frequency component increases then gradually, finally all is switched into the picture signal of scenario B with the end picture of cross compound turbine.
If not image is switched completely to scenario B from scenario A, always but want it to be shown with the state that two images are synthesized, as long as then control any one synthetic state of the state of the B～E that makes data synthesis unit 104 keep Fig. 6.
And then, image processing apparatus of the present invention is not limited to the cross compound turbine of two images, can be corresponding will carry out band shared data set chronologically or the structure of switching randomly from the 1st memory cell the 103, the 2nd memory cell 203 and 204, three of external input terminals.Even certainly a plurality of external input terminals 204 are set, perhaps increase the image input unit, also can carry out correspondence by image processing apparatus of the present invention.
In addition, data synthesis unit 104 carries out as cross compound turbine, replace the processing of the sub-band of a plurality of image correspondences, be not limited to as above-mentioned Fig. 6 A～F the method that begins to replace in order from the sub-band of radio-frequency component, also can begin to replace from the sub-band (for example LL) of low-frequency component.
And then, under the situation of replacing sub-band, in above-mentioned Fig. 6 A～B, whole sub-band compositions of once replacing HH1 are illustrated, make that further sub-band HH1 being carried out refinement according to the process (progress of frame) of time like that shown in Figure 11 A～E is that unit sequence is replaced with the code character but also can control as other method.
In addition, also can control the process (progress of frame) that makes shown in Figure 12 A～E like that according to the time while the addition rate of change of the sub-band composition of two image correspondences is replaced.
As above such, by using this illustrated in this form of implementation image processing apparatus, just can in the processing of input picture, provide this new special-effect, and have the effect that can utilize as the effect of video camera or editing device etc.
＜the 2 form of implementation 〉
In the 2nd form of implementation, to image processing apparatus illustrated in the 1st form of implementation, describe about the structure (playback system) of decoding function.Decoding function is used to reproduce the JPEG2000 file that is recorded in tape deck 108, and reproduces the situation of this document.At this moment, establishing tape deck 108 is the record reproducing devices with representational role, and in addition, reproduced jpeg file is included in the live image that cross compound turbine illustrated in the 1st form of implementation is handled.
Fig. 7 is the block diagram of structure example that the image processing system of image processing apparatus, tape deck and display unit in the present invention's the 2nd form of implementation has been made up in expression.
Among Fig. 7, the 108th, the tape deck with representational role, the 701st, code input unit, the 702nd, entropy decoding unit, the 703rd, inverse quantization unit, the 704th, the decoding memory cell, the 705th, anti-discrete wavelet transform unit, the 706th, show and use buffer storage location, the 707th, the image output unit, the 708th, discrete wavelet transform data output unit, the 709th, the display unit of LCD etc.
The view data (JPEG2000 file) that is recorded in the record cell 108 is carried out reproduction, and is input to code input unit 701.Therefore, analysis package is contained in the management information (head) in the view data and extracts the necessary parameter of subsequent treatment.
Afterwards, 702 pairs of coefficient values that are quantized of entropy decoding unit are decoded.The coefficient value that has carried out decoding is output to the inverse quantization unit 703 of back level.703 pairs of coefficient values of being imported of inverse quantization unit carry out inverse quantization, and as a result of resulting discrete wavelet transform coefficient is saved in decoding memory cell 704.
Then, in anti-discrete wavelet transform unit 705, carry out anti-discrete wavelet transform with the conversion coefficient in the memory 704, this process is described below being kept at decoding.
Fig. 8 is the figure of the detailed structure of the anti-discrete wavelet transform of expression unit 705.Anti-discrete wavelet transform unit 705 at first, when in vertical direction among being stored in the sub-band of decoding with the data set the frame memory 704, LL2 and LH2 read conversion coefficient and have carried out after the up-samples with 1: 2, LL2 is applied the Filtering Processing of utilizing G0 LH2 is applied the Filtering Processing of utilizing G1 and carries out addition.In addition HL2, HH2 are also carried out same processing.Then, read top result in the horizontal direction, to the result that before obtained from LL2 and LH2 at the after-applied G0 of up-samples, to the result that obtains from HL2 and HH2 at the after-applied G1 of up-samples and carry out addition.By above processing, the end of synthesis of 1 grade.By carrying out top processing, just can restore the picture signal of base band from data set with whole grades.
Then, the picture signal of 705 outputs is imported into demonstration buffer storage location 706 from anti-discrete wavelet transform unit in Fig. 7.Image output unit 707 can be with from showing picture signal with buffer storage location 706 as visual image, and the display unit 709 that outputs to LCD monitor (LCD) etc. shows.
In addition, the 708th, discrete wavelet transform data output unit, employing self-demarking code in the future outputs to outside structure with the output of memory cell 704.
If owing to use the image processing apparatus of the 2nd form of implementation of forming by above-mentioned the sort of structure, then can decode, reproduce and show the view data of having carried out from tape deck 108 reproducing, so about in reproducing live image, comprising the video image part of cross compound turbine, can be same with the 1st form of implementation, as Fig. 9 and Figure 10, the cross compound turbine live image is reproduced and show.
And then, if will be from the discrete wavelet transform unit data set of reproduced images of 705 outputs be used for input at the outside input unit 204 of the illustrated Fig. 1 of the 1st form of implementation, then also can implement from the cross compound turbine of the input picture of the reproduced image of tape deck 108 and the 1st image input unit 101 or the 2nd image input unit 201.
As above such, by using this illustrated in this form of implementation image processing apparatus, use reproduced image also can provide new special-effect, and have the effect that can utilize as the effect of video camera or editing device etc.
＜the 3 form of implementation 〉
Figure 15 is expression as in the record reproducing device of an example of image processing apparatus of the present invention, and the block diagram of the structure example of register system 300, Figure 16 are the block diagrams of structure example of the playback system 400 of the same device of expression.
(structure of register system 300)
At first, with reference to Figure 15 the structure and the action of register system 300 are described.Among the figure, the 301st, the lens unit that constitutes by lens and aperture etc., the 302nd, the image unit that constitutes by imaging apparatuss such as CCD, cmos sensors.The 303rd, temporary transient storage is from the memory of data unit of image unit 302 outputs.The 304th, carry out the view data creating unit of predetermined process to temporarily being stored in view data in the memory cell 303.The 305th, control the camera control unit of lens unit 301～view data creating unit 304 blanketly.
(action of register system 300)
The light of scioptics unit 301 incidents is imaged on the imaging apparatus of image unit 302.Then, in image unit 302, drive imaging apparatus according to drive control signal from camera control unit 305.The output signal of imaging apparatus is stored in memory cell 303 carried out the A/D conversion in image unit 302 after.The view data of being stored is imported into view data creating unit 304 according to reading control signal from the memory of camera control unit 305.
In view data creating unit 304, carry out the processing of pixel interpolation processing, colored calculation process, gamma processing etc., and create the baseband images data of 1 picture amount.The view data of being created by view data creating unit 304 is imported into image compression unit 320 described later.
In this form of implementation, the image compression encoding that carries out as image compression unit 320 is handled, and establishes to follow view data is divided into compressed encoding that a plurality of frequency bands compress the so-called JPEG2000 mode of expansion handles and describe.In addition, at length illustrated in ISO/IEC 15444 about the compression coding mode of JPEG2000 mode, so here only illustrate the necessary part of the understanding of this form of implementation.
By the view data that view data creating unit 304 is created, be imported into the discrete wavelet transform unit 306 of image compression unit 320, implement the two-dimensional discrete wavelet transform.In this form of implementation, input signal is carried out band shared discrete wavelet transform unit 306, can utilize the structure of Fig. 2 A illustrated in the 1st form of implementation.
The sub-band conversion coefficient of these 306 outputs from the discrete wavelet transform unit is output to the quantifying unit 307 of Figure 15.The sub-band conversion coefficient of 307 pairs of 306 outputs from the discrete wavelet transform unit of quantifying unit quantizes.The quantized character of the quantifying unit 307 in this form of implementation also can be with illustrated in fig. 3 identical.Like this, quantifying unit 307 antithetical phrase frequency band transformation coefficients carry out equal interval quantizing, are transformed into quantization index (index) and output to follow-up entropy coding unit 308.
(structure of playback system 400)
Then, the playback system 400 to the record reproducing device of this form of implementation describes.Figure 16 is that the block diagram by the structure example of the playback system 400 of the expansion of the coded view data of register system shown in Figure 15 300, reproduction is carried out in expression.
Among Figure 16, the 310th, tape deck shown in Figure 15, the 401st, the code sensing element, the 402nd, the entropy decoding unit, the 403rd, inverse quantization unit, the 404th, buffer storage location is used in decoding, and the 405th, anti-discrete wavelet transform unit, 402～405 constitute the image that compressed view data is launched launches unit 410.The 406th, show and to use buffer storage location, the 407th, the image output unit of LCD, CRT etc., the 408th, control the playback control unit of playback system 400 blanketly.On playback control unit 408, be connected with the console switch 409 that the user specifies various reproducing movement to use.
(action when reproducing usually)
Action during at first, to common reproduction (constant speed reproduction) describes.Carried out compressed encoding and recorded the view data of tape deck 310 by above-mentioned register system 300, read by code sensing element 401.401 pairs of heads that are included in the data flow of code sensing element are analyzed, and extract the necessary parameter of follow-up decoding processing.
Afterwards, entropy decoding unit 402 carries out the decoding processing opposite with entropy coding unit 308, and the coefficient value that has quantized is decoded.Decoded coefficient value is output to follow-up inverse quantization unit 403.Inverse quantization unit 403 uses the quantized character identical with quantifying unit 307 that the coefficient value of being imported is carried out inverse quantization, and as a result of resulting discrete wavelet transform coefficient is saved in decoding buffer storage location 404.
Anti-discrete wavelet transform unit 405 has in the 1st form of implementation with reference to the same structure in the illustrated anti-discrete wavelet transform unit of Fig. 8 705, carry out anti-discrete wavelet transform to being kept at decoding with the conversion coefficient in the buffer storage 404, and the view data of base band is decoded.
The baseband images data of being decoded by anti-discrete wavelet transform unit 405 are imported into demonstration buffer storage location 406.Image output unit 407 will be from showing that the data of using buffer storage location 406 as visual image, output to the display unit of LCD or CRT monitor etc.
Like this, playback control unit 408 is when reproducing usually, and carry out following control: the whole sub-band conversion coefficients to each frame shown in Figure 17 A are decoded, and make its decoding and demonstration like that in chronological order shown in Figure 17 B.In Figure 17 A, B n, n+1 ... represent the order of continuous frame images, in this example, show the image of (n+1) frame every 1/30 second interval since the image of n frame.
(action during high-speed retrieval)
Then, the playback system 400 of use Figure 16 illustrates the action when carrying out high-speed retrieval (search).
For example, when sending the indication of high-speed retrieval (being made as 2 times of speed here reproduces) from console switch 409,408 pairs of code sensing elements 401 of playback control unit, notice is carried out 2 times of speed and is reproduced.Code sensing element 401, the correspondence of the frame reading method during according to reproduction speed of being notified and the high-speed retrieval that is predetermined is read sub-band conversion coefficient about each frame from tape deck 310.
That is, when usually reproducing, each frame is read the conversion coefficient of whole sub-bands, reproductions of decoding, but when high-speed retrieval exists whole sub-bands read the frame of sub-band conversion coefficient and only read the frame of the sub-band conversion coefficient of a part of sub-band.In this form of implementation, when indicating the reproduction (m is the integer more than or equal to 2) of m times of speed, every m frame only 1 frame is read the sub-band conversion coefficient of whole sub-bands, remaining (m-1) frame is only read the sub-band conversion coefficient of the LL sub-band (owing in this form of implementation, carrying out the discrete wavelet transform of 2 grades, so be the LL2 sub-band) of the lowest class.
Since be that 2 times of speed are reproduced now, so as shown in Figure 18, the mutual existence of frame (n+1, n+3, n+5) of only reading the frame (n, n+2, n+4) of LL2 sub-band conversion coefficient and reading whole sub-band conversion coefficients.Owing to only read LL2 sub-band conversion coefficient time necessary and compare very short with the situation of the conversion coefficient of reading whole sub-bands, so do not need the Japanese Patent Application Publication spy to open that the 2001-352524 communique put down in writing, say so m during doubly speed is reproduced the m speed doubly during with common reproduction read in such high speed and read, even fast a little speed was read when code sensing element 401 when high-speed retrieval, needed only than common reproduction.
For only reading LL2 sub-band conversion coefficient, analyze the head of the view data that is write down, the sub-band transform coefficient data of the sub-band (oblique line part) beyond the LL2 sub-band of Figure 18 A is carried out the processing of not reading from tape deck 308.
The data of each sub-band conversion coefficient of being read are decoded by 402 pairs of coefficient values that are quantized of entropy decoding unit.The coefficient value that has carried out decoding is output to follow-up inverse quantization unit 403.403 pairs of coefficient values of being imported of inverse quantization unit carry out inverse quantization, and resulting discrete wavelet transform coefficient is saved in decoding buffer storage location 404.
Afterwards, playback control unit 408, is synthesized continuous m frame, and is generated a synthetic frame with in the buffer storage location 404 in decoding.That is, synthetic 1 frame of only reading conversion coefficient (m-1) frame of LL2 sub-band and reading the conversion coefficient of whole sub-bands, and generate a synthetic frame.
In this form of implementation, synthetic frame is weighted on average by the LL2 sub-band conversion coefficient to the LL2 sub-band conversion coefficient of the frame of the conversion coefficient of reading whole sub-bands and remaining (m-1) frame and is generated.
Promptly, such shown in Figure 18 B, to the LL2 composition LL2 (n+1) of (n+1) frame and the LL2 composition LL2 (n) of n frame, multiply by predetermined weight coefficient α, β respectively and make its addition, to calculate LL (n+1) ' as the LL sub-band conversion coefficient after synthetic.That is, carry out LL (n+1) '=α * LL (n+1)+this weighted average computing of β * LL (n).
In this form of implementation, establish factor alpha, β satisfies the relation of alpha+beta=1.0.Thereby, for example in that situation that simple mean value synthesizes is next to be α=0.5, β=0.5 by getting, under the situation of the LL composition of wanting to emphasize (n+1) frame, for example as α=0.7, β=0.3, make α＞β decide coefficient value to be weighted average calculating operation.Each sub-band transform coefficient values that LL sub-band conversion coefficient has been carried out average weighted 1 frame amount is transformed into the view data of base band by anti-discrete wavelet transform unit 405 like this.Afterwards, view data is imported into demonstration and is output to image output unit 407 with buffer storage location 406 as visual image as Figure 18 C.
Like this, playback system 400 is according to the control of playback control unit 408, it is such shown in Figure 18 A in 2 times of speed are reproduced, the frame of the conversion coefficient (being the LL sub-band in this form of implementation) of specific sub-band is only read in mutual setting, with the frame of the conversion coefficient of reading whole sub-bands, the sub-band conversion coefficient to 2 continuous frames is that unit is weighted on average to generate a synthetic frame with the sub-band like that shown in Figure 18 B.Then, the picture signal that this synthetic frame is decoded into base band makes its demonstration.By carry out just can as Figure 18 C, realizing 2 times of speed reproductions in 1 frame interval (1/30) for the reading of these sub-band coefficients of 2 continuous frames, synthetic and decoding processing.
Such shown in Figure 18 C, because shown image in the high-speed retrieval in this form of implementation, the n frame that to lose fully with simple interframe access method information and the iconic element of (n+2) frame will be presented at respectively on the reproduced image of (n+1) frame, (n+3) frame as after image (afterimage), so even make 2 times of speed reproduce the display effects that also can obtain nature under the situation about showing.
In addition as described above, since in this form of implementation at playback record under the situation of the sub-band conversion coefficient of tape deck, 1 to the m frame in the consecutive image that is write down when m times of speed is reproduced is the sub-band conversion coefficient of reading whole sub-bands, (m-1) frame of having got between other have been carried out gets final product its sub-band conversion coefficient of only reading a part (being LL2 sub-band conversion coefficient in this form of implementation), so need not read whole conversion coefficients to whole m frames, also have the effect that reduces employed memory span and consumption power with m speed doubly.
Apparent such high-speed retrieval method is not limited to 2 times of speed and reproduces, and also reproduces applicable to m doubly fast (m is the integer more than or equal to 2).
For example, under the situation of carrying out 3 times of speed reproductions, shown in Figure 19 A～C, handle getting final product like that.Processing when reproducing with shown in Figure 18 A～C 2 times speed is more such, when 3 times of speed are reproduced, except 2 frames in 3 continuous frames become the frame of only reading LL2 sub-band conversion coefficient, the LL2 sub-band conversion coefficient of 3 frame amounts is weighted average this point and all carries out common processing outward.
The average weighted action of decoding in the buffer storage location 404, for such shown in Figure 19 B, to the LL2 composition LL2 (n+2) of (n+2) frame, the LL2 composition LL2 (n+1) of (n+1) frame and the LL2 composition LL2 (n) of n frame, multiply by predetermined coefficient α, β, γ respectively and carry out the processing of addition.That is the LL2 (n+2) ' of the LL2 sub-band conversion coefficient after, synthetic tries to achieve by LL2 (n+2) '=α * LL2 (n+2)+β * LL2 (n+1)+γ * LL2 (n).
In the case, become the relation of alpha+beta+γ=1.0.For example next in the situation of calculating simple mean value is α=0.33, β=0.33, γ=0.33, for example is weighted average calculating operation with α=0.5, β=0.3, γ=0.2 under the situation of the LL composition of wanting to emphasize (n+2) frame.Carry out average weighted each sub-band transform coefficient values like this and be transformed into the view data of base band by anti-discrete wavelet transform unit 405.Afterwards, view data is imported into demonstration and is output to image output unit 407 with buffer storage location 406 as visual image as Figure 19 C.
By carry out just can realizing 3 times of speed reproductions in 1 frame interval (1/30) for the reading of these sub-band coefficients of 3 continuous frames, synthetic and decoding processing.
Even in the display image shown in Figure 19 C, the n frame that to lose fully with simple interframe access method information, the iconic element of (n+1) frame will be presented at as after image on the decoded picture of (n+2) frame, so even make 3 times of speed reproduce under the situation about showing also can with nature feel make it reproduce demonstration.
Like this too under the m situation that doubly speed is reproduced later on, use m weighted average coefficient to carry out the weighted average computing of the LL composition of m frame amount, moving of interframe become the view data of base band just to obtain afterimage effect thus resulting sub-band transformation of coefficient by anti-discrete wavelet transform unit 405, even also can be carried out doubly speed reproduction of the little m of visual sense of discomfort greatly.
The variation of＜the 3 form of implementation 〉
In above-mentioned example, for the frame (get) of the sub-band conversion coefficient of only reading a part, only read LL2 sub-band conversion coefficient, generate LL2 sub-band conversion coefficient has been carried out average weighted synthetic frame.
But, to getting kind and the number of frame, in it does not become the scope of whole sub-band coefficients, can at random not set the sub-band conversion coefficient read.
That is, also can except LL2 sub-band conversion coefficient, the sub-band conversion coefficient of HL2, LH2, HH2 also be read shown in Figure 20 A, B like that, these sub-band conversion coefficients are weighted on average to generate synthetic frame and to reproduce.
In the case, the weighted average computing is that unit carries out with the sub-band.In addition, Figure 20 A, B represent when 2 times of speed are reproduced, even but handle similarly in 3 times of high-speed retrieval more than the speed and get final product.
As described above, according to this form of implementation, when high-speed retrieval has been carried out the image of compressed encoding to each of a plurality of band component, generate by sub-band conversion coefficient from a plurality of continuous frames, the part of the sub-band conversion coefficient of the image of the 1 frame amount that expression will be reproduced, use simple processing, no matter and moving of interframe how all can realize the level and smooth high-speed retrieval that visual sense of discomfort is little.
＜the 4 form of implementation 〉
In the 3rd form of implementation, sub-band conversion coefficient public in becoming each frame of synthetic object is weighted on average to generate synthetic frame.With respect to the spy of this this form of implementation be loaded in: read different sub-band conversion coefficient mutually from each frame that becomes synthetic object, to generate a synthetic frame.
In addition, owing to the high-speed retrieval method that illustrates in this form of implementation also can be implemented by record reproducing device illustrated in the 3rd form of implementation, so the explanation of the structure of relevant record reproducing device is just omitted.In addition because the processing when waiting the reproduction of speed doubly is also common with the 3rd form of implementation, so below action during only to high-speed retrieval describe.
Same with the 3rd form of implementation, at first 2 times of speed reproductions are described.For example when send the indication that 2 times of speed are reproduced as the console switch among Figure 16 409, then 408 pairs of code sensing element notices of playback control unit are carried out 2 times of fast reproductions.Code sensing element 401, the correspondence of the frame reading method during according to reproduction speed of being notified and the high-speed retrieval that is predetermined is read sub-band conversion coefficient about each frame from tape deck 310.
That is, when usually reproducing, each frame is read the conversion coefficient of whole sub-bands, reproductions of decoding, but when high-speed retrieval, mutual different sub-band in becoming the frame of synthetic object is read the sub-band conversion coefficient.In this form of implementation, when indicating the reproduction (m is the integer more than or equal to 2) of m times of speed, read to obtain the sub-band conversion coefficient of mutually different sub-bands from continuous m frame, promptly obtain sub-band conversion coefficient about the sub-band of all categories according to the sub-band conversion coefficient of reading from these m frames.
Thereby, when 2 times of speed are reproduced, for example such shown in image pattern 21A, the 21B, read the sub-band conversion coefficient of the 2nd grade (LL2, HL2, HH2, LH2) from a frame of the coded image data of 2 continuous frames, read the sub-band conversion coefficient of the 1st grade (HL1, HH1, LH1) from another frame, just from the view data of 2 continuous frame amounts, obtain sub-band conversion coefficient thus into the synthetic necessary whole sub-bands of a frame.
Promptly, if for n frame and (n+1) frame, the sub-band transform coefficient data of then reading HL1 (n), HH1 (n), LH1 (n) about the view data of n frame, the sub-band transform coefficient data that the view data of (n+1) frame of following is then read LL2 (n+1), HL2 (n+1), HH2 (n+1), LH2 (n+1).
Like this, from the view data of certain 2 continuous frame, read the data of different mutually sub-band conversion coefficients, so that whole sub-band conversion coefficients of such 1 picture amount are made up shown in Figure 21 B.
The data of each sub-band conversion coefficient of being read are decoded by 402 pairs of coefficient values that are quantized of entropy decoding unit.The coefficient value that has carried out decoding is output to the inverse quantization unit 403 of back level.403 pairs of coefficient values of being imported of inverse quantization unit carry out inverse quantization, and resulting discrete wavelet transform coefficient is saved in decoding memory cell 404.
Afterwards, the sub-band transform coefficient values that has carried out 1 picture amount of combination shown in Figure 21 B like that is transformed into the view data of base band by anti-discrete wavelet transform unit 405.Then, the baseband images data are imported into and show with buffer storage location 406, are output to image output unit 407 as visual image as Figure 21 C.
Like this, playback system 400 is according to the control of playback control unit 408, in 2 times of speed are reproduced,, shown in Figure 21 B, obtain the sub-band conversion coefficient of whole sub-bands like that, and read the sub-band conversion coefficient of different sub-bands from each frame from continuous 2 frames of each frame shown in Figure 21 A.Then, create the synthetic frame shown in Figure 21 B, and the picture signal that is decoded into base band makes its demonstration.By carry out just can as Figure 21 C, realizing 2 times of speed reproductions in 1 frame interval (1/30) for the reading of these sub-band coefficients of 2 continuous frames, synthetic and decoding processing.
Such shown in Figure 21 C, because shown image in the high-speed retrieval in this form of implementation, the n frame that to lose fully with simple interframe access method information and the iconic element of (n+2) frame will be presented at respectively as after image on the reproduced image of (n+1) frame, (n+3) frame, so even make 2 times of speed reproduce the display effects that also can obtain nature under the situation about showing.
In addition in this form of implementation when doubly speed is reproduced at m, during the sub-band conversion coefficient of playback record in tape deck, analyze the head of data, from the view data of continuous m frame, read the mutual different sub-band conversion coefficient of every frame, and the mutual different sub-band conversion coefficient read from the m frame of combination is with the sub-band conversion coefficient of whole sub-bands of constituting 1 frame amount.Thereby, to each frame its sub-band conversion coefficient of reading a part is got final product, so do not need with m speed doubly whole m frames to be read whole conversion coefficients as prior art, same with the 3rd form of implementation, also have the effect that reduces employed memory span and consumption power.
The method of apparent this form of implementation is also same with the 3rd form of implementation, is not limited to 2 times of speed and reproduces, and also reproduces applicable to m doubly fast (m is the integer more than or equal to 2).
For example, under the situation of carrying out 3 times of speed reproductions, shown in Figure 22 A～C, handle getting final product like that.Processing when reproducing with shown in Figure 21 A～C 2 times speed is more such, when 3 times of speed are reproduced, except from the view data of 3 continuous frames, reading, and all carry out common processing outside the sub-band conversion coefficient this point of the sub-band conversion coefficient read respectively from 3 frames of combination with whole sub-bands of constituting 1 frame amount to the mutual different sub-band conversion coefficient of every frame.
In this example, such shown in Figure 22 A, read the frame of the sub-band conversion coefficient of the 1st grade (HL1, HH1, LH1) in turn repeatedly, read the frame of the sub-band conversion coefficient of the 2nd grade (HL2, HH2, LH2), only read the frame of LL2 sub-band conversion coefficient except LL2.Thus, just can obtain the sub-band conversion coefficient of one group of whole sub-band by the sub-band conversion coefficient of reading from 3 continuous frames.
The data (Figure 22 A) of each sub-band conversion coefficient of reading from each frame are decoded into the coefficient value that has quantized by entropy decoding unit 402.And output to the inverse quantization unit 403 of back level.403 pairs of coefficient values of being imported of inverse quantization unit carry out inverse quantization, and resulting discrete wavelet transform coefficient is saved in decoding memory cell 404.Afterwards, the sub-band transform coefficient values that has carried out 1 picture amount of combination shown in Figure 22 B like that is transformed into the view data of base band by anti-discrete wavelet transform unit 405.Afterwards, the baseband images data are imported into and show with buffer storage location 406, and are output to image output unit 407 as visual image as Figure 22 C.
Like this, playback system 400 is according to the control of playback control unit 408, in 3 times of speed are reproduced,, shown in Figure 22 B, obtain the sub-band conversion coefficient of whole sub-bands like that, and read the sub-band conversion coefficient of different sub-bands from each frame from continuous 3 frames of each frame shown in Figure 22 A.Then, create the synthetic frame shown in Figure 22 B, and the picture signal that is decoded into base band makes its demonstration.By carry out just can as Figure 22 C, realizing 3 times of speed reproductions in 1 frame interval (1/30) for the reading of these sub-band coefficients of 3 continuous frames, synthetic and decoding processing.
Such shown in Figure 22 C, in this form of implementation with the shown image of high-speed retrieval, the n frame that to lose fully with simple interframe access method information and the iconic element of (n+1) frame will be presented at as after image on the reproduced image of (n+2) frame, so even make 3 times of speed reproduce the display effects that also can obtain nature under the situation about showing.
Later on like this too under the m situation that doubly speed is reproduced, control from the view data of m frame and read data the different sub-band conversion coefficient of every frame.Then, become the view data of base band just to obtain the afterimage effect of radio-frequency component thus resulting sub-band transformation of coefficient by anti-discrete wavelet transform unit 405, though interframe move also can carry out greatly the little m of visual sense of discomfort doubly speed reproduce.
As described above, according to this form of implementation, when high-speed retrieval has been carried out the image of compressed encoding to each of a plurality of band component, repeatedly generate by sub-band conversion coefficient from a plurality of continuous frames, the sub-band conversion coefficient of the image of the 1 frame amount that expression will be reproduced, use simple processing, no matter and moving of interframe how all can realize the level and smooth high-speed retrieval that visual sense of discomfort is little.
" other forms of implementation "
Also can make up the above-mentioned the 3rd and the 4th form of implementation.In the case, for example in the 4th form of implementation, read control so that the part of the sub-band conversion coefficient of reading from the data of continuous m frame when doubly speed is reproduced at m repeats, in decoding is handled with the synthetic/assembling in the buffer storage location 404, the weighted average that the sub-band conversion coefficient that repeats carries out being carried out in the 3rd form of implementation is handled, generated a synthetic frame thus and get final product.
In addition in above-mentioned form of implementation as image segmentation being become the processing of a plurality of frequency bands and compressed image, be illustrated as example in the JPEG2000 mode of having used discrete wavelet transform, but just can use the present invention so long as picture signal is divided into the compression method that a plurality of frequency bands compress, self is not particularly limited compress mode.
And then in above-mentioned form of implementation, in order to make explanation and understanding of the present invention easy, the view data of carrying out compressed encoding with the two-dimensional discrete wavelet transform that uses 2 grades is illustrated as example, but also can carry out the above conversion of 3 grades.
In the above-described embodiments, only the image processing apparatus that is made of 1 equipment is illustrated, but also can realizes the function that is equal to by the system that constitutes by a plurality of equipment.
In addition, following situation is also contained among the present invention, promptly, the program of software of function that realizes the foregoing description is direct from recording medium, perhaps use wire/wireless communication to offer and have system for computer or the device that to carry out this program, carry out the program of being supplied with by the computer of this system or device and reach the function that is equal to.
Thereby, be with computer realization function treatment of the present invention, and the program code itself of supplying with and be installed in this computer also will be realized the present invention.Just, be used for realizing that the computer program of function treatment of the present invention itself is also contained in the present invention.
In the case,, then keep aloof from the form of program as long as have the function of program, can use object identification code, by the performed program of interpretive program, supply with the script data of OS etc.
As the recording medium that is used to the program of supplying with, light/optomagnetic storage medium, the nonvolatile semiconductor memory etc. such as magnetic recording media, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-R, DVD-RW of floppy disk, hard disk, tape etc. are for example arranged.
Supply method as the program of having used wire/wireless communication, can list in the server on computer network and store, form the method etc. that computer program of the present invention itself or the compressed file that comprises automatic installation function etc. can be used as on client computer the data file (program data file) that forms computer program of the present invention and program data file downloaded to the client computer of connection.At this moment, also program data file can be divided into a plurality of segmental files, and segmental file is disposed at different servers.
Just, the program data file used by computer realization function treatment of the present invention is also contained among the present invention to the server unit of a plurality of user's download.
In addition, also can encrypt and be stored in the storage medium of CD-ROM etc. and issue to the user to program of the present invention, to satisfying the user of predetermined condition, for example from homepage it is downloaded the key information of untiing password is provided, its installation is realized on computers by using this key information to carry out encrypted program through the Internet.
Make the function of the above embodiments is achieved except that carry out the program read by computer, indication according to this program, on computers Yun Zhuan OS etc. carry out actual treatment partly or entirely, by this processing the function of the foregoing description is achieved.
And then, after function expansion board that the program of reading from recording medium is written to computer to be inserted and memory that the function expansion unit that is connected on computers has, indication based on this program, the CPU that have on this function expansion board and the function expansion unit etc. carry out the part or all of of actual treatment, by this processing the function of the foregoing description are achieved.
As top illustrated, according to the present invention, by image segmentation being become a plurality of frequency bands, and the view data of replacing the frequency band of a plurality of image correspondences, just can have the switching of and image synthetic at the image of the visual effect that did not have in the past.
And then according to the present invention, reproducing at high speed when being divided into a plurality of frequency bands and having carried out the image of record, by at least to the frame of a part, only read the sub-band conversion coefficient of a part, just can carry out high-speed retrieval and not have the frame of information dropout, also can realize the level and smooth reproduction output that visual sense of discomfort is little even have in interframe in high-speed retrieval under the situation of mobile image.
Because without departing from the spirit and scope of the present invention, can realize the present invention's form of implementation in a large number and widely, be construed as the present invention and be not limited to specific form of implementation, but by additional claim defined.
Claims (12)
1. image processing apparatus carries out the synthetic and output of frequency band to being divided into the view data that a plurality of band component write down, and comprising:
Obtain the data acquisition device of above-mentioned view data;
Above-mentioned obtained view data is carried out the reproduction output device that frequency band is synthetic and export; And
Control the control device of above-mentioned data acquisition device and above-mentioned reproduction output device according to reproducing condition;
Above-mentioned control device, in that 1 image to be carried out frequency band synthetic and reproduce under the situation of output from constituting view data that continuous M opens image,
Control so that by above-mentioned data acquisition device, respectively from constituting the data that each view data that M-1 opens image obtains a part of band component above-mentioned a plurality of band component, obtain the data of band component of at least a portion of above-mentioned a plurality of band component from the view data that constitutes remaining 1 image;
Make above-mentioned reproduction output device based on these data that obtain, above-mentioned 1 image is carried out the synthetic and output of frequency band, wherein, M is the integer more than or equal to 2.
2. image processing apparatus according to claim 1 is characterized in that:
Above-mentioned control device, it is synthetic and reproduce under the situation of output 1 image to be carried out frequency band in the view data of opening image from the continuous M of above-mentioned formation,
Control and make above-mentioned data acquisition device, each view data of opening image from above-mentioned formation M-1 obtains the data of identical frequency band composition, obtain the data of whole band component of above-mentioned a plurality of band component from the view data of remaining 1 image of above-mentioned formation, wherein, M is the integer more than or equal to 2.
3. image processing apparatus according to claim 2 is characterized in that:
Above-mentioned identical frequency band composition is minimum band component.
4. image processing apparatus according to claim 2 is characterized in that:
Above-mentioned identical frequency band composition is a plurality of band component that comprise minimum band component.
5. image processing apparatus according to claim 2 is characterized in that:
Above-mentioned reproduction output device, in each data to above-mentioned identical frequency band composition, with the view data of above-mentioned identical frequency band composition in the view data that is included in above-mentioned remaining 1 image, the computing of being scheduled to and synthetic after, above-mentioned 1 image is carried out the synthetic and output of frequency band.
6. image processing apparatus according to claim 5 is characterized in that:
Above-mentioned predetermined computing is the weighted average computing.
7. image processing apparatus according to claim 1 is characterized in that:
Open the data of the band component that each view data of image obtains from above-mentioned formation M-1, data with the band component that obtains from the view data that constitutes above-mentioned remaining 1 image, be the data of all different band component, and the combination of the data of obtained band component is the data of whole band component of above-mentioned a plurality of band component.
8. image processing apparatus according to claim 1 is characterized in that:
Above-mentioned view data is divided into above-mentioned a plurality of band component by the two-dimensional discrete wavelet transform of a plurality of grades.
9. image processing apparatus according to claim 1 is characterized in that:
Also comprise the camera that image is photographed; And
To become the go forward side by side converting means of line item of a plurality of frequency contents by the image segmentation that above-mentioned camera is photographed.
10. image processing apparatus according to claim 1 is characterized in that:
Above-mentioned control device, under above-mentioned reproducing condition is situation with the reproduction of the speed also faster than common reproduction speed, above-mentioned data acquisition device and above-mentioned reproduction output device are controlled, 1 image is carried out the synthetic and reproduction output of frequency band so that open the view data of image from the continuous M of above-mentioned formation.
11. image processing apparatus according to claim 10 is characterized in that:
Above-mentioned reproducing condition is by being imported by the operating means of user's operation.
12. one kind is carried out the image processing method of the synthetic and output of frequency band to being divided into view data that a plurality of band component write down, comprising:
The data that obtain above-mentioned view data obtain step;
Above-mentioned obtained view data is carried out frequency band reproduction synthetic and output export step; And
Control the controlled step that above-mentioned data obtain step and above-mentioned reproduction output step according to reproducing condition;
Above-mentioned controlled step, in that 1 image to be carried out frequency band synthetic and reproduce under the situation of output from constituting view data that continuous M opens image,
Control so that obtain step by above-mentioned data, respectively from constituting the data that each view data that M-1 opens image obtains a part of band component above-mentioned a plurality of band component, obtain the data of band component of at least a portion of above-mentioned a plurality of band component from the view data that constitutes remaining 1 image;
Make above-mentioned reproduction output step based on these data that obtain, above-mentioned 1 image is carried out the synthetic and output of frequency band, wherein, M is the integer more than or equal to 2.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2002326935A JP2004165800A (en) | 2002-11-11 | 2002-11-11 | Image processing apparatus |
| JP326935/2002 | 2002-11-11 | ||
| JP032267/2003 | 2003-02-10 |
Related Parent Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200310113275.8A Division CN1263290C (en) | 2002-11-11 | 2003-11-10 | Image processor and method thereof |
| CN03101132.7 Division | 2003-11-10 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1691766A true CN1691766A (en) | 2005-11-02 |
| CN100407784C CN100407784C (en) | 2008-07-30 |
Family
ID=32805733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2005100722297A Expired - Fee Related CN100407784C (en) | 2002-11-11 | 2003-11-10 | Image processing apparatus and method |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP2004165800A (en) |
| CN (1) | CN100407784C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106101807A (en) * | 2016-07-12 | 2016-11-09 | 青岛海信电器股份有限公司 | Image data processing method, device and intelligent television |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9544562B2 (en) * | 2013-10-17 | 2017-01-10 | Northrop Grumman Systems Corporation | Converting an image from a dual-band sensor to a visible color image |
| JP6453694B2 (en) * | 2015-03-31 | 2019-01-16 | 株式会社モルフォ | Image composition apparatus, image composition method, image composition program, and recording medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5287187A (en) * | 1989-05-15 | 1994-02-15 | Canon Kabushiki Kaisha | Video signal processing apparatus displaying image and process information |
| JPH07162851A (en) * | 1993-12-06 | 1995-06-23 | Matsushita Electric Ind Co Ltd | Special reproducing method |
| JPH08140083A (en) * | 1994-11-09 | 1996-05-31 | Sony Corp | Image display device |
| JPH0970038A (en) * | 1995-08-31 | 1997-03-11 | Matsushita Electric Ind Co Ltd | Image data processing method |
| JPH11196262A (en) * | 1997-11-07 | 1999-07-21 | Matsushita Electric Ind Co Ltd | Digital information imbedding extracting device/method, and medium recording program to execute the method |
| JP3321402B2 (en) * | 1997-12-08 | 2002-09-03 | 株式会社日立製作所 | Video data editing method and device |
| JP3691247B2 (en) * | 1998-04-27 | 2005-09-07 | 株式会社リコー | Image processing apparatus and image processing method |
| JP4323685B2 (en) * | 2000-06-09 | 2009-09-02 | キヤノン株式会社 | Playback apparatus and playback method |
-
2002
- 2002-11-11 JP JP2002326935A patent/JP2004165800A/en not_active Withdrawn
-
2003
- 2003-11-10 CN CN2005100722297A patent/CN100407784C/en not_active Expired - Fee Related
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106101807A (en) * | 2016-07-12 | 2016-11-09 | 青岛海信电器股份有限公司 | Image data processing method, device and intelligent television |
| CN106101807B (en) * | 2016-07-12 | 2019-03-08 | 青岛海信电器股份有限公司 | Image data processing method, device and smart television |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100407784C (en) | 2008-07-30 |
| JP2004165800A (en) | 2004-06-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1221927C (en) | Image processing device and method | |
| CN1076932C (en) | Method and apparatus for coding video signal, and method and apparatus for decoding video signal | |
| CN1242620C (en) | Transcoder-based adaptive quantization for digital video recording | |
| CN1260946C (en) | Imaging equipment, image processing method and recording medium | |
| CN1135708C (en) | Picture signal processing system, decoder, picture signal processing method, and decoding method | |
| CN1199304A (en) | Additional information insertion and regeneration apparatus, method and record medium | |
| CN1784011A (en) | Image coding apparatus and image coding method | |
| CN1941913A (en) | Method and apparatus for skipping pictures | |
| CN1717058A (en) | Method and apparatus for coding images and method and apparatus for decoding the images | |
| CN1086070A (en) | The method and system of Code And Decode picture intelligence and relevant picture-signal recording media | |
| CN1788496A (en) | Image processing device, image processing method, information processing device, information processing method, information recording device, information recording method, information reproduction dev | |
| CN1221287A (en) | Apparatus for coding and decoding digital video signal derived from motion picture film source | |
| CN1211373A (en) | Digital image coding method and digital image coder, and digital image decoding method and digital image decoder, and data storage medium | |
| CN1229223A (en) | Image processing device, composite system device and method, speration method and customer device | |
| CN1489391A (en) | Space predicting method and apparatus for video encoding | |
| CN1428781A (en) | Method and equipment for producing recording information signal | |
| CN1652608A (en) | Data processing device and method of same, and encoding device and decoding device | |
| CN1808469A (en) | Image searching device and method, program and program recording medium | |
| CN1270541C (en) | Decoder and method thereof, coding device and method thereof, image processing system and method thereof | |
| CN1881452A (en) | Recording apparatus, recording method, program, recording medium, and data recording medium | |
| CN1263290C (en) | Image processor and method thereof | |
| CN1767648A (en) | Information processing apparatus, information processing method, recording medium and program | |
| CN1358029A (en) | Modifying source material | |
| CN1116480A (en) | Animation encoding method, animation decoding method, animation recording medium and animation encoder | |
| CN1650629A (en) | Encoding device and method, decoding device and method, edition device and method, recording medium, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CI01 | Correction of invention patent gazette |
Correction item: Other Correct: Original application number: 200310113275.8 False: Original application number: 3101132.7 Number: 31 Page: 1468 Volume: 24 |
|
| CI03 | Correction of invention patent |
Correction item: Other Correct: Original application number: 200310113275.8 False: Original application number: 3101132.7 Number: 31 Volume: 24 |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: OTHER; FROM: ORIGINAL APPLICATION NUMBER OF DIVISION:03101132.7 TO: ORIGINAL APPLICATION NUMBER OF DIVISION:200310113275.8 |
|
| ERR | Gazette correction |
Free format text: CORRECT: OTHER; FROM: ORIGINAL APPLICATION NUMBER OF DIVISION:03101132.7 TO: ORIGINAL APPLICATION NUMBER OF DIVISION:200310113275.8 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080730 Termination date: 20171110 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |