PROCESS METHOD FOR COMPRESSING MOVING PICTURES AND APPARATUS FOR PERFORMING SAID METHOD
FIELD OF THE INVENTION
The present invention relates to a process method for compressing moving pictures and
an apparatus for performing said method and, more particularly, to process methods for
compressing moving pictures and apparatus for performing said methods configured to
markedly improve picture quality even at the same compression rate.
BACKGROUND OF THE INVENTION
A conventional method of compressing moving pictures is to convert spatial
frequencies to time frequencies via a block segmentation method.
However, there is a drawback in the conventional method for compressing the moving
pictures thus described in that the number of calculations required for conversion via a
block segmentation method can be decreased. Further, block noise generated by the conventional block segmentation method such as a mosaic-like picture can be generated by the discontinuity of data at a borderline to deteriorate the picture quality
such that the compression rate cannot be increased beyond a certain level.
SUMMARY OF THE INVENTION
The present invention is disclosed to solve the aforementioned drawbacks and it is an
object of the present invention to provide a process method for compressing moving
pictures and an apparatus for performing said method configured to markedly improve
the picture quality at the same compression rate.
In accordance with one embodiment of the present invention, there is provided' a
process method for compressing moving pictures wherein original images are
compressed via encoding and are outputted, the method comprising the steps of:
reducing an original image by a predetermined fraction to conform to an output
resolution and establishing the reduced original image as a reference image, and
copying the reference image in plural numbers for establishment as sub-images
(setting step); erasing pixels per predetermined order from horizontal and vertical
resolutions of each sub-image while alternating the pixel-erasing order at each sub-
image to thereby allow the pixels erased per sub-image to be alternated (alternating
step); allowing the reference image and the alternated sub-images to be arranged in
multi-frames along a time axis (arranging step); differentiating the respective sub-
images from the reference image (differentiating step); deleting, from each
differentiated sub-image, areas of vertical and horizontal resolutions whose pixels are
erased in the alternating step and coupling the remaining non-deleted areas to thereby
reduce the overall resolutions of each sub-image (reducing step); and arranging the
reference image and the reduced sub-images on a two-dimensional space and
synthesizing the same into an image of a single frame, and compressing and outputting
the synthesized image (compressing and outputting step).
In accordance with another embodiment of the present invention, there is provided a
process method for decoding a compressed image and outputting the same, wherein a
reference image and a plurality of sub-images each having a value differentiated from
the reference image and a resolution lower than that of the reference image are
synthesized to be compressed into the compressed image of a single frame, the method
comprising the steps of: separating the reference image and respective sub-images
from the compressed image for arrangement on a time axis (arranging step);
expansively interpolating the respective separated sub-images to the same resolution as
that of the reference image (expansively interpolating step); adding the value differentiated from the reference image to each expanded sub-image (adding step); and insertedly coupling the pixels of the horizontal and vertical resolutions of each sub- image to which the differentiated values are added and the reference image onto a
single frame according to a set order, and outputting by synthesizing the same into an
image of original resolution (synthesizing and outputting step).
In still another embodiment of the present invention, there is provided a process
method for compressing moving pictures wherein original images are compressed and
outputted by encoding, the method comprising the steps of: reducing an original
image by a predetermined fraction to conform to an output resolution and setting same
as a reference image, and copying the reference image in plural numbers and setting
same as sub-images (setting step); erasing pixels from horizontal and vertical
resolutions of each sub-image per set order wherein the pixel erasing order is
alternated for each sub-image such that a pixel erased for each sub-image can be
alternated (alternating step); arranging the reference image and the alternated sub-
images along a time axis in multiple frames (arranging step); differentiating the
respective sub-images from the reference image (differentiating step); deleting, from
each sub-image, areas of vertical and horizontal resolutions whose pixels are erased in
the alternating step and coupling the remaining non-deleted areas to thereby reduce the overall resolutions of each sub-image (reducing step); and arranging the reference image and the reduced sub-images in multiple frames along a time axis and compressing and outputting the same (arranging and compressing step).
In accordance with still a further embodiment of the present invention, there is
provided a process method for decoding compressed images and outputting the same,
wherein a reference image and a plurality of sub-images each having a value
differentiated from the reference image and a resolution lower than that of the
reference image are compressed into respective frames and the compressed respective
frames are arranged along a time axis, the method comprising the steps of: separating
the reference image and the respective sub-images from the compressed images
(separating step); extensively interpolating the respective separated sub-images to the
same resolution as that of the reference image (extensively interpolating step); adding
the value differentiated from the reference image to each expanded sub-image (adding
step); and alternatively coupling the pixels of the horizontal and vertical resolutions of
each sub-image to which the differentiated values are added and the reference image
onto a single frame according to a set order and outputting by synthesizing the same in
an image of original resolution (synthesizing and outputting step).
In accordance with still a further embodiment of the present invention, there is provided a process method for compressing moving pictures wherein an original image is encoded, compressed and outputted, the method comprising the steps of: reducing an original image of a current frame by a predetermined fraction to conform to an output resolution and setting same as a reference image, and copying the reference
image in plural numbers and setting same as sub-images (setting step); reducing
original images of a previous frame and a next frame by a predetermined fraction and
copying same in plural numbers and setting same as sub-images (setting step);
erasing pixels from horizontal and vertical resolutions of each sub-image at a
predetermined order wherein the order of erasing the pixels are alternated at every sub-
image such that pixels are alternated in being erased at each sub-image (alternating
step); arranging the reference image and the alternated sub-images in multiple frames
along a time axis (arranging step); differentiating the respective sub-images from the
reference image (differentiating step); and arranging the reference image and the
respective differentiated sub-images in multiple frames along the time axis, and
compressing and outputting same (compressing and outputting step).
In accordance with still further embodiment of the present invention, there is provided a process method for decoding compressed images and outputting the same, wherein a
reference image and a plurality of sub-images each having a differentiated value from the reference image and a resolution lower than that of the reference image are compressed into respective frames and the compressed respective frames are arranged along a time axis, the method comprising the steps of: separating the reference image and the respective sub-images from the compressed image (separating step); adding the value differentiated from the reference image to each said separated sub-image
(adding step); and alternatively coupling the pixels of horizontal and vertical
resolutions of each sub-image to which the differentiated values are added and the
reference image onto a single frame according to a predetermined order and
compressing and outputting same as an image of original resolution (compressing and
outputting step).
BRIEF DESCRIPTION OF THE DRAWINGS
For a better understanding of the nature and objects of the present invention, reference
should be made to the following detailed description with the accompanying drawings,
in which:
FIG. 1 is a schematic block diagram for illustrating encoding means out of apparatuses for performing a process method for compressing moving pictures according to a first embodiment of the present invention;
FIG. 2 is a schematic block diagram for illustrating decoding means out of apparatuses for performing the process method for compressing the moving pictures according to the first embodiment of the present invention;
FIG. 3 is a schematic drawing for explaining a step for dividing an original image into
a reference image and a plurality of sub-images for encoding;
FIG. 4 is a schematic drawing for explaining a step of alternatively separating pixels
from each sub-image and erasing same;
FIGS. 5 and 6 are schematic drawings for explaining a step of comparing each sub-
image with a reference image for differentiation;
FIG. 7 is a schematic drawing for explaining a step of erasing a pixel of level having a
difference within a predetermined scope in the respective differentiated sub-images;
FIG. 8 is a schematic drawing for explaining a step of arranging and synthesizing a reference image and each sub-image in a single frame;
FIG. 9 is a schematic drawing for explaining a step of separating a synthesized image into a reference image and each sub-image for decoding;
FIG. 10 is a schematic drawing for explaining a step of expanding a resolution of each sub-image to that of a reference image;
FIG. 11 is a schematic drawing for explaining a step of interpolating pixels relative to a
sub-image of a current frame using frames before and after the current frame;
FIG. 12 is a schematic drawing for explaining a step of composing an image of an
original resolution using a reference image and sub-images;
FIG. 13 is a schematic block drawing of encoding means out of apparatuses for
performing a process method for compressing moving pictures according to a second
embodiment of the present invention;
FIG. 14 is a schematic block drawing of decoding means out of apparatuses for
performing a process method for compressing moving pictures according to the second
embodiment of the present invention;
FIG. 15 is a schematic drawing for explaining a step of dividing an original image into a reference image and a plurality of sub-images for encoding;
FIG. 16 is a schematic drawing for explaining a step of pairing up sub-images and synthesizing the paired sub-images along a time axis;
FIG. 17 is a schematic drawing for explaining a step of arranging and synthesizing a
reference image and sub-images along a time axis;
FIG. 18 is a schematic drawing for explaining a step of separating a synthesized image
into a reference image and respective sub-images for decoding;
FIG. 19 is a schematic block drawing of encoding means out of apparatuses for
performing a process method for compressing moving pictures according to a third
embodiment of the present invention;
FIG. 20 is a schematic block drawing of decoding means out of apparatuses for
performing a process method for compressing moving pictures according to a
preferred embodiment of the present invention;
FIG. 21 is a schematic drawing for explaining a step of dividing an original image into a reference image and a plurality of sub-images for encoding;
FIG. 22 is a schematic drawing for explaining a step of pairing up sub-images and synthesizing the paired-up sub-images along a time axis;
FIG. 23 is a schematic drawing for explaining a step of arranging and synthesizing a
reference image and sub-images along a time axis;
FIG. 24 is a schematic drawing for explaining a step of adding a differentiated value to
a sub-image for restoration; and
FIG. 25 is a schematic drawing for explaining a step of using a reference image and
sub-images to synthesize an image having an original resolution.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
The preferred embodiment of the present invention will now be described in detail
with reference to the annexed drawings, where the present embodiment is not limiting
the scope of the present invention but is given only as an illustrative purpose.
The present invention is configured to markedly improve a picture quality by allocating a data of a scale addition information being added and coded to a sub channel of an original image while having the same compression rate as that of the known compression technique.
[FIRST EMBODIMENT]
An apparatus for performing a first embodiment of the present invention includes
encoding means and decoding means. The encoding means includes an image space
separator (10), an image frame generator (20), a reference image resolution sheer (30),
a sub-image subtracter (40), a sub-image level detector (50), a sub-image slicer (60), a
sub-image resolution slicer (70) and an image synthesizer (80), as shown in FIG.1.
The image space separator (10) performs a processing work (hereinafter referred to as
alternative erasion) where an original image is reduced by a predetermined fraction to
conform to an output resolution whereby the reduced image is established as a
reference image, and the reference image is plurally copied and the plurally-copied
images are established as sub-images, wherein pixels are erased from horizontal and
vertical resolutions of each sub-image according to a predetermined order while the
pixel erasing order is alternated for each sub-image such that the pixels being erased at
each sub-image can be alternated.
The image frame generator (20) arranges the reference image and the sub-images
processed at the image space separator (10) along a time axis, and the reference image
resolution slicer (30) separates the reference image from the sub-images for
transmission to the image synthesizer (80).
The sub-image subtracter (40) differentiates each sub-image from the reference image,
while the sub-image level detector (50) detects a pixel value of each differentiated sub-
image.
The sub-image slicer (60) compares pixel values of each differentiated sub-images to
erase pixel values each having a difference within a predetermined scope, and again
compares a pixel value of each sub-image with that of the reference image to erase a
pixel value having a difference within a predetermined scope.
The sub-image resolution slicer (70) deletes, for each sub-image, areas of the vertical and horizontal resolutions whose pixels were erased in the alternative erasion step and couples the remaining non-deleted areas to thereby reduce the overall resolutions of each sub-image.
The image synthesizer (80) arranges the reference image and the reduced sub-images on a two-dimensional space, and synthesizes and compresses same to an image of a single frame to be outputted.
Referring to FIG. 2, the decoding means for performing the first embodiment of the
present invention includes an inverse image separator (110), a reference image
resolution expander (120), a sub-image resolution expander (130), a three-dimensional
frame comparator (140), a three-dimensional frame adder (150), a three-dimensional
frame synthesizer (160) and an image synthesizer (170).
The inverse image separator (110) separates the reference image and the sub-images
from the compressed images outputted from the image synthesizer (80) of the afore-
mentioned encoding means and arranges same on a time axis.
The reference image resolution expander (120) transmits to the image synthesizer
(170) the reference image separated from the inverse image separator (110). The
sub-image resolution expander (130) expands the resolution of each sub-image
separated from the inverse image separator (110) to the same resolution of the
reference image for interpolation.
The three-dimensional frame comparator (140) compares the expansively interpolated sub-images with the reference image to detect the same and different pixels between the sub-images and the reference image and detect pixels each having a difference
therebetween, i.e., detect a differentiated status.
The three-dimensional frame adder (150) adds to each sub-image a value differentiated
from the reference image in response to the differentiated status detected by the three-
dimensional frame comparator (140), and the three-dimensional frame synthesizer
(160) buffers each sub-image added by the differentiated value for image synthesis of
the original resolution.
The image synthesizer (170) alternatively and insertedly couples the pixels of vertical
and horizontal resolutions of the reference image and the sub-images on a single frame
in response to a predetermined order to synthesize and output an image of original
resolution.
The operational example of the first embodiment of the present invention thus constructed will now be described in detail with reference to the accompanying drawings.
First, an encoding process will be described.
A resolution of an original image (lo, i.e., an image of a currently- worked frame out of
moving pictures comprising a plurality of frames) is comprised of a two-dimensional
plane of M*N having a horizontal direction (x) and a vertical direction (y), as shown in
FIG. 3. The original image thus described is reduced and copied in plural numbers in
accordance with an M/m*N/n image which is an image to be finally outputted via the
image space separator (10). The reduced original image is set as a reference image (Ir)
while the copied plural images are set as sub-images (Is).
The sub-images (Is) are erased of their pixels of horizontal and vertical resolutions
according to a predetermined order based on the reference image (Ir) via the image
space separator (10) as shown in FIG. 4, wherein the pixel erasing order is alternated
for each sub-image such that pixels erased at each sub-image are alternatively
processed.
For example, a first sub-image (Is) erases the even-numbered pixels at the horizontal
resolution and the even- numbered pixels at the vertical resolution, and then, the next sub-image (Is) erases the odd-numbered pixels at the horizontal resolution and the even-numbered pixels at the vertical resolution, and then, the following sub-image (Is) erases the even-numbered pixels at the horizontal resolution and the odd-numbered pixels at the vertical resolution, and then, the next following sub-image (Is) erases odd- numbered pixels at the horizontal resolution and the odd-numbered pixels at the
vertical resolution.
The reference image (Ir) and the sub-images (Is) thus processed are arranged in plural
frames along the time axis by the image frame generator (20). In other words, the
frame of the reference image (Ir) is added by the frames of the plurality of sub-images
(Is) to thereby exceed the number of the original frame of the moving pictures (for
example, approximately 29.97 frames per second under NTSC broadcasting
regulation).
Referring now to FIGS. 5 and 6, the sub-images (Is) are differentiated from the
reference image (Ir) via the sub-image subtracter (40) to thereby remove the spatial
correlation on the two-dimensional plane having the M*N frequency via the differentiation thus described.
The correlation between the reference image (Ir) and the sub-images (Is) thus differentiated is greater than that between two timely neighboring frames (original image frames). This is because components having time lag do not exist between the reference image (Ir) and the sub-images (Is) but there is only a frequency difference within the same space in the present invention, while time lag between moving components exists in the case of an image taking up a differentiation between
neighboring frames via a general compression method.
Therefore, an image data of sub-images taking up the differentiation according to the
present invention can have a higher correlation to thereby enable to heighten a
compression rate while maintaining the same picture quality than that of a case of an
image taking up a differentiation between images of neighboring frames according to
the general compression method.
For example, even the resolution of the sub-image (Is) is more compressed by
approximately a quarter or above than that of the reference image (Ir) by the sub-image
resolution slicer (70), an image of less picture quality deterioration can be embodied
compared with the general compression method.
Next, each pixel value of respective differentiated sub-images (Is) is detected by the sub-image level detector (50), and the sub-image slicer (60) performs a first slicing process where a zero pixel value is reallocated to any pixel of the sub-image (Is) whose pixel value lies between a predetermined upper and lower limit based on a zero pixel value. In other words, although the differentiated pixel having a pixel value between the upper and lower limits appropriately predetermined according to the brightness or darkness of the relevant image sections is sliced, a human's eye is unable
to easily discern a difference before and after the slicing).
Particularly, because a human cannot distinguish the difference between an original
image and a sliced image in a very small value, there would be no problem if all image
data within approximately 8 pixel values out of plus (+) and minus (-) components in
the differentiated values are treated identically as the reference image data.
Consequently, values lying between the upper and lower limit out of pixel values of the
sliced sub-images (Is) can be substituted by zero values because a human's eye cannot
distinguish the difference. The slicer (60) performs a second slicing process for the
firstly sliced sub-images (Is) thus explained above where the firstly sliced sub-images
(Is) are compared with the reference image (Ir) in terms of pixel values, and the pixel
values of the firstly sliced sub-images (Is) within the scope of the predetermined upper
and lower limits based on the relevant pixel values of the reference image (Ir) are substituted by zero values. Most of the pixels of each sub-image (Is) now come to have zero pixel values after the second slicing process as shown in FIG. 7.
Data of the sub-images (Is) thus reconstructed through the first and second slicing processes may be deleted because it does not matter even if the pixels of such sliced sub-images (Is) having zero pixel values are substituted by those of the reference
image (Ir) during decoding.
In this case, the horizontal and vertical resolutions of sub-images (Is) other than the
reference image (Ir) can be reduced by approximately 1/2 or above. This is because the
scope of data size reducible without any loss to an original image signal is
approximately 1/2 when differentiation is made from neighboring frames. Data
approximately half the size has large overlapped portions with neighboring frames.
Meanwhile, it is preferred that an extra weight be added to the upper and lower limit
values of the horizontal resolution instead of those of the vertical resolution during the
first and second slicing processes. This is because the horizontal resolution responds
more sensitively than the vertical resolution to a human's eyes. It is also preferred that
the data of horizontal resolution be given with such upper and lower limit values that slicing width can be smaller than that of the vertical resolution to thereby increase the accuracy of the data.
Meanwhile, it is also preferred that the pixel values of the vertical resolution be placed further near to zero by widening the slicing scope, and in this case, the sub-images (Is) having data of vertical resolution have a relatively inferior fidelity such that sub- images (Is) having mostly data of vertical resolution are removed.
Furthermore, sub-images (Is) having data of horizontal resolution are also removed of
lots of data during differentiation with the reference image (Ir), which means that
spatial and timely overlap in a screen is effectively removed as much as the removed
sub-images (Is) of the horizontal resolution.
Consequently, the present invention can further increase the compression efficiency
compared with that of the conventional block compression techniques because
frequency correlation of the overall images is effectively erased, resulting in a
remarkable improvement from the decreased picture quality.
Next, each sliced sub-image (Is) is reduced in overall resolution thereof because areas
of vertical and horizontal resolutions previously removed of pixels at the image space
separator (10) step are deleted and coupled by the sub-image resolution slicer (70), and
the resolution at this time is preferred to be reduced to a size of even fraction (not odd
fraction), for example, approximately 1/4 of the reference image (Ir).
Referring now to FIG. 8, the reference image (Ir) and the reduced sub-images (Is) are arranged on a two-dimensional space by the image synthesizer (80) to perform a time compression of being synthesized of a single frame. For example, the reference image
(Ir) is allocated to a left or a right side of the image while sub-images (Is) are allocated
to a remaining space of the reference image (Ir).
The sub-images (Is) synthesized along with the reference image (Ir) have been already
removed of lots of data having large overlap via differentiation with the reference
image (Ir), and compressed by the image synthesizer (80) via known Run Length
Coding (RLC) method or the like for storage and outputting.
In other words, an image composed of images synthesized with the reference image
(Ir) reduced to as much as a predetermined fraction of the original image (lo), the sub-
images (Is) and an image having a differentiated data between the sub-image (Is) and
the reference image (Ir) is finally compressed and outputted.
As mentioned, although the finally-outputted compressed image is reduced by as much
as the predetermined fraction compared with the original image (lo), the control of the compression rate depends on the accuracy of time-compressed sub-image (Is), i.e., how much data of a sub-image having small loss rate of picture quality out of the sub- images (Is) is reduced or removed. In this case, the loss of data occurs in accordance with the order of less importance with regard to the fidelity of the original image. However, deterioration of picture quality of an image is not that severe compared with
that of the existing compression technique.
Next, the decoding process will be explained.
Referring to FIG. 9, the compressed image outputted via the image synthesizer (80) of
the encoding means is divided into a reference image (Ir) and sub-images (Is) by the
inverse image separator (110) and arranged along the time axis. Each sub-image (Is)
separated from the inverse image separator (110) is expanded to the same resolution as
that of the reference image (Ir) by the sub-image resolution expander (130) (see FIG.
10). The resolution expansion of the sub-images (Is) is conducted by a pixel
interpolation method for increasing the number of pixels by copying neighboring
pixels. Most of the sub-images (Is) having the horizontal resolution are restored to
original images, but some of the sub-images having vertical resolution include frames
that are not restored.
As exemplified in FIG. 11, the sub-image resolution expander (130) applies to a lost pixel (op) of the sub-image (Is), an average (A) of a pixel value (f) of a frame thereof, a pixel value (f-1) relative to the same position of a previous frame of the current frame and a pixel value (f+1) relative to the same position of a next frame of the current frame to restore the loss. However, if the average (A) is smaller than f/2, (f- 1 )/2
or (f+l)/2, "f is compared with "f-1" and "f+1", and the smallest of the three is
substituted into the pixel (op).
Each sub-image (Is) expanded by the sub-image resolution expander (130) is
compared with the reference image (Ir) via the three-dimensional comparator (140) to
detect a differentiated status, i.e., the same and different pixels between the sub-images
and the reference image.
The three-dimensional frame adder (150) adds to each sub-image a value differentiated
from the reference image (Ir) in response to the differentiated status detected by the
three-dimensional frame comparator (140) to restore the image data of each sub-image,
and the restored sub-images (Is) are buffered by the three-dimensional frame
synthesizer (160) for image synthesis on a two-dimensional plane.
Referring now to FIG. 12, the respective sub-images (Is) inputted to the image synthesizer (170) after buffering at the three-dimensional frame synthesizer (160) are synthesized along with the reference image (Ir) into an image of original resolution at the image synthesizer (170) by alternatively and insertedly coupling the pixels of horizontal and vertical resolutions in accordance with a predetermined order onto a single frame of a two-dimensional plane, and then outputted.
There is an advantage in the first embodiment of the present invention thus explained
in that a space of an original image is reduced to a predetermined fraction for an
establishment of a reference image, and a plurality of sub-images having timely
continuity are copied from the reference image and are allocated, and in order to
remove the correlation within a space, differentiations are mutually performed and data
imperceptible to a human's visual characteristic is removed to compress data of overall
images, such that deterioration of picture quality resulting from a data discontinuity at
a borderline generated from the conventional block compression method can be
markedly attenuated to bring about an improvement of visual characteristics. In other
words, the visual characteristic is improved to allow the data to be compressed as
much such that picture quality can be further improved at the same compression rate.
[SECOND EMBODIMENT]
An apparatus for implementing a second embodiment of the present invention includes encoding means and decoding means.
As illustrated in FIG. 13, the encoding means further includes an image space separator (210), an image frame generator (220), a reference image resolution slicer (230), a sub-
image subtracter (240), a sub-image level detector (250), a sub-image slicer (260), a
sub image time compressor (270), a sub-image space converter (280) and an image
synthesizer (290).
The image space separator (210) performs a processing work (hereinafter referred to as
alternative erasion) where an original image is reduced by a predetermined fraction to
conform to an output resolution whereby the reduced image is established as a
reference image, and the reference image is plurally copied and the plurally-copied
images are established as sub-images, wherein pixels are erased from horizontal and
vertical resolutions of each sub-image according to a predetermined order while the
pixel erasing order is alternated for each sub-image such that the pixels being erased at
each sub-image can be alternated..
The image frame generator (220) arranges the reference image and the sub-images
processed at the image space separator (210) along a time axis, and the reference image resolution slicer (230) separates the reference image from the sub-images for transmission to the image synthesizer (290)
The sub-image subtracter (240) differentiates each sub-image from the reference image, while the sub-image level detector (250) detects a pixel value of each differentiated
sub-image.
The sub-image slicer (260) compares pixel values of each differentiated sub-images to
erase pixel values each having a difference within a predetermined scope, and again compares a pixel value of each sub-image with that of the reference image to erase a pixel value having a difference within a predetermined scope.
The sub-image time compressor (270) pairs off with neighboring frames relative to each sub-image to time-compress and synthesizes the paired-off frames. The sub image
• space converter (280) deletes, for each sub-image, areas of the vertical and horizontal resolutions whose pixels were erased in the alternative erasion and couples the remaining non-deleted areas to thereby reduce the overall resolutions of each sub- image.
The image synthesizer (290) arranges the reference image and the reduced sub-images on a three-dimensional space along a time axis, and compresses the same for output.
Next, the decoding means for conducting the second embodiment of the present invention includes, as shown in FIG. 14, an inverse image separator (310), a reference image resolution expander (320), a sub-image space converter (330), a sub-image
frame detector (340), a three-dimensional frame synthesizer (350), a sub-image adder
(360), a three-dimensional image generator (370) and an image synthesizer (380).
The inverse image separator (310) separates the reference image and the sub-images
from the compressed images outputted from the image synthesizer (290) of the afore¬
mentioned encoding means and arranges same on a time axis.
The reference image resolution expander (320) transmits to the three-dimensional
image generator (370) the reference image separated from the inverse image separator
(310). The sub-image resolution expander (330) expands the resolution bf each sub-
image separated from the inverse image separator (310) to the same resolution of the
reference image for interpolation.
The three-dimensional frame comparator (350) detects a level of pixel value of each
expansively interpolated sub-image and compares the pixels of the sub-images with
the reference image, and detects a differentiated status, that is, different pixels between the sub-images and the reference image.
The three-dimensional frame synthesizer (350) compares the correlation of pixels between the current frame and each of neighboring previous and next frames per
horizontal and vertical resolutions for omitted data of each sub-images discovered by
the result detected by the sub-image frame detector (340), and takes a pixel of a higher
correlation for correction.
The sub-image frame adder (360) adds to each sub-image a value differentiated from
the reference image in response to the differentiated status detected by the sub-image
frame detector (340), and the three-dimensional image generator (370) performs a
buffering process for the reference image and each sub-image for image synthesis.
The image synthesizer (380) alternatively and insertedly couples the pixels of vertical
and horizontal resolutions of the reference image and the sub-images on a single frame in response to a predetermined order to synthesize and output an image of original resolution.
The operational example of the second embodiment of the present invention thus constructed will now be described in detail with reference to the accompanying drawings.
First, an encoding process will be described.
A resolution of an original image (lo, i.e., an image of a currently-worked frame out of
moving pictures comprising a plurality of frames) is comprised of a two-dimensional
plane of M*N having a horizontal direction (x) and a vertical direction (y), as shown in
FIG. 15. The original image thus described is reduced and copied in plural numbers in
accordance with an M/m*N/n image which is an image of a final output end via the
image space separator (210). The reduced original image is set as a reference image
(Ir) while the copied plural images are set as sub-images (Is).
The sub-images (Is) are erased of their pixels of horizontal and vertical resolutions
according to a predetermined order based on the reference image (Ir) via the image
space separator (210), wherein the pixel erasing order is alternated for each sub-image
such that pixels erased at each sub-image are alternatively processed.
For example, a first sub-image (Is) erases the even-numbered pixels at the horizontal
resolution and the even-numbered pixels at the vertical resolution, and then, the next sub-image (Is) erases the odd-numbered pixels at the horizontal resolution and the even-numbered pixels at the vertical resolution, and then, the following sub-image (Is) erases the even-numbered pixels at the horizontal resolution and the odd-numbered pixels at the vertical resolution, and then, the next following sub-image (Is) erases the odd-numbered pixels at the horizontal resolution and the odd-numbered pixels at the
vertical resolution.
The reference image (Ir) and the sub-images (Is) thus processed are arranged in plural
frames along the time axis by the image frame generator (220). In other words, the
frame of the reference image (Ir) is added by the frames of the plurality of sub-images
(Is) to thereby exceed the number of the original frame of the moving pictures (for
example, approximately 29.97 frames per second under NTSC broadcasting
regulation).
The sub-images (Is) thus described are differentiated from the reference image (Ir) via
the sub-image subtracter (240) to thereby remove the spatial correlation on the two-
dimensional plane having the M*N frequency via the differentiation thus described
(see FIGS. 5 and 6 of the first embodiment).
The correlation between the reference image (Ir) and the sub-images (Is) thus differentiated is greater than that between two timely neighboring frames (original image frames). This is because components having time lag do not exist) between the reference image (Ir) and the sub-images (Is) but there is only a frequency difference within the same space in the present invention, while time lag between moving components exists in a case of an image taking up a differentiation between
neighboring frames via a general compression method.
Therefore, an image data of sub-images taking up the differentiation according to the
present invention can have a higher correlation to thereby enable to heighten a
compression rate while maintaining the same picture quality as an image taking up a
differentiation between images of neighboring frames according to the general
compression method.
For example, even the resolution of the sub-image (Is) is more compressed by
approximately a quarter or above than that of the reference image (Ir) by the sub-image
space converter (280), an image of less picture quality deterioration can be embodied
compared with the general compression method.
Next, each pixel value of respective differentiated sub-images (Is) is detected by the
sub-image level detector (250), and the sub-image slicer (260) performs a first slicing
process where a zero pixel value is reallocated to any pixel of the sub-image (Is)
whose pixel value lies between a predetermined upper and lower limit based on a zero
pixel value). In other words, although the differentiated pixel having a pixel value
between the upper and lower limits appropriately predetermined according to the
brightness or darkness of the relevant image sections is sliced, a human's eye is unable
to easily discern a difference before and after the slicing.
Particularly, because a human cannot distinguish the difference between an original
image and a sliced image in a very small value, there would be no problem if all image
data within approximately 8 pixel values out of plus (+) and minus (-) components in
the differentiated values are treated identically as the reference image.
Consequently, values lying between the upper and lower limit out of pixel values of the
sliced sub-images (Is) can be substituted to zero values because a human's eye cannot
distinguish the difference. The slicer (60) performs a second slicing process for the
firstly sliced sub-images (Is) thus explained above where the firstly sliced sub-images
(Is) are compared with the reference image (Ir) in terms of pixel values, and the pixel
values of the firstly sliced sub-images (Is) within the scope of the predetermined upper
and lower limits based on the relevant pixel values of the reference image (Ir) are
substituted by zero values. Most of the pixels of each sub-image (Is) now come to have
zero pixel values after the second slicing process (see FIG. 7 of the first embodiment)
Data of the sub-images (Is) thus reconstructed through the first and second slicing processes may be deleted because it does not matter even if the pixels of such sliced sub-images (Is) having zero pixel values are substituted by those of the reference
image (Ir) during decoding.
In this case, the horizontal and vertical resolutions of sub-images (Is) other than the
reference image (Ir) can be reduced by approximately 1/2 or above. This is because the
scope of data size reducible without any loss to an original image signal is
approximately 1/2 when differentiation is made from neighboring frames. Data
approximately half the size has large overlapped portions with neighboring frames.
Meanwhile, it is preferred that an extra weight be added to the upper and lower limit
values of the horizontal resolution instead of those of the vertical resolution during the
first and second slicing processes. This is because the horizontal resolution responds
more sensitively than the vertical resolution to a human's eyes. It is also preferred that
the data of horizontal resolution be given with such upper and lower limit values that
slicing width can be smaller than that of the vertical resolution to thereby increase the
accuracy of the data.
Meanwhile, it is also preferred that the pixel values of the vertical resolution be placed further near to zero by widening the slicing scope, and in this case, the sub-images (Is) having data of vertical resolution have a relatively inferior fidelity such that sub- images (Is) having mostly data of vertical resolution are removed.
Furthermore, sub-images (Is) having data of horizontal resolution are also removed of
lots of data during differentiation with the reference image (Ir), which means that
spatial and timely overlap in a screen is effectively removed as much as possible.
Consequently, the present invention can further increase the compression efficiency
compared with the conventional block compression techniques because frequency
correlation of the overall images is effectively erased, resulting in a remarkable
improvement from the decreased picture quality.
Referring now to FIG. 16, the respective sliced sub-images pair off with timely
neighboring frames via the sub-image time compressor (270) and an average (A) is
calculated from pixel values of paired-off sub-images (Is) and is substituted into the
synthesized pixel values to generate a new synthesized sub-image (Is).
If the average (A) is less than a half value of the pixel value (f) of the current frame,
less than a half value of the pixel value (f-1) relative to the same position of the previous frame or less than a half value of the pixel value (f+1) relative to the same position of the next frame, a representative value extraction technique is applied for correction where "f ', "f-1" and "f+1" are compared thereamong and the smallest value
thereof is substituted into the synthesized pixel value (see FIG. 11 of the first
embodiment). At this time, if the neighboring plural sub-images are synthesized into a
single frame, there is a fear of generating a negative (-) component, such that an
appropriate value is added to remove the negative (-) component (for example, adding
a value of approximately 64).
The reason for synthesizing the sub-images along the time axis is to satisfy a number
of frames when the number of sub-images (Is) exceeds the number of frames of
outputted images in cases where the number of frames of images outputted via
encoding is fixed. In other words, the number of sub-images (Is) is increased more
than the number of frames of images outputted at the step of the image space separator
(210) to increase the accuracy of images, and the number of frames is decreased by the
time compression to meet the number of frames of outputted images.
Next, each time-compressed sub-image (Is) is reduced in overall resolution thereof
because areas of vertical and horizontal resolutions previously removed of pixels at the image space separator (210) step are deleted and coupled by the sub-image space converter (280), and the resolution at this time is preferred to be reduced to a size of even fraction (not odd fraction), for example, approximately 1/4 of the reference image (Ir).
Referring now to FIG. 17, the reference image (Ir) and the reduced sub-images (Is) are
arranged on a three-dimensional space along a time axis by the image synthesizer
(290).
The sub-images (Is) synthesized along with the reference image (Ir) have been already
removed of lots of data having large overlap via differentiation with the reference
image (Ir), and compressed by the image synthesizer (80) via known Run Length
Coding (RLC) method or the like for storage and outputting.
In other words, an image composed of images synthesized with the reference image
(Ir) reduced to as much as a predetermined fraction of the original image (lo), the sub-
images (Is) and an image having a differentiated data between the sub-image (Is) and
the reference image (Ir) is finally compressed and outputted.
As mentioned, although the finally-outputted compressed image is reduced by as much
as the predetermined fraction compared with the original image (lo), the control of the
compression rate depends on the accuracy of time-compressed sub-image (Is), i.e.,
how much data of a sub-image having small loss rate of picture quality out of the sub- images (Is) is reduced or removed. In this case, more data is lost in an image having
less importance. However, deterioration of picture quality of an image is not that
severe compared with that of the existing compression technique.
Next, the decoding process will be explained.
Referring to FIG. 18, the compressed image outputted via the image synthesizer (290)
of the encoding means is divided into a reference image (Ir) and sub-images (Is) by the
inverse image separator (310) and arranged along the time axis. Each sub-image (Is)
separated from the inverse image separator (310) is expanded to the same resolution as
that of the reference image (Ir) by the sub-image space converter (330) (see FIG. 10 of
the first embodiment). The resolution expansion of the sub-images (Is) is conducted by
a pixel interpolation method for increasing the number of pixels by copying
neighboring pixels.
Each sub-images (Is) expanded by the sub-image space converter (330) is detected of levels of each pixel value via the sub-image frame detector (340) for comparison with the reference image (Ir), from which a pixel having a difference from the same pixel between the sub-images and the reference image is detected, that is, a differentiated status is detected. When pixel values of each sub-image are detected by the sub-image space converter (330), there may be data that are not restored.
In order to restore the data, the three-dimensional frame synthesizer (350) compares
correlation of pixels between the current frame and each of neighboring previous and
next frames per horizontal and vertical resolutions for omitted data of each sub-images,
and takes a pixel of a higher correlation for restoration.
In other words, the current frame is compared with timely neighboring previous and
next frames, and a determination is made as to which of the relevant portions of the
neighboring frames has a higher correlation with pixels of vertical or horizontal
resolution omitted from the current frame, and the determination therefrom uses
information provided from the differentiated erasion process conducted by the image
space separator (210) during encoding and resolution reducing process by the sub-
image space converter (280).
For example, if the reduction of resolution of sub-images (Is) of a current frame has been affected by deletion of even pixels of horizontal resolution and odd pixels of vertical resolution during encoding, the former may be restored by taking the even pixels of horizontal resolution from the previous frame, and the latter may be restored by taking the odd pixels of vertical resolution from the next frame.
Each sub-image (Is) is added by a differentiated value from the reference image (Ir) by
the sub-image adder (360) in response to the differentiated status detected by the sub-
image frame detector (340) to restore the image data of each sub-image (Is), and the
reference image (Ir) transmitted via the reference image resolution expander (320) and
the respective restored sub-images (Is) are buffered by the three-dimensional image
generator (370) for image synthesis on the two-dimensional plane.
The reference image (Ir) and each sub-image (Ir) buffered by the three-dimensional
image generator (370) and inputted to the image synthesizer (380) are synthesized into
an image of original resolution by alternatively and insertedly coupling the pixels of
horizontal and vertical resolutions in accordance with a predetermined order onto a
single frame of a two-dimensional plane, and then outputted (see FIG. 12 of the first
embodiment).
There is an advantage in the second embodiment of the present invention thus explained in that a space of an original image is reduced to a predetermined fraction for an establishment of a reference image, and a plurality of sub-images having timely continuity are copied from the reference image and are allocated, and in order to remove the correlation within a space, differentiations are mutually performed and data imperceptible due to a human's visual characteristic is removed to compress data of
overall images, such that deterioration of picture quality resulting from data
discontinuity at a borderline generated from the conventional block compression
method can be markedly attenuated to bring about an improvement of a human's visual
characteristic. In other words, the visual characteristic is improved to allow the data to
be compressed as much as possible, so that picture quality can be further improved at
the same compression rate.
[THIRD EMBODIMENT]
An apparatus for implementing a third embodiment of the present invention includes
encoding means and decoding means. As shown in FIG.19, the encoding means
includes an image space separator (410), a reference image resolution slicer (420), a
three-dimensional image frame generator (430), a sub-image subtracter (440), a sub-
image time compressor (450) and a three-dimensional frame synthesizer (460).
The image space separator (410) performs a processing work (hereinafter referred to as alternative erasion) where an original image of the current frame is reduced by a predetermined fraction to conform to an output resolution whereby the reduced image is established as a reference image, and the reference image is plurally copied and the plurally-copied images are established as sub-images, wherein pixels are erased from
horizontal and vertical resolutions of each sub-image according to a predetermined
order while the pixel erasing order is alternated for each sub-image so that the pixels
being erased at each sub-image can be alternated.
The reference image resolution slicer (420) separates the reference image from the
sub-images for transmission to the three-dimensional frame synthesizer (460). The
three-dimensional image frame generator (430) arranges the sub-images of a previous
frame, the current frame and a next frame processed by the image space separator
(410) on a time axis according to a time order.
The sub-image subtracter (440) differentiates the respective sub-images from the
reference image, and the sub-image time compressor (450) pairs off the neighboring
frames for the respective sub-images and time-compresses the same to be outputted.
The three-dimensional frame synthesizer (460) arranges the reference image and the
reduced sub-images on a three-dimensional space along a time axis for compression
and outputting.
Next, the decoding means for implementing the third embodiment of the present invention includes an inverse image separator (510), a reference image resolution expander (520), a three-dimensional image adder (530), a three-dimensional image
comparator (540) and a three-dimensional frame synthesizer (550).
The inverse image separator (510) separates the reference image and the sub-images
from the compressed images outputted from the three-dimensional frame synthesizer
(460) of the afore-mentioned encoding means and arranges same on a time axis.
The reference image resolution expander (520) transmits to the three-dimensional
frame synthesizer (550) the reference image separated from the inverse image
separator (510). The three-dimensional image adder (530) detects the differentiated
status of each sub-image to add the differentiated value of the reference image to the
respective sub-images, and the three-dimensional image comparator (540) detects data
differentiated relative to the respective sub-images.
The three-dimensional frame synthesizer (550) alternatively and insertedly couples the
pixels of horizontal and vertical resolutions of the reference image and the sub-images according to a predetermined order onto a single frame in response to the differentiated data detected by the three-dimensional image comparator (540) for synthesis to an image of original resolution and outputting.
The operational example of the third embodiment of the present invention thus
described will now be explained with reference to the accompanying drawings.
First, an encoding process will be described.
A resolution of an original image (lo, i.e., an image of a currently-worked frame out of
moving pictures comprising a plurality of frames) is comprised of a two-dimensional
plane of M*N having a horizontal direction (x) and a vertical direction (y), as shown in
FIG. 21. The original image thus described is reduced and copied in plural numbers in
accordance with an M/m*N/n image which is an image to be finally outputted via the
image space separator (210). The reduced original image is set as a reference image
(Ir) while the copied plural images are set as sub-images (Is).
At the same time, the image space separator (410) reduces the original image of a
previous and a next frame from the current frame by a predetermined fraction and the
reduced original image is copied in plural numbers, which are established as respective sub-images. For reference, the number of sub-images of the previous and next frames is preferably half the number of the sub-images (Is) of the current frame.
The respective sub-images (Is) of the previous frame, the current frame and the next frame are erased of their pixels of horizontal and vertical resolutions according to a
predetermined order based on the reference image (Ir) via the image space separator
(410), wherein the pixel erasing order is alternated for each sub-image so that pixels
erased at each sub-image are alternatively processed.
For example, a first sub-image (Is) erases the even-numbered pixels at the horizontal
resolution and the even-numbered pixels at the vertical resolution, and then, the next
sub-image (Is) erases the odd-numbered pixels at the horizontal resolution and the
even-numbered pixels at the vertical resolution, and then, the following sub-image (Is)
erases the even-numbered pixels at the horizontal resolution and the odd-numbered
pixels at the vertical resolution, and then, the next following sub-image (Is) erases odd-
numbered pixels at the horizontal resolution and the odd-numbered pixels at the
vertical resolution.
The reason for generating the sub-images (Is) of previous and next frames in addition
to the current frame is to allow the sub-images (Is) of the previous and next frame
adjacent to the current frame to have lots of pixel information having a small
difference from the current frame during the differentiation from the reference image
(Ir) while there is almost no time lag from the current frame, thereby enabling to compensate the pixel loss during the decoding process for prevention of deterioration of picture quality.
The reference image (Ir) and the sub-images (Is) thus processed are arranged in plural
frames along the time axis by the image frame generator (430). In other words, the
frame of the reference image (Ir) is added by the frames of the plurality of sub-images
(Is) to thereby exceed the number of the original frame of the moving pictures (for
example, approximately 29.97 frames per second under NTSC broadcasting
regulation).
The sub-images (Is) thus described are differentiated from the reference image (Ir) via
the sub-image subtracter (440) to thereby remove the spatial correlation on the two-
dimensional plane having the M*N frequency via the differentiation thus described
(see FIGS. 5 and 6 of the first embodiment).
Referring now to FIG. 22, the respectively differentiated sub-images pair off with timely neighboring frames via the sub-image time compressor (450) and an average (A) is calculated from pixel values of paired-off sub-images (Is) and is substituted into the synthesized pixel values to generate a new synthesized sub-image (Is).
If the average (A) is less than a half value of the pixel value (f) of the current frame, less than a half value of the pixel value (f-1) relative to the same position of the front
frame or less than a half value of the pixel value (f+1) relative to the same position of
the rear frame, a representative value extraction technique is applied for correction
where "f, "f-1" and "f+1" are compared thereamong and a smallest value thereof is
substituted into the synthesized pixel value. At this time, if the neighboring plural sub-
images are synthesized into a single frame, there is a fear of generating a negative (-)
component, such that an appropriate value is added to remove the negative (-)
component (for example, adding a value of approximately 64).
The reason for synthesizing the sub-images along the time axis is to satisfy a number
of frames when the number of sub-images (Is) exceeds the number of frames of
outputted images in cases where the number of frames of images outputted via
encoding is fixed. In other words, the number of sub-images (Is) is increased more
than the number of frames of images outputted at the step of the image space separator (410) to increase the accuracy of images, and the number of frames is decreased by the time compression to meet the number of frames of outputted images.
Referring now to FIG. 22, the time-compressed sub-images (Is) are arranged on a three-dimensional space along with the reference image (Ir) on the time axis by the three-dimensional frame synthesizer (460).
The sub-images (Is) arranged along with the reference image (Ir) have been already
removed of lots of data having large overlap via differentiation with the reference
image (Ir), such that the sub-images are compressed by the three-dimensional frame
synthesizer (460) via the known Run Length Coding (RLC) method or the like for
storage and outputting.
In other words, an image composed of images synthesized with the reference image
(Ir) reduced to as much as a predetermined fraction of the original image (lo), the sub-
images (Is) and an image having a differentiated data between the sub-image (Is) and
the reference image (Ir) is finally compressed and outputted.
As mentioned, although the finally-outputted compressed image is reduced by as much
as the predetermined fraction compared with the original image (lo), the control of the
compression rate depends on the accuracy of the time-compressed sub-image (Is), i.e.,
how much data of a sub-image having a small loss rate of picture quality out of the sub-images (Is) is reduced or removed. In this case, data loss occurs in accordance
with the order of importance with regard to the fidelity of the original image. However, deterioration of picture quality of an image is not that severe compared with that of the existing compression technique.
Next, the decoding process will be explained.
The compressed image outputted via the three-dimensional frame synthesizer (460) of
the encoding means is divided into a reference image (Ir) and sub-images (Is) by the
inverse image separator (510) and arranged along the time axis. Each sub-image (Is)
separated from the inverse image separator (510) is added by a differentiated value
from the reference image (Ir) at the three-dimensional image adder (530) to restore the
image data of the respective sub-images (Is) (see FIG. 24).
For each sub-image (Is) whose differentiated value was restored, the data differentiated
during the encoding process are detected by the three-dimensional image comparator
(540), and pixels of horizontal and vertical resolutions of the reference image and the
sub-images are alternatively and insertedly coupled by the three-dimensional frame
synthesizer (550) in accordance with the differentiated data detected by the three- dimensional comparator (540) in response to a predetermined order onto the single frame and then synthesized into an image of original resolution to be outputted ( see FIG. 25).
There is an advantage in the third embodiment of the present invention thus explained in that a space of an original image is reduced to a predetermined fraction for an
establishment of a reference image, and a plurality of sub-images having timely
continuity are copied from the reference image and are allocated, and in order to
remove the correlation within a space, differentiations are mutually performed and a
data imperceptible due to a human's visual characteristic is removed to compress data
of overall images, such that deterioration of picture quality resulting from a data
discontinuity at a borderline generated from the conventional block compression
method can be markedly attenuated to bring about an improvement of a human's visual
characteristic. In other words, the visual characteristic is improved to allow the data to
be compressed as much as possible, such that picture quality can be further improved
at the same compression rate.
The foregoing description of the preferred embodiment of the present invention has
been presented for the purpose of illustration and description. It is not intended to be
exhaustive or to limit the invention to the precise form disclosed, and modifications
and variations are possible in light of the above teachings or may be acquired from practice of the invention. It is intended that the scope of the invention be defined by the claims appended hereto, and their equivalents.