WO2017145752A1

WO2017145752A1 - Image processing system and image processing device

Info

Publication number: WO2017145752A1
Application number: PCT/JP2017/004512
Authority: WO
Inventors: 影山　昌広
Original assignee: 株式会社日立産業制御ソリューションズ
Priority date: 2016-02-24
Filing date: 2017-02-08
Publication date: 2017-08-31
Also published as: JP6516695B2; JP2017152902A

Abstract

The present invention is provided with: an image position shifting unit (103) for shifting the position of an inputted image; an image reduction unit (104) for reducing the size of the position-shifted image by cutting the number of pixels thereof; an image enlargement unit (301) which increases the number of pixels of a decoded image to enlarge the same, the decoded image being the reduced image which was encoded, sent via a communication network (107), and decoded; an image position shifting unit (302) for position-shifting the enlarged image so as to cancel out and negate the position shifting performed by the image position shifting unit (103); and a folding distortion reduction unit (303) for performing folding distortion reduction processing for reducing folding distortion of the image having been positioned-shifted by the image position shifting unit (302), by using information on another image having been position-shifted by the image position shifting unit (302).

Description

Image processing system and image processing apparatus

Import by reference

This application claims the priority of Japanese Patent Application No. 2016-033145, which was filed on February 24, 2016, and is incorporated herein by reference.

The present invention relates to an image processing system and an image processing apparatus that perform processing related to image data size and image quality.

Images (moving images and still images) taken by a camera, such as a videophone, a remote conference, or a network surveillance camera, are simultaneously transmitted via a communication network such as the Internet, an intranet, or a public network, and are received by an image receiving terminal. It is generally done to display.

On the other hand, the number of pixels of a camera used for imaging has increased with the advance of technology, and at present, for example, a full HD size image composed of horizontal 1920 pixels × vertical 1080 pixels is generally used. However, when an image is transmitted in an image transmission application that exceeds the limit of the transmission bandwidth of the communication network, the received image may deteriorate or freeze due to an application communication error or data delay. Sometimes. In addition, when the transmitted image data is recorded and stored on a medium such as a hard disk (magnetic disk) or an optical disk, if the data size is large, it may be possible to record only a short time on a medium with a limited recording capacity. In addition, there is a problem that the cost of the recording apparatus is increased due to the necessity of a large-capacity recording medium.

Therefore, in a system that realizes an application for transmitting and recording images, the image data size is reduced in accordance with the transmission band and recording capacity. For example, the number of pixels in the captured image frame is reduced to reduce the image size. In general, the data size is reduced by reducing the overall size and then encoding and compressing, and when the image is reproduced, the reduced image is enlarged and displayed.

However, there is a correlation between the encoded data size of the image and the image quality, and the image quality deteriorates as the encoded data size is reduced. Therefore, a technique for reducing the encoded data size while suppressing a decrease in image quality has been developed. As a technique related to image sharpening, for example, Patent Document 1 discloses two series of digital data. A technique is disclosed in which a wideband output signal exceeding the Nyquist frequency of an input signal is obtained by canceling aliasing distortion in a one-dimensional direction using the signal. In Non-Patent Document 1, a plurality of image frames are combined into one frame, and a high-frequency component exceeding the Nyquist frequency of the input image is restored by performing a successive approximation process with iterative calculation, and high-resolution output is performed. A technique for obtaining an image is disclosed.

Patent Document 1: Japanese Patent Laid-Open No. 10-280685

Non-patent document 1: S.M. C. Park, M.M. K. Park, and M.M. G. King, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, 2003.

By the way, as described above, a personal computer (hereinafter abbreviated as “PC”), a mobile terminal, etc. as means for realizing an application for transmitting, recording, and displaying an image of a video phone, a remote conference, a network surveillance camera, or the like. Is often used. In these terminals, signal processing is performed by performing software processing using a general-purpose CPU (Central Processing Unit), MPU (Micro Processing Unit), GPGPU (General Purpose Computing Computing on Graphics Processing Unit), DSP (Digital Signal Processor), etc. Even without using special hardware, it is possible to realize encoding, decoding, reduction, enlargement, sharpening, and the like of an image.

On the other hand, the super-resolution processing exemplified in the above-described prior art requires a large amount of calculation for processing compared to edge enhancement processing that only amplifies high-frequency components included in an image. For example, in a super-resolution technique for obtaining an output image by successive approximation processing using an input image of a plurality of frames as described in Non-Patent Document 1, a high-precision motion search in units of subpixels for each pixel is performed. The accompanying inter-frame alignment (registration), image enlargement, enlargement image reduction, difference detection between the reduced image and the input image, and output image correction processing must be repeated many times (iteration). In other words, in the conventional technology, the motion search and iteration for each pixel has a very large amount of calculation, so that it exceeds the processing limit of calculation resources such as CPU and memory, and frame dropping occurs that makes the motion of the image jerky. Or the entire process may stop, or user input from a mouse, keyboard, or the like may not be accepted.

The present invention has been made in view of the above, and provides an image processing system and an image processing apparatus capable of reducing the encoded data size while suppressing an increase in the amount of calculation related to image processing and a decrease in image quality. For the purpose.

To achieve the above object, the present invention provides a first image position shift unit that shifts an image position to any one of a plurality of predetermined shift positions for each of a plurality of temporally continuous images. An image reduction unit that reduces the number of pixels of the image that has been position shifted by the first image position shift unit, and an encoding unit that encodes the image reduced by the image reduction unit to generate an encoded image A decoding unit that decodes the encoded image transmitted via the communication network to generate a decoded image, and an image that is enlarged by increasing the number of pixels of the decoded image decoded by the decoding unit An enlargement unit, a second image position shift unit that shifts the position of the image enlarged by the image enlargement unit so as to cancel the position shift performed by the first image position shift unit, and the second image position Position shift at shift section The aliasing distortion of the image, and that a folded distortion reduction unit which performs aliasing distortion reduction processing for reducing by using information on the position shifted other images in the second image position shifting unit.

The encoded data size can be reduced while suppressing an increase in the amount of computation related to image processing and a decrease in image quality, and high-speed image transmission and image sharpening processing can be performed even with relatively inefficient calculation resources.

1 is a functional block diagram schematically showing an overall configuration of an image processing system according to a first embodiment. It is a figure which shows an example of an image position shift process roughly. It is a figure which shows schematically another example of an image position shift process. It is a functional block diagram which shows roughly an example of a structure of the image expansion and sharpening part of an image receiver. It is a functional block diagram which shows the structure of an image reduction part roughly. It is a functional block diagram which shows roughly the structure of the horizontal processing part and vertical processing part of an image reduction part. FIG. 5 is a diagram schematically illustrating an example of a configuration for realizing a function of image enlargement / clearing processing in an image enlargement / clearing unit illustrated in FIG. 4. It is a figure which shows typically another example of the structure which implement | achieves the function of the image expansion and sharpening process in the image expansion and sharpening part shown in FIG. It is a figure which shows typically another example of the structure which implement | achieves the function of the image expansion and sharpening process in the image expansion and sharpening part shown in FIG. It is a figure which shows typically another example of the structure which implement | achieves the function of the image expansion and sharpening process in the image expansion and sharpening part shown in FIG. It is a figure which shows the structure which carried out equivalent conversion of the structure of FIG. It is a block diagram which shows an example of a structure of the one-dimensional asymmetric filter which the image expansion and sharpening part shown in FIG. 11 has. It is a figure which shows the filter coefficient of the one-dimensional asymmetric filter in the structure of FIG. FIG. 10 is a block diagram illustrating an example of a configuration of a two-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG. 9. It is a block diagram which shows the other example of a structure of the image expansion and sharpening part in the image receiver of FIG. It is a block diagram which shows an example of a structure in the motion detection part which the image expansion and sharpening part of FIG. 15 has. It is a figure which shows the structural example in the case of synchronizing using the frame phase information generation part of an image transmitter, and the frame phase information acquisition part of an image receiver. It is a figure which shows the structural example in the case of synchronizing using the frame phase information generation part of an image receiver, and the frame phase information acquisition part of an image transmitter. It is a figure which shows the structural example in the case of synchronizing using the frame phase information estimation part of an image receiving part. It is explanatory drawing which shows an example of the operation | movement of the frame phase information estimation process of the frame phase information estimation part shown in FIG. It is a flowchart which shows the frame phase estimation process in a frame phase information estimation part. It is a figure which shows the mode of the arrangement | positioning of the pixel in a Bayer array color filter. It is a functional block diagram which shows typically the example of 1 structure of an image storage device. It is a figure explaining an example of the filter coefficient used in the image position shift function of FIG. It is a functional block diagram which shows roughly the example of 1 structure of the image transmission apparatus which concerns on 2nd Embodiment. It is a figure which shows an example of the menu display screen which the control part which an image receiving device or an image storage device has produces | generates and displays on a display part. It is a functional block diagram which shows roughly the example of 1 structure of the image transmission apparatus which concerns on 3rd Embodiment. It is a functional block diagram which shows roughly the example of 1 structure of the image receiver which concerns on 3rd Embodiment. It is a figure explaining an example of the filter coefficient used in the image position shift part of FIG. It is a flowchart which shows an example of the process in the image receiver which concerns on 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
A first embodiment of the present invention will be described with reference to FIGS.

First, an overview of image sharpening realized by the present embodiment will be described.
When the image signal is sampled, a frequency component higher than the Nyquist frequency that is uniquely determined according to the number of pixels constituting one frame of the image becomes aliasing distortion. In an image processing system to be described later (see FIG. 1 and the like later), when the image is reduced in the image reduction unit (104) of the image transmission apparatus (101), a low-pass filter (see FIG. 5 and the like later) is cut off. Setting the frequency lower than the Nyquist frequency can be expected to reduce the aliasing distortion, but on the other hand, the reduced image tends to be a blurred image because the high frequency component is attenuated. On the other hand, when the cut-off frequency of the low-pass filter is set higher than the Nyquist frequency, the reduced distortion component is mixed with the baseband component originally included in the pre-reduction image. Since the aliasing distortion appears to be moire (interference fringe) noise, the image quality may be greatly degraded.

Therefore, in the present embodiment, the image enlargement / clearing unit (111) of the image receiving device (109) performs aliasing distortion using a plurality of image frames when enlarging the reduced transmission image. By reducing the frequency, a baseband component having a wide band is extracted, and the image is sharpened. In order to reduce the aliasing distortion, the property that the phase of aliasing distortion that occurs when the sampling phase of the same input signal is changed and the sampling is changed is used. This property is used in interlaced scanning. For example, in a general 2: 1 interlace, at the time of imaging, an image is scanned by scanning every scanning line from the top, and a first field consisting of an image of only odd-numbered scanning lines and only an even-numbered image are scanned. It is divided into the second field consisting of and transmitted. At the time of display, conversion from interlaced scanning to progressive scanning is performed by signal processing, or the first field and the second field are synthesized and perceived in the brain by the afterimage effect of the human eye. , Getting clear frame images.

In addition, since only the concept of interlaced scanning can be converted into only the “divisor” of the number of captured pixels, that is, “1 / N” (where N is a positive integer), for example, As in the case of conversion from the full HD size (horizontal 1920 pixels × vertical 1080 pixels) to the D1 size (horizontal 704 pixels × vertical 480 pixels), the reduction to the “non-divisor” number of pixels cannot be performed.

Therefore, in the present embodiment, first, the image position shift unit (103) shifts the position of the image for each frame, and then the image reduction unit (104) reduces the image, thereby reducing the reduced image (transmission image). ) Intentionally change the phase of the aliasing distortion included. At this time, if the subject is stationary, the phase of the aliasing distortion can be uniquely determined from the image shift method (shift direction and shift amount). For example, if the position of an image of full HD size (horizontal 1920 pixels × vertical 1080 pixels) is shifted by 1 pixel in the horizontal direction, an image of D1 size (horizontal 704 pixels × vertical 480 pixels) after image reduction is 0 in the horizontal direction. .37 (= 704/1920) pixels shift, so the phase of the aliasing distortion rotates by 11π / 15 (= 2π × 704/1920) radians. Since the phase rotation amount is constant and can be determined in advance for the entire frame, the above-described motion search in units of subpixels becomes unnecessary. Thereafter, by reducing the aliasing distortion based on the operation principle described later, the image can be sharpened even when the number of pixels is reduced to a non-divisor.

Details of the present embodiment based on the above knowledge will be described below.

FIG. 1 is a functional block diagram schematically showing the overall configuration of the image processing system according to the present embodiment.

In FIG. 1, an image processing system includes an image transmission device (101), a communication network (107), an image storage device (108), an image reception device (109), an operation unit (114), an image reception device (109), and The display unit (113) is a schematic configuration.

The image transmission device (101) includes a camera (102) that captures an image (still image, moving image), an image position shift unit (103) that performs image position shift processing on an image captured by the camera (102), and an image position An image reduction unit (104) that performs image reduction processing on an image that has undergone shift processing, an encoding unit (105) that performs encoding processing on an image that has undergone image reduction processing, and an image via a communication network (107) Based on the command and data transmitted from the reception device (109), the image transmission device (101) including the camera (102), the image position shift unit (103), the image reduction unit (104), and the encoding unit (105). And a control unit (106) for controlling the entire operation. The image receiving device (109) performs the decoding process with the decoding unit (110) that performs decoding processing on the image transmitted from the image transmitting device (101) via the communication network (107). An image enlargement / clearing unit (111) that performs image enlargement / clarification processing on the image and a control unit (112) that controls the operation of the entire image receiving device (109) are provided.

In the image processing system according to the present embodiment, the transmission data size is reduced by reducing the number of pixels constituting one frame of the image in the image reduction unit (104) of the image transmission device (101), and the image reception device. By enlarging the image in the image enlarging / clearing unit (111) included in (109), a clear image with less blur can be displayed. It should be noted that the image having the third pixel number output from the image receiving device (109) is further enlarged or reduced using an image enlargement unit or image reduction unit (not shown) to obtain an image having the fourth pixel number (not shown). You may comprise so that it may display on a display part (113) after converting.

The camera (102) includes, for example, an imaging unit (not shown) composed of a photoelectric conversion element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) and a lens, signal level adjustment, contrast adjustment, brightness adjustment, and white balance. An image (still image, moving image) captured by the camera (102) is sent to a subsequent image position shift unit (103). In the following description, the number of pixels of an image is represented by a set of the number of pixels in the horizontal direction (the number of horizontal pixels) and the number of pixels in the vertical direction (the number of vertical pixels). In the following description, the frame is assumed to be composed of the first number of pixels.

The image position shift unit (103) performs image position shift processing that shifts (shifts) the position of the image obtained by the camera (102) vertically and horizontally in units of integer pixels. Here, the image position shift processing in the image position shift unit (103) will be described with reference to FIGS.

FIG. 2 is a diagram schematically showing an example of the image position shift process.

In FIG. 2, an image in which three persons are subjects is schematically shown, the broken line is the original image frame (200) taken by the camera (102), and the solid line is shifted by image position shift processing. The obtained image frames (201) to (204) are shown. Moreover, the arrow has shown the state transition for every frame time (for example, 1/30 second). In FIG. 2, an image position shift process for shifting an image at a cycle of 4 frames is performed, which is referred to as “4-frame type image position shift”.

In the state of the image frame (201) after the image position shift process, the position of the original image frame (200) is shifted to the upper left, and predetermined pixel values (for example, pixel value representing black = 0) are set at the right end and the lower end of the image. ) Is inserted. Similarly, in the state of the image frame (202), the position of the original image frame (200) is shifted to the upper right and black (pixel value = 0) is inserted at the left end and the lower end of the image, and the image frame (203) In the state, the position of the original image frame (200) is shifted to the lower right and black (pixel value = 0) is inserted at the left end and the upper end of the image. In the state of the image frame (204), the original image frame ( 200) is shifted to the lower left, and black (pixel value = 0) is inserted at the right and upper ends of the image. That is, in the 4-frame type image position shift, the image position shift process is performed by performing state transition in the order of the image frames (201) to (204) every frame time. The shift direction and the shift amount in the image position shift unit (103) are controlled by the control unit (106).

FIG. 3 is a diagram schematically showing another example of the image position shift process.

In FIG. 3, as in FIG. 2, the broken line indicates the original image frame (200) taken by the camera (102), and the solid line indicates the image frames (205) and (205) shifted by the image position shifting process. ing. Moreover, the arrow has shown the state transition for every frame time (for example, 1/30 second). In FIG. 3, an image position shift process for shifting an image at a cycle of 2 frames is performed, which is referred to as “2-frame type image position shift”.

In the state of the image frame (205) after the image position shift processing, the position of the original image frame (200) is shifted to the left and black (pixel value = 0) is inserted at the right end of the image. Similarly, in the state of the image frame (206), the position of the original image frame (200) is shifted to the right, and black (pixel value = 0) is inserted at the left end of the image. That is, in the two-frame type image position shift, the image position shift process is performed by performing state transition in the order of the image frames (205) and (206) every frame time.

Note that the image position shifting process is not limited to the examples shown in FIGS. For example, in the 4-frame position shift shown in FIG. 2, the order of “image frame (201) → image frame (204) → image frame (203) → image frame (202) → upper left (201) →. Position shift or “image frame (201) → image frame (202) → image frame (204) → image frame (203) → image frame (201) →... (201) → Image frame (204) → Image frame (202) → Image frame (203) → Image frame (201) →..., Or position shift in a random (random) order You may do it. Further, for example, the position of the image frame (200) is set to one state of position shift, and “image frame (200) → position shift to right side → position shift to lower right → position shift to lower side → image frame (200) → · .. ”may be used to shift the position.

Similarly, for example, in the two-frame type position shift shown in FIG. 3, in the state of the image frame (205), the position is not shifted (that is, the position is fixed as the image frame (200)). The position may be shifted as follows: (200) → image frame (206) → image frame (200) →.

Further, image position shift processing including position shift in the vertical direction and the oblique direction (not shown) may be performed.

Furthermore, it is not limited to those that perform position shift with a 4-frame cycle, such as a 4-frame position shift, or those that perform position shift with a 2-frame cycle, such as a 2-frame position shift. A position shift of “6 frames cycle” combining position shift of 3 frames in the horizontal direction and position shift of 2 frames in the vertical direction, or position shift of 3 frames in the horizontal direction Further, a position shift of “9 frame periods” combining the position shift of 3 frames in the vertical direction may be performed.

Note that, in the image enlargement and sharpening process in the image enlargement and sharpening unit (111), which will be described in detail later, the image sharpening state changes according to the direction of the position shift by the image position shift process. For example, when the “4-frame type position shift” shown in FIG. 2 is performed, the resolution (clearness) in the horizontal and vertical directions of the image after the image enlargement and sharpening processing is improved, as shown in FIG. When “two-frame position shift” is performed, only the resolution (clearness) in the horizontal direction is improved. In other words, the resolution in a desired direction can be improved by performing an image position shift process that shifts the image frame in the direction in which the resolution is desired to be improved.

FIG. 5 is a functional block diagram schematically showing the configuration of the image reduction unit. FIG. 6 is a functional block diagram schematically illustrating the configuration of the horizontal processing unit and the vertical processing unit of the image reduction unit.

In FIG. 5, an image reduction unit (104) performs an image reduction process for converting the number of pixels constituting one frame of an image so as to be a second number of pixels smaller than the first number of pixels. This configuration is also referred to as an interpolation filter, and a horizontal processing unit (401-H) that performs horizontal image reduction processing on an image input from the image position shift unit (103) and vertical image reduction processing. And a vertical processing unit (401-V). Note that the processing order for the input image may be switched between the horizontal processing unit (401-H) and the vertical processing unit (401-V).

In FIG. 6, the horizontal processing unit (401-H) and the vertical processing unit (401-V) can be realized with the same configuration, and a pixel insertion unit (402) that performs m-times upsampling, A low-pass filter (403) for reducing and a pixel thinning unit (404) for performing 1 / n downsampling are provided.

In the horizontal processing unit (401-H) and the vertical processing unit (401-V), m-times upsampling (zero insertion) is performed on the input image by the pixel insertion unit (402), and then the low-pass filter (403 ) To remove or reduce unnecessary high-frequency components, and 1 / n downsampling is performed by the pixel thinning unit (404), so that the number of pixels in the horizontal direction (or vertical direction) of the image is m / n times (where m , N can be increased or decreased to a positive integer). Note that image enlargement processing by the image enlargement unit (301), which will be described later, can be realized with the same configuration by setting the constants m and n of the image reduction unit 104.

That is, for example, when considering the full HD size (horizontal 1920 pixels × vertical 1080 pixels) and D1 size (horizontal 704 pixels × vertical 480 pixels) as the number of pixels constituting the image frame, m = 11 and n = 30 If the setting is used, the horizontal reduction (1920 pixels → 704 pixels) in the image reduction unit (104) can be realized. If m = 2 and n = 1 are set, the horizontal reduction in the image enlargement unit (301) is realized. Direction expansion (704 pixels → 1408 pixels) can be realized. The same applies to reduction / enlargement in the vertical direction. Note that the number of output pixels (second pixel number) in the image reduction unit (104) is controlled by setting m and n by the control unit.

The low-pass filter (403) is multiplied by a window function (such as a Hanning window) by a sinc function (= sin (x) / x) obtained by inverse Fourier transform of an ideal frequency characteristic having a Nyquist frequency as a cutoff frequency. Filter coefficients are used. Further, when the image position shift with decimal pixel accuracy is performed, the center position of the sinc function is moved by t to be sin (xt) / (xt). If it is assumed that the processing for reducing the aliasing distortion is performed in the subsequent processing, it is possible to set the cutoff frequency of each low-pass filter (403) in the horizontal direction and the vertical direction to a frequency higher than the Nyquist frequency. It is done. Further, an operation equivalent to the configuration shown in FIG. 6 can be realized by a polyphase filter (polyphase filter).

The encoding unit (105) encodes and compresses the image input from the image reduction unit (104), and transmits the image to the image reception device (109) and the image storage device (108) via the communication network 107. is there. Note that the processing configuration for connecting the image transmission device (101) or the image reception device (109) to the communication network (107) uses a general technique and is not shown. The decoding unit 110 of the image receiving device (109) performs a decoding process according to the encoding method used by the encoding unit (105).

Examples of encoding methods used in the encoding unit 105 include MPEG (Moving Picture Expert Group) -1, MPEG-2, MPEG-4, H.264, and the like. H.264, H.C. H.265, VC-1, JPEG (Joint Photographic Experts Group), Motion JPEG, JPEG-2000, or other standardized encoding may be performed, or non-standard encoding may be performed. The encoding method in the encoding unit (105) is controlled by the control unit (106).

The communication network (107) may be either wired or wireless, and is a network for communicating digital data using a communication protocol such as a general IP (Internet Protocol).

FIG. 4 is a functional block diagram schematically showing an example of the configuration of the image enlargement / clearing unit of the image receiving apparatus.

In FIG. 4, an image enlargement / clearing unit (111) includes an image enlargement unit (301) that performs image enlargement processing on an image having the second number of pixels decoded by the decoding unit (110), and an image enlargement unit. An image position shift unit (302) that performs an image position shift process on the processed image and an aliasing distortion reduction unit (303) that performs an aliasing distortion reduction process on the image that has undergone the image position shift process.

In the following signal processing, the same processing may be performed on all color image signals (RGB (red, green, blue), YUV (luminance Y, color difference UV), etc.). Alternatively, the following signal processing may be performed only on the luminance signal Y and not on the color difference signal (UV), but only on the image enlargement processing.

The image enlargement unit (301) converts an image having the second number of pixels (for example, D1 size (horizontal 704 pixels × vertical 480 pixels)) input from the decoding unit (110) into an image having the third number of pixels. The image enlargement process to be converted into the image reduction processing is performed, and can be realized by the same configuration as the image reduction unit (104) as described above. The purpose of the image enlargement process is to increase the number of pixels of the image (that is, increase the sampling frequency) in order to prevent the aliasing distortion reduced in the subsequent processing block from being aliased again in the frequency domain. The number of pixels may be larger than the second number of pixels. Therefore, for the sake of simplicity of explanation, the third pixel number is assumed to be twice the second pixel number (that is, horizontal 1408 pixels × vertical 960 pixels).

The image position shift unit (302) converts the image that has been subjected to the image enlargement process by the image enlargement unit (301) in the direction opposite to the image position shift process by the image position shift unit (103) of the image transmission device (101). By shifting the position (that is, shifting the position of the image so as to cancel the position shift performed by the image position shift unit (103)), image blur between frames is suppressed. For example, when image position shift processing is performed such that the image position shift unit (103) shifts one pixel in the left direction and one pixel in the upward direction, the image position shift unit (302) is set to 0. 0 in the right direction. If an image position shift process is performed such that 73 (= 1408/1920) pixels are shifted downward by 0.89 (= 960/1080) pixels, the image returns to the original position, and image blur between frames is lost. Is suppressed. In the image position shift process in the image position shift unit (302), the image shift may be performed so as to cancel the image position shift process in the image position shift unit (103). For example, “4” illustrated in FIG. The image position shift unit (in order to shift all images to the center positions of the “frame type image position shift” and the “two frame type image position shift” shown in FIG. 3 (that is, the position of the image frame (200)). What is necessary is just to determine the shift direction and shift amount in 302). Further, the shift direction and shift amount in the image position shift unit (302) may be determined so that all images are shifted to fixed positions that are not the respective centers in the image position shift process of the image position shift unit (103). Good. The shift direction and shift amount of the image position shift unit (302) are controlled by the control unit (112).

The aliasing distortion reduction unit (303) performs aliasing distortion reduction processing for reducing aliasing distortion of the image subjected to the image position shift processing by the image position shift unit (302), and outputs an image from the image enlargement / clearing unit (111). It is what. Note that details of the aliasing reduction processing will be described later.

FIG. 7 is a diagram schematically showing an example of a configuration for realizing the function of the image enlargement / clearing process in the image enlargement / clearing unit shown in FIG.

In FIG. 7, the image enlargement / clearing unit (501) obtains a wideband output signal exceeding the Nyquist frequency of the input signal by canceling the aliasing distortion in the one-dimensional direction using two series of digital signals. One-dimensional enlargement unit (502- # 0), (502- # 1), one-dimensional interpolation filter (503- # 0), (503- # 1), Hilbert transformer (506), etc. It is comprised by.

In the image enlarging / sharpening unit (501), two series of digital signals (that is, input # 0 and input # 1) are converted to 1 by each one-dimensional enlarging unit (502- # 0) and (502- # 1). After being expanded in the dimensional direction (horizontal direction or vertical direction), each one-dimensional interpolation filter (503- # 0), (503- # 1) uses each position shift amount (α0, α1). The position of the image frame is adjusted, the output signal of the adder (504), the image signal after being multiplied by the coefficient K by the multiplier (507) after passing through the subtracter (505) and the Hilbert transformer (506) Are converted so that the phase of the aliasing distortion contained in both of them is 180 degrees (opposite phase), and then added by an adder (508) to correct the aliasing distortion in the image. Can be canceled.

Here, the image enlargement unit (301) shown in FIG. 4 corresponds to the one-dimensional enlargement units (502- # 1) and (502- # 2) in FIG. 7, and the image position shift unit (302) This corresponds to the dimension interpolation filters (503- # 0) and (503- # 1). Note that the image enlargement unit (301) shown in FIG. 4 is equivalent to the interpolation filter (401) as described above, and the sinc function used to determine the coefficient of the low-pass filter (403) is sin (x−t). ) / (Xt) and the one-dimensional interpolation filters (503- # 0) and (503- # 1) together with the image position shift unit (302), the one-dimensional enlargement unit (502- #) 1) and (502- # 2) can be omitted. 4 is realized by the adder (504), subtracter (505), Hilbert transformer (506), multiplier (507), and adder (508) in FIG. Therefore, the image enlargement / clearing unit (111) in FIG. 4 and the image enlargement / clearing unit (501) in FIG. 7 can be regarded as equivalent.

The configuration of the image enlargement / clearing unit (501) shown in FIG. 7 illustrates the case where the one-dimensional image enlargement process and the aliasing reduction process are performed. The case of performing the direction image enlargement process and the aliasing reduction process will be described below with reference to FIG.

FIG. 8 is a diagram schematically showing another example of a configuration for realizing the function of the image enlargement / clearing process in the image enlargement / clearing unit shown in FIG.

In FIG. 8, processing blocks (501-HA), (501-HB), (501-V) having the same configuration as the image enlargement / clearing unit (501) shown in FIG. In this configuration, the outputs of (501-HA) and (501-HB) are connected in series as the input of the processing block (501-V). The horizontal processing unit (601) including processing blocks (501-HA) and (501-HB) performs horizontal image enlargement processing and aliasing distortion reduction processing, and vertical processing including processing blocks (501-V). In the part (602), vertical image enlargement processing and aliasing reduction processing are performed. That is, by configuring the image enlargement / clearing unit (111) in this way, two-dimensional image enlargement processing and aliasing distortion reduction processing are performed using four systems of signals (ie, inputs # 0 to # 3). Can be realized. Note that the same processing can be realized even if the processing order of the horizontal processing unit (601) and the vertical processing unit (602) is interchanged.

Here, the image position shift shown in FIGS. 2 and 3, the position shift amount (α0, α1) shown in FIG. 7, the horizontal position shift amount (α0, α1, α2, α3) shown in FIG. The relationship with the vertical position shift amount (β0, β1) will be described. Note that an image having the first number of pixels (for example, horizontal 1920 pixels × vertical 1080 pixels) shown in FIG. 1 is the center position (image frame) of the rotational motion in the “4-frame position shift” shown in FIG. The two-dimensional coordinates (position of (200)) are set as the reference (0, 0), and the two-dimensional coordinates (−h, −v) → (h, −v) → (h, v) → (−h, v) → It is assumed that the position is shifted as (−h, −v) →... (Where h = 2m, v = 2n, and m and n are positive integers). Similarly, in the “two-frame position shift” shown in FIG. 3, the two-dimensional coordinates (− The following description will be made on the assumption that the position is shifted as h, 0) → (h, 0) → (−h, 0) →.

First, the case of “4-frame position shift” shown in FIG. 2 will be described. When this “4-frame position shift” is supported, the two-dimensional processing block shown in FIG. 8 is used. As described above, after an image having a first number of pixels (for example, horizontal 1920 pixels × vertical 1080 pixels) is reduced to a second number of pixels (for example, horizontal 704 pixels × vertical 480 pixels) and transmitted. When the aliasing distortion reduction process is performed by enlarging the third pixel number (for example, horizontal 1408 pixels × vertical 960 pixels), (−h, −v) → (h, −v) → (h, v) → (−h, v) → (−h, −v) →... In the third pixel number image, p = 1408/1920, q = 960/1080 corresponds to a position shift of (−ph, −qv) → (ph, −qv) → (ph, qv) → (−ph, qv) → (−ph, −qv) →. . Therefore, by setting the horizontal position shift amount (α0, α1, α2, α3) = (ph, −ph, −ph, ph) and the vertical position shift amount (β0, β1) = (qv, −qv). The image position shift process in the image position shift unit (103) on the image transmission apparatus (101) side and the effect of the image position shift process (302) on the image reception apparatus (109) side are canceled and canceled. And image blurring can be stopped.

Accordingly, in the horizontal processing unit (601) of FIG. 8, the coefficient Kα = −1 / tan (π (α0−α1)) = − 1 / tan (2πph) = − 1 / tan (2πh × 1408/1920) = − If it is set to 1 / tan (πh × 22/15), the folding distortion in the horizontal direction can be reduced. Further, in the vertical processing unit (602) of FIG. 8, the coefficient Kβ = −1 / tan (π (β0−β1)) = − 1 / tan (2πqv) = − 1 / tan (2πv × 960/1080) = − Setting 1 / tan (πv × 16/9) can reduce the folding distortion in the vertical direction.

Next, the case of “2-frame type position shift” shown in FIG. 3 will be described. When this “two-frame position shift” is supported, the one-dimensional processing block shown in FIG. 7 is used as horizontal processing, and horizontal image enlargement processing and aliasing reduction processing are performed. On the other hand, regarding the vertical processing, only the image enlargement in the vertical direction is performed using the interpolation filter shown in FIG. 6, and these horizontal processing and vertical processing are connected in series to perform two-dimensional processing. As described above, an image having a first number of pixels (for example, horizontal 1920 pixels × vertical 1080 pixels) is reduced to a second number of pixels (for example, horizontal 704 pixels × vertical 480 pixels), and the communication network (107 ), And when the aliasing distortion reduction processing is performed by enlarging the third pixel number (for example, horizontal 1408 pixels × vertical 960 pixels), (−h , 0) → (h, 0) → (−h, 0) →..., (−ph, 0) → (ph) in the image with the third pixel number, p = 1408/1920. , 0) → (−ph, 0) →... Therefore, if the horizontal position shift amount (α0, α1) = (ph, −ph) is set, the image position shift process by the image position shift unit (103) on the image transmitting apparatus (101) side and the image receiving apparatus (109) The effect of the image position shift processing by the image position shift unit (302) on the side can be canceled and canceled, and image blur can be stopped.

Accordingly, in the image enlarging / sharpening unit (501) of FIG. 7, the coefficient K = −1 / tan (π (α0−α1)) = − 1 / tan (2πph) = − 1 / tan (2πh × 1408/1920) ) = − 1 / tan (πh × 22/15), the folding distortion in the horizontal direction can be reduced.

Note that even when the image is reduced to a non-divisor pixel number by the above-described image reduction processing, the aliasing distortion included in the image can be reduced, and the image can be sharpened without performing iteration.

Here, in the configuration of FIG. 8 described above, the amount of calculation is about three times that of the configuration of FIG. Therefore, a configuration capable of realizing an equivalent operation while reducing the calculation amount of the configuration shown in FIG. 8 will be described with reference to FIGS. 9 and 10.

9 and 10 are diagrams schematically showing another example of a configuration for realizing the function of the image enlargement / clearing processing in the image enlargement / clearing unit shown in FIG.

In the configuration of FIG. 7 described above, focusing on the fact that the Hilbert transform is a fixed coefficient odd-symmetric filter and that the coefficient Kα and the coefficient Kβ are constant, “adder (504), subtracter (505)”. , Hilbert transformer (506), multiplier (507), and adder (508) "are all linear operations composed of fixed coefficients, and can be expressed as one" one-dimensional asymmetric filter "(described later). . That is, in the configuration of FIG. 7, the signal passing through the one-dimensional interpolation filter (503- # 0) for the input # 0 and then passing through the “one-dimensional asymmetric filter” and the one-dimensional interpolation for the input # 1 After passing through the filter (503- # 1), the signal passing through the “one-dimensional asymmetric filter” in which the polarity (positive / negative) of the subtractor (505) is inverted may be added together at the end. The same output is obtained.

Therefore, in FIG. 9, the order of the “one-dimensional asymmetric filter” in the horizontal processing unit (601) and the “one-dimensional enlargement unit and one-dimensional interpolation filter” in the vertical processing unit (602) in FIG. The vertical one-dimensional enlargement units are combined into a two-dimensional expansion unit (702), the horizontal / vertical one-dimensional interpolation filters are combined into a two-dimensional interpolation filter (703), and the horizontal / vertical one-dimensional asymmetric filters are combined. Collectively, a two-dimensional processing unit (701) is configured as a two-dimensional asymmetric filter (704).

In this two-dimensional processing unit (701), two-dimensional interpolation is performed in accordance with the horizontal position shift amounts (α0, α1, α2, α3) and vertical position shift amounts (β0, β1) corresponding to the inputs # 0 to # 3. The coefficients of the filter (703) and the two-dimensional asymmetric filter (704) are set, and the two-dimensional processing units (701- # 0 to # 3) are set.

Subsequently, each signal passing through each two-dimensional processing unit (701- # 0 to # 3) is passed through each frame memory (705- # 0 to # 3), and then added to all signals by an adder (706). If added, an output equivalent to the configuration of FIG. 8 can be obtained.

In this way, in the configuration shown in FIG. 8, no output can be obtained unless all calculations of the horizontal processing unit (601) and the vertical processing unit (602) are performed using the inputs # 0 to # 3. In the configuration of FIG. 9, the processes for the inputs # 0 to # 3 are calculated independently of the other inputs by the respective two-dimensional processing units (701- # 0 to # 3). Each input # 0 to # 3 is input in the order of input # 0, input # 1, input # 2, input # 3, input # 0,... For each image frame. For the above, the calculation result may be stored in the frame memory (705- # 0 to # 3). Further, since there is no need for each two-dimensional processing unit (701- # 0 to # 3) for an image frame that is not input, only one two-dimensional processing unit (701) is installed as shown in FIG. In accordance with the amount (α 0, α 1, α 2, α 3) and the vertical position shift amount (β 0, β 1), the internal coefficient is set while changing according to the frame number, and the switch (707) is used. The frame memory (705- # 0 to # 3) may be written while being selected for each frame.

Therefore, by configuring the image enlargement / clearing unit (111) as shown in FIG. 10, the calculation amount is about 1/6 (= 2 (horizontal / vertical) / 3 (horizontal) compared to the calculation amount in the configuration of FIG. (Horizontal / vertical) × 1/4 (ratio of the number of input frames to be processed))).

Similarly, the configuration of the image enlargement / clearing unit (111) in the case of “2-frame position shift” can be equivalently converted from the configuration of FIG. 7 to the configuration of FIG. That is, the one-dimensional enlargement units (502- # 0) and (502- # 1) in FIG. 7 are combined into the one-dimensional enlargement unit (801) in FIG. 11, and the one-dimensional interpolation filters (503- # 0) in FIG. (503- # 1) are combined into a one-dimensional interpolation filter (802) in FIG. 11, and “adder (504), subtracter (505), Hilbert transformer (506), multiplier (507)” in FIG. The adder (508) "is integrated into the one-dimensional asymmetric filter (803) in FIG. 11, and is set while changing the internal coefficient in accordance with the frame number in accordance with the horizontal position shift amount (α0, α1). Then, the frame memory (805- # 0), (805- # 1) is written while being selected for each frame using the switch (804), and both signals are output by the adder (806). If added, an output equivalent to the configuration of FIG. 7 can be obtained.

That is, by configuring the image enlargement / clearing unit (111) as shown in FIG. 11, the calculation amount for one one-dimensional interpolation filter can be reduced compared to the calculation amount in the configuration of FIG.

FIGS. 12 to 14 are a block diagram and an operation explanatory diagram showing an example of a configuration of an asymmetric filter included in the image enlargement / clearing unit of FIGS. 5 and 11. FIG.

FIG. 12 is a block diagram showing an example of the configuration of a one-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG.

In FIG. 7 described above, the input # 0 passes through the one-dimensional enlargement unit (502- # 0) and the one-dimensional interpolation filter (503- # 0), and then “signal that has passed through the adder (504)”. Then, "the signal that has passed through the subtracter (505), the Hilbert transformer (506), and the multiplier (507)" is added by the adder (508) to be an output. At this time, if the input # 1 = 0 (no signal) is set, for the input # 0, both the adder (504) and the subtracter (505) have a coefficient of 1 (that is, the signal does not change). I understand. Therefore, as shown in FIG. 12, the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 0 is obtained by multiplying the signal obtained by multiplying the coefficient K through the Hilbert transformer (506) and the input signal by the adder (508). The added signal is output.

On the other hand, if input # 0 = 0 (no signal), the coefficient of the adder (504) is 1 and the coefficient of the subtracter (505) is −1 for the input # 1. . Therefore, the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 1 is a value obtained by inverting the polarity (positive / negative) of the coefficient K compared to the coefficient of the filter coefficient of the one-dimensional asymmetric filter (704) for the input # 0. It becomes.

The Hilbert transformer (506) is an odd symmetric filter. When t = 2m (where m is an integer), the coefficient C (t) = 0, and when t = 2m + 1 (where m is an integer), the coefficient C (t) = 2 / (πt). Therefore, in the configuration of FIG. 12, the filter coefficients of the one-dimensional asymmetric filter (704) are “asymmetric” operations that are neither even nor odd symmetric as shown in FIG. The filter coefficient (C (t)) shown in FIG. 13 is an example. For example, the coefficient C (t) (where t ≠ 0) is obtained by multiplying the coefficient of the Hilbert transform by a general window function (such as Hanning window). ), The influence of the filter end may be reduced.

FIG. 14 is a block diagram illustrating an example of a configuration of a two-dimensional asymmetric filter included in the image enlargement / clearing unit in FIG.

The two-dimensional asymmetric filter (704) is obtained by connecting the above-described one-dimensional asymmetric filter (803) in series. The horizontal processing unit (803-H) converts the number of pixels in the horizontal direction, and the vertical processing unit ( By converting the number of pixels in the vertical direction at 803-V), two-dimensional pixel number conversion can be realized.

Thus, by performing independent signal processing for each input image frame using an asymmetric filter, it is possible to reduce the amount of computation in reducing aliasing distortion.

Next, a process corresponding to an image including movement in the subject will be described.

The above-described configuration of the image enlargement / clearing unit (111) reduces the amount of calculation by eliminating the motion search and iteration for each pixel and realizes image sharpening capable of high-speed processing. Among these, in order to eliminate the need for motion search for each pixel, the position shift amounts (αn, βn) used in the configurations of FIGS. 7 to 11 are fixed values for each frame. At this time, there is no problem if the subject in the image frame does not move, but there is a problem that multiple images are generated by calculation between the frames if the subject has movement. Therefore, a configuration example for solving such a problem will be described below.

FIG. 15 is a block diagram showing another example of the configuration of the image enlargement / clearing unit in the image receiving apparatus of FIG.

15, a processing unit (1001) including an image enlargement unit (301), an image position shift unit (302), and a aliasing distortion reduction unit (303) has the same configuration as that shown in FIG. A signal in which image blur between frames is suppressed by the image position shift unit (302) is passed through a low-pass filter (1002) for suppressing unnecessary aliasing distortion, and an image pixel is detected by a motion detection unit (1003) described later. A control signal (m, where 0 (no motion) ≦ m ≦ 1 (with motion)) corresponding to the magnitude of each motion is generated, and the low-pass filter (1002) is generated using the weighted mixing unit (1004). The passed signal (p1) and the signal (p0) passed through the aliasing distortion reduction unit (303) are weighted and mixed in accordance with the control signal m to obtain an output signal. Here, the weighted mixing unit (1004) includes a subtracter (1005) that generates a control signal (1-m) from the control signal m, a multiplier (1006) (1007), and an adder (1008). Every time, “output = p1 × m + p0 × (1−m)” is performed. That is, in the region where the value of the control signal m is close to 0 (the region close to static), the signal (p0) passed through the aliasing reduction unit (303) is output, and the region where the value of the control signal m is close to 1 (the motion is In the large region, control is performed so that the signal (p1) that has passed through the low-pass filter (1002) is output.

FIG. 16 is a block diagram illustrating an example of the configuration of the motion detection unit included in the image enlargement / clearing unit of FIG.

In the case of the “4-frame position shift type” described above in the motion detection unit (1003), four consecutive frame images input from the low-pass filter (1002) are converted into frame numbers (# 0, #) by the switch (1101). 1, # 2, # 3, # 0,...) And switching to frame memory (1102). Subsequently, the image read from each frame memory (1102) is averaged for each pixel by using the averaging unit (1103), and the average image and each frame by using the subtracter (1104) and the absolute value unit (1105). After calculating the absolute value of the difference for each pixel, the maximum value is obtained for each pixel by the maximum value unit (1106) and falls within the range of “0 ≦ m ≦ 1” by the normalizer (1107). Thus, after normalizing the value of m, the control signal m is obtained. This normalization can be realized by a general technique in which a predetermined fixed value is subtracted or multiplied by the output signal of the maximum value unit (1106), and thus detailed illustration and description are omitted.

By the motion detection unit (1003) configured in this way, an area in which all the values of the four-frame images (# 0, # 1, # 2, # 3) input to the motion detection unit (1003) match (that is, In the area where the image is still), the value of the control signal m is 0, and the area where the values of the images of 4 frames (# 0, # 1, # 2, # 3) do not match (that is, 1 out of 4 frames). In the area where the image is moving over the frame), the control signal m has a value of 0 <m ≦ 1 according to the magnitude of the absolute value of the difference signal described above.

In the case of the “two-frame position shift type” described above, among the configuration of the motion detection unit (1003) shown in FIG. 16, the switcher (1101), frame memory (1102), average unit (1103), subtractor (1104) and the absolute value unit (1105) can be easily realized simply by changing from 4 frames to 2 frames.

With the configuration described above, even when the subject in the image moves, it is possible to obtain a high-quality image in the still region of the image while suppressing the occurrence of the multiple images described above in the moving region of the image. For example, if this configuration is applied to a surveillance camera, a parked vehicle becomes a still image area, so that a license plate or the like can be visually recognized with a fine image.

Next, frame synchronization between the image transmission device and the image reception device will be described.

17 to 19 are block diagrams respectively showing an example of the configuration of the control unit included in the image transmission device and the image reception device in FIG.

The image position shift unit (103) included in the image transmission device (101) and the image position shift unit (302) included in the image reception device (109) operate in synchronization with each other accurately in units of frames and are in opposite directions. It is necessary to control to shift the position. For obtaining the frame phase information used for this synchronization, any one of the configurations shown in FIGS. 17 to 19 is used. The frame phase information is, for example, one of the frame numbers “# 0, # 1, # 2, # 3” in the case of “4 frame type position shift”, and “2 frame type position shift”. In this case, the frame number is “# 0, # 1”.

FIG. 17 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information generation unit of the image transmission device and the frame phase information acquisition unit of the image reception unit.

In FIG. 17, the frame phase information generation unit (1201) included in the control unit (106) of the image transmission apparatus (101) generates frame phase information, and the multiplexing unit (1202) is used to encode the encoding unit (105). Is multiplexed with the image data output from. The multiplexed data is transmitted to the image receiving device (109) via the communication network (107) and separated by the separating unit (1203), and then the image data is sent to the decoding unit (110) to receive frame phase information. Are respectively input to the control unit (112). Accordingly, the frame phase information acquisition unit (1204) of the control unit (112) can obtain the same information as the frame phase information generated by the frame phase information generation unit (1201) of the image transmission device (101). The frame phase information generation unit (1201), the multiplexing unit (1202), the separation unit (1203), and the frame phase information acquisition unit (1204) can be realized by a general technique, and thus detailed illustration is omitted.

FIG. 18 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information generation unit of the image reception unit and the frame phase information acquisition unit of the image transmission apparatus.

In FIG. 18, the frame phase information generating unit (1206) included in the control unit (112) of the image receiving apparatus (109) generates frame phase information, which is transmitted to the image transmitting apparatus (101) via the communication network (107). To transmit. Accordingly, the frame phase information acquisition unit (1205) included in the control unit (106) of the image transmission device (101) is the same as the frame phase information generated by the frame phase information generation unit (1206) of the image reception device (109). Information can be obtained. The frame phase information generation unit (1206) and the frame phase information acquisition unit (1205) can be realized by a general technique, and thus detailed illustration is omitted.

FIG. 19 is a diagram illustrating a configuration example when synchronization is performed using the frame phase information estimation unit of the image reception unit.

In FIG. 19, the frame phase information generation unit (1201) included in the control unit (106) of the image transmission apparatus (101) generates frame phase information, but this information is not transmitted to the image reception apparatus (109). In the image receiving device (109), the encoded image data transmitted from the image transmitting device (101) is decoded by the decoding unit (110), and the frame included in the control unit (112) based on the decoded image. Frame phase information is obtained by performing a frame phase information estimation process in the phase information estimation unit (1207).

FIG. 20 is an explanatory diagram showing an example of the operation of the frame phase information estimation process of the frame phase information estimation unit shown in FIG.

In FIG. 20, an image (1301) is a D1-size image obtained by reducing the image frame (201) shifted in the upper left direction in FIG. 2 by the image reduction unit (104) included in the image transmission apparatus (101) shown in FIG. This is an image (horizontal 704 pixels × vertical 480 pixels), and is taken as an example for explanation. In FIG. 20, assuming that the two-dimensional coordinates on the image (1301) are (x, y) (where x is a horizontal coordinate and y is a vertical coordinate), an example of a series of pixel values at a position of y = 0, y = An example of a series of pixel values at position 1 and an example of a series of pixel values at position y = 2 are shown.

The position of the image frame (201) is shifted in the upper left direction, and black (pixel value = 0) is inserted at the right end and the lower end of the image as described above. Therefore, in the reduced image (1301), at the left end where the image originally exists, the pixel value P (0, y) at the position of x = 0 and the pixel value P (1, y) at the position of x = 1. The magnitude relationship of changes randomly according to the pattern. On the other hand, the pixel value P (703, y) at the position x = 703 is larger than the pixel value P (704, y) at the position x = 704 at the right end where the image originally does not exist because black is inserted. . Using this property, frame phase information estimation processing is performed to estimate whether the image is shifted to the left or right.

FIG. 21 is a flowchart showing frame phase estimation processing in the frame phase information estimation unit.

In FIG. 21, when the frame phase information estimation process is started, the frame phase information estimation unit (1207) first prepares each data (L, R) of the left end counter and the right end counter on the memory, In addition to initializing the value to 0, data (y) indicating the vertical coordinate is prepared on the memory and initialized to y = 0 (step S1401). Subsequently, the values of the two pixels at the left end of the image (P (0, y) and P (1, y)) and the values of the two pixels at the right end (P (703, y) and P (704, y)) Each counter value (L, R) is updated according to the magnitude relationship (step S1402). That is, in step S1402, if “P (0, y) ≧ P (1, y)”, the value (L) of the counter for the left end is incremented by 1, and if “P (704, y) ≧ P (703, y) ", the value (R) of the right end counter is incremented by one. Subsequently, it is determined whether or not the value of the vertical coordinate to be processed has reached the lower end of the image, that is, whether or not y = 479 (step S1403). If the determination result is NO, data (y ) Is incremented by one to set y ← y + 1 (step S1404), and the processing of steps S1404 and S1402 is repeated until the determination result of step S1403 becomes YES. If the determination result in step S1403 is YES, the image shift direction is estimated according to the value of each counter value (L, R) (step S1405). That is, in step S1405, if “the value of the counter for the left end (L) ≧ the value of the counter for the right end (R)”, it is estimated that the image is shifted leftward. Estimated to be shifted in the direction. When step S1405 ends, the process ends.

By the above processing, it can be estimated whether the image is shifted to the left or right. In FIG. 21, only the frame phase information estimation process in the left-right direction is illustrated for simplicity of explanation, but it is possible to perform frame phase information estimation by performing the same process in the vertical direction. is there. That is, if the procedure in which each data value and numerical value update direction shown in FIG. 21 is rotated by 90 degrees with respect to the image is used, it can be similarly estimated whether the image is shifted in the vertical direction. In addition, it is clear that the same estimation can be performed not by comparing the pixel values at the edge of the image, but by comparing the variances of the pixel values. An estimation may be performed.

Here, the image position shift processing in the image position shift unit (103) will be further considered. That is, depending on the type of image captured by the camera (102) of the image transmission device (101), it is necessary to consider the contents of the image position shift process. For example, in the case where the camera (102) includes image data of a single-plate image sensor on which an image Bayer array color filter is arranged, it is necessary to restrict the operation of the image position shift process of the image position shift unit (103).

FIG. 22 is a diagram showing a state of pixel arrangement in the Bayer array color filter.

As shown in FIG. 22, in the Bayer array color filter (1501), R: red, G: green, and B: blue color filters are regularly arranged with a two-pixel interval as one period in both the horizontal and vertical directions. ing. That is, when an image signal (hereinafter referred to as a Bayer image signal) obtained by photoelectrically converting light transmitted through the Bayer array color filter (1501) by a single-plate image sensor is input to the image position shift unit (103), If the position of either or both of the direction and the vertical direction is shifted in units of odd pixels, the positional relationship of R: red, G: green, B: blue changes, and the color of the image after the position shift is Will change.

Therefore, when the Bayer image signal is input to the image position shift unit (103), the position is shifted in units of two pixels (even number of pixels) in both the horizontal direction and the vertical direction, so that R: red, G: green, B : Image position shift processing is performed so that the positional relationship of blue is not different from that before the position shift. In addition, after the position shift in units of 2 pixels (even pixel units) as described above, only the color signal (RGB, YUV, etc.) converted from the Bayer image signal or the luminance signal Y converted from the Bayer image signal, Each signal processing shown in this embodiment can be applied as it is.

FIG. 23 is a functional block diagram schematically showing a configuration example of the image storage device.

In FIG. 23, an image storage device (108) is a device for storing (recording) or reproducing encoded image data, and includes a communication interface (1601), a control unit (1602), a memory (1603). ), Storage (1604), output interface (1605), input interface (1606), and other processing blocks, and a bus (1607) connecting each processing block.

The image storage device (108) stores an application program in the storage (1604), the control unit (1602) expands the program from the storage (1604) to the memory (1603), and the control unit (1602) By executing the program, various functions such as recording, reproduction, and search can be realized. In the following description, for the sake of simplicity, various functions realized by the execution of each application program by the control unit (1602) are mainly realized by "various program function units" expanded in the memory (1603). However, it is also possible to use hardware as a processing unit having the same function and implement each function mainly by each processing unit.

The application program may be stored in the storage (1604) in advance when the image storage device (108) is shipped, or a medium such as a CD (Compact Disc) / DVD (Digital Versatile Disc) or a semiconductor memory. And installed in the image storage device (108) via a medium connection unit (not shown). It is also possible to download and install from the communication network (107) via the communication interface (1601).

The communication interface (1601) is connected to the communication network (107), receives image data from the image transmission apparatus (101) shown in FIG. 109) also has a function of transmitting image data. The control unit (1602) controls the communication interface (1601), the memory (1603) (various program function units), the storage (1604), and the input / output interfaces (1605, 1606). The control unit (1602) also has a function of executing various signal processing according to a processing procedure described later. In the memory (1603), the function part of the application program stored in the storage (1604) is expanded under the control of the control part (1602). The storage (1604) accumulates image data from the image transmission apparatus (101), and stores application programs and various types of information created by the application programs. The output interface (1605) has a function of outputting an image obtained as a result of signal processing by the control unit to an external device via the bus (1607). The output image is displayed on an external display unit (1608). The input interface (1606) has a function of receiving a signal from the operation unit (1609) and transmitting the signal to the control unit (1602) via the bus (1607).

The image storage device (108) configured in this manner follows the following operation sequence during recording. That is, at the time of recording, the image storage device (108) reads a recording application program (not shown) stored in the storage (1604) into the memory (1603), and follows the procedure described in the recording application program according to the procedure described in the recording application program. 1602) controls each part. First, a connection is established with the image transmission apparatus (101) shown in FIG. 1 via the communication interface (1601) and the communication network (107). Thereafter, the encoded image data transmitted from the image transmission device (101) is received via the communication interface (1601) and the communication network (107), and is encoded into the storage (1604) via the bus (1607). Accumulated image data is stored. At this time, the received encoded image data may be decoded by the control unit (1602), output an image via the output interface (1605), and displayed on the display unit (1608). Next, an operation sequence at the time of reproduction in the image storage device (108) will be described.

Also, the image storage device (108) follows the following operation sequence during reproduction. That is, at the time of playback, the image storage device (108) reads a playback application program (not shown) stored in the storage (1604) into the memory (1603), and follows the procedure described in the playback application program according to the procedure described in the playback application program. 1602) controls each part. Thereafter, the encoded image data stored in the storage (1604) is read out via the bus (1607) and transmitted to the image receiving device (109) via the communication interface (1601) and the communication network (107). .

At the time of transmission, the encoded image data to be transmitted may be decoded by the control unit (1602), and the image may be output and displayed on the display unit (1608) via the output interface (1605). Also, the encoded image data stored in the storage (1604) is read out only by the image storage device (108) alone, decoded by the control unit (1602), and displayed via the output interface (1605) (1608). ) May be output and displayed. However, as described above, if an image whose image position is shifted by the image position shift unit (103) of the image transmission apparatus (101) is displayed on the display unit (1608) as it is, the image is periodically blurred. Become. Therefore, the various program function units developed in the memory (1603) include the frame phase information estimation function (1610) having the same function as the frame phase information estimation unit (1207) and the same as the image position shift unit (302). An image position shift function (1611) having a function is provided, and image blurring is canceled by performing a process of canceling out the effect of the image position shift process. That is, the image position shift function (1611) uses the third pixel number in the operation of the image position shift unit (302) in FIG. 4 as the pixel number of the encoded image data accumulated in the storage (1604) (second In this case, the operation of the image position shift unit (302) can be used as it is.

Also, the image position shift function (1611) can be realized by using a convolution process with a linear filter having asymmetric coefficients.

FIG. 24 is a diagram for explaining an example of filter coefficients used in the image position shift function of FIG.

24, coefficients (a) to (d) indicate filter coefficients that are convoluted with the decoded image. In each coefficient (a) to (d), a symbol or a numerical value in parentheses (that is, 0, α, 1-α, β, 1-β, etc.) represents the value of each coefficient, and the right shoulder of the parenthesis The symbol “T” represents transposition. The symbol “*” represents a convolution operation. That is, the coefficients (a) to (d) in FIG. 24 represent the coefficients of a two-dimensional filter of horizontal 3 taps × vertical 3 taps.

Here, when α> 0 and β> 0, the center of gravity of the two-dimensional filter represented by the coefficient (a) is shifted in the lower right direction, and an image obtained by convolving this filter coefficient is in the lower right direction. The position shifts. Similarly, the image convolved with coefficient (b) is shifted in the lower left direction, the image convolved with coefficient (c) is shifted in the upper right direction, and the image convolved with coefficient (d) is in the upper left. The position shifts in the direction.

For example, an image having the first number of pixels in FIG. 1 (for example, horizontal 1920 pixels × vertical 1080 pixels) is based on the two-dimensional coordinates of the center position of the rotational motion in “4-frame position shift” in FIG. 0,0), two-dimensional coordinates (−h, −v) → (h, −v) → (h, v) → (−h, v) → (−h, −v) →. In the case of shifting, in the image of the number of pixels (second pixel number) of the encoded image data accumulated in the storage (1604), p = 704/1920, q = 480/1080, (−ph, − This corresponds to a position shift of qv) → (ph, −qv) → (ph, qv) → (−ph, qv) → (−ph, −qv) →.

Accordingly, the filter coefficients are determined as α = ph and β = qv, and are switched for each frame in the order of coefficients (a) → (b) → (d) → (c) → (a) →. However, blurring of the image can be suppressed by convolving it with the stored image. With the above configuration, image blurring can be suppressed when the image stored in the image storage device (108) is played back on the display unit (1608).

The “two-dimensional filter of horizontal 3 taps × vertical 3 taps” described above is an example for explaining the operation, and the present invention is not limited to this, and a two-dimensional filter having a different number of taps. Obviously, a filter may be used. In addition, it is obvious that the image in which blurring is suppressed in this way may be transmitted to the outside via the communication interface (1601) and the communication network (107).

The effect of the present embodiment configured as described above will be described.

The super-resolution processing exemplified in the prior art requires a large amount of calculation for processing compared to edge enhancement processing that only amplifies high-frequency components included in an image. For example, in a super-resolution technique for obtaining an output image by successive approximation processing using an input image of a plurality of frames as described in Non-Patent Document 1, a high-precision motion search in units of subpixels for each pixel is performed. The accompanying inter-frame alignment (registration), image enlargement, enlargement image reduction, difference detection between the reduced image and the input image, and output image correction processing must be repeated many times (iteration). In other words, in the conventional technology, the motion search and iteration for each pixel has a very large amount of calculation, so that it exceeds the processing limit of calculation resources such as CPU and memory, and frame dropping occurs that makes the motion of the image jerky. Or the entire process may stop, or user input from a mouse, keyboard, or the like may not be accepted.

On the other hand, in the present embodiment, for each of a plurality of temporally continuous images, an image position shift unit (103) (103) that shifts the position of the image to any of a plurality of predetermined shift positions. A first image position shift unit), an image reduction unit (104) for reducing the number of pixels of the image whose position has been shifted by the image position shift unit (103), and an image reduced by the image reduction unit (104) A decoding unit (105) that generates a decoded image by decoding the encoded image sent via the communication network (107), An image enlargement unit (301) that increases the number of pixels of the decoded image decoded by the conversion unit (110) and enlarges the image, and an image position shift unit ( 103) An image position shift unit (302) (second image position shift unit) that shifts the position so as to cancel and cancel the shift, and an aliasing distortion of the image shifted by the image position shift unit (302) Since it is configured to include the aliasing distortion reduction unit (303) that performs aliasing distortion reduction processing that reduces using the information of the other image whose position is shifted in (302), an increase in the amount of calculation related to image processing and image quality It is possible to reduce the encoded data size while suppressing the decrease in the above. That is, by eliminating the need for motion search and iteration for each pixel that makes up the image, the amount of computation can be reduced, and image sharpening and transmission data that can be processed at high speed even with relatively inefficient computing resources Image transmission with a small size can be realized.

<Second Embodiment>
A second embodiment of the present invention will be described with reference to FIGS. 25 and 26. FIG.

In this embodiment, the image transmission apparatus according to the first embodiment is configured to have a function of switching presence / absence of image position shift processing, and performs image position shift processing that cancels and cancels image position shift processing. Even when displayed via an image receiving device or an image storage device that does not have means for performing (that is, means for shifting the position in the reverse direction), an image that is not periodically blurred can be provided.

FIG. 25 is a functional block diagram schematically showing a configuration example of the image transmission apparatus according to the present embodiment. In the figure, the same members as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

In FIG. 25, an image transmission device (1802) includes a camera (102) that captures an image (still image, moving image), and an image position shift unit (103) that performs an image position shift process on the image captured by the camera (102). The image signal from the camera (102) is input to the image reduction unit (104), or the image signal from the camera (102) is passed through the image position shift unit (103) and then to the image reduction unit (104). A switcher (1803) for switching between input and image, an image reduction unit (104) for performing an image reduction process on an image input via the switcher (1803), and an encoding process for the image subjected to the image reduction process On the basis of commands and data transmitted from the image receiving device (109) and the image storage device (108) via the communication unit (105) and the communication network (107). A control unit that controls the overall operation of the image transmission apparatus (1802) including the mela (102), the image position shift unit (103), the switch (1803), the image reduction unit (104), and the encoding unit (105). 106).

FIG. 26 is a diagram illustrating an example of a menu display screen generated by the control unit included in the image receiving device or the image storage device and displayed on the display unit.

In FIG. 26, the menu display screen (1901) generated by the control unit (112, 1602) of the image receiving device (109) or the image storage device (108) and displayed on the display unit (113, 1608) is an image transmission. A message portion (1902) indicating that the display is a menu display for selecting an operation related to the image position shift processing of the apparatus (1802), a message portion (1903) indicating that the image position shift processing is not performed, and an image position shift A selection unit (1905) for selecting not to perform processing, a message unit (1904) indicating that image position shift processing is performed, and a selection unit (1906) for selecting execution of image position shift processing And have. FIG. 26 illustrates the case where the user of the image receiving device (109) or the image storage device (108) has selected to perform image position shifting. The message portion (1903) indicating that the image position shift process is not performed may include a message indicating that resolution improvement cannot be expected. The message portion (1904) indicating that the image position shift process is to be performed may include a message indicating that resolution improvement can be expected.

When the image transmission device (1802) is connected to the image reception device (109) or the image storage device (108) having the image position shift unit (302) via the communication network (107), the user can display a menu. By selecting the selection unit (1906) of the screen (1901), the switch (1803) is switched to the lower side in the figure, and the image that has passed through the image position shift unit (103) is displayed by the image reduction unit (104). The image is reduced, encoded by the encoding unit (105), and transmitted to the image reception device (109) and the image storage device (108) via the communication network (107). As a result, the image receiving device (109) can output a high-quality image with reduced aliasing distortion.

On the other hand, when the image transmission apparatus (1802) is connected to an image reception apparatus or image storage apparatus that does not have the image position shift unit (302) via the communication network (107), the user can display a menu display screen ( 1901) by selecting the selection unit (1905), the switch (1803) is switched to the upper side in the figure, and the image reduction unit (104) reduces the image without passing through the image position shift unit (103). The encoding unit (105) encodes the data and transmits the data to the image reception device (109) and the image storage device (108) via the communication network (107). As a result, the image receiving apparatus can output an image without blurring.

The switch (1803) is controlled in accordance with a control signal sent from the control unit (1804) according to the setting of the menu display screen (1901). At this time, the image is transmitted via the communication network (107). You may enable it to control according to the command automatically transmitted from the receiver (109) or the image storage device (108). That is, the image receiving device (109) or the image storage device (108) having the image position shift unit (302) is set in advance so as to send a command indicating that image position shift processing is performed, and this command is transmitted to the control unit ( When the signal is not received in 1804), it may be determined that an image receiving apparatus that does not have the image position shift unit (302) is connected, and the switch (1803) may be switched to the upper side in the drawing.

Other configurations are the same as those in the first embodiment.

Also in the present embodiment configured as described above, the same effect as in the first embodiment can be obtained.

Further, in the image subjected to the image position shift process by the image position shift unit (103), means for performing image position shift processing that cancels and cancels the image position shift process (that is, means for position shifting in the reverse direction) In an image receiving apparatus that does not have an image, the displayed image is periodically blurred. And any other image receiving device that does not include an image position shift unit, and the like, can display a suitable image for each.

<Third Embodiment>
A third embodiment of the present invention will be described with reference to FIGS.

In the present embodiment, even when the image position shift process is displayed via an image receiving apparatus that does not have a means for performing an image position shift process that cancels and cancels the image position shift process (that is, a means for shifting the position in the reverse direction). Another embodiment in which an image without blurring can be provided is shown.

FIG. 27 is a functional block diagram schematically showing a configuration example of the image transmission apparatus according to the present embodiment. FIG. 28 is a functional block diagram schematically showing a configuration example of the image receiving apparatus according to the present embodiment. In the figure, the same members as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

In FIG. 27, an image transmission device (2001) includes a camera (102) that captures an image (still image, moving image), and an image position shift unit (103) that performs an image position shift process on the image captured by the camera (102). The image reduction unit (104) that performs the image reduction process on the image that has been subjected to the image position shift process, and cancels out the position shift performed by the image position shift unit (103) on the image that has undergone the image reduction process. An image position shift unit (2002) that shifts the position as described above, an encoding unit (105) that performs an encoding process on an image that has undergone image position shift processing by the image position shift unit (2002), and a communication network (107) Based on the command and data received via the camera, the camera (102), the image position shift unit (103), the image reduction unit (104), and the image position shift unit (2002) Control unit for controlling an image transmission apparatus (2001) the whole operations including the fine coding unit (105) and a (2003).

The image position shift unit (2002) is for shifting the image in the direction opposite to the image blur caused by the image position shift unit (103) under the control of the control unit (2003). ) Operates in the same manner as the image position shift function (1611). That is, the operation of the image position shift unit (302) is performed by replacing the third pixel number in the operation of the image position shift unit (302) in FIG. 4 with the pixel number (second pixel number) of the image data to be transmitted. Can be used as they are. The asymmetric filter described above can also be used as the image position shift unit (2002).

In FIG. 28, an image receiving device (2004) performs a decoding process on the image sent from the image transmitting device (2001) via the communication network (107), and a decoding process. An image position shift unit (2005) that performs the same image position shift process as the image position shift unit 103 on the performed image, and an image enlargement / clearing process on the image that has been subjected to the image position shift process by the image position shift unit (2005) An image enlarging / sharpening unit (111) for performing the above and a control unit (2006) for controlling the entire operation of the image receiving apparatus (2004).

The image position shift unit (2005) operates under the control of the control unit (2006), and cancels the position shift performed by the image position shift unit (2002) of the image transmission device (2001) so as to cancel the position shift. To do.

FIG. 29 is a diagram illustrating an example of filter coefficients used in the image position shift unit in FIG.

29, coefficients (a) to (d) indicate filter coefficients that are convoluted with the decoded image. In each coefficient (a) to (d), a symbol or a numerical value in parentheses (that is, 0, α, 1-α, β, 1-β, etc.) represents the value of each coefficient, and the right shoulder of the parenthesis “−1” of the symbol represents an inverse characteristic, and the symbol “T” on the right shoulder of the parenthesis represents transposition. The symbol “*” represents a convolution operation. That is, the coefficients (a) to (d) in FIG. 29 represent the coefficients of the inverse filter of the two-dimensional filter shown in FIG. In addition, since each coefficient of an inverse filter can be calculated | required with a general technique, concrete description is abbreviate | omitted.

Other configurations are the same as those in the first embodiment.

Also in the present embodiment configured as described above, the same effects as those of the first embodiment can be obtained.

Further, when the image transmission device (2001) and the image reception device (2004) are used in combination, the image reception device (2004) can output a clear image by the same operation as the first embodiment. In addition, even when the image transmission apparatus (2001) is connected to an image reception apparatus or an image storage apparatus that does not include an image position shift unit, it is possible to suppress image blurring that occurs during display.

<Fourth embodiment>
A fourth embodiment of the present invention will be described with reference to FIG.

In the present embodiment, the image storage device (108) in the first embodiment is used as the image receiving device (1612).

In the image processing system according to the present embodiment, the image transmission device (101) is used as the image transmission device, and is described in an image reception application program (not shown) stored in the storage (1604) of the image storage device (108). The control unit (1602) controls each unit in accordance with the processing (described later in detail with reference to FIG. 30), whereby the image storage device (108) is also used as the image reception device (1612) (see FIG. 23).

FIG. 30 is a flowchart showing an example of processing in the image receiving apparatus according to the present embodiment.

In FIG. 30, the image receiving device (1612) first acquires the encoded image data via the communication network (107) (step S2201), and the control unit (1602) decodes the image data. It stores in the memory (1603) (step S2202). Subsequently, frame phase information is acquired (step S2205). In parallel with step S2205, the image is enlarged (step S2203), and the position of the image is shifted (step S2204). These processes correspond to the processing contents of the image enlargement unit (301) and the image position shift unit (302) shown in FIG.

Subsequently, aliasing distortion reduction processing is performed (step S2206). In step S2206, two-dimensional processing is performed (step S2207), the image is stored in a predetermined frame memory (memory (1603) in FIG. 23) (step S2208), and the values of all frame memories are added for each same pixel position. (Step S2209). Note that these processes correspond to the processing contents in the configuration shown in FIGS. 9 and 10. In parallel with step S2206, low-pass filter processing for suppressing unnecessary aliasing distortion is performed (step S2210). Subsequently, a motion detection process is performed to obtain a control signal m (step S2211). This processing corresponds to the processing content in the configuration shown in FIG.

Subsequently, weighted mixing is performed using the image after the aliasing reduction obtained in step S2206, the image after the low-pass filter obtained in step S2210, and the control signal m obtained in step S2211, thereby obtaining an output image ( Step S2212). This processing corresponds to the operation of the weighted mixing unit (1004) shown in FIG. The output image signal obtained in step S2212) is output via the output interface (1605) (step S2213), and the process ends.

Note that the processing shown in steps S2201 to S2213 is controlled by the control unit (1602) so that it is executed each time image data is transmitted from the image transmission apparatus (101) to the image reception apparatus (1612). Good. Alternatively, every time the processing shown in steps S2201 to S2213 is completed, a command instructing transmission of image data may be transmitted from the image receiving apparatus (109) to the image transmitting apparatus (101). Further, by providing the various program function units developed in the memory (1603) of FIG. 23 with another image position shift function (not shown), the operation of the image position shift unit (2005) described in FIG. It is obvious that the operation shown in FIG. 29 can be realized, and it is obvious that the same operation as that shown in FIG. 28 can be realized using the configuration of the image receiving device (1612).

Other configurations are the same as those in the first embodiment.

Also in the present embodiment configured as described above, the same effects as in the first embodiment can be obtained.

In addition, this invention is not limited to each above-mentioned embodiment, Various modifications are included. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to one having all the configurations described. Moreover, you may implement | achieve part or all of said each structure, function, etc., for example by designing with an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.

That is, the functions of the embodiments of the present invention can be realized by software program codes. In this case, a storage medium in which the program code is recorded is provided to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention. As a storage medium for supplying such program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. are used.

Also, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. May be. Further, after the program code read from the storage medium is written in the memory on the computer, the computer CPU or the like performs part or all of the actual processing based on the instruction of the program code. Thus, the functions of the above-described embodiments may be realized.

Further, by distributing the program code of the software that realizes the functions of the embodiment via a network, the program code is stored in a storage means such as a hard disk or a memory of a system or apparatus, or a storage medium such as a CD-RW or CD-R And the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage means or the storage medium when used.

Finally, it should be understood that the processes and techniques described herein are not inherently related to any particular equipment, and can be implemented by any suitable combination of components. In addition, various types of devices for general purpose can be used in accordance with the teachings described herein. It may prove useful to build a dedicated device to perform the method steps described herein. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined. Although the present invention has been described with reference to specific examples, these are in all respects illustrative rather than restrictive. Those skilled in the art will appreciate that there are numerous combinations of hardware, software, and firmware that are suitable for implementing the present invention. For example, the described hardware may be implemented by ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc., and the described software is assembler, C / C ++, perl, Shell, PHP, Python. , Java (registered trademark) or a wide range of programs or script languages may be used.

Furthermore, in the above-described embodiment, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

In addition, other implementations of the invention will become apparent to those skilled in the art from consideration of the specification and embodiments of the invention disclosed herein. Various aspects and / or components of the described embodiments can be used singly or in any combination in a computerized storage system capable of managing data.

Claims

A first image position shift unit that shifts the position of the image to any of a plurality of predetermined shift positions for each of a plurality of temporally continuous images;
An image reduction unit for reducing the number of pixels of the image shifted in position by the first image position shift unit;
An encoding unit that encodes the image reduced by the image reduction unit to generate an encoded image;
A decoding unit that decodes the encoded image sent via a communication network to generate a decoded image;
An image enlarging unit for enlarging by increasing the number of pixels of the decoded image decoded by the decoding unit;
A second image position shift unit that shifts the position of the image enlarged by the image enlargement unit so as to cancel the position shift performed by the first image position shift unit;
An aliasing distortion reducing unit that performs aliasing distortion reduction processing for reducing aliasing distortion of an image that has been position-shifted by the second image position shifting unit using information of another image that has been position-shifted by the second image position shifting unit. And an image processing system.
The image processing system according to claim 1,
A third image position shift unit that shifts the image to a position that cancels the position shift by the first image position shift unit with respect to the image reduced by the image reduction unit, and sends the image to the encoding unit;
A fourth image position shift unit that shifts the image to a position where the position shift by the third image position shift unit is canceled with respect to the decoded image decoded by the decoding unit, and sends the image to the image enlargement unit; An image processing system comprising:
The image processing system according to claim 1,
The image reduction system is characterized in that the image reduction unit reduces the image to an image having a pixel number that is a non-divisor (a fraction of a non-integer) of the number of pixels of the original image.
The image processing system according to claim 1,
The image processing system, wherein the aliasing reduction unit uses an asymmetric filter having an asymmetric coefficient.
The image processing system according to claim 1,
The image processing system, wherein the second image position shift unit performs position shift based on frame phase information generated based on the position shift in the first image position shift unit.
The image processing system according to claim 1,
The image processing system, wherein the second image position shift unit performs position shift based on the decoded image decoded by the decoding unit.
The image processing system according to claim 1,
The image processing system, wherein the first image position shift unit performs position shift based on command information received via the communication network.
The image processing system according to claim 7.
An image processing system comprising: a determination unit that determines the command information.
The image processing system according to claim 1,
The aliasing distortion reduction unit includes a motion adaptation processing unit that performs aliasing reduction processing on a still area of the image and outputs an image that is position-shifted by the second image position shifter in the moving area of the image. An image processing system characterized by that.
Codes that are a plurality of temporally continuous images, each shifted to one of a plurality of predetermined shift positions, reduced in number of pixels, reduced, encoded, and sent via a communication network A decoding unit for decoding the decoded image to generate a decoded image;
An image enlarging unit for enlarging by increasing the number of pixels of the decoded image decoded by the decoding unit;
An image position shift unit that shifts the position of the image enlarged by the image enlargement unit so as to cancel the position shift;
A aliasing distortion reducing unit that performs aliasing distortion reduction processing for reducing aliasing distortion of the image that has been position shifted by the image position shifting unit using information of another image that has been position shifted by the image position shifting unit; An image processing apparatus.
A first image position shift unit that shifts the position of the image to any of a plurality of predetermined shift positions for each of a plurality of temporally continuous images;
An image reduction unit for reducing the number of pixels of the image shifted in position by the first image position shift unit;
An encoding unit that encodes the image reduced by the image reduction unit to generate an encoded image;
A decoding unit that decodes the encoded image to generate a decoded image, an image enlarging unit that increases the number of pixels of the decoded image decoded by the decoding unit, and an enlargement by the image enlarging unit A second image position shift unit that shifts the position of the image shifted by the first image position shift unit to cancel the position shift performed by the first image position shift unit; The processing apparatus including a aliasing distortion reducing unit that performs aliasing distortion reduction processing for reducing distortion using information of another image whose position is shifted by the second image position shifting unit is configured to transmit the code via a communication network. An image processing apparatus for sending a digitized image.