JP5103314B2 - Image signal processing apparatus, image signal processing method, and video display apparatus - Google Patents

Description

  The present invention relates to a technique for increasing the resolution of an image signal.

  Recent television receivers have become larger in screen size and do not display image signals input from broadcasting, communication, storage media, etc., as they are, but display them by increasing the number of pixels in the horizontal and vertical directions by digital signal processing. It is generally done. In this case, the resolution cannot be increased only by increasing the number of pixels by a generally known low-pass filter using a sinc function, a spline function, or the like.

  Therefore, Patent Document 1 discloses a technique for increasing the number of pixels while increasing the resolution by combining a plurality of input image frames (hereinafter abbreviated as “frames”) into one frame.

  Similarly, Patent Document 2 discloses a technique for realizing high resolution by performing weighted addition of each image signal based on each phase difference of three input image frames and removing aliasing components.

  Also, a technique for realizing high resolution in a two-dimensional direction composed of a horizontal direction and a vertical direction by weighting and adding each image signal based on each phase difference of nine input image frames and removing aliasing components. 3 is disclosed.

JP 2007-324789 JP-A-8-336046 JP-A-9-69755

  In the current digital television broadcasting using terrestrial waves and satellites (BS, CS), programs are broadcast with HD (High Definition) image signals in addition to conventional SD (Standard Definition) image signals. Even a broadcast with a high pixel number signal such as an HD image signal is not an image signal obtained by shooting all programs with an HD camera (high resolution camera). For example, in some programs and scenes, it is often the case that an image signal shot with a low pixel number camera such as an SD camera is converted (up-converted) into a high pixel number signal such as an HD image signal before broadcasting. Are known.

  Even if the conventional resolution conversion processing shown in Patent Documents 1, 2, and 3 is performed on such an up-converted signal, there is a problem that the effect of increasing the resolution is small.

  The present invention has been made in view of the above-described problems of the prior art, and an object thereof is to improve the resolution of an image signal more suitably.

  In order to achieve the above object, an embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, it is possible to more suitably increase the resolution of an image signal.

  Embodiments of the present invention will be described below with reference to the drawings.

Moreover, in each drawing, the component to which the same code | symbol is attached | subjected shall have the same function.

  Further, the expression “phase” in each description and each drawing of this specification includes the meaning of “position” on a two-dimensional image when used in a two-dimensional image space. The said position means the position of decimal pixel precision.

  In addition, the expression “up-conversion” in each description and each drawing of the present specification means conversion processing for increasing the number of pixels of an image (pixel number increase processing) or conversion processing for enlarging an image (image enlargement conversion processing). To do.

  Further, the expression “motion compensation” in each description and each drawing of the present specification includes a meaning of performing alignment by calculating a phase difference or a sampling phase difference, that is, a spatial position difference.

  In addition, the expression “Re axis” in each description and each drawing in this specification means a real number axis.

  In addition, the expression “Im axis” in each description and each drawing in this specification means an imaginary axis.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an image signal processing apparatus according to Embodiment 1 of the present invention, and features thereof will be described. The image signal processing apparatus according to this embodiment is applied to a video display apparatus such as a television receiver. In the following description of the present embodiment, a video display device will be described as an example of the image signal processing device.

  In FIG. 1, an image signal processing apparatus according to the present embodiment includes an input unit (1) to which a frame sequence of a moving image such as a television broadcast signal is input, and a frame input from the input unit (1). A resolution conversion unit (2) for increasing the resolution and a display unit (3) for displaying an image based on the frame whose resolution is increased by the resolution conversion unit (2) are further provided. Here, the input unit (1) may have a channel selection, demodulation, and stream separation function, if necessary. Also, an input terminal for inputting video from another device may be used. As the display section (3), for example, a plasma display panel, a liquid crystal display panel, or an electron / electrolytic emission display panel is used. Details of the resolution conversion unit (2) will be described below.

  The resolution converter (2) converts the SD-to-HD converted (up-converted) video on the broadcast station side and the low-resolution video that has undergone pixel number conversion processing on the receiving side, such as an LSI chip, into high-resolution video. To do. That is, in the example of FIG. 1, the video input to the input unit (1) may already be converted from SD to HD (up-conversion) on the broadcast station side, and the video signal input to the input unit (1) For example, after the pixel number conversion processing by the pixel number conversion unit (107), it may be input to the resolution conversion unit (2). Hereinafter, a specific configuration and operation of the resolution conversion unit (2) will be described in detail.

In FIG. 1, first, the original component extraction unit (101) uses the signals of frame # 1 and frame # 2 input to the original component extraction unit (101) to generate the original component of frame # 1 (for example, in FIG. The component (301) shown) is extracted. At this time, in the original component extraction process by the original component extraction unit (101), information on the enlargement ratio (m / n) is used. The information on the enlargement ratio (m / n) may be input through the input unit (5), but may be stored in advance in the storage unit (108). Further, it may be input together with frame # 1 and frame # 2 in a state included in the stream data.

  Subsequently, the aliasing component extraction unit (103) extracts the aliasing component of frame # 1 (for example, the component (305) shown in FIG. 3). At this time, if the subtracter (102) is used to subtract the output signal of the original component extraction unit (101) from the signal of frame # 1 input to the original component extraction unit (101), the aliasing component extraction unit (103) can be realized. The frequency conversion unit (104) returns the extracted aliasing component to the original high frequency component of the aliasing component of frame # 1. Here, in the process of the frequency conversion unit (104), information on the reference phase (δ) is used. Here, the reference phase (Δ) is phase information indicating a position in an image that is a reference for enlargement processing in the above-described SD → HD conversion on the broadcasting station side and pixel number conversion processing on the receiving side. The information on the reference phase (δ) may be input from the input unit (5), but may be stored in advance in the storage unit (108). Further, it may be input together with frame # 1 and frame # 2 in a state included in the stream data. Next, the high frequency component is added or mixed with the original component by the adder (105) to obtain an output signal of the high frequency component reproduction unit (106). Note that the addition and mixing processing in the adder (105) may be simple addition or may be added by multiplying the original component and the high frequency component by respective coefficients. That is, any function may be used as long as it is a mixture using a function having the original component and the high frequency component as variables.

  Here, as shown in FIG. 2, the original component extraction unit (101) removes or reduces the aliasing component without changing the number of pixels. Details of the operation of the original component extraction unit (101) will be described below.

In FIG. 2, the original component extraction unit (101) receives two image frames # 1 and # 2 from the input unit (3), removes or reduces aliasing components, and removes or reduces aliasing components. The original component of is output. The original component extraction unit (101) performs phase shift on each image signal of two input image frames by position estimation (201) and phase shift (216), and generates two signals from each image signal. To do.
Thereby, four signals can be generated from the image signals of the two input image frames. Based on the phase difference between the two input image frames, a coefficient for canceling and synthesizing the aliasing components of the four signals is calculated for each pixel based on the phase difference between the two input image frames. To do. For each pixel of the image to be generated, the sum of the pixel value of the corresponding pixel included in each of the four signals is multiplied by each coefficient to calculate the pixel value of the original component image. By performing this for each pixel of the generated image, an original component image can be generated. Thereby, an image with fewer aliasing components can be generated using two input image frames.

  As described above, the processing target signal assumed in the processing of the resolution conversion unit (2) according to the present embodiment is SD → HD converted (up-converted) on the broadcasting station side and input to the input unit (1). Low-resolution video that has undergone pixel number conversion, such as video or video that has been input to the input unit (1) and has been increased in number of pixels by a pixel number conversion unit (107) such as an LSI chip (i.e., an enlarged video) is there. For example, when a signal in which the number of pixels of a low-resolution video is increased by m / n times (where m and n are integers) is input, the sampling frequency (fs) before pixel number conversion and the sampling after pixel number conversion It is assumed that a relational expression of fs = fs ′ * n / m holds between the frequency (fs ′). At this time, the number of pixels is calculated by the phase difference θ virtually estimated based on the sampling frequency (fs) before the pixel number conversion and the position estimation unit (201) in the original component extraction unit (101) shown in FIG. Similarly, a relational expression of θ = θ ′ * n / m is established between the phase difference θ ′ (1202) actually estimated based on the converted sampling frequency (fs ′).

  In the position estimation unit (201) shown in FIG. 2, the correspondence on the frame # 2 is determined based on the sampling phase (sampling position) of the pixel to be processed on the frame # 1 input to the original component extraction unit (101). The position of the target pixel is estimated, the sampling phase difference θ ′ (1202) is obtained for each pixel based on the sampling frequency (fs ′) after the number of pixels conversion, and the motion compensation unit (1201) moves the frame # 2 After compensation and alignment with frame # 1, the aliasing component is removed or reduced using the phase shift unit (216) and aliasing component removal unit (217).

  FIG. 3 shows the relationship between the aliasing component and the original component in the above operation of the original component extraction unit (101). Here, the frequency fs shown in the figure is the sampling frequency before performing the pixel number conversion such as up-conversion, and fs = fs between the sampling frequency (fs ′) after the pixel number conversion as described above. It is assumed that the relational expression '* n / m holds. This figure shows the outputs of the delay units (205) and (207) and the π / 2 phase shifters (206) and (208) shown in FIG. 2 in a one-dimensional frequency domain. In the figure (a), the signals of frame # 1 and frame # 2 output from the delay units (205) and (207) are respectively folded back from the original components (301) and (302) and the original sampling frequency (fs). The signal obtained by adding the folded components (305) and (306). At this time, the phase of the aliasing component (306) is rotated by the phase difference θ (202). Here, the phase difference θ (202) is the sampling phase (sampling) of the pixel to be processed on frame # 1 input to the original component extraction unit (101) by the position estimation unit (201) shown in FIG. Position) as a reference, the phase difference θ ′ (1202) as a result of estimating the position of the corresponding pixel on the frame # 2, that is, the “deviation amount” between the corresponding pixels is expressed as θ = θ ′ * n / It is a value normalized by the pixel interval (= sampling interval = 2π) after conversion by the relational expression of m.

  On the other hand, the signals of frame # 1 and frame # 2 output from the π / 2 phase shifter (206) (208) are the original components (303) (304) after π / 2 phase shift and π / 2, respectively. The signal is obtained by adding the aliasing components (307) and (308) after the phase shift. (B) and (c) show the original component and the aliasing component extracted for easy understanding of the phase relationship between the components shown in (a).

  Here, when the vector sum of the four components shown in FIG. 4B is taken, the Re-axis component is set to 1, the Im-axis component is set to 0, and the four components shown in FIG. When the vector sum is taken, the coefficients to be multiplied by each component are determined so that both the Re-axis and Im-axis components are set to 0, and the weighted sum is taken. Only the components can be extracted.

  A low pass filter is generally used for SD → HD conversion (up-conversion) on the broadcasting station side and pixel number conversion in the pixel number conversion unit (107). At this time, the signal intensity for each frequency of each component changes according to the frequency characteristics of the low-pass filter, but between the original components (301) and (302) shown in FIG. 3 or the aliasing components (305) (306). Between (307) and (308), the degree of change in signal intensity for each frequency is the same, so that the coefficient determination unit (209) replaces the phase difference θ used in the conventional resolution conversion unit (4). The aliasing component can be canceled by determining the coefficients (C00, C10, C20, C30) using the phase difference (θ ′ * n / m).

  Here, the determination method of each coefficient (C00, C10, C20, C30) generated by the coefficient determiner (209) will be described with reference to FIG.

  First, FIG. 13 shows the operation and specific example of the coefficient determiner (209). As shown in FIG. 3A, when the vector sum of the four components shown in FIG. 3B is taken, the Re-axis component is set to 1, the Im-axis component is set to 0, and FIG. If the coefficients to be multiplied by each component are determined so that both the Re-axis and Im-axis components are 0 when the vector sum of the four components shown in (c) is taken, two frame images It is possible to realize an image signal processing apparatus that extracts an image with a small amount of aliasing components in the one-dimensional direction.

  Here, as shown in FIG. 2, the coefficient for the output of the delay unit (205) (the sum of the original and folded components of frame # 1) is C00, and the output of the π / 2 phase shifter (206) (frame # 1 The coefficient for the original component and the folded component of each π / 2 phase shift result) is C10, and the output of the delay unit (207) (the position estimation unit (201) and the motion compensation unit (1201) As a reference, the coefficient for the frame # 2 original component of frame # 2 compensated with decimal pixel accuracy for each pixel and the sum of the folded components is C20, and the output of the Hilbert transformer (208) (the original of frame # 2 subjected to motion compensation as described above) The coefficient for the sum of the π / 2 phase shift results of the component and the aliasing component is C30. As described above, the phase difference θ virtually estimated based on the sampling frequency (fs) before the pixel number conversion and the position estimation unit (201) in the original component extraction unit (101) shown in FIG. A relational expression of θ = θ ′ * n / m holds between the phase difference θ ′ (1202) actually estimated based on the sampling frequency (fs ′) after the pixel number conversion. For this reason, as shown in FIG. 13 (a), (1) Sum of Re-axis of original component = 0, (2) Sum of Im-axis of original component = 0, (3) Sum of Re-axis of folded component = 1 (4) When the condition that the sum of the Im axis of the aliasing components = 0 is satisfied, the phase difference θ based on the sampling frequency (fs) before conversion of the number of pixels is used, and FIG. From the phase relationship of each component shown in 3 (c), simultaneous equations shown in FIG. 13 (b) can be obtained. When this is solved, the result shown in FIG. 13C can be derived. The coefficient determiner (209) outputs coefficients C00, C10, C20, and C30 that satisfy any one of FIGS. 13 (a), 13 (b), and 13 (c).

  As an example, the values of the coefficients C00, C10, C20, and C30 when the phase difference θ based on the sampling frequency (fs) before pixel number conversion is changed from 0 to 2π every π / 8 are shown in FIG. Shown in d).

  As described above, the coefficient determiner (209) generates each coefficient (C00, C10, C20, C30), and the aliasing component removal unit (217) of the original component extraction unit (101) shown in FIG. It is possible to cancel the aliasing component by using.

  Next, an overall operation example of the resolution conversion unit (2) in FIG. 1 will be described in comparison with a conventional resolution conversion example of a video signal.

  First, an example of resolution conversion processing of a conventional video signal will be described with reference to FIGS. In the following example, a case where video is subjected to SD → HD conversion (up-conversion) on the broadcast station side will be described.

  In current digital television broadcasting using terrestrial waves and satellites (BS, CS), programs are broadcast with HD (High Definition) image signals in addition to SD (Standard Definition) image signals. As shown in FIG. 4, not all programs are image signals captured by the HD camera (401), but the image signals captured by the SD camera (402) are converted into an SD → HD converter (403). It is well known that the signal is converted (up-converted) into a signal having the same number of pixels as HD, switched for each program or scene by the switch (404), and output to the transmission line (405). Yes. The transmission path (405) includes not only broadcasting but also communication, storage (recording), etc., and also includes signal processing such as image encoding and multiplexing.

  As described above, when an image signal whose number of pixels has been converted in advance is input, even if the conventional resolution conversion processing as disclosed in Patent Document 1 is applied as it is, the effect of increasing the resolution is small. The details will be described below.

  FIG. 5 shows a general configuration of the SD → HD converter (403). For example, when changing the number of pixels to m / n times (where m and n are integers), the up-rate unit (501) first up-times m times (i.e., arranges input data for each m pixels in order). Then, insert (0) into the (m-1) pixels in the meantime, and then apply a low-pass filter (502) with the specified characteristics, and reduce it to 1 / n times with the down rate device (503). Down rate (that is, one pixel is selected at equal intervals for every n pixels and thinned out). By performing this pixel number conversion process independently in the horizontal direction and the vertical direction, a two-dimensional input image frame can be converted into a size of an arbitrary number of pixels.

  FIG. 6 shows the frequency spectrum of the output signal of each part shown in FIG. 5, assuming that the horizontal enlargement ratio when converting from an SD image to an HD image is 8/3 (= 1920/720) as an example of the operation. It is. 6A to 6E, the horizontal axis indicates the frequency f, the vertical axis indicates the intensity of the signal component, and the upward arrow (601) indicates the position (frequency) of the sampling carrier (sine wave). Further, the horizontal sampling frequency of the SD image is fs, and the horizontal sampling frequency of the HD image after SD → HD conversion is fs ′ (= 8/3 fs).

  FIG. 6 (a) shows the frequency spectrum of the output signal of the SD camera (402), and it is assumed that it originally contains a frequency component equal to or higher than the Nyquist frequency (= fs / 2). In order to enlarge this signal by 8/3 times in the horizontal direction, each process of 8 times up rate, low pass filter and 1/3 times down rate is performed in the SD → HD conversion (403).

  FIG. 6 (b) shows the frequency spectrum of the output signal of the 8-times up-rate device (501) in the SD → HD conversion (403). The sampling carrier interval shown in FIG. It shows that it spreads.

  FIG. 6 (c) shows the frequency spectrum of the output signal of the low-pass filter (502) in the SD → HD conversion (403). Assuming that the cut-off frequency of the low-pass filter (502) is fs / 2, only the frequency components in the vicinity of f = 0 and f = 8fs are left as shown in the figure, and the other frequency components are removed or reduced. At this time, the frequency component equal to or higher than the Nyquist frequency (= fs / 2) remains as a folded component.

  FIG. 6 (d) shows the frequency spectrum of the output signal of the 1/3 times down-rate device (503) in the SD → HD conversion (403), and the sampling frequency fs ′ (= 8 fs / after SD → HD conversion). A new sampling carrier is generated at an integer multiple of 3). This shows that the frequency component that has passed through the low-pass filter (502) is convoluted with respect to this sampling carrier.

  FIG. 6 (d) ′ is a diagram in which the scale of the horizontal axis in FIG. 6 (d) is converted to make it easy to see the range of 0 ≦ f ≦ fs. The SD flowing through the transmission line (405) shown in FIG. → Shows the frequency spectrum of the signal after HD conversion. Here, regarding the original component shown in FIG. 6 (d) ′, the frequency component equal to or higher than the Nyquist frequency (= fs / 2) is attenuated by the low-pass filter (502) and does not remain in the original position. This is very different from the frequency spectrum of the original signal shown in FIG. That is, even if the conventional resolution conversion processing is performed on this signal as it is, there is only an effect of removing or reducing the aliasing component, so that the frequency spectrum shown in FIG. 6 (e) is obtained, and the LPF (502) The frequency component exceeding the pass band (602) cannot be reproduced, and the effect of increasing the resolution is small.

  Next, an example of the overall operation of the resolution conversion unit (2) in FIG. 1 will be described with reference to FIG.

  Here, as an example, it is assumed that a signal obtained by enlarging (up-converting) the signal from the SD camera (402) by 8/3 times by the SD → HD converter (403) is input with the configuration shown in FIG. The operation of each unit in the resolution conversion unit (2) will be described below.

  FIG. 7 (a) shows the frequency spectrum of the input signal to the resolution conversion unit (2), which is the same spectrum as FIG. 6 (d ') used for the description of the operation of the prior art described above. Here, if the sampling frequency before being expanded (up-converted) by 8/3 is fs, the high-frequency component of the original signal exceeding the fs / 2 frequency is blocked by the low-pass filter (502) described above. And converted to a folded component with fs / 2 as the axis of symmetry. Depending on the cutoff characteristic of the low-pass filter (502), a high-frequency component may remain, but this residual component does not significantly affect the operation of the present embodiment, so illustration and description thereof are omitted.

  The original component extraction unit (101) shown in FIG. 1 receives the signal of FIG. 7 (a) as an input, removes or reduces the aliasing component, and outputs the signal of FIG. 7 (b). Further, the aliasing component extraction unit (103) subtracts the signal of FIG. 7 (b) from the signal of FIG. 7 (a) and outputs the signal of FIG. 7 (c).

  The frequency converter (104) shown in FIG. 1 returns the signal shown in FIG. 7 (c) to the original frequency spectrum by folding it back to a high frequency with fs / 2 as the axis of symmetry. The detailed configuration and operation of the frequency conversion unit (104) will be described later.

  In the high frequency component reproduction unit (106), the signal of FIG. 7 (b) and the signal of FIG. 7 (d) are added to obtain the signal of FIG. 7 (e), which is output from the resolution conversion unit (2). At this time, a higher frequency component can be reproduced than the spectrum of FIG. 6 (e) used for explaining the operation of the prior art.

  Next, FIG. 8 shows a configuration example of the frequency conversion unit (104). As described above, the frequency conversion unit (104) inverts the low frequency side and the high frequency side of the frequency spectrum of the input signal with fs / 2 as the axis of symmetry, and returns and outputs the original frequency spectrum. In order to realize this operation, as is generally known, a sine wave of amplitude 1 (= cos (2πfs * t + δ)) with 1 / fs as one cycle as shown in the figure is output. After preparing a carrier (carrier wave) generator (803) and multiplying an input signal and a carrier using a multiplier (801), unnecessary frequency components may be removed or reduced by a low-pass filter (802).

  Further, the operation of the frequency conversion unit (104) will be described in detail with reference to FIG. FIG. 9A shows the frequency spectrum of the input signal of the frequency conversion unit 104, that is, the output signal of the aliasing component extraction unit 103, which is the same as the frequency spectrum shown in FIG. . FIG. 9 (b) shows the frequency spectrum of the carrier for frequency inversion, which is a sine wave having 1 / fs as one cycle, and thus a line spectrum of frequency ± fs. FIG. 5C shows the frequency spectrum of the output signal of the multiplier 801, which is the result of convolving the signals of FIG. 5A and FIG. 5B in the frequency domain. FIG. 4D shows the pass band of the low-pass filter 802, and only the frequency component of −fs ≦ f ≦ fs in the signal of FIG. The frequency spectrum of the output of the low-pass filter (802) is as shown in Fig. (E), and the low and high frequencies of the input signal (Fig. (A)) are inverted with fs / 2 as the axis of symmetry. Results are obtained. It is obvious that the same effect can be obtained by using a band-pass filter instead of the low-pass filter (802) and passing only the frequency component shown in FIG.

  In FIG. 7 (e) described above, in order to correctly connect the original component and the high frequency component reproduced from the folded component, including the phase, the reference phase of the carrier generated by the carrier generator (803) ( The relationship between δ) and the positions of the pixels before and after enlargement may be set as shown in FIG. This will be described in detail below.

  (A) Before enlargement when the signal from the SD camera (402) is enlarged (up-converted) by 8/3 times by the SD → HD converter (403) according to the configuration shown in FIG. 10 (a), (b), and (c) show the positional relationships of the above signal, (b) the signal after enlargement, and (c) the carrier signal. As shown in the figure (a), when the pixel (1001-1) (1001-2) (1001-3) (1001-4) of the signal before enlargement is used, the signal after enlargement to 8/3 times These pixel positions are assumed to be the pixels (1002-1), (1002-2), (1002-3),... (1002-9) shown in FIG. That is, it is assumed that the positions of the pixel (1001-1) and the pixel (1002-1) and the positions of the pixel (1001-4) and the pixel (1002-9) are enlarged so as to match. At this time, using the positions of the pixels (1001-1) and (1002-1) and the positions of the pixels (1001-4) and (1002-9) as reference phases, the carrier signal shown in FIG. cos) is set to zero. By setting in this way, the value of the carrier signal becomes 1 (= cos0) at the position of the reference phase, and the output of the frequency converter (104) shown in FIG. 8 (that is, the aliasing component shown in FIG. 7 (d)). And the phase of the original component shown in FIG. 7B can be matched.

  The operation shown in FIG. 9 does not consider the influence of the sampling frequency (fs ′) after the pixel number conversion, but this can be solved by a generally well-known oversampling technique. That is, when the aliasing component newly generated by the sampling frequency (fs ′) after the pixel number conversion becomes an obstacle and a part of the frequency spectrum is overlapped, the desired component shown in FIG. Sampling of fs' → fs' 'using another sampling frequency (fs'', but fs' '>> fs and fs'' >> fs') that is sufficiently higher than the sampling frequency (fs, fs') If the operation shown in FIG. 9 is performed after frequency conversion (pixel number conversion) and fs ″ → fs ′ sampling frequency conversion is performed again, the above-described frequency inversion operation can be realized.

  Patent Document 4 discloses a technique for improving the above oversampling technique and performing frequency conversion by multiplication with a complex sine wave. According to this technique, the method using a sine wave multiplication and a filter cannot distinguish between positive and negative frequencies because of real signal processing, and when a wideband signal is shifted by a small amount of frequency, or when the aliasing component due to sampling becomes an interference, Problems such as the need to shift in two steps are solved. In addition, since frequency conversion is performed by multiplication of complex sine waves, even if the sum signal and difference signal overlap in a wideband signal as in the case of real signal processing, or if aliasing due to sampling interferes, Processing can be performed by frequency conversion, and the number of components can be reduced.

FIG. 11 shows an example in which the frequency conversion unit (104) is configured using the technique described in Japanese Patent Application Laid-Open No. 1-292904. In the configuration shown in FIG. 8, the carrier (cos wave) generated by the carrier generator (803) cannot distinguish between positive and negative frequencies. However, in the configuration shown in FIG. 11, the carrier generator (1105) (1106) ) Simultaneously generates a cos wave and a sin wave having a phase difference of π / 2 (i.e., orthogonal), and generates a quadrature component of the input signal by a π / 2 phase shifter (1101), and a multiplier (1102) ( 1103) and adder (1104) can be used to simultaneously perform frequency inversion and removal or reduction of the aliasing component (interference) by sampling as described above. Since this operation is described in detail in Patent Document 4, a description thereof is omitted here. As the π / 2 phase shifter (1101), the Hilbert transformer described in Patent Document 1 may be used as it is.

  In the above description, the case where the input signal includes a horizontal folding component is described as an example. However, the present invention is not limited to this, and the input signal is folded in the vertical direction or the time direction. It is clear that the present invention can be applied to a case where a component is included or a case where a folded component in a two-dimensional or three-dimensional direction is included. In this case, the filter, phase shift, carrier (sinusoidal wave), etc. used in each part may be read and applied in the direction in which the aliasing component is generated. For example, when the input signal includes a folding component in the vertical direction, a carrier having a vertical filter, a vertical phase shift, and a vertical frequency may be used. For example, the process of converting an interlaced scan format signal into a progressive scan format signal (IP conversion) is an up-conversion process that doubles the number of pixels in the vertical direction (number of scan lines), so it is IP-converted. As an input image frame of the image signal processing apparatus according to the first embodiment, if m = 2 and n = 1 at the above-described enlargement ratio m / n (where m and n are integers), the vertical direction Resolution conversion processing can be realized.

  The example of the conventional resolution conversion processing of the video signal described with reference to FIGS. 4 to 6 and the operation and configuration example of the resolution conversion unit (2) according to the present embodiment described with reference to FIGS. Also, the case where the video is converted from SD to HD (up-conversion) on the broadcasting station side has been described. However, even when the video is converted at the receiving side, that is, the pixel number converting unit (107) of the image signal processing unit according to the present embodiment, the resolution conversion process only changes from the broadcasting station side to the receiving side, Other explanations are the same as the above explanation.

  The image signal processing apparatus according to the first embodiment described above has an effect that a high resolution can be realized even for an image signal whose number of pixels has been converted in advance.

  Next, Embodiment 2 of the present invention will be described with reference to FIGS. In the second embodiment, an operation equivalent to that of the resolution conversion unit (2) according to the first embodiment is performed by changing to another configuration. Hereinafter, the specific configuration and operation will be described in detail.

  In FIG. 12, the operation of each unit of the original component extraction unit (101) is the same as the operation of each unit of the original component extraction unit (101) according to the first embodiment, and thus the description thereof is omitted. On the other hand, the aliasing component extraction units (1203) and (1210) have the same configuration as the aliasing component removal unit (217) described above, but the coefficients (C02, C12, C22, C32) (C03, C13, C23, C33) are set as described below, and the aliasing components are extracted in such a way that the phases of the output signals of the aliasing component extraction units (1203) and (1210) are orthogonal to each other. To do. That is, the output of the folding component extraction unit (1203) is made substantially the same as the folding component output from the subtractor (102) shown in FIG. 1 according to the first embodiment, and the output of the folding component extraction unit (1210). Is substantially the same as the aliasing component output from the π / 2 phase shifter (1101) of FIG. 11 according to the first embodiment. The configurations of the frequency conversion unit (104) and the high frequency component reproduction unit (106) are the same as the configurations of the units shown in FIGS.

  Subsequently, with reference to FIGS. 13 to 15, the coefficients (C00, C10, C20, C30) (C02, C12, C22, C32) (C03, C30, C30) generated by the coefficient determiners (209) (1209) (1216) are used. A method for determining C13, C23, and C33) will be described.

  First, the coefficient determination operation of the coefficient determiner (209) is the same as the coefficient determination operation of FIG.

  Next, FIG. 14 shows an operation and a specific example of the coefficient determiner (1203). In the coefficient determiner (1203), as shown in FIG. 14 (a), (1) the sum of the Re-axis of the original component = 0, (2) the sum of the Im-axis of the original component = 0, (3) By setting the sum of the Re axes = 1 and (4) the sum of the Im axes of the folded components = 0, the simultaneous equations shown in FIG. 14B can be obtained. Solving this, it is possible to derive coefficients (C02, C12, C22, C32) for extracting only the Re-axis signal of the folded component as shown in FIGS.

  FIG. 15 shows the operation and specific example of the coefficient determiner (1216). Similarly to the above, in the coefficient determiner (1216), as shown in FIG. 15 (a), (1) the sum of the Re-axis of the original component = 0, (2) the sum of the Im-axis of the original component = 0, The simultaneous equations shown in FIG. 15B can be obtained by setting 3) the sum of the Re-axis of the folding component = 0 and (4) the sum of the Im-axis of the folding component = 1. Solving this, it is possible to derive coefficients (C03, C13, C23, C33) for extracting only the Re-axis signal of the aliasing component as shown in FIGS. 15 (c) and 15 (d).

  By using the coefficients thus obtained, the outputs (folded components) of the folded component extraction units (1203) and (1216) shown in FIG. 12 are in phase with each other. Accordingly, the π / 2 phase shift (1101) in the frequency conversion unit (104) shown in FIG. 11 can be eliminated, and the multipliers (1102) and (1103) are added to the frequency conversion unit (104) shown in FIG. The above-described frequency conversion operation can be realized by inputting the outputs of the aliasing component extraction units (1203) and (1216) as they are.

  As can be seen from FIGS. 13 to 15, each coefficient (C00, C10, C20, C30) (C02, C12, C22, C32) (C03, C13, C23, C33) is equal to the same phase difference θ. Since the arrangement order of the coefficients is different, it is clear that the circuit may be simplified by unifying the coefficient determiners (217), (1209), and (1216) in FIG. Further, as can be seen from FIGS. 13 and 14, since the relationship of C02 = 1−C00, C12 = −C10, C22 = −C20, C32 = −C30 is satisfied, the sum of the re-axis of the aliasing component = frame # 1 This is the total relationship between the signal and the original component Re axis. Accordingly, by using a subtracter (102) such as the aliasing component extraction unit (103) shown in FIG. 1, the aliasing component extraction is performed by subtracting the output signal of the aliasing component removal unit (217) from the output of the delay unit (205). The circuit of the aliasing component extraction unit (1203) may be simplified instead of the output signal of the unit (1203).

  When the calculation formulas shown in FIGS. 13 to 15 are combined into one, the matrix calculation formula shown in FIG. 16 is obtained. That is, the left side partial matrices (1601), (1602), and (1603) in FIG. 16 (a) represent the values on the right side of each (a) in FIGS. Further, the matrix (1604) of the first term on the right side in FIG. 16 (a) is the coefficient (C00, C10, C20, C30) (C02, C12, C22, C32) (C03, C13, C23, and C33) are multiplied. In addition, the submatrix (1605) (1606) (1607) of the second term on the right side in FIG. 16 (a) is represented by the coefficients (C00, C10, C20, C30) (C02, C12, C22, C32) (C03, C13). , C23, C33) itself. When this matrix arithmetic expression is solved, each coefficient can be obtained as shown in FIG. Depending on the value of the phase difference θ, when the solution of each coefficient becomes indefinite and cannot be obtained, the coefficient C00 = 1 is forcibly set and other coefficients (C10, C20, C30, C02, C12, C22, The signal of frame # 1 input to the resolution conversion unit (2) may be output as it is, for example, by setting all of C32, C03, C13, C23, and C33) to 0.

  In addition to the effects of the image signal processing apparatus according to the first embodiment, the image signal processing apparatus according to the second embodiment described above includes oversampling for preventing unnecessary aliasing components (interferences) due to sampling in the frequency converter. π / 2 phase shift (Hilbert transform) is not required, and it is possible to reduce the number of components and the processing time (delay time) caused by oversampling or π / 2 phase shift.

The third embodiment of the present invention includes an original component extraction unit (101) in the configuration of the image signal processing apparatus shown in FIG.
Instead of this, an original signal extraction unit (2014) shown in FIG. 20 is used to realize an image signal processing apparatus that increases the resolution by using three input frame images.

  The operation of the original component extraction unit (2014) is an improvement of the technique described in Patent Document 2. Therefore, first, the operation of the prior art described in Patent Document 2 will be described with reference to FIG. In the technique described in Patent Document 2, as shown in FIGS. 3A, 3B, and 3C, the aliasing component is canceled using signals of three frames (# 1, # 2, and # 3), Only the original signals (1701) (1702) (1703) are taken out. At this time, with reference to the phase of the aliasing component (1704) included in frame # 1, the phase difference α of the aliasing component (1705) of frame # 2 \ and the phase difference β of the aliasing component (1706) of frame # 3 The coefficients (C00, C10, C20) to be multiplied by the signals of the respective frames (# 1, # 2, # 3) are obtained by estimating for each pixel and solving the matrix arithmetic expression shown in FIG. This coefficient (C00, C10, C20) is calculated from the signal of frame # 1 (sum of original component (1701) and aliasing component (1704)) and the signal of frame # 2 (original component (1702) and aliasing component (1705)). Sum) and frame # 3 signal (the sum of the original component (1703) and the aliasing component (1706)), respectively, and weighted addition to cancel the aliasing components (1704), (1705) and (1706). Only components (1701) (1702) (1703) are extracted. Since this operation is described in detail in Patent Document 2, detailed description thereof is omitted.

  Similarly to the conventional technique described in Patent Document 1 described above, the conventional technique described in Patent Document 2 has only an effect of removing or reducing the aliasing component. When input, the effect of increasing the resolution is small.

  On the other hand, in the original component extraction unit (2014) shown in FIG. 20 according to the third embodiment of the present invention, the input signal of frame # 1 is sent to the aliasing component removal unit (2013) via the delay unit (2001). input. In addition, the input signal of frame # 2 is detected by the position estimation unit (2002) by the phase difference α ′ (2011) between corresponding pixels of frame # 2 and frame # 1, that is, the “deviation amount” between the corresponding pixels. Is calculated for each pixel, and a signal in which the shift amount is compensated with decimal pixel accuracy is generated by the motion compensation unit (2003) and input to the aliasing component removal unit (2013). Similarly, the input signal of frame # 3 is output from the position estimation unit (2004) by the phase difference β ′ (2012) between corresponding pixels of frame # 3 and frame # 1, that is, the “deviation amount between corresponding pixels”. ”Is obtained for each pixel, and a signal in which the shift amount is compensated with decimal pixel accuracy is generated by the motion compensation unit (2005) and input to the aliasing component removal unit (2013). At this time, as in the first embodiment, the up-rate (broadband interpolation) described in Patent Document 2 is not necessary, and it is not necessary to interpolate pixels (sampling points) to increase the density of image data.

  The aliasing component removal unit (2013) converts the phase difference (α ′, β ′) described above into a phase difference (α, β) estimated based on the sampling frequency (fs) before the pixel number conversion. The coefficient (C00, C10, C20) is determined for each pixel by the coefficient determiner (2006), and each signal is weighted and added using the multipliers (2007) (2008) (2009) and the adder (2010). As a result, the aliasing component is removed or reduced and used as the output of the original component extraction unit (2014).

  Here, each phase difference (α, β) used in the coefficient determiner (2006) is the sampling frequency (fs) before performing the pixel number conversion that increases the number of pixels by m / n times (where m and n are integers). ) As a reference, and α = α ′ * n / m between the phase difference (α ′, β ′) estimated based on the sampling frequency (fs ′) after conversion of the number of pixels. And β = β '* n / m. If this phase difference (α, β) is used, the coefficients (C00, C10, C20) can be determined using the matrix calculation formula shown in FIG.

  The image signal processing apparatus according to the third embodiment can use the output (original component) of the original component extraction unit (2014) as it is as the output (original component) of the original component extraction unit (101) shown in FIG. . Therefore, the configuration and operation of each part other than the original component extraction unit (101) in the resolution conversion unit (2) shown in FIG. 1, that is, the configuration and operation of the aliasing extraction unit (103) and the high frequency component reproduction unit (106) are as follows. Since this is the same as the image signal processing apparatus according to the first embodiment, the description thereof is omitted.

  According to the image signal processing apparatus according to the third embodiment described above, three images included in the low-resolution video whose number of pixels has been converted in the preceding process, such as a video that has been converted from SD to HD (up-conversion) on the broadcast station side. Using this image, the aliasing component can be reduced and the high frequency component can be reproduced to generate a high-resolution image.

  Next, Embodiment 4 of the present invention will be described with reference to FIGS. In the fourth embodiment, an operation equivalent to that of the resolution conversion unit (2) according to the third embodiment is performed by changing to another configuration. Hereinafter, the specific configuration and operation will be described in detail.

  First, the matrix computing equation shown in FIG. 18 according to the third embodiment is expanded to the matrix computing equation shown in FIG. 19A, and each coefficient (C00, C10, C20) (C01, C11, C21) (C02, C12, C22) is obtained. Here, the submatrix (1801) on the left side in FIG. 19 (a) is (1) the sum of the Re-axis of the original component = 1, (2) the sum of the Re-axis of the aliasing component = 0, and (3) of the aliasing component. Indicates that the sum of the Im axis is 0. Similarly, the submatrix (1901) on the left side in FIG. 19 (a) is (1) the sum of the Re-axis of the original component = 0, (2) the sum of the Re-axis of the folded component = 1, and (3) the folded component. In FIG. 19 (a), the submatrix (1902) on the left-hand side of FIG. 19A represents (1) the sum of the Re-axis of the original component = 0, and (2) the Re-axis of the folded component. It represents that the sum is 0, and (3) the sum of the folding axis of the Im axis is 1. The matrix (1802) of the first term on the right side in FIG. 19 (a) is the same as the matrix (1802) in FIG. Each phase difference (α, β) is a phase difference estimated on the basis of the sampling frequency (fs) before the pixel number conversion that increases the number of pixels by m / n times (where m and n are integers). Yes, α = α ′ * n / m and β = β ′ * n / m between the phase difference (α ′, β ′) estimated based on the sampling frequency (fs ′) after pixel number conversion The following relational expression holds.

  Each frame (# 1, # 2, # 3) after motion compensation of each coefficient (C00, C10, C20) (C01, C11, C21) (C02, C12, C22) obtained as shown in FIG. 19 (b) By multiplying this signal, it is possible to extract each component of the sum of the Re axis of the original component, the sum of the Re axis of the folded component, and the sum of the Im axis of the folded component.

  FIG. 21 shows the configuration of the resolution conversion unit (2) according to the fourth embodiment. In the figure, the original component extraction unit (2014) multiplies the signals of each frame (# 1, # 2, # 3) by the coefficients (C00, C10, C20) obtained as shown in FIG. The total component of the Re axis of the component is extracted. Since the operation of each unit of the original component extraction unit (2014) is the same as the operation of each unit of the original component extraction unit (2014) shown in FIG. 20 according to the third embodiment, description thereof is omitted.

  The aliasing component extraction unit (2111) in FIG. 21 uses the multipliers (2102) (2103) (2104) and the adder (2105) to obtain the coefficients (C01, C11, C21) obtained as shown in FIG. ) Is multiplied by the signal of each frame (# 1, # 2, # 3) to extract the total component of the aliasing component on the Re axis.

  Similarly, the aliasing component extraction unit (2112) in FIG. 21 uses the multipliers (2107), (2108), (2109), and the adder (2110) to obtain the coefficients (C02, C12, C22) is multiplied by the signal of each frame (# 1, # 2, # 3) to extract the sum component of the aliasing component on the Im axis.

  Each component of the total of the Re axis of the original component extracted in this way, the total of the Re axis of the folded component, and the total of the Im axis of the folded component is represented by the high frequency component reproduction unit ( 106) and output a high resolution signal. Since the operation of each unit of the high frequency component reproduction unit (106) is the same as the operation of each unit of the high frequency component reproduction unit (106) according to the second embodiment, the description thereof is omitted.

  As can be seen from FIG. 19 (b), the relationship is C01 = 1−C00, C11 = −C10, C21 = −C20, and therefore, as in the aliasing component extraction unit (103) shown in FIG. 21. Instead of the aliasing component extraction unit (2111) shown in FIG. 21, by subtracting the original component (output of the adder (2010)) from the input signal (output of the delay device (2001)) of frame # 1, It is obvious that the total component of the Re axis may be used.

  In addition to the effects of the image signal processing apparatus according to the third embodiment, the image signal processing apparatus according to the fourth embodiment described above includes oversampling for preventing unnecessary aliasing components (interferences) due to sampling in the frequency converter. π / 2 phase shift (Hilbert transform) is not required, and it is possible to reduce the number of components and the processing time (delay time) caused by oversampling or π / 2 phase shift.

  It should be noted that the enlargement ratio (m / n) and the reference phase (δ) in each embodiment of the present invention are parameterized, and the video signal (frame # 1, # 2, # 3, etc.) is transmitted from the transmission side to the reception side. And may be transmitted in association with each other.

  Further, the enlargement ratio (m / n) and the reference phase (δ) are determined in advance so as to be the same and predetermined values between transmission and reception, for example, and the predetermined values are respectively stored in the transmission device and the reception device. May be stored in advance and used together. In this way, transmission of the enlargement ratio (m / n) and the reference phase (δ) between the transmission device and the reception device, detection at the reception or reception device, and the like become unnecessary.

  If the enlargement ratio (m / n) or the reference phase (δ) cannot be detected or reproduced on the receiving side, the resolution conversion process described above is stopped and, for example, the input signal of frame # 1 is the output signal of the image signal processing device. May be configured to be output as they are.

  Each embodiment of the present invention can be similarly applied to, for example, a DVD player, a magnetic disk player, or a semiconductor memory player in addition to the devices described in the above embodiments. For example, the present invention can also be applied to a portable image display terminal (for example, a mobile phone) for receiving 1-segment broadcasting.

  As the image frame, an image frame of a signal other than the television broadcast signal may be used. Further, for example, a streaming image transmitted via the Internet or an image frame of an image reproduced from a DVD player or an HDD player may be used.

  Further, in each of the above-described embodiments, the description has been given by taking as an example high resolution in frame units. However, the target for higher resolution is not necessarily the entire frame. For example, a part of the frame of the input image or input video may be set as the resolution target. That is, if the image processing according to the embodiment of the present invention described above is performed for a plurality of frames of a part of the frame of the input video, a high-quality enlarged image of the input image or a part of the input video can be obtained. . This can be applied to, for example, an enlarged display of a part of an image.

  According to each embodiment of the present invention described above, it is possible to perform processing for suitably converting a low-resolution image to high resolution, and it is possible to suitably obtain a high-resolution high-resolution image.

It is explanatory drawing of the structure which concerns on Example 1 of this invention. It is explanatory drawing of the structure which concerns on Example 1 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 1 of this invention. It is explanatory drawing of the resolution conversion process of a prior art. It is explanatory drawing of the resolution conversion process of a prior art. It is explanatory drawing of the resolution conversion process of a prior art. It is explanatory drawing of the operation | movement which concerns on Example 1 of this invention. It is explanatory drawing of the structure which concerns on Example 1 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 1 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 1 of this invention. It is explanatory drawing of another structure which concerns on Example 1 of this invention. It is explanatory drawing of the structure which concerns on Example 2 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 1 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 2 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 2 of this invention. It is explanatory drawing of the operation | movement which concerns on Example 2 of this invention. It is explanatory drawing of operation | movement of a prior art. It is explanatory drawing of operation | movement of a prior art. It is explanatory drawing of the operation | movement which concerns on Example 3 of this invention. It is explanatory drawing of the structure which concerns on Example 3 of this invention. It is explanatory drawing of the structure which concerns on Example 4 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1,5 ... Input part; 2,4 ... Resolution conversion part; 3 ... Display part; 101,2014 ... Original component extraction part; 102 ... Subtractor; 103,1203,1210,2111,2112 ... Folding component extraction part; 104 ... frequency conversion unit; 105, 214, 1104, 1208, 1215, 2010, 2105, 2110 ... adder; 106 ... high frequency component reproduction unit; 107 ... pixel number conversion unit; 108 ... storage unit; 201, 2002, 2004 ... position estimation 202, 1202, 2011, 2012 ... phase difference; 203, 204 ... up-rate device; 205, 207, 2001 ... delay device; 206, 208, 1101 ... π / 2 phase shifter; 209, 1209, 1216, 2006, 2101, 2106 ... coefficient Determiner; 210, 211, 212, 213, 801, 1102, 1103, 1204, 1205, 1206, 1207, 1211, 1212, 1213, 1214, 2007, 2008, 2009, 2102, 2103, 2104, 2107, 2108, 2109 ... Multiplier; 216 ... Phase shift Part; 217, 2013 ... aliasing component removal part; 301, 302, 1701, 1702, 1703 ... original component; 303, 304 ... original component after π / 2 phase shift; 305, 306, 307, 308, 1704, 1705, 1706 ... aliasing component; 401 ... HD camera; 402 ... SD camera; 403 ... SD to HD converter; 404 ... switcher; 405 ... transmission path; 501 ... up-rate device; 502,802 ... low-pass filter; 503 ... down-rate device; 601 ... Sampling carrier; 602 ... Low pass filter (LPF) passband; 803, 1105, 1106 ... Carrier generator; 1001, 1002 ... Pixel; 1201, 2003, 2005 ... Motion compensation unit; 1601, 1602, 1603, 1604, 1605,1606,1607,1801,1802,1803,1901,1902,1903,1904 ... coefficient;

Claims (19)

An original component signal generation unit that generates an original component signal obtained by removing or reducing the aliasing component from the video signal;
A high frequency component signal is generated based on the aliasing component included in the video signal, and a high resolution image signal is generated using the original component signal generated by the original component signal generation unit and the high frequency component signal. An image signal processing apparatus comprising: a high-resolution image signal generation unit. The video signal is a video signal obtained by converting the number of pixels at a predetermined magnification,
The original component signal generation unit performs phase shift processing on the plurality of image signals included in the video signal or on the plurality of image signals, and on the plurality of image signals before and after the phase shift processing, 2. The image signal processing according to claim 1, wherein an original component signal is generated by multiplying and adding a coefficient calculated using a phase difference between a plurality of image signals in a video signal and the predetermined enlargement ratio. apparatus. The high-resolution image signal generator is
A folded component signal generating unit that extracts a folded component from the video signal and generates a folded component signal;
A frequency converter that generates a high-frequency component signal by frequency-converting the aliasing component signal generated by the aliasing component signal generator;
The image signal processing according to claim 1, further comprising: an addition / mixing unit that adds or mixes the high-frequency component signal generated by the frequency conversion unit and the original component signal generated by the original component signal generation unit. apparatus. The video signal is a video signal that has been enlarged at a predetermined magnification,
The frequency converter is
A sine wave is generated using reference phase information which is phase information serving as a reference for the enlargement process and the predetermined enlargement ratio, and the generated sine wave and the folded component signal generated by the folded component signal generation unit 4. The image signal processing apparatus according to claim 3, wherein the high frequency component signal is generated based on a multiplication result.   The image signal processing apparatus according to claim 4, wherein the reference phase information is input from the outside to the image signal processing apparatus or stored in advance in a storage unit included in the image signal processing apparatus. An original component signal generating step for generating an original component signal in which the aliasing component is removed or reduced from the video signal;
A high frequency component signal is generated based on the aliasing component included in the video signal, and a high resolution image signal is generated using the original component signal and the high frequency component signal generated in the original component signal generation step. An image signal processing method comprising: a high-resolution image signal generation step. The video signal is a video signal obtained by converting the number of pixels at a predetermined magnification,
The original component signal generation step performs phase shift processing on the plurality of image signals included in the video signal or on the plurality of image signals, and on the plurality of image signals before and after the phase shift processing, 7. The image signal processing according to claim 6, wherein an original component signal is generated by multiplying and adding a coefficient calculated using a phase difference between a plurality of image signals in a video signal and the predetermined enlargement ratio. Method. The high-resolution image signal generation step includes
A folding component signal generation step of generating a folding component signal by extracting the folding component from the video signal;
A frequency conversion step of generating a high-frequency component signal by frequency-converting the folding component signal generated in the folding component signal generation step;
7. The image signal processing according to claim 6, further comprising an addition / mixing step of adding or mixing the high-frequency component signal generated in the frequency conversion step and the original component signal generated in the original component signal generation step. Method. The video signal is a video signal that has been enlarged at a predetermined magnification,
The frequency conversion step includes
A sine wave is generated using reference phase information which is phase information serving as a reference for the enlargement process and the predetermined enlargement ratio, and the generated sine wave and the folded component signal generated in the folded component signal generation step 9. The image signal processing method according to claim 8, wherein the high frequency component signal is generated based on a multiplication result. An input unit for inputting a video signal in which the number of pixels is converted at a predetermined magnification on the broadcast side;
Generating an original component signal in which the aliasing component included in the video signal is removed or reduced, generating a high frequency component signal based on the aliasing component included in the video signal, and the original component signal, the high frequency component signal, A resolution converter that generates a high-resolution video signal using
A video display device that displays the high-resolution video signal generated by the resolution converter.   The resolution converter performs phase shift processing on the plurality of image signals included in the video signal or on the plurality of image signals, and the video signal on the plurality of image signals before and after the phase shift processing. The video display device according to claim 10, wherein the original component signal is generated by multiplying and adding a coefficient calculated using the phase difference between the plurality of image signals and the predetermined magnification.   The resolution conversion unit extracts a folding component from the video signal to generate a folding component signal, converts a frequency of the folding component signal to generate a high frequency component signal, and generates the high frequency component signal and the original component signal. The video display device according to claim 10, wherein a high-resolution video signal is generated by adding or mixing the two.   The resolution conversion unit generates a sine wave using reference phase information which is phase information serving as a reference for the enlargement process and the predetermined enlargement factor, and a multiplication result of the generated sine wave and the aliasing component signal The video display apparatus according to claim 12, wherein the high frequency component signal is generated based on the image.   The video display device according to claim 13, wherein the reference phase information is input from the outside to the image signal processing device or stored in advance in a storage unit included in the video display device. An input unit for inputting a video signal;
A pixel number conversion unit that converts the number of pixels of the video signal input to the input unit at a predetermined magnification;
A video signal after pixel number conversion is input from the pixel number conversion unit, an original component signal in which aliasing components included in the video signal after pixel number conversion are removed or reduced is generated, and the video signal after pixel number conversion A resolution conversion unit that generates a high-frequency component signal based on the aliasing component included in the signal, and generates a high-resolution video signal using the original component signal and the high-frequency component signal;
A video display device that displays the high-resolution video signal generated by the resolution converter.   The resolution converter performs a phase shift process on the plurality of image signals included in the video signal after the pixel number conversion, or on the plurality of image signals before and after the phase shift process. The original component signal is generated by multiplying and adding the coefficient calculated using the phase difference between the plurality of image signals included in the video signal after the pixel number conversion and the predetermined enlargement ratio. The video display device according to claim 15.   The resolution conversion unit extracts a folding component from the video signal after the pixel number conversion to generate a folding component signal, frequency-converts the folding component signal to generate a high frequency component signal, and the high frequency component signal 16. The video display apparatus according to claim 15, wherein a high-resolution video signal is generated by adding or mixing the original component signal and the original component signal.   The resolution conversion unit generates a sine wave using reference phase information which is phase information serving as a reference for the enlargement process and the predetermined enlargement factor, and a multiplication result of the generated sine wave and the aliasing component signal The video display device according to claim 17, wherein the high-frequency component signal is generated based on the image.   19. The video display device according to claim 18, wherein the reference phase information is input from the outside to the image signal processing device or stored in advance in a storage unit included in the video display device.

Images