JP4759159B2 - Image processing apparatus, scanning method conversion method, storage medium, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention is, for example, an image processing apparatus and a scanning method conversion used in an apparatus or system for converting an image signal (interlaced image signal) by an interlace scanning method into an image signal (sequential scanning image signal) by a sequential scanning method. The present invention relates to a method, a program for causing a computer to execute processing steps for implementing the method, and a computer-readable storage medium storing the program.
[0002]
[Prior art]
Conventionally, for example, in television broadcasting by a television system such as the NTSC system or the PAL system, an interlace scanning system has been adopted as a screen scanning system.
[0003]
Interlaced scanning is a scan that saves bandwidth and realizes high-efficiency transmission by alternately transmitting a screen having information on odd-numbered scanning lines and a screen having information on even-numbered scanning lines. It is a method.
[0004]
However, since the interlace scanning method has problems such as line flicker, for example, in a display device such as a display, a gap between image signals (interlace image signals) by the interlace scanning method is interpolated to sequentially scan image signals. Is converted to output.
[0005]
On the other hand, digital broadcasting conforms to the MPEG2 standard (hereinafter also simply referred to as “MPEG2”), but MPEG2 is compatible with interlaced structure signals. As a result, even in digital broadcasting, conversion processing from an interlaced image signal to a sequentially scanned image signal is necessary.
[0006]
Here, MPEG2 is a standard of a motion compensation type compression method. Therefore, in MPEG2, motion compensation is performed at the time of encoding and decoding, and the data obtained thereby already has a motion vector. That is, motion detection required when converting an interlaced image signal by a television system such as the NTSC system into a sequentially scanned image signal (hereinafter, this conversion is also referred to as “scan line conversion” or “scan system conversion”). Is not necessary.
[0007]
As techniques focusing on the above points, for example, Japanese Patent Laid-Open No. 10-126749 discloses inter-field interpolation that performs interpolation from pixels in the previous field, and intra-field interpolation that performs interpolation from pixels above and below the position to be interpolated. Is described on the basis of a motion vector in MPEG2. According to this configuration, the image signal scanning method can be converted without using a motion detection circuit, and the circuit scale can be reduced.
[0008]
[Problems to be solved by the invention]
By the way, the digital broadcasting as described above requires much less bandwidth per channel than, for example, the conventional NTSC television broadcasting, and thus can be multi-channeled and multiplexed with data broadcasting. It becomes.
[0009]
However, if an attempt is made to increase the number of channels or multiplexing in digital broadcasting, the amount of data processed by the microprocessor in the digital broadcasting receiver (digital television device or the like) increases accordingly. Further, when scanning line conversion processing is further performed while processing such a large amount of data, there is a problem in that image quality deterioration due to dropped frames occurs because the processing is not in time.
[0010]
In order to solve the above problem, for example, in Japanese Patent Application Laid-Open No. 11-146398, a digital television apparatus using a microprocessor is configured to detect the load of the microprocessor by some method and suppress dropping of frames. Has been proposed.
[0011]
However, the conventional configuration described in Japanese Patent Application Laid-Open No. 11-146398 or the like is a configuration that reduces the processing load by changing the processing of the decoder, and is not a configuration that considers the scanning line conversion processing load.
[0012]
For example, when the processing from decoding processing to scanning line conversion is executed by a microprocessor, the scanning line conversion method is monitored separately from the decoding processing and the scanning line conversion method is changed according to the load. It is conceivable that a means for doing this is necessary, but conventionally, such a configuration has not been realized.
[0013]
As described above, in a conventional image processing apparatus such as a digital television apparatus using a microprocessor, when the load on the microprocessor is increased due to simultaneous viewing of multiple channels, a frame drop or the like occurs, resulting in image quality degradation. Has occurred. Although the processing load on the microprocessor can be reduced by changing the decoding process, the processing load of one scanning line conversion is not small depending on the configuration. Therefore, for example, when the processing load on the microprocessor is large, it is considered that a configuration for changing to a scanning line conversion method with a small processing load is necessary. However, in such a configuration, the processing load is simply large. If all the scanning line conversion is not performed at this time, the image quality deteriorates rapidly.
[0014]
Therefore, the present invention has been made to eliminate the above-described drawbacks. When the processing load on the microprocessor increases, scanning line conversion with a large processing load can be changed to scanning line conversion with a low processing load. By adopting the configuration, an image processing apparatus, a scanning method conversion method, and a processing step for carrying out the processing can be executed by a computer, which can continuously execute scanning line conversion without causing abrupt image quality degradation. It is an object to provide a program and a computer-readable storage medium storing the program.
[0015]
[Means for Solving the Problems]
The image processing apparatus according to the present invention obtains a second image signal by the second scanning method from an encoded signal obtained by encoding the first image signal by the first scanning method by motion compensation prediction encoding. A processing device, a first image storage means for storing a first image signal obtained by decoding the encoded signal, and a motion for storing a motion vector obtained by decoding the encoded signal Vector storage means and area dividing means for dividing the screen constituted by the first image signal in the first image storage means into a moving area or a static area based on the motion vector in the motion vector storage means Estimating means for estimating the load of conversion processing of the first image signal in the first image storage means to the second image signal, and the load estimated by the estimating means is lower than a predetermined value If the scan As a method of method conversion, an intra-field interpolation method is executed for the dynamic region, an inter-field interpolation method is executed for the static region, and when the load estimated by the estimation unit is higher than a predetermined value, A scanning method conversion means for executing an inter-field interpolation method for the moving region and the stationary region, wherein the scanning method conversion means is a case where the load estimated by the estimation means is higher than a predetermined value; Even if inter-field interpolation is performed on the moving area and the stationary area If the load does not become lower than the predetermined value, a specific value is inserted at the position to be interpolated instead of executing the inter-field interpolation method.
According to another aspect of the image processing apparatus of the present invention, a second image signal according to the second scanning method is obtained from an encoded signal obtained by encoding the first image signal according to the first scanning method according to motion compensation prediction encoding. A first image storage means for storing a first image signal obtained by decoding the encoded signal, and a motion vector obtained by decoding the encoded signal Based on the motion vector in the motion vector storage means, and the screen composed of the first image signal in the first image storage means is a moving area, a static area or an intermediate area. An area dividing means for dividing the image into two areas, an estimating means for estimating a load of conversion processing of the first image signal in the first image storage means to the second image signal, and the estimation means. Based on load In the divided region by the region dividing unit, the intra-field interpolation method, the inter-field method, which is the method of the scanning method conversion process for the first image signal in the first image storage unit to the second image signal, is performed. Scanning method conversion means for selecting and executing an interpolation method, a motion compensation interpolation method, or a method of inserting a specific value at a position to be interpolated instead of the interpolation method.
[0016]
The scanning method conversion method of the present invention obtains a second image signal by the second scanning method from an encoded signal obtained by encoding the first image signal by the first scanning method by motion compensation prediction encoding. An image storage step for storing a first image signal obtained by decoding the encoded signal, and a motion vector obtained by decoding the encoded signal An area for dividing a screen composed of the first image signal stored in the image storage step into a moving area or a still area based on the motion vector storage step and the motion vector stored in the motion vector storage step A dividing step, an estimating step for estimating a load of conversion processing to the second image signal with respect to the first image signal stored in the image storing step, When the load estimated by the constant step is lower than a predetermined value, as a scanning method conversion method, an intra-field interpolation method is executed for the moving region, and an inter-field interpolation method is executed for the stationary region, When the load estimated by the estimation step is higher than a predetermined value, the scanning method conversion step of performing an inter-field interpolation method for the moving region and the stationary region, the scanning method conversion step, The load estimated by the estimating step is higher than a predetermined value, Even if inter-field interpolation is performed on the moving area and the stationary area If the load does not become lower than the predetermined value, a specific value is inserted at the position to be interpolated instead of executing the inter-field interpolation method.
According to another aspect of the scanning method conversion method of the present invention, a second image by the second scanning method is obtained from an encoded signal obtained by encoding the first image signal by the first scanning method by motion compensation predictive coding. A scanning method conversion method for obtaining a signal, an image storing step for storing a first image signal obtained by decoding the encoded signal, and a motion obtained by decoding the encoded signal Based on the motion vector storage step storing the vector and the motion vector stored in the motion vector storage step, the screen constituted by the first image signal stored in the image storage step is a moving region or a still region. A region dividing step for dividing the region into intermediate regions, and a load of the conversion process for converting the first image signal stored in the image storing step into the second image signal are estimated. And a scanning method for scanning the second image signal with respect to the first image signal stored in the image storing step in the divided region based on the region dividing step based on the load estimated in the estimating step. A scanning method conversion step for selecting and executing an intra-field interpolation method, an inter-field interpolation method, a motion compensation interpolation method or a method for inserting a specific value at a position to be interpolated instead of the interpolation method, which is a conversion processing method; Is provided.
[0017]
Of the present invention A computer-readable storage medium records a program for causing a computer to execute the processing steps of the scanning method conversion method. .
[0018]
Of the present invention The program causes a computer to execute the processing steps of the scanning method conversion method. .
[0033]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0034]
[First embodiment]
The present invention is applied to, for example, a scanning system conversion apparatus 100 as shown in FIG.
Scanning system conversion apparatus 100 according to the present embodiment is used in a digital television apparatus or the like, and in particular, an image signal (hereinafter referred to as “MPEG2 image data”) encoded by an MPEG2 compliant encoder is used as an input signal. By decoding this MPEG2 image data, interlaced structure image data (interlaced image data) and motion vectors are obtained, and interlaced image data is converted into still region data and motion region data by region integration based on the motion vectors. The image data is divided and rearranged for each of these areas, the scan data for the still area is sequentially acquired by performing the scan method conversion for the still area, and the moving area image data is used for the moving area. Scanning image data is acquired sequentially by performing the scanning method conversion. Estimate the current overall processing load from the amount of target data, and if the result of this estimation is that the processing load is heavy, change the scanning method conversion method for moving regions from intra-field interpolation to inter-field interpolation. Is configured to change. As a result, the overall processing load can be reduced without abrupt degradation of image quality.
Hereinafter, the configuration and operation of the scanning method conversion apparatus 100 according to the present embodiment will be specifically described.
[0035]
In the present embodiment, it is assumed that intra-field interpolation requires more time for calculation processing and processing load is larger than intra-field interpolation.
[0036]
As shown in FIG. 1, the scanning system conversion apparatus 100 includes an input terminal 101 to which MPEG2 image data is input, an MPEG2 decoding unit 102 for decoding MPEG2 image data from the input terminal 101, and an MPEG2 decoding unit 102. Reference memory 104 that stores image data of a reference field necessary for decoding, display memory 103 that stores interlaced image data obtained by the MPEG2 decoding unit 102, and motion vector data obtained by the MPEG2 decoding unit 102 The motion vector 105 for storing the image, the area dividing unit 106 for dividing the screen area based on the motion vector stored in the motion vector storage memory 105, and the scanning method based on the division result (integration result) of the area dividing unit 6 A scanning method selection unit 107 and a method of scanning image data in the display memory 103 A signal distribution unit 108 that distributes the image data of the still region and the image data of the moving region based on the selection instruction from the selection unit 107, and a signal storage memory 109 that stores the image data of the still region obtained by the signal distribution unit 108 A signal storage memory 110 that stores the image data of the moving region obtained by the signal distribution unit 108, a scanning method conversion unit 111 that converts the scanning method of the image data of the still region stored in the signal storage memory 109, and a signal A scanning method conversion unit 112 that converts the scanning method of moving area image data stored in the storage memory 110, and an output memory that stores image data after the scanning method conversion by the scanning method conversion unit 111 and the scanning method conversion unit 112 113 and an output terminal 114 for outputting image data in the output memory 113 to the outside.
[0037]
In the scanning method conversion apparatus 100 as described above, first, MPEG2 image data is input to the input terminal 101.
Specifically, for example, MPEG2 data given to a digital television apparatus from a storage medium such as a digital broadcast tuner or a DVD is distributed to MPEG2 audio data and MPEG2 image data by a demultiplexer in the apparatus. Of these data, MPEG2 image data is input to the input terminal 101.
[0038]
The MPEG2 decoding unit 102 decodes MPEG2 image data input to the input terminal 101. At this time, since the reference memory 104 stores the image data of the reference field necessary for decoding the current field, the MPEG2 decoding unit 102 uses the image data in the reference memory 104 to perform the decoding process. To do.
[0039]
The MPEG2 decoding unit 102 supplies the interlaced image data acquired by the decoding process to the display memory 103 and supplies the motion vector data acquired together with the interlaced image data to the motion vector storage memory 105.
[0040]
In MPEG2, an image composed of a luminance signal and a color difference signal is subjected to DCT processing (orthogonal transformation processing) in units of 8 × 8 pixels. Further, in many cases, a total of 6 blocks, each of which has 4 blocks of luminance signals and 1 block of color difference signals, are handled as one macro block, and a motion vector is detected in units of the macro block.
In this embodiment, since the scanning method conversion is performed using the detected motion vector, the scanning method conversion is also performed in units of macroblocks.
[0041]
When the MPEG2 decoding unit 102 completes decoding of the MPEG2 image data for one screen, the interlaced image data for the current field for one screen is stored in the display memory 103. The motion vector storage memory 105 also stores a motion vector for one screen.
[0042]
The area dividing unit 106 divides the area of the screen based on the motion vector for one screen stored in the motion vector storage memory 105.
In the present embodiment, as a method of scanning method conversion, a method of applying inter-field interpolation to a still region and applying intra-field interpolation to a moving region is used, so that the region dividing unit 106 uses a motion vector as a reference. When performing the area division to be performed, it is sufficient that the area can be divided into a stationary area and a moving area.
[0043]
Specifically, for example, FIGS. 2A and 2B show an example (example 1) of area division in the area dividing unit 106. FIG.
[0044]
First, the region dividing unit 106 tentatively divides the macroblock corresponding to the target motion vector into a moving region or a still region by comparing the target motion vector in the motion vector storage memory 105 with an arbitrary threshold value.
[0045]
FIG. 2A shows the provisional division.
In FIG. 2A, one screen 201 is divided into 5 × 5 macro blocks. Motion vectors corresponding to these macroblocks are stored in the motion vector storage memory 105.
Therefore, the area dividing unit 106 divides the screen 201 into a still area (see white block) and a moving area based on the result of whether or not the motion vector corresponding to each macroblock is larger than an arbitrary threshold value. Divide into black blocks (see black block).
[0046]
Next, the region dividing unit 106 compares the motion vector of the target macro block (target macro block) with the motion vectors of the eight surrounding macro blocks for each macro block on the temporarily divided screen 201. As a result, when the motion vector of the target macroblock is clearly different in tendency from the motion vectors of the eight surrounding macroblocks, the target macroblock is integrated into the same area as the eight surrounding macroblocks. To do.
[0047]
For example, if the target macroblock is a macroblock determined as a moving area, and six macroblocks among the surrounding eight macroblocks are macroblocks determined as a still area, the target macroblock is Even if it is determined to be a moving region, it is integrated as a still region around it.
[0048]
FIG. 2B shows the state of the above integration. That is, FIG. 2B is a result of the above integration on the provisional division screen 201 shown in FIG.
In FIG. 2B, the macroblock indicated by “A” is originally determined to be a moving area because the surrounding seven macroblocks are determined to be a static area. Integrated into.
[0049]
In the present embodiment, as an example, when six or more macroblocks out of eight macroblocks around the target macroblock are the determination results of a different area from the target macroblock, Although the target macroblocks are integrated, the number of macroblocks (here, 6) that is a criterion for determining the integration is not limited to this.
[0050]
FIGS. 3A and 3B show another example (example 2) of area division in the area dividing unit 106.
[0051]
First, the region dividing unit 106 clarifies a macroblock corresponding to the target motion vector by comparing the target motion vector in the motion vector storage memory 105 with the first threshold value and the second threshold value. Are temporarily divided into any one of three areas: an area that is stationary, an area that is clearly moving, and an intermediate area in which the motion vector is moving slightly.
[0052]
FIG. 3A shows the provisional division.
In FIG. 3A, one screen 301 is divided into 5 × 5 macroblocks. Motion vectors corresponding to these macroblocks are stored in the motion vector storage memory 105.
Therefore, the region dividing unit 106 determines whether or not the motion vector corresponding to each macroblock is larger than the first threshold and further larger than the second threshold. The screen 301 is divided into a still area (see white block), a moving area (see black block), and an intermediate area (see shaded block).
[0053]
Next, the area dividing unit 106 applies eight macro blocks around the target macro block (target macro block) to each macro area in the intermediate area (shaded block area) in the temporarily divided screen 301. Referencing, recognizing the number of macroblocks determined as a still area and the number of macroblocks determined as a moving area, and integrating the target macroblock into a still area or moving area having a larger number of macroblocks To do.
[0054]
At this time, when the number of macroblocks determined to be a still area and the number of macroblocks determined to be a moving area are the same, the area dividing unit 106 determines that the target macroblock is a predetermined area (still area or Integrate into the dynamic domain
[0055]
If all the macroblocks around the macroblock of interest are macroblocks determined to be intermediate regions, the region dividing unit 106 suspends the integration of the macroblock of interest and processes the macroblocks that can be merged first. After that, the macroblocks that have been reserved are integrated. As a result, all macroblocks determined as intermediate areas in the provisional division are integrated into the still area or the moving area.
[0056]
FIG. 3B shows the state of the above integration. That is, FIG. 3B is a result of the above integration on the provisional division screen 301 shown in FIG.
In FIG. 3B, the macroblock indicated by “A” is a macroblock determined to be an intermediate area at the time of provisional division, but there are many macroblocks determined to be a static area around the macroblock. Integrated into the quiesce area. The macroblock indicated by “B” is a macroblock determined as an intermediate area at the time of provisional division, but since there are many macroblocks determined as a dynamic area around it, the macroblock is integrated into the dynamic area. Yes.
[0057]
As described above, the region dividing unit 106 performs region division (region integration) on the screen, and the scanning method selection unit 107 is set as a processing target by the signal distribution unit 108 based on the integration result of the region dividing unit 106. A signal indicating whether a macroblock belongs to a moving area or a stationary area, that is, a macroblock that should perform scanning area conversion for a moving area or a scanning area conversion for a stationary area A signal (conversion method selection signal) indicating whether it is a macro block is supplied to the signal distribution unit 108.
[0058]
Based on the conversion method selection signal from the scanning method selection unit 107, the signal distribution unit 108 selects the target macroblock as a signal for a still region for each macroblock constituting the interlaced image data stored in the display memory 103. This is distributed to the storage memory 111 or the signal storage memory 112 for the moving area.
[0059]
At this time, the signal distribution unit 108 adds information indicating the position in the screen to the target macroblock with an arbitrary configuration, and the signal storage memory 111 for the still area or the signal storage memory for the moving area 112.
[0060]
For example, the signal distribution unit 108 adds numbers based on the screen position to the target macroblock in order from “0”. In this case, in the case of a screen (image data) composed of 720 pixels × 480 pixels, the number of macroblocks is 1350, and the added numbers are “0” to “1350”. Examples of the number addition method include a method of adding 10-bit position information bits to the head of the target macroblock. Alternatively, the 10-bit position information added to the head of the target macroblock may be information indicating the horizontal position with the first 6 bits and the vertical position with the last 4 bits.
[0061]
In addition, the signal distribution unit 108 monitors the availability of the signal storage memory 109 and the signal storage memory 110, and estimates a processing load with a configuration described later.
[0062]
Each of the signal storage memory 109 and the signal storage memory 110 is a memory having a storage capacity for two screens, for example.
The signal storage memory 109 for the still area sequentially stores the macro blocks determined to be the still area supplied from the signal distribution unit 108 together with the position information added to the macro blocks. Similarly, the moving area signal storage memory 110 sequentially stores macroblocks supplied from the signal distributor 108 and determined to be moving areas together with position information added to the macroblocks.
[0063]
The scanning method conversion unit 111 for the still area converts the scanning method of the image data of the macroblock stored in the signal storage memory 109 for the still area according to the configuration described later, and the position information added to the macroblock Based on the above, the macro block after the scanning system conversion is stored in the corresponding position in the output memory 113.
[0064]
The moving region scanning method conversion unit 112 converts the scanning method of the image data of the macroblock stored in the signal storage memory 110 for the moving region according to the configuration described later, and the position information added to the macroblock. Based on the above, the macro block after the scanning system conversion is stored in the corresponding position in the output memory 113.
[0065]
Here, FIG. 4A shows the scanning method conversion by the scanning method conversion unit 111 for the still region, and FIG. 4B shows the scanning by the scanning method conversion unit 112 for the moving region. This shows the method conversion.
[0066]
As shown in FIG. 4A, the scanning method conversion unit 111 for the still area corresponds to the corresponding image data 401 and 402 from the reference memory 103 and the corresponding image data 403 and 403 from the signal storage memory 109. The pixel value of the previous field (see “◯” mark) is interpolated to a position where interpolation is required (see “Δ” mark).
[0067]
As shown in FIG. 4B, the moving area scanning method conversion unit 112 receives input of the corresponding image data 411 and 412 from the signal storage memory 110 and receives a position (“Δ” mark) that requires interpolation. Interpolate with the average value of the pixels located above and below (see “◯”).
[0068]
Through the processing as described above, conversion from interlaced image data to progressively scanned image data is performed.
[0069]
In addition, the scanning method conversion unit 111 and the scanning method conversion unit 112 are processed macros so that the signal distribution unit 108 can monitor whether or not the macro blocks in the signal storage memory 109 and the signal storage memory 110 have already been processed. A flag indicating that processing has been completed is added to the block.
For example, the scanning method conversion unit 111 and the scanning method conversion unit 112 set all the bits of the position information added to the processed macro block to “1”.
[0070]
The signal distribution unit 108 monitors whether the macroblocks in the signal storage memory 109 and the signal storage memory 110 have been processed by referring to the position information, and estimates the processing load based on the result. .
[0071]
Specifically, for example, when the signal distribution unit 108 distributes image data at the head of the screen in the display memory 103 to the signal storage memory 109 or the signal storage memory 110, the signal distribution unit 108 stores the signal data in the signal storage memory 109 and the signal storage memory 110. By referring to the flag indicating that processing has been performed by the scanning method conversion unit 111 and the scanning method conversion unit 112 with respect to the macro block, it is recognized how many macro blocks that have not been processed exist.
[0072]
Then, the signal distribution unit 108 determines that the processing load is large and the processing is not in time when there is a certain number or more of unprocessed macroblocks, for example, when one block of macroblocks remains unprocessed. Then, when distributing the image data of the screen to be distributed next, the scanning method conversion method is switched from the intra-field interpolation to the inter-field interpolation for the macro block in the moving area. For example, the signal distribution unit 108 supplies a signal (scanning method conversion method switching signal) indicating that the scanning method conversion method is switched to the moving region scanning method conversion unit 112. Thereby, the scanning method conversion unit 112 for the moving region switches the scanning method conversion method from intra-field interpolation to inter-field interpolation based on the scanning method conversion method switching signal from the signal distribution unit 108.
[0073]
The signal distribution unit 108 also includes the signal storage memory 109 and the signal storage memory 110 while the scanning method conversion unit 112 for the moving region substitutes the inter-field interpolation for the scanning method conversion for the moving region. Monitoring is continued, and when the remaining amount of macro blocks that have not been processed decreases, it is determined that the processing load has been reduced, and the method of moving area scanning method conversion is returned to intra-field interpolation. For example, the signal distribution unit 108 supplies a signal (scanning method conversion method switching signal) indicating that the scanning method conversion method is returned to the moving region scanning method conversion unit 112. Accordingly, the scanning method conversion unit 112 for the moving area restores the scanning method conversion method based on the scanning method conversion method switching signal from the signal distribution unit 108.
[0074]
In addition, the signal distribution unit 108, when the scanning method conversion unit 112 for the moving region replaces the scanning method conversion method for the moving region with inter-field interpolation, the processing load is not reduced. The scanning method conversion is not performed for both. For example, the signal distribution unit 108 instructs the scanning method conversion unit 111 and the scanning method conversion unit 112 to insert “0” or an appropriate numerical value at a position to be interpolated. In this case, the output memory 113 stores interlaced image data.
[0075]
As described above, based on the instruction according to the processing load from the signal distribution unit 108, the moving area scanning method conversion unit 112 and the still region scanning method conversion unit 111 include the signal storage memory 109 and the signal storage unit 109. Scan mode conversion is performed on the macroblock of the memory 110. The converted macroblock is stored in the corresponding position in the output memory 113.
[0076]
The output memory 113 outputs the macroblock image data stored in the corresponding positions by the moving area scanning method conversion unit 112 and the still region scanning method conversion unit 111 via the output terminal 114.
[0077]
For example, when the number “0” to “1350” indicating the position on the screen is given as the position information added to the macroblock, the number value is converted into the address of the output memory 113, and the address The image data is written in and output from the output terminal 114. As a result, the scanned image data is sequentially output from the output terminal 114.
[0078]
As described above, in this embodiment, based on a motion vector obtained from MPEG2 image data, a screen composed of the image data is divided into a still area and a moving area into two areas. Inter-field interpolation is performed by the scanning method conversion unit 111 for the still area for the image data of the moving region, and intra-field interpolation is performed by the scanning method conversion unit 112 for the moving region for the image data of the moving region. At this time, the signal distribution unit 108 monitors the use status (number of unprocessed macroblocks) of each of the conversion unit memories 109 and 110 (monitors the processing load), while scanning the scan method conversion unit 111 for the still area and The moving area scanning method conversion unit 112 distributes macro blocks to be processed. When the signal distribution unit 108 determines that the processing load has increased, the inter-field interpolation is used instead of the scanning method conversion method for the moving region.
With this configuration, when decoding and scanning method conversion of MPEG2 image data using a microprocessor, even when the processing load increases, scanning method conversion can be performed without causing rapid image quality degradation. It can be carried out.
[0079]
In this embodiment, as an example, the intra-field interpolation has a larger processing load than the inter-field interpolation. However, depending on the apparatus or system, it takes a lot of time to refer to the memory for inter-field interpolation. For example, inter-field interpolation may increase the processing load. In this case, when it is determined that the processing load is large, the inter-field interpolation is changed to intra-field interpolation, contrary to the present embodiment.
[0080]
Further, for example, when the processing load is almost the same between the inter-field interpolation and the intra-field interpolation, when the processing load increases, one of the methods may be changed to the other method. In this case, it is also possible not to perform the scanning system conversion for the moving area by inserting “0” or an appropriate numerical value at the position to be interpolated.
[0081]
In the present embodiment, when the processing load increases, the scanning method conversion method for the moving region is set to inter-field interpolation to reduce the processing load. The scanning method conversion may not be performed by inserting 0 "or an appropriate numerical value.
[0082]
[Second Embodiment]
The present invention is applied to, for example, a scanning method conversion apparatus 500 as shown in FIG.
In the scanning method conversion apparatus 500 of FIG. 5, the same reference numerals are given to the portions that operate in the same manner as the scanning method conversion apparatus 100 of FIG. 1, and the detailed description thereof is omitted.
[0083]
First, in the first embodiment, two types of intra-field interpolation and inter-field interpolation are used as the scanning method conversion method, and the region division is also performed in accordance with the scanning method conversion method. By dividing the scan method into two, and changing the scan method conversion process such as normal scan method conversion, conversion substituting inter-field interpolation for the moving region, and no scan method conversion, based on the processing load, The processing load is reduced.
[0084]
On the other hand, in the present embodiment, three types of intra-field interpolation, inter-field interpolation, and motion compensation interpolation are used as the scanning method conversion method, and region division is also performed in accordance with the scanning method conversion method. The area is divided into three areas: a moving area, a moving area, and an intermediate area. In this way, by configuring so as to use a large number of scanning method conversion methods, it becomes possible to increase the number of stages of processing change when the processing load becomes large, and to that extent, the processing load is reduced with less sense of incongruity. Can be achieved.
[0085]
In the present embodiment, it is assumed that the inter-field interpolation has the smallest processing load, and the intra-field interpolation and motion compensation interpolation have the processing load in order.
[0086]
Therefore, as shown in FIG. 5, the scanning method conversion apparatus 500 according to the present embodiment includes a signal storage memory 119 and a scanning method conversion unit 120 in addition to the configuration shown in FIG. The output (motion vector) of the memory 105 is also supplied to the scanning method conversion 120.
[0087]
The area dividing unit 106 performs area division based on the motion vector for one screen stored in the motion vector storage memory 105. As described with reference to FIG. And the second threshold value, it is divided into three areas: an apparently stationary area (stationary area), an apparently moving area (moving area), and an intermediate area that cannot be determined in any way To do.
[0088]
In addition, for each macroblock constituting one screen, the region dividing unit 106 includes up to six of the eight macroblocks around the target macroblock (target macroblock) and the target macroblock. If the area is different from the block, a process of integrating the macro block of interest into the area is performed.
[0089]
Based on the integration result of the area dividing unit 106, the scanning method selection unit 107 determines whether the macroblock to be processed by the signal distribution unit 108 is a macroblock belonging to a moving area, a stationary area, or an intermediate area. Signal, i.e., a macroblock to be subjected to a scanning method conversion for a moving area, a macroblock to be subjected to a scanning method conversion for a still area, or a macroblock to be subjected to a scanning method conversion for an intermediate area Is supplied to the signal distribution unit 108 (conversion method selection signal).
[0090]
Based on the conversion method selection signal from the scanning method selection unit 107, the signal distribution unit 108 selects the target macroblock as a signal for a still region for each macroblock constituting the interlaced image data stored in the display memory 103. The data is distributed to the storage memory 111, the signal storage memory 112 for the moving area, or the signal storage memory 119 for the intermediate area.
[0091]
At this time, the signal distribution unit 108 monitors the memories 109, 110, and 119 and estimates the processing load as in the first embodiment.
[0092]
If the signal distribution unit 108 determines that the processing load is large as a result of the estimation of the processing load, the signal distribution unit 108 first stops the scanning method conversion for the intermediate region, and the region is divided into two instead of three from the next screen. As shown in FIG. 5, the area division unit 116 and the conversion method selection unit 107 are instructed by an interrupt signal or the like.
[0093]
In this case, the region dividing unit 106 switches the method of dividing the region into three based on the instruction from the signal distributing unit 108 to the method of dividing into two as in the first embodiment.
Further, the scanning method selection unit 107 switches from a method of selecting one from three scanning method conversions to a method of selecting one from two scanning method conversions similar to those in the first embodiment.
[0094]
Here, FIG. 6 shows scanning method conversion in the scanning method conversion unit 120 for the intermediate region.
As shown in FIG. 6, the intermediate area scanning method conversion unit 120 inputs the corresponding image data 601 and 602 from the reference memory 103 and the corresponding image data 603 and 603 from the signal storage memory 119. In the case where the motion vector of the macroblock surrounded by the dotted line is a thick broken line arrow, the interpolated pixel (see “Δ”) is the pixel in the field paired with the reference field (“ ○ ”), and if there is no pixel at the position pointed to by the vector, the closest pixel is interpolated.
[0095]
The configuration and operation as described above are different from the first embodiment.
In other words, in the present embodiment, three types of scanning method conversion methods are used, and the divided areas are set to three areas, that is, a stationary area, a moving area, and an intermediate area, and three types of scanning system conversion are performed based on the processing load. In addition, it is configured to execute the corresponding processing among the four-stage processing of conversion of the scanning method of the still region and the moving region, conversion that substitutes the moving region by inter-field interpolation, and no scanning method conversion. As a result, it is possible to reduce image quality deterioration due to switching of scanning method conversion.
[0096]
For example, when switching from three types of scanning method conversion to still region and moving region scanning method conversion rather than image quality degradation when switching from still region and moving region scanning method conversion to still region only scanning method conversion Since the image quality is less deteriorated, this embodiment has a great effect when the processing load can be reduced by switching from three types of scanning method conversion to two types of scanning method conversion. become.
[0097]
Note that if the processing load is not reduced even if the scanning method conversion for the intermediate region is stopped and the scanning method conversion for only the stationary region and the moving region is performed, as in the first embodiment, The scanning method conversion for both may not be performed. Conversely, when the processing load is reduced and the scanning method conversion of the intermediate area is restarted, this is also the same as in the first embodiment, and thus detailed description thereof is omitted.
[0098]
Also in the present embodiment, as in the first embodiment, the scanning method conversion method for the moving region is switched from inter-field interpolation to intra-field interpolation. For example, the scanning method conversion method is performed. This may be omitted as in the first embodiment.
[0099]
Further, in the present embodiment, it is assumed that inter-field interpolation has the smallest processing load, and then intra-field interpolation and motion compensation interpolation have a sequential processing load. It cannot be considered from the amount of calculation that the motion compensation interpolation has a smaller load than other scanning method conversion methods.
[0100]
Also, in the case where intra-field interpolation has a smaller processing load than inter-field interpolation, or in the case where the processing load of inter-field interpolation and intra-field interpolation is almost the same, etc. Obviously, the same can be done.
[0101]
[Third embodiment]
The present invention is applied to, for example, a scanning method conversion apparatus 700 as shown in FIG.
In the scanning method conversion apparatus 700 of FIG. 7, the same reference numerals are given to the portions that operate in the same manner as the scanning method conversion apparatus 100 of FIG. 1, and the detailed description thereof is omitted.
[0102]
First, in the first embodiment, in order to estimate the size of the processing load, the signal storage memories 109 and 110 (in the second embodiment, in addition to this, the signal storage memory 119) is monitored. did.
[0103]
In contrast, in the present embodiment, the display memory 103 is monitored in order to estimate the size of the processing load.
[0104]
For this reason, as shown in FIG. 7, the scanning method conversion apparatus 700 according to the present embodiment is configured to include a load estimation unit 121 in front of the display memory 103 in addition to the configuration shown in FIG. 1. Has been.
[0105]
The load estimation unit 121 monitors the remaining amount of the display memory 103 when writing the image data supplied from the MPEG2 decoding unit 102 (decoded image data to be displayed next) to the display memory 103.
[0106]
Specifically, for example, in the display memory 103, a processing end monitoring bit of 1 bit is assigned to each macro block constituting one screen, and an unprocessed macro block is set to “1”. “0” is set for a processed macroblock.
[0107]
The load estimation unit 121 determines whether or not there are a certain number of unprocessed macroblocks in the display memory 103 by referring to the processing end monitoring bit, and there is a certain number or more of unprocessed macroblocks. If so, it is determined that the processing load is large. In this case, the load estimation unit 121 instructs the MPEG2 decoding unit 102 to stop outputting the motion vector to the motion vector storage memory 105 from the processing of the image data of the next screen, and also to the region dividing unit 106. It instructs the area division to stop. Furthermore, the load estimation unit 121 notifies the signal distribution unit 108 that the processing load is large.
[0108]
Based on the instruction from the load estimation unit 121, the region dividing unit 106 instructs the scanning method conversion unit 112 for the moving region to switch from intra-field interpolation to intra-field interpolation.
Further, when the processing load is large according to the notification from the load estimation unit 121, the signal distribution unit 108 alternately receives the macroblock image data received from the display memory 103 and the signal storage memory 109 for the still region and the moving region use. The signal storage memories 110 are each distributed.
[0109]
If the processing load is not reduced by the processing as described above, the processing load is reduced by not performing the scanning method conversion in both the stationary region and the moving region, as in the first embodiment. It may be.
[0110]
As described above, the first embodiment has the point that the region division is stopped when the processing load becomes large, and the inter-field interpolation is applied in this case without distinguishing between the stationary region and the moving region. Unlike the configuration of the embodiment, by adopting such a configuration, the effect of reducing the processing load can be expected more.
[0111]
In the first to third embodiments, the reference field is used for the inter-field interpolation. For example, when the processing load increases, the image quality deteriorates when the moving area is substituted by inter-field interpolation. Because of the large value, the screen one field before may be stored in an arbitrary configuration, and inter-field interpolation may be performed with the pixels of the one field.
[0112]
Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the host and terminal according to the first to third embodiments to a system or apparatus, and a computer of the system or apparatus. Needless to say, this can also be achieved by (or CPU or MPU) reading and executing the program code stored in the storage medium.
In this case, the program code itself read from the storage medium realizes the functions of the first to third embodiments, and the storage medium storing the program code and the program code constitute the present invention. It becomes.
A ROM, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, or the like can be used as a storage medium for supplying the program code.
Further, by executing the program code read by the computer, not only the functions of the first to third embodiments are realized, but also an OS running on the computer based on the instruction of the program code. Needless to say, the present invention includes a case where the functions of the first to third embodiments are realized by performing part or all of the actual processing.
Further, after the program code read from the storage medium is written to the memory provided in the extension function board inserted in the computer or the function extension unit connected to the computer, the function extension is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or function expansion unit performs part or all of the actual processing, and the functions of the first to third embodiments are realized by the processing.
[0113]
FIG. 8 shows an example of the configuration of the computer function 800.
As shown in FIG. 8, the computer function 800 includes a CPU 801, a ROM 802, a RAM 803, a keyboard controller (KBC) 805 for a keyboard (KB) 809, and a CRT controller (CRT display (CRT) 810 as a display unit ( CRTC) 806, a hard disk (HD) 811 and a floppy disk (FD) 812 disk controller (DKC) 807, and a network interface card (NIC) 808 for connecting to the network 820 are connected to each other via a system bus 804. It is configured to be communicable.
[0114]
The CPU 801 generally controls each component connected to the system bus 804 by executing software stored in the ROM 802 or the HD 811 or software supplied from the FD 812.
That is, the CPU 801 reads out a processing program according to a predetermined processing sequence from the ROM 802, HD 811, or FD 812 and executes it, thereby performing control for realizing the operations in the first to third embodiments. Do.
[0115]
The RAM 803 functions as a main memory or work area for the CPU 801.
The KBC 805 controls an instruction input from the KB 809 or a pointing device (not shown).
The CRTC 806 controls the display of the CRT 810.
The DKC 807 controls access to the HD 811 and the FD 812 that store a boot program, various applications, an edit file, a user file, a network management program, the processing program in the present embodiment, and the like.
The NIC 808 exchanges data bidirectionally with devices or systems on the network 820.
[0116]
【The invention's effect】
As described above, in the present invention, the second scanning is performed from the encoded signal obtained by encoding the first image signal by the first scanning method (interlaced scanning method or the like) by the motion compensation predictive encoding (MPEG2 or the like). When acquiring the second image signal by the method (sequential scanning method or the like), the screen constituted by the first image signal is divided into regions based on the motion vector, and each divided region is based on the processing load. A scanning method conversion process is executed by the method. Thereby, for example, when the processing load on the microprocessor increases, the scanning line conversion without changing the scanning line conversion with a high processing load is changed to the scanning line conversion with a low processing load without causing a sudden deterioration in image quality. Can be continued.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a scanning method conversion apparatus to which the present invention is applied in a first embodiment.
FIG. 2 is a diagram for explaining an example (Example 1) of region division in the scanning method conversion apparatus.
FIG. 3 is a diagram for explaining another example (example 2) of the region division.
FIG. 4 is a diagram for explaining an example of a scanning method conversion method in the scanning method conversion apparatus.
FIG. 5 is a block diagram showing a configuration of a scanning method conversion apparatus to which the present invention is applied in a second embodiment.
FIG. 6 is a diagram for explaining an example of a scanning method conversion method in the scanning method conversion apparatus.
FIG. 7 is a block diagram showing a configuration of a scanning method conversion apparatus to which the present invention is applied in a third embodiment.
FIG. 8 is a block diagram showing a configuration of a computer function that reads and executes a processing program for realizing the functions of the first to third embodiments from a computer-readable storage medium.
[Explanation of symbols]
100 Scanning conversion device
101 Input terminal
102 MPEG2 decoding unit
103 Display memory
104 Reference memory
105 Motion vector memory
106 Area division unit
107 Conversion method selector
108 Signal distributor
109,110 Signal storage memory
111, 112 Scanning system converter
113 Output memory
114 Output terminal

Claims (6)

An image processing apparatus for obtaining a second image signal by a second scanning method from an encoded signal obtained by encoding a first image signal by a first scanning method by motion compensation prediction encoding,
First image storage means for storing a first image signal obtained by decoding the encoded signal;
Motion vector storage means for storing a motion vector obtained by decoding the encoded signal;
Area dividing means for dividing a screen constituted by the first image signal in the first image storage means into a moving area or a still area based on a motion vector in the motion vector storage means;
Estimating means for estimating a load of conversion processing of the first image signal in the first image storage means to the second image signal;
When the load estimated by the estimating means is lower than a predetermined value, the intra-field interpolation method is executed for the moving region and the inter-field interpolation method is executed for the stationary region as the scanning method conversion method. When the load estimated by the estimating means is higher than a predetermined value, the scanning method converting means for performing an inter-field interpolation method between the moving area and the stationary area,
The scanning method converting means is a case where the load estimated by the estimating means is higher than a predetermined value, and even if the inter-field interpolation method is executed between the moving area and the stationary area, the load is less than the predetermined value. An image processing apparatus characterized by inserting a specific value at a position to be interpolated instead of executing the inter-field interpolation method when it does not become low. An image processing apparatus for obtaining a second image signal by a second scanning method from an encoded signal obtained by encoding a first image signal by a first scanning method by motion compensation prediction encoding,
First image storage means for storing a first image signal obtained by decoding the encoded signal;
Motion vector storage means for storing a motion vector obtained by decoding the encoded signal;
Area dividing means for dividing the screen constituted by the first image signal in the first image storage means into a moving area, a still area or an intermediate area based on the motion vector in the motion vector storage means; ,
Estimating means for estimating a load of conversion processing of the first image signal in the first image storage means to the second image signal;
Based on the load estimated by the estimation means, a method of a scanning method conversion process for the first image signal in the first image storage means to the second image signal in the divided area by the area dividing means Scanning method conversion means for selecting and executing an intra-field interpolation method, an inter-field interpolation method, a motion compensation interpolation method, or a method for inserting a specific value at a position to be interpolated instead of the interpolation method. A featured image processing apparatus. A scanning method conversion method for obtaining a second image signal by a second scanning method from an encoded signal obtained by encoding a first image signal by a first scanning method by motion compensation predictive coding. ,
An image storing step of storing a first image signal obtained by decoding the encoded signal;
A motion vector storage step for storing a motion vector obtained by decoding the encoded signal;
An area dividing step of dividing the screen constituted by the first image signal stored in the image storing step into a moving area or a still area based on the motion vector stored in the motion vector storing step;
An estimation step for estimating a load of a conversion process for converting the first image signal stored in the image storage step into the second image signal;
When the load estimated by the estimation step is lower than a predetermined value, an intra-field interpolation method is executed for the moving region and an inter-field interpolation method is executed for the stationary region as a scanning method conversion method. When the load estimated by the estimation step is higher than a predetermined value, a scanning method conversion step for performing an inter-field interpolation method on the moving region and the stationary region includes:
The scanning method conversion step is a case where the load estimated by the estimating step is higher than a predetermined value, and even if the inter-field interpolation method is performed on the moving region and the stationary region, the load is less than the predetermined value. A scanning method conversion method characterized by inserting a specific value at a position to be interpolated in place of execution of the inter-field interpolation method when not lowered. A scanning method conversion method for obtaining a second image signal by a second scanning method from an encoded signal obtained by encoding a first image signal by a first scanning method by motion compensation predictive coding. ,
An image storing step of storing a first image signal obtained by decoding the encoded signal;
A motion vector storage step for storing a motion vector obtained by decoding the encoded signal;
An area dividing step for dividing the screen constituted by the first image signal stored in the image storing step into a moving area, a stationary area, or an intermediate area based on the motion vector stored in the motion vector storing step. When,
An estimation step for estimating a load of a conversion process for converting the first image signal stored in the image storage step into the second image signal;
Based on the load estimated in the estimation step, a method of a scanning method conversion process for the first image signal stored in the image storage step to the second image signal in the divided region in the region division step. A scanning method conversion step for selecting and executing an intra-field interpolation method, an inter-field interpolation method, a motion compensation interpolation method, or a method for inserting a specific value at a position to be interpolated instead of the interpolation method. Scanning method conversion method.   A computer-readable storage medium storing a program for causing a computer to execute the processing steps of the scanning method conversion method according to claim 3.   The program for making a computer perform the process step of the scanning system conversion method of Claim 3 or 4.

Images