JP2005142986A - Moving image encoding method, moving image encoding apparatus and moving image encoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for reducing an inter-frame prediction throughput by stopping detecting a motion vector using an intra-frame prediction evaluation value. <P>SOLUTION: In the image encoding apparatus comprising a motion detection unit ME for calculating an inter-frame prediction evaluation value InterCost, an intra-frame prediction evaluation value calculation unit IntraCostCalc, a mode selection unit ModeSel for selecting either inter-frame prediction or intra-frame prediction as a prediction mode, and a variable length encoding unit VLC for outputting an encoded signal BitStm, the image encoding apparatus includes an inter-frame prediction evaluation value estimation unit for calculating an inter-frame prediction evaluation estimate before the end of the calculation in InterCost, and an intra-frame prediction determination unit IntraDet for instructing the end of calculation of the inter-frame prediction evaluation value in the motion detection unit ME by selecting intra-frame prediction as a prediction mode when the intra-frame prediction evaluation value and the inter-frame prediction evaluation value estimate fulfill predetermined conditions. Thus, a problem is solved that huge throughput is required for motion detection since it has to detect a motion vector from a number of motion vector candidates in motion detection processing used to inter-frame prediction of moving image encoding. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a method for speeding up an inter-frame prediction process used for moving picture coding, and a program for implementing the method by software.

  In recent years, with the development of multimedia applications, it has become common to handle all media information such as images, sounds and texts in a unified manner. At this time, the media can be handled uniformly by digitizing all the media. However, since a digitized image has an enormous amount of data, image information compression technology is indispensable for storage and transmission. On the other hand, in order to interoperate compressed image data, standardization of compression technology is also important. Image compression technology standards include H.264 of ITU (International Telecommunication Union Telecommunication Standardization Sector). 261, H.H. H.263, ISO (International Organization for Standardization) MPEG (Moving Picture Experts Group) -1, MPEG-2, MPEG-4, and the like.

  As a technique common to these video coding systems, there is inter-frame prediction with motion compensation. Inter-frame prediction is a prediction method using correlation between frames, and is a method of predicting input pixels from pixel values of an encoded image. At this time, in order to correspond to the motion of the object in the screen, motion vector information indicating which pixel in the encoded image to be referred (hereinafter referred to as a reference image) is the predicted pixel is added to the encoded signal. Keep it. A motion vector is added to each rectangular area (hereinafter referred to as a block) obtained by dividing an input image into a predetermined size in MPEG or the like.

  However, there are cases where there is no corresponding part in the reference image, such as when a new object enters the frame or when a background that has been hidden behind when the object moves appears. In such a case, the prediction efficiency of intra-frame prediction may be higher than inter-frame prediction.

  In order to cope with this, in MPEG or the like, inter-frame prediction and intra-frame prediction can be switched for each block. Since a signal indicating the selected prediction mode (prediction mode information) is stored in the encoded signal, the image decoding apparatus can select the same prediction mode as the image encoding apparatus from this information. In the image coding apparatus, the prediction error in each mode is calculated and compared, and the prediction mode is selected. As the evaluation value of the inter-frame prediction for the motion vector MV, for example, the square error shown in Equation 1 is used or the absolute value error shown in Equation 2 is used.

  On the other hand, as the evaluation value for intra-frame prediction, for example, the variance value shown in Expression 3 and the value shown in Expression 4 are used.

  Since these evaluation values represent prediction errors, the smaller the value, the more accurate the prediction and the higher the encoding efficiency. Therefore, for example, as shown in Equation 5, if the evaluation values of inter-frame prediction and intra-frame prediction are compared and a mode with a smaller evaluation value is selected, the encoding efficiency becomes higher.

  Expressions 1 to 4 are examples of evaluation values, and evaluation values defined by other calculation expressions may be used.

  Formula 5 is an example of the mode determination formula, and other determination formulas suitable for selecting a prediction mode with high encoding efficiency using the inter-frame prediction evaluation value and the intra-frame prediction evaluation value as variables may be used. Good.

  FIG. 10 shows a conventional image coding apparatus. The conventional image encoding device shown in FIG. 10 divides the image signal Img into blocks and performs processing for each block. The subtracter Sub subtracts the predicted image signal Pred from the image signal Img input to the image encoding device, and outputs it as a residual signal Res. The residual encoding unit ResEnc receives the residual signal Res, performs encoding processing such as DCT transformation and quantization, and outputs a residual encoded signal ERes including quantized DCT coefficients. The residual decoding unit ResDec receives the residual encoded signal ERes, performs image decoding processing such as inverse quantization and inverse DCT transform, and outputs a residual decoded signal DRes. The adder Add adds the residual decoded signal DRes and the predicted image signal Pred, and outputs the result as a decoded image signal DecImg. In the decoded image signal DecImg, a signal that may be referred to in the subsequent inter-frame prediction is stored in the frame memory FM. The pixel prediction unit Predictor inputs the encoded image stored in the frame memory FM as a reference image signal Ref and outputs a predicted image signal Pred. The pixel prediction unit Predictor performs pixel prediction and outputs prediction mode information Mode indicating a prediction mode and a motion vector MV at the time of inter-frame prediction. The variable length coding unit VLC performs variable length coding on the residual signal Res, the prediction mode information Mode, and the motion vector MV, and outputs the result as a coded signal BitStrm.

  Next, the pixel prediction unit Predictor will be described in detail. FIG. 11 is a block diagram of the pixel prediction unit Predictor. The motion detection unit ME detects and outputs a motion vector MV from the reference image signal Ref and the image signal Img, and also outputs an inter-frame prediction evaluation value InterCost that is an evaluation value when inter-frame prediction is performed using the motion vector MV. The intra-frame prediction evaluation value calculation unit IntraCostCalc outputs an intra-frame prediction evaluation value IntraCost that is an evaluation value when the image signal Img is input and intra-frame prediction is performed. The mode determination unit ModeSel inputs and compares the inter-frame prediction evaluation value InterCost and the intra-frame prediction evaluation value IntraCost, selects a prediction mode, and outputs the selected prediction mode as prediction mode information Mode. The motion compensation unit MC performs inter-frame prediction using the motion vector MV, and outputs a prediction result as an inter-frame prediction image signal MCPred. The intra-frame prediction unit IntraPredictor performs intra-frame prediction and outputs a prediction result as an intra-frame prediction image signal IPred. When the prediction mode information Mode indicates inter-frame prediction, the switch SW1 switches to the “1” side, outputs the inter-frame prediction image signal MCPred as the prediction image signal Pred, and the prediction mode information Mode indicates intra-frame prediction. The switch SW1 switches to the “0” side and outputs the intra-frame predicted image signal IPred as the predicted image signal Pred.

  In inter-frame prediction, one motion vector is selected from a large number of motion vector candidates. Therefore, the motion detection unit ME selects one motion vector having the smallest evaluation value from a plurality of motion vector candidates. For each motion vector candidate, for example, the evaluation values shown in Equations 1 and 2 are calculated, and the motion vector is detected by selecting the motion vector candidate that minimizes the evaluation value. The evaluation value corresponding to the selected motion vector is the inter-frame prediction evaluation value InterCost.

  FIG. 12 is a diagram illustrating a processing flow of the pixel prediction unit Predictor. In S11, a motion vector is detected by the motion detection unit ME to obtain a motion vector MV and an inter-frame prediction evaluation value InterCost. In S12, the intra-frame prediction evaluation value IntraCost is calculated by the intra-frame prediction evaluation value calculation unit IntraCostCalc. In S13, the mode determination unit ModeSel compares the inter-frame prediction evaluation value InterCost and the intra-frame prediction evaluation value IntraCost to determine the prediction mode.

  FIG. 13 is a block diagram showing a configuration of an image decoding apparatus corresponding to the conventional image encoding apparatus of FIG. The variable length decoding unit VLD receives the encoded signal BitStrm, performs variable length decoding, and outputs a residual encoded signal ERes, a motion vector MV, and prediction mode information Mode. The residual decoding unit ResDec receives the residual encoded signal ERes, performs decoding processing such as inverse quantization and inverse DCT transform, and outputs a residual decoded signal DRes. The motion compensation unit MC inputs a decoded image from the frame memory FM as a reference image signal Ref, and outputs a result of inter-frame prediction with motion compensation using the motion vector MV as an inter-frame prediction image signal MCPred. The intra-frame prediction unit IntraPredictor outputs an intra-frame prediction image signal IPred. When the prediction mode information Mode indicates intra-frame prediction, the switch SW1 switches to “0” and outputs the intra-frame prediction image signal IPred as the prediction image signal Pred. When the prediction mode information Mode indicates inter-frame prediction, the switch SW1 switches to the “1” side and outputs the inter-frame prediction image signal MCPred as the prediction image signal Pred. The adder Add adds the residual decoded signal DRes and the predicted image signal Pred, and outputs the result as a decoded image signal DecImg outside the image decoding apparatus. The frame memory FM stores the decoded image signal DecImg for inter-frame prediction. As described above, the image decoding apparatus decodes the encoded signal BitStrm and outputs it as a decoded image signal DecImg by the processing described above.

  By the way, in the inter-frame prediction, since the evaluation values as shown in Expression 1 and Expression 2 must be calculated for each motion vector candidate, if there are many motion vector candidates, the calculation time becomes enormous. For example, if all pixel positions within the range of −16 to 16 in each component of the motion vector are motion vector candidates, the number of candidates is 33 × 33 = 1089. When real-time encoding processing is performed with an arithmetic unit having low processing capability, it may be impossible to evaluate all candidates from the viewpoint of processing amount. Therefore, methods for reducing the amount of motion detection processing by reducing the number of motion vector candidates to be evaluated have been studied. In this case, since not all candidates are evaluated, it is not always possible to select a motion vector that minimizes the evaluation value, and the prediction error increases and coding efficiency decreases. Therefore, it is desirable to use a motion detection method in which the coding efficiency hardly deteriorates even if the number of motion vectors to be evaluated is reduced. One of the methods is motion detection by local search. Hereinafter, motion detection by local search will be described.

  The local search uses a tendency that the prediction error increases according to the distance, centering on a motion vector with a true prediction error (motion vector that minimizes the prediction error), and motion vector candidates in order of decreasing prediction error. It is a method to evaluate. Here, an example of local search is shown. FIG. 14 is a conceptual diagram of local search. In the local search, a search start point (initial value of the search center) is first selected and evaluated. Here, the point of number 0 (point indicated by the motion vector PMV) is set as the search start point. Next, a motion vector indicating a pixel position around the search center is evaluated. The figure shows the case where four neighborhoods of the search center are evaluated. When the evaluation value at the point other than the search center is the minimum among the search center and the four neighboring points, the search center is moved to that point. If the evaluation value at the search center is minimum, the motion vector detection is terminated at that time, and a motion vector candidate indicating the search center is set as a motion vector MV. In the illustrated example, the search center moves in the order of 0, 2, 5, and 9. In order to reduce the number of motion vector candidates to be evaluated in the local search, it is important to select a search start point that is as close to the true motion vector as possible. As an example of such a search start point, a prediction vector (a motion vector predicted from motion vectors of surrounding blocks) or the like is used. Further, when selecting a search start point, there is also a method of evaluating a plurality of candidates such as a prediction vector and a zero vector, and setting a point that minimizes an evaluation value as a search start point.

  FIG. 15 is a block diagram illustrating a pixel prediction unit Predictor that performs local search. Units that operate in the same manner as in FIG. 11 and signals that operate in the same manner are denoted by the same symbols, and description thereof is omitted. The search start point selection unit ME1 selects a search start point, and outputs a motion vector indicating the selected search start point as an initial motion vector MV1. Local search unit ME2 receives initial motion vector MV1, performs local search, and outputs motion vector MV and an evaluation value for motion vector MV as inter-frame prediction evaluation value InterCost. FIG. 16 is a diagram illustrating a processing flow of the pixel prediction unit Predictor of FIG. The same processing steps as those in FIG. 12 are denoted by the same symbols, and description thereof is omitted. In S14, the search start point is selected by the search start point selection unit ME1. In S15, the local search is performed by the local search unit ME2.

In addition, in an image encoding apparatus that switches between a plurality of prediction modes, a conventional technique for a method for speeding up mode determination is known (see, for example, Patent Document 1). In this prior art, the estimated value of the encoding efficiency in the first encoding mode is compared with the set value, and when the comparison result satisfies a condition that the code amount can be sufficiently reduced in the first encoding mode, This shows a method of reducing the processing amount by selecting one encoding mode and not estimating the encoding efficiency of the second encoding mode.
JP-A-8-186722 (Example 7)

  In the conventional image coding apparatus, the amount of processing of inter-frame prediction processing is reduced by focusing only on the motion vector detection processing. In the present invention, however, motion vector detection is performed using the evaluation value of intra-frame prediction. A method of reducing the processing amount of inter-frame prediction by truncating is provided.

  In order to solve the conventional problem, in the present invention, motion detection means for calculating an inter-frame prediction evaluation value, intra-frame prediction evaluation value calculation means for calculating an intra-frame prediction evaluation value, inter-frame prediction evaluation value, A mode determination unit that selects either inter-frame prediction or intra-frame prediction from the intra-frame prediction evaluation value as a prediction mode, and a variable that multiplexes prediction mode information indicating the prediction mode selected by the mode determination unit and outputs it as an encoded signal In the image encoding device, comprising: a long encoding unit; an inter-frame prediction evaluation value estimation unit that calculates an estimation value of an inter-frame prediction evaluation value before the end of calculation of an inter-frame prediction evaluation value; When the intra prediction evaluation value and the estimated value of the inter-frame prediction evaluation value satisfy a predetermined condition, intra-frame prediction is selected as the prediction mode, and motion detection is performed. Having a frame prediction judgment means for instructing termination of the calculation of the inter-frame prediction evaluation value means.

  With this configuration, when there is a high possibility that a block will be finally determined to be intra-frame prediction, the motion detection process can be aborted at that point, and the coding efficiency is almost degraded from the conventional image coding device. The amount of motion detection processing can be reduced without doing so.

  According to the image encoding device of the present invention, the processing amount of inter-frame prediction can be reduced, and as a result, the processing amount of the image encoding device can be reduced.

  Embodiments of the present invention will be described below with reference to the drawings. The point of the present invention is to estimate and use the evaluation value of inter-frame prediction. When there is a high possibility that the block is finally determined to be intra-frame prediction and mode determination, the motion detection process is terminated at that point. is there. As a result, it is possible to reduce the amount of motion detection processing with almost no deterioration in encoding efficiency from the conventional image encoding apparatus. The present invention is particularly effective when there are a large number of blocks determined to be intra-frame prediction. For example, when the camera is moved greatly or when a large object enters the frame, the inter-frame correlation becomes low, and the number of blocks for intra-frame prediction increases. In such a case, the effect of the present invention is enhanced.

(Embodiment 1)
The image coding apparatus of the present invention differs from the conventional image coding apparatus shown in FIG. 10 only in the pixel prediction unit Predictor. Therefore, here, the pixel prediction unit Predictor will be described, and the other parts are assumed to be the same as those of the conventional image encoding device, and description thereof will be omitted.

  FIG. 1 is a block diagram illustrating a pixel prediction unit Predictor according to the first embodiment. Units that operate in the same manner as in FIG. 11 and signals that operate in the same manner are denoted by the same symbols, and description thereof is omitted. The difference from the pixel prediction unit Predictor of the conventional image encoding device in FIG. 11 is that an inter-frame prediction evaluation value estimation unit InterCostEst, an intra-frame prediction determination unit IntraDet, and a switch SW2 are added. The inter-frame prediction evaluation value estimation unit InterCostEst estimates an inter-frame prediction evaluation value, and outputs an inter-frame prediction evaluation value estimation value EInterCost. The intra-frame prediction determination unit IntraDet inputs the intra-frame prediction evaluation value IntraCost and the inter-frame prediction evaluation value estimation value EInterCost, and instructs the motion detection unit ME not to perform motion detection when a predetermined condition is satisfied, Switch SW2 is set to “0”. When the predetermined condition is not satisfied, the switch SW2 is switched to “1” and the same processing as that of the conventional image encoding device is performed.

  FIG. 2 is a diagram illustrating a processing flow of the pixel prediction unit Predictor of FIG. The same processing steps as those in FIG. 12 are denoted by the same symbols, and description thereof is omitted. In the present invention, the intra-frame prediction evaluation value calculation of S12 and the inter-frame prediction evaluation value estimation of S16 are performed before the motion vector search of S11, and when the predetermined condition is satisfied in the intra-frame prediction determination of S17, the prediction mode is set there. The determination is made as intra-frame prediction, and the process ends. Here, the processing is performed in the order of S12 and S16, but this order may be reversed.

  As a condition used in the intra-frame prediction determination, for example, a condition represented by Expression 6 can be considered.

  Here, the inter-frame prediction evaluation value estimation value E is an estimation value of the inter-frame prediction evaluation value. Expression 6 corresponds to the inter-frame prediction evaluation value of the conditional expression determined as intra-frame prediction in the determination expression (Expression 5) used in the mode determination unit ModeSel replaced with the estimated value E. Therefore, if a value close to the inter-frame prediction evaluation value of the block can be selected as the estimated value E, the selected prediction mode should have almost the same result as that of the conventional image encoding device. If the calculation processing amount of the inter-frame prediction evaluation value estimation value E is smaller than the calculation processing amount of the inter-frame prediction evaluation value by the motion detection unit ME, the processing amount of the pixel prediction unit Predictor decreases. As an example of such an estimated value E of the inter-frame prediction evaluation value, it is conceivable to use a value obtained by encoding a block encoded in the past, such as an inter-frame prediction evaluation value of the immediately preceding block.

  As described above, according to the present embodiment, it is possible to reduce the amount of motion detection processing with almost no deterioration in encoding efficiency from the conventional image encoding apparatus.

  In the method of Patent Document 1 shown in the background art, an intra-frame prediction evaluation value is compared with a preset value. In the present invention, an inter-frame prediction evaluation value is estimated from information of a block encoded in the past. Therefore, intra-frame prediction can be determined in advance with higher accuracy.

  Expression 6 is an example of an intra-frame prediction determination expression, and other determination expressions using the inter-frame prediction evaluation value estimated value and the intra-frame prediction evaluation value as variables may be used.

  In addition, although this Embodiment demonstrated the case of only two types of prediction modes, also in the case of three or more types of prediction modes, an evaluation value is estimated with respect to one or more types of prediction modes, By comparing the evaluation value or the estimated value of the evaluation value, the same processing as in the present embodiment can be performed.

  In the present embodiment, inter-frame prediction and intra-frame prediction have been described as examples of the prediction mode, but other prediction methods may be used.

(Embodiment 2)
A part of the motion detection process is performed on the estimated value of the inter-frame prediction described in Equation 6 of Embodiment 1, and the estimation of the inter-frame prediction evaluation value is estimated from the result, thereby further estimating the inter-frame prediction evaluation value. A method of increasing the accuracy is also conceivable. This will be described below.

  The image coding apparatus of the present invention is different from the conventional image coding apparatus shown in FIG. 10 only in the pixel prediction unit Predictor. Therefore, here, the pixel prediction unit Predictor will be described, and the other parts are the same as those of the conventional image encoding device, and description thereof will be omitted.

  FIG. 3 is a block diagram illustrating the pixel predictor Predictor according to the second embodiment. Units that operate in the same manner as in FIG. 1 and FIG. The intra-frame prediction determination unit IntraDet inputs the inter-frame prediction evaluation value InterCost1 and the intra-frame prediction evaluation value IntraCost based on the initial motion vector MV1, and when the predetermined condition is satisfied, the motion detection unit ME does not perform motion detection. And switch SW2 to “0”. When the predetermined condition is not satisfied, the switch SW2 is switched to “1” and the same processing as that of the conventional image encoding device is performed. The inter-frame prediction evaluation value InterCost1 based on the initial motion vector MV1 corresponds to the inter-frame prediction evaluation value estimated value EInterCost in the first embodiment.

  FIG. 4 is a diagram showing a processing flow of the pixel prediction unit Predictor of FIG. The same processing steps as those in FIG. 16 are denoted by the same symbols, and description thereof is omitted. When the intraframe prediction evaluation value calculation in S12 and the interframe prediction evaluation value estimation in S16 are performed before the local search in S15, and the predetermined condition is satisfied in the intraframe prediction determination in S17, the prediction mode is set in the frame It is determined as prediction and the process ends. Here, the processes are performed in the order of S14, S12, and S16, but the order of these processes may be changed.

  The predetermined condition of the present invention can be determined by Expression 6 as in the first embodiment. When it is finally determined to be intraframe prediction, Equation 6 should always be intraframe prediction regardless of the position of the search start point. Therefore, the problem is when the intra-frame prediction is determined by Equation 6 regardless of the final determination of inter-frame prediction. However, when it is finally determined to be inter-frame prediction, it can be expected that the inter-frame prediction evaluation value interCost1 takes a sufficiently small value with respect to the intra-frame prediction evaluation value IntraCost at the time of the initial motion vector. For this reason, the above-described problems hardly occur, and therefore there is almost no deterioration in encoding efficiency with respect to the conventional image encoding apparatus.

  As described above, according to the present embodiment, it is possible to reduce the amount of motion detection processing with almost no deterioration in encoding efficiency from the conventional image encoding apparatus.

  Expression 6 is an example of an intra-frame prediction determination expression, and other determination expressions using the inter-frame prediction evaluation value estimated value and the intra-frame prediction evaluation value as variables may be used.

  Note that the determination in Embodiment 1 and the determination in Embodiment 2 may be combined. For example, the intra-frame prediction may be performed only when both determination formulas indicate intra-frame prediction.

  In addition, although this Embodiment demonstrated the case of only two types of prediction modes, also in the case of three or more types of prediction modes, an evaluation value is estimated with respect to one or more types of prediction modes, By comparing the evaluation value or the estimated value of the evaluation value, the same processing as in the present embodiment can be performed.

  In the present embodiment, inter-frame prediction and intra-frame prediction have been described as examples of the prediction mode, but other prediction methods may be used.

(Embodiment 3)
Furthermore, by recording a program for realizing the configuration of the image encoding method shown in each of the above embodiments on a storage medium such as a flexible disk, the processing shown in each of the above embodiments is performed. It can be easily implemented in an independent computer system.

  FIG. 5 is an explanatory diagram of a storage medium for storing a program for realizing the image coding methods of the first and second embodiments by a computer system.

  FIG. 5B shows an appearance, a cross-sectional structure, and a flexible disk as viewed from the front of the flexible disk, and FIG. 5A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, an image encoding method as the program is recorded in an area allocated on the flexible disk FD.

  FIG. 5C shows a configuration for recording and reproducing the program on the flexible disk FD. When recording the program on the flexible disk FD, the image encoding method as the program is written from the computer system Cs via the flexible disk drive FDD. When the image encoding method is built in a computer system by a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

(Embodiment 4)
As a fourth embodiment of the present invention, application examples of the moving picture coding method, the moving picture coding apparatus, and the moving picture coding program described above and a system using the same will be described with reference to FIGS. .

  FIG. 6 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.

  The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

  However, the content supply system ex100 is not limited to the combination as shown in FIG. 6 and may be connected in combination. Also, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs data transmission processing. The moving image data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, the moving image data may be encoded by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile phone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In the content supply system ex100, the content (for example, a video image of music live) captured by the user with the camera ex113, the camera ex116, and the like is encoded and transmitted to the streaming server ex103, while the streaming server ex103. Distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, and a mobile phone ex114 that can decode the encoded data. In this way, the content supply system ex100 can receive and play back the encoded data at the client, and can also receive a private broadcast by receiving, decoding, and playing back at the client in real time. It is a system that becomes possible.

  In the content encoding process or decoding process, the moving picture encoding method, the moving picture encoding apparatus, and the moving picture encoding program described in the above embodiment may be used. For example, the computer ex111, the PDA ex112, the camera ex113, the mobile phone ex114, and the like may be provided with the moving picture coding apparatus described in the above embodiment and realize the moving picture coding method and the moving picture coding program.

A mobile phone will be described as an example.
FIG. 7 is a diagram showing a mobile phone ex115 using the moving picture coding method, the moving picture coding apparatus, and the moving picture coding program according to the embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video from a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording media ex207 and mobile phone ex115 with recording media ex207 And a slot portion ex206 to ability. The recording medium ex207 stores a flash memory element, which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory), which is a nonvolatile memory that can be electrically rewritten and erased, in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The mobile phone ex115 controls the main control unit ex311 which controls the respective units of the main body unit including the display unit ex202 and the operation key ex204. The power supply circuit unit ex310, the operation input control unit ex304, and the image encoding Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

  When the end of call and the power key are turned on by a user operation, the power supply circuit ex310 starts up the camera-equipped digital mobile phone ex115 in an operable state by supplying power from the battery pack to each unit. .

  The cellular phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305 After conversion into a signal, this is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 converts the image data supplied from the camera unit ex203 into encoded image data by compression encoding, and sends this to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 via the voice processing unit ex305 as digital voice data.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit Spread spectrum processing is performed in ex306, digital analog conversion processing and frequency conversion processing are performed in the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.

  When receiving data of a moving image file linked to a homepage or the like in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Data is sent to the demultiplexing unit ex308.

  In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 generates reproduction moving image data by decoding the encoded bit stream of the image data, and supplies this to the display unit ex202 via the LCD control unit ex302. The moving image data included in the moving image file linked to the home page is displayed. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  In the above configuration, the image encoding unit ex312 includes the moving image encoding apparatus of the above embodiment.

  It should be noted that the present invention is not limited to the above system, and recently, digital broadcasting by satellite and terrestrial has become a hot topic, and as shown in FIG. A moving picture coding apparatus and a moving picture coding program can be incorporated. Specifically, in the broadcasting station ex409, a coded bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. Here, a device such as the television (receiver) ex401 or the set-top box (STB) ex407 may include the moving image encoding device described in the above embodiment. Moreover, you may use the moving image encoding method of the said embodiment. Furthermore, a moving image encoding program may be provided. The playback apparatus ex403 that reads and decodes the encoded bitstream recorded on the storage medium ex402 such as a CD or DVD that is a recording medium also includes the moving picture encoding method, the moving picture encoding apparatus, and the like described in the above embodiment. It is possible to implement a moving image encoding program. In this case, the reproduced video signal is displayed on the monitor ex404. In addition, the moving picture coding method, the moving picture coding apparatus, and the moving picture coding program described in the above embodiment are included in the set-top box ex407 connected to the cable ex405 for cable television or the antenna ex406 for satellite / terrestrial broadcasting. A configuration is also possible in which the above is implemented and this is played back on a television monitor ex408. At this time, the moving picture coding apparatus described in the above embodiment may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce the moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Furthermore, an image signal can be encoded and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records image signals on a DVD disk ex421 and a disk recorder that records images on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the decoding device of the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be interpolated and reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 8, and the same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

  In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats, that is, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

  As described above, the moving picture coding method, the moving picture coding apparatus, and the moving picture coding program described in the above embodiment can be used in any of the devices and systems described above, and the effects described in the above embodiment. Can be obtained.

  The device according to the present invention can reduce the processing amount of inter-frame prediction, and as a result, can reduce the processing amount of the image encoding device, and is useful for, for example, a moving image encoding device.

Block diagram of the pixel prediction unit of the first embodiment Process Flow of Pixel Prediction Unit of Embodiment 1 Block diagram of a pixel prediction unit according to the second embodiment Process Flow of Pixel Prediction Unit of Embodiment 2 Explanatory drawing about the storage medium for storing the program for implement | achieving the image coding method of Embodiment 1- Embodiment 2 by a computer system Block diagram explaining the entire configuration of the content supply system Example of a mobile phone equipped with the moving picture encoding apparatus of the present invention Block diagram explaining the structure of a mobile phone Example of digital broadcasting system Block diagram of a conventional image encoding device Block diagram of a conventional pixel prediction unit Processing flow of conventional pixel prediction unit Block diagram of image decoding device Conceptual diagram of local search Block diagram of a pixel prediction unit in a conventional image encoding device that performs local search Processing flow of conventional pixel prediction unit that performs local search

Explanation of symbols

Img image signal
BitStrm encoded signal
Res residual signal
ERes residual encoded signal
DRes residual decoded signal
DecImg Decoded image signal
Ref reference image signal
Pred Predictive image signal
MV motion vector
Mode Prediction mode information
InterCost interframe prediction evaluation value
IntraCost Intra-frame prediction evaluation value
MCPred interframe prediction image signal
IPred intra-frame prediction image signal
MV1 initial motion vector
InterCost1 Initial motion vector interframe prediction evaluation value
EInterCost Interframe prediction evaluation value estimate
Sub subtractor
Add Adder
ResEnc Residual encoder
ResDec residual decoder
FM frame memory
Predictor pixel prediction unit
VLC variable length encoder
SW1, SW2 switch
ME motion detector
IntraCostCalc Intra-frame prediction evaluation value calculator
ModeSel Mode judgment part
MC motion compensation unit
IntraPredictor Intraframe prediction unit
VLD variable length decoder
ME1 Search start point selection section
ME2 Local search unit
IntraDet Intraframe prediction determination unit
InterCostEst inter-frame prediction evaluation value estimation unit
Cs computer system
FD flexible disk
FDD flexible disk drive

Claims (6)

In a video encoding method for selecting one prediction mode from two or more types of prediction modes and encoding type information indicating the selected one prediction mode,
If one or more of the two or more types of prediction modes are set as the first prediction mode and a mode other than the first prediction mode is set as the second prediction mode, the first prediction mode is set before the evaluation value calculation in the first prediction mode is completed. When the estimated value of the evaluation value and the evaluation value of the second prediction mode are calculated, and the estimated value of the evaluation value of the first prediction mode and the evaluation value of the second prediction mode satisfy a predetermined condition The moving picture coding method characterized in that the evaluation value calculation in the first prediction mode is terminated and the second prediction mode is selected as the one prediction mode.   The moving image encoding method according to claim 1, wherein an estimated value of the evaluation value of the first prediction mode and an evaluation value of the second prediction mode are calculated before the evaluation value calculation of the first prediction mode is started, When the estimated value of the evaluation value of the first prediction mode and the evaluation value of the second prediction mode satisfy a predetermined condition, the evaluation value calculation of the first prediction mode is terminated, and the second prediction mode is changed to the second prediction mode. A moving picture encoding method, characterized by selecting as one prediction mode.   The video encoding method according to claim 1, wherein the evaluation value calculation in the first prediction mode is interrupted halfway, the evaluation value in the first prediction mode is estimated from the calculation intermediate result, and the evaluation value in the first prediction mode is determined. If the estimated value and the evaluation value of the second prediction mode satisfy a predetermined condition, the evaluation value calculation of the first prediction mode is terminated and the second prediction mode is selected as the one prediction mode. A video encoding method characterized by the above.   4. The moving image encoding method according to claim 1, wherein a prediction mode indicating inter-frame prediction is the first prediction mode, and a prediction mode indicating intra-frame prediction is the second prediction mode. Encoding method. Motion detection means for calculating an inter-frame prediction evaluation value, intra-frame prediction evaluation value calculation means for calculating an intra-frame prediction evaluation value, inter-frame prediction and intra-frame prediction based on the inter-frame prediction evaluation value and the intra-frame prediction evaluation value A mode determination unit that selects one of predictions as a prediction mode; and a variable-length encoding unit that multiplexes prediction mode information indicating the prediction mode selected by the mode determination unit and outputs the multiplexed signal as an encoded signal. In the image encoding device,
Inter-frame prediction evaluation value estimation means for calculating an estimated value of the inter-frame prediction evaluation value before the calculation of the inter-frame prediction evaluation value, and the intra-frame prediction evaluation value and the estimated value of the inter-frame prediction evaluation value An intra-frame prediction determining unit that selects intra-frame prediction as a prediction mode when the predetermined condition is satisfied, and instructs the end of calculation of the inter-frame prediction evaluation value of the motion detection unit; A moving image encoding device. A program for selecting a prediction mode from two or more types of prediction modes by a computer and performing a moving picture encoding method for encoding type information indicating the selected one prediction mode,
If one or more of the two or more types of prediction modes are set as the first prediction mode and a mode other than the first prediction mode is set as the second prediction mode, the first prediction mode is set before the evaluation value calculation in the first prediction mode is completed. When the estimated value of the evaluation value and the evaluation value of the second prediction mode are calculated, and the estimated value of the evaluation value of the first prediction mode and the evaluation value of the second prediction mode satisfy a predetermined condition A moving image characterized by causing a computer to perform a moving image encoding method, wherein the evaluation value calculation in the first prediction mode is terminated and the second prediction mode is selected as the one prediction mode. Encoding program.

Images