JP2014042139A - Coding device, coding method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a suppressed capacity of a reference image storage unit, an improved coding efficiency and a suppressed readout data amount from the storage unit, with satisfactory balance.SOLUTION: Among a plurality of blocks obtained when dividing a processing target frame image into blocks of processing units for a second motion search, a block in which a higher coding efficiency (Yes in step 104) is obtained by using a first motion vector than using a second motion vector is determined to be a specific block for the second motion search (step 112) within a range not exceeding an upper limit value in regard to the total value of a direct readout data amount when a reference frame image is read out directly without intervention of a reference image storage unit. Other blocks than the specific block are determined to be an unspecific block for the second motion search using the second motion vector (step 114).

Description

  The disclosed technology relates to a moving image encoding device, an encoding method, and a program.

  The motion search process is a process of searching a reference image for a block similar to a block to be processed (here, called an original block). For example, first, a large number of candidate blocks having the same size as the original block are extracted from the reference image. Next, for each candidate block, an evaluation index (SAD (sum of absolute differences), encoding cost obtained from a motion vector, or a value obtained by combining a plurality of evaluation indices) is calculated. . Then, based on the calculated evaluation index value, a motion vector indicating the candidate block evaluated to be the best is output.

  In order to improve the coding efficiency by inter-frame prediction, it is required to extract candidate blocks more similar to the original block in the motion search process. For that purpose, it is desirable that the image area (hereinafter referred to as the search range) from which candidate blocks are extracted is wider. However, when the search range is expanded, the number of times of calculating the similarity evaluation index with the original image increases, and the amount of calculation required for motion search processing and when extracting candidate blocks from the reference image stored in the frame memory The amount of read data increases, and the memory bandwidth is reduced. The calculation amount of the motion search process and the read process from the memory occupy a considerable proportion in the entire moving picture encoding process, and various techniques exist for reducing this.

  A technique is known in which a reference buffer memory (hereinafter referred to as a reference image storage unit) is provided between a main storage unit that stores a reference image and a motion search unit that performs a motion search. In the reference image storage unit, a reference image partially read from a frame buffer memory (hereinafter simply referred to as a storage unit) is transferred and stored. The motion search unit reads a reference image from the reference image storage unit and performs a motion search. Thereby, the bandwidth of the storage unit required for reading the reference image is suppressed.

  A hierarchical search method is also known as one of motion search methods. For example, in the first-stage motion search, an approximate motion vector is derived for a wide search range using an image obtained by reducing the original image. In the second stage motion search, using the original image and the reference image, motion prediction is performed in detail for a narrow search range centered on the point indicated by the motion vector of the first stage motion search result. In this hierarchical search method, since the first-stage motion prediction for calculating the similarity index between a large number of candidate regions is performed with the reduced image, the amount of calculation can be greatly reduced.

JP 2009-152710 A JP 2012-4710 A JP-A-7-203462

  However, in the technique using the reference image storage unit, it is necessary to hold a plurality of processing block lines in the reference image in the reference image storage unit, so that the capacity is large. In order to reduce the memory capacity, it is desirable to narrow the search range of the motion search process. On the other hand, it is desirable to widen the search range from the viewpoint of coding efficiency.

  An object of the disclosed technique is to achieve, in a balanced manner, suppression of the capacity of the reference image storage unit, improvement of encoding efficiency, and suppression of the amount of read data from the storage unit.

  In the disclosed technology, the first motion search unit performs the first motion search including the first process and the second process for each block that is the processing unit of the first motion search in the processing target frame image of the moving image. Do. In the first process, the first motion search unit searches for the first motion vector with reference to the first search range in the reduced reference frame image to obtain the encoding efficiency. In the second process, the first motion search unit searches for a second motion vector by referring to a second search range smaller than the first search range in the reduced reference frame image and included in the first search range. The encoding efficiency is obtained.

  The reference image storage unit stores a reference image partially read from a storage unit that stores a reference frame image referred to in the second motion search performed based on the processing result of the first motion search. .

  The control unit stores the reference image in the reference image storage unit before starting the second motion search of each block for each block that is a processing unit of the second motion search in the processing target frame image. Let At this time, the control unit causes the reference image storage unit to store the reference image including the search range of the second motion search set based on the second motion vector for each block.

  The determination unit encodes the first motion vector rather than the second motion vector among a plurality of blocks when the processing target frame image is divided into blocks that are processing units of the second motion search. A block with high efficiency is determined as a specific block. The specific block is a block that performs the second motion search using the first motion vector. The determination unit determines the specific block within a range in which the total value of the directly read data amount when the reference frame image is directly read from the storage unit without going through the reference image storage unit does not exceed the upper limit value. judge.

  The determination unit determines a block other than the specific block as a non-specific block that performs the second motion search using the second motion vector.

  The second motion search unit refers to a search range image set based on the first motion vector in the reference frame image for the block determined as the specific block by the determination unit, and determines a motion vector. The second motion search to be searched is performed. The second motion search unit refers to the image directly read from the storage unit without going through the reference image storage unit when referring to at least an image not stored in the reference image storage unit.

  In addition, the second motion search unit is configured to search for the block determined to be the non-specific block based on the second motion vector among the reference images stored in the reference image storage unit. A second motion search for searching for a motion vector with reference to the image is performed.

  As one aspect, the disclosed technology has an effect that each of suppression of the capacity of the reference image storage unit, improvement of encoding efficiency, and suppression of the amount of read data from the storage unit can be realized in a balanced manner. .

It is a schematic block diagram of the encoding apparatus which concerns on embodiment. It is a schematic block diagram of a 1st motion search part. It is a schematic block diagram of a reference image memory circuit. It is a schematic block diagram of a 2nd motion search part. It is a figure which shows typically the memory | storage state of the reference image memory just before the process start of the 2nd motion search about the block B1 to be processed. It is a figure which shows typically the memory | storage state of the reference image memory just before the process start of the 2nd motion search about the block B2 made into a process target next to the block B1. FIG. 7 is a diagram schematically illustrating the storage state of the reference image memory immediately before the start of the second motion search process for the processing target block B1 in an update control example different from the update control examples illustrated in FIGS. 5 and 6. In the update control example different from the update control examples shown in FIGS. 5 and 6, the storage state of the reference image memory immediately before the start of the second motion search process for the block B2 to be processed next to the block B1 is schematically shown. FIG. It is a flowchart which shows an example of the determination process performed by the determination part. It is a flowchart which shows an example of a specific block determination process. It is a figure which shows an example of the timing chart which shows the timing of the process of each component of an encoding apparatus. It is a flowchart which shows the other example of a specific block determination process. It is the figure which illustrated the synthetic | combination range. It is a flowchart which shows the other example of a specific block determination process. It is the schematic diagram which showed typically the content of the specific block determination process shown in FIG. 14 for every step. It is a figure which shows the example of arrangement | positioning of four blocks w, x, y, and z. It is a figure which shows an example of arrangement | positioning of three blocks, and an example of the search range of each block. It is the figure which represented typically the area | region hold | maintained at a reference buffer for every processing period of each block shown to FIG. 17A. It is a figure which shows a mode that a motion search and the access to the main memory are processed in a pipeline. It is a flowchart which shows the other example of a specific block determination process. It is a block diagram in case the function of the determination part of an encoding apparatus is implement | achieved by a computer, and the function of a 1st motion search part and a main encoding process part is implement | achieved by the encoding part provided in the computer.

  Hereinafter, an example of an embodiment of the disclosed technology will be described in detail with reference to the drawings.

  FIG. 1 shows an encoding apparatus 1 according to this embodiment. The encoding device 1 is, for example, MPEG2 or H.264. This is a device that encodes a moving image according to a standard such as H.264, and includes a reduction unit 10, a first motion search unit 11, a determination unit 12, a main encoding processing unit 13, a memory controller 14, and a main memory 15. Each functional unit of the encoding device 1 can be realized by, for example, an electronic circuit, and may be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit).

  The main memory 15 can be a memory module such as SDRAM (Synchronous Dynamic Random Access Memory). The main memory 15 is provided with storage areas of a reduced image memory 16, an original image memory 17, and a frame memory 18.

  The main memory 15 is connected to the memory controller 14. The memory controller 14 receives a write request or a read request, and performs reading / writing (memory control) with respect to the main memory 15 in accordance with the received request.

  The reduction unit 10 and the first motion search unit 11 are components for performing a first-stage motion search (hereinafter referred to as a first motion search) in an inter-frame prediction encoding process including a two-stage motion search process. is there. Here, inter-frame predictive coding is a well-known method of generating a predicted image based on frame images at different times and encoding a difference (error) image between the image to be encoded and the predicted image.

  A frame image included in a moving image before encoding processing (hereinafter, referred to as an original image to be distinguished from a frame image after encoding processing) is input to the reduction unit 10. The reduction unit 10 generates a reduced image obtained by reducing the original image, and stores the reduced image in the reduced image memory 16 of the main memory 15 via the memory controller 14.

  The original image is also stored in the original image memory 17 of the main memory 15 via the memory controller 14.

  The first motion search unit 11 reads a reduced image from the reduced image memory 16 via the memory controller 14, and performs a motion search using the read reduced image. Hereinafter, the motion search performed by the first motion search unit 11 is referred to as a first motion search.

  The first motion search unit 11 performs a first motion search for each block (region) predetermined as a processing unit of the first motion search. The first motion search unit 11 performs a first motion search by partially referring to the reference frame image. In the present embodiment, the reference frame image is a reduced image obtained by reducing an original image at a different time from the original image including the processing target block (processing target frame image). In the first motion search, a first process for searching for a motion vector with reference to an image in the first search range in the reduced image, and a first process for searching for a motion vector with reference to an image in the second search range in the reduced image. Two processes are performed. Here, the motion vector refers to data indicating the relative positional relationship (deviation amount and deviation direction) between the block selected from the reference frame image and the block to be processed. A block refers to each area when an image is divided into a plurality of areas, or an image of each area. Note that the second search range is smaller than the first search range and is included in the first search range. The size and position of the first search range and the size and position of the second search range are determined in advance according to the size and position of the block to be processed.

  In the first motion search, the blocks to be processed are sequentially switched in the direction from left to right. When the processing is completed up to the rightmost block of the image, the processing target is switched to the leftmost block in the line one line below. Further, the block to be processed in the first motion search can be sequentially switched by a switching signal generated at a predetermined time interval by the signal generating unit provided in the encoding device 1, for example.

  FIG. 2 shows an example of a schematic configuration diagram of the first motion search unit 11. The first motion search unit 11 includes a holding unit 36, a control unit 37, a calculation unit 38, an original image buffer 39, and a reference buffer 40.

  When the switching timing arrives, for example, by receiving a switching signal generated at a predetermined time interval from the signal generation unit, the control unit 37 controls each component constituting the first motion search unit 11 to perform the first motion search. The processing target is switched to the next block. In addition, a read request is output to the memory controller 14. As a result, the image of the block in the reduced image of the original image to be processed next for the first motion search is read out from the reduced image memory 16 of the main memory 15 and stored in the original image buffer 39. Further, an image corresponding to the first search range in the reduced image of the original image that has been processed before switching is stored in the reference buffer 40 as a reference image from the main memory 15.

  Specifically, the control unit 37 outputs the read request to the memory controller 14 including the range to be read and the storage destination address of the read image, and updates the original image buffer 39 and the reference buffer 40. The memory controller 14 reads the reduced image in the range to be read designated by the read request and stores it in the storage destination address designated by the read request.

  The original image buffer 39 and the reference buffer 40 may be, for example, a FIFO memory (a first-in first-out memory). In this case, the storage area of the block that has become unnecessary is set as an unused area and can be a storage destination for storing an image read from the main memory 15.

  Further, the control unit 37 sequentially generates motion vectors to be evaluated by the calculation unit 38 and held by the holding unit 36, and sends the motion vectors to the calculation unit 38 and the holding unit 36.

  The calculation unit 38 controls the control unit 37 from the image of the block to be processed stored in the original image buffer 39 and the first search range in the reference frame image at a time different from the processing target stored in the reference buffer 40. Is compared with a block selected based on the motion vector sent from (hereinafter referred to as a candidate block). Then, the calculation unit 38 calculates SAD (Sum of Absolute Differences) as an evaluation value of the similarity between each candidate block and the block to be processed.

  Since the second search range is included in the first search range, the calculation unit 38 does not have to separately calculate SAD for each candidate block included in the second search range. Although the example in which the calculation unit 38 calculates SAD has been described here, other evaluation values such as SSD (Sum of Squared Differences) and SATD (Sum of Absolute Transformed Differences) are calculated. May be. Further, an evaluation value considering the influence of the motion vector may be calculated. Other known techniques may be used.

  The holding unit 36 unconditionally holds the motion vector sent from the control unit 37 first in each block and the evaluation value calculated by the calculation unit 38 based on this motion vector as the first processing result candidate. . Thereafter, each time an evaluation value is sent from the calculation unit 38, it is compared with the evaluation value of the held first processing result candidate. As a result of the comparison, if the newly sent evaluation value is better than the stored evaluation value, the first processing result candidate is updated with the evaluation value and the motion vector corresponding to the evaluation value. This process is repeated for all the vectors sent from the control unit 37 and the evaluation values calculated by the calculation unit 38 for the vectors, so that the best evaluation value among the motion vector candidates of this block is finally obtained. Since the first processing result candidate is held with the motion vector indicated and the corresponding evaluation value, the held first processing result candidate is used as the motion vector and evaluation value of the first processing of the first motion search in this block. Selected.

  Further, in the second process, the holding unit 36 includes only the motion vectors sent from the control unit 37 that belong to the second search range and the evaluation values calculated by the calculation unit 38 for the motion vectors. Then, “holding the first candidate” and “updating with a value indicating a better evaluation value” described in the explanation of the operation of the first process of the holding unit 36 are performed. Also in the second process, a candidate to be held and updated (second process result candidate) is a set of a motion vector and an evaluation value, like the first process result candidate. Eventually, the motion vector indicating the best evaluation value among the motion vector candidates in the second search range of this block and the corresponding evaluation value are held as the second processing result candidates. The second processing result candidate is selected as the motion vector and evaluation value of the second processing of the first motion search in this block.

  Hereinafter, the motion vector searched in the first process is referred to as a first motion vector, and the motion vector searched in the second process is referred to as a second motion vector. In addition, the SAD corresponding to the first motion vector is referred to as a first evaluation value, and the SAD corresponding to the second motion vector is referred to as a second evaluation value. Each of the first motion vector, the first evaluation value, the second motion vector, and the second evaluation value is output to the determination unit 12.

  Note that the first evaluation value and the second evaluation value output to the determination unit 12 indicate coding efficiency such as SSD, SATD, an evaluation value that considers the influence of a motion vector, and other well-known techniques, not SAD. If it is a value considered that, it will not specifically limit.

  As described above, since the first motion search unit 11 uses the reduced image for the first motion search, the amount of calculation is greatly reduced, and the motion search becomes efficient.

  In the present embodiment, in the first motion search, a reduced image of a frame image at a time different from that of the processing target frame image including the processing target block is used as the reference frame image. However, the present invention is not limited to this. For example, in the first motion search, a reduced image of a decoded image of a frame image at a time different from that of the processing target frame image including the processing target block may be used as the reference frame image. The decoded image is generated in the main encoding processing unit 13 described later.

  Either the first motion vector or the second motion vector searched by the first motion search unit 11 is used for the second motion search performed in the main encoding processing unit 13 to be described later. Which motion vector is used? Depends on the determination result of the determination unit 12.

  The determination unit 12 includes a block (hereinafter, referred to as a specific block) in which each of a plurality of blocks serving as a second motion search processing unit included in one frame image performs a second motion search using the first motion vector. Then, it is determined which of the blocks (hereinafter, referred to as non-specific blocks) is subjected to the second motion search using the second motion vector. A specific example of the determination method will be described later. As a determination result, the determination unit 12 outputs the first motion vector to the main encoding processing unit 13 for the block determined as the specific block, and the second motion vector as the main code for the block determined as the non-specific block. To the processing unit 13. Hereinafter, of the first motion vector and the second motion vector, the motion vector output as a determination result from the determination unit 12 is referred to as a determination motion vector.

  The main encoding processing unit 13 includes a prediction error image generation unit 20, an orthogonal transform / quantization unit 21, an inverse orthogonal transform / inverse quantization unit 22, a decoded image generation unit 23, a filter unit 24, a second motion search unit 25, And an inter-frame prediction unit 26. The main encoding processing unit 13 further includes a reference image memory circuit 27, an intra-frame prediction mode determination unit 28, an intra-frame prediction unit 29, a switching unit 30, and an entropy encoding unit 31.

  The prediction error image generation unit 20 is supplied with the original image as an encoding target.

  The prediction error image generation unit 20 is supplied with a prediction image from the inter-frame prediction unit 26 or the intra-frame prediction unit 29. The predicted image will be described later. The prediction error image generation unit 20 calculates difference information indicating the difference between the image to be encoded and the prediction image, and supplies this difference information to the orthogonal transform / quantization unit 21 as prediction error image information. As described above, the prediction error image generation unit 20 generates a prediction error image.

  The orthogonal transform / quantization unit 21 performs conversion processing on the supplied prediction error image information, and separates the prediction error image into frequency components in the horizontal and vertical directions. Further, the orthogonal transform / quantization unit 21 quantizes information on each frequency component obtained by the transform process. The orthogonal transform / quantization unit 21 reduces the code amount of the prediction error image by quantizing the information. The orthogonal transform / quantization unit 21 supplies the quantized information to the inverse orthogonal transform / inverse quantization unit 22 and the entropy coding unit 31.

  The inverse orthogonal transform / inverse quantization unit 22 performs inverse transform processing after dequantizing the information supplied from the orthogonal transform / quantization unit 21. Here, the inverse transform process refers to a process for transforming in the opposite direction to the transform process performed by the orthogonal transform / quantization unit 21.

  By performing decoding processing by the inverse orthogonal transform / inverse quantization unit 22, information of an error image corresponding to a prediction error image before encoding is obtained. This information is called a reproduction prediction error image.

  The decoded image generation unit 23 is supplied with the reproduction prediction error image generated by the inverse orthogonal transform / inverse quantization unit 22. The decoded image generation unit 23 is supplied with a predicted image from the inter-frame prediction unit 26 or the intra-frame prediction unit 29. The decoded image generation unit 23 generates an image (decoded image) obtained by decoding the image to be encoded by adding the reproduction prediction error image and the predicted image. In addition to being output to the filter unit 24, the generated decoded image is also output to the intra-frame prediction mode determination unit 28 and the intra-frame prediction unit 29 in intra-frame prediction encoding (described later).

  The filter unit 24 performs a filtering process on the decoded image generated by the decoded image generation unit 23 and stores it in the frame memory 18 of the main memory 15 via the memory controller 14. Since the encoding process is performed in units of blocks, noise may occur at the boundary between blocks. The filter unit 24 reduces noise generated by the encoding.

  The frame memory 18 is a memory in which each decoded image supplied from the filter unit 24 is accumulated. The decoded image stored in the frame memory 18 is referred to by a second motion search unit 25 and an inter-frame prediction unit 26, which will be described later, for motion vector detection and motion compensation when encoding other frame images. When referring to the decoded image, there are a case where it is referred to via a reference image memory circuit 27 to be described later, and a case where it is directly read from the frame memory 18 for reference. Details will be described later.

  The intra-frame prediction mode determination unit 28 and the intra-frame prediction unit 29 are components for generating a predicted image for each block when encoding is performed using the intra-frame prediction encoding method. Here, the intra-frame predictive encoding method is a method of encoding / decoding an image using only information of a frame image to be encoded without using another frame image. Specifically, the motion of a neighboring image along a predetermined direction is estimated from an image of a block around the block to be encoded when one frame image is divided into a plurality of blocks, and the object to be encoded The block image is predicted, and the difference from the encoding target block image is encoded.

  The intra-frame prediction mode determination unit 28 determines a prediction mode when encoding is performed by the intra-frame prediction encoding method according to the original image to be encoded.

  The intra-frame prediction unit 29 predicts the image of the block to be encoded based on the original image input via the intra-frame prediction mode determination unit 28 in the prediction mode determined by the intra-frame prediction mode determination unit 28. Then, the predicted image is output.

  The second motion search unit 25, the inter-frame prediction unit 26, and the reference image memory circuit 27 perform the second-stage motion search (hereinafter referred to as the second motion search) in the inter-frame prediction encoding process including the two-stage motion search process. For example). In the second motion search, the above-described decoded image is used as a reference frame image, and the reference frame image is partially referenced to perform a motion search. The decoded image is used at the same magnification without being reduced.

  Also, in the second motion search, similarly to the first motion search, the processing target blocks are sequentially switched in the direction from left to right in the processing target frame. When the processing is completed up to the rightmost block of the image, the processing target is switched to the leftmost block in the line one line below. In addition, for example, the second motion search processing block is provided with a signal generation unit in the second motion search unit 25 and is sequentially switched by a switching signal generated at a predetermined time interval by the signal generation unit. Also good.

  Note that the size when the block of the first motion search processing unit is returned to the original magnification and the size of the block of the second motion search processing unit may be the same or different. Hereinafter, a case where the size of the block of the first motion search processing unit returned to the original magnification and the size of the block of the second motion search processing unit are the same will be described as an example.

  As shown in FIG. 3, the reference image memory circuit 27 includes a control unit 41 and a reference image memory 42. The reference image memory 42 may be a memory module such as an SRAM (Static Random Access Memory), for example, and an image partially read from the frame memory 18 in which the decoded image is stored is stored as a reference image. The reference image memory 42 is connected to the control unit 41. The control unit 41 performs reading / writing (memory control) with respect to the reference image memory 42.

  In the reference image memory 42, an image in a range (hereinafter referred to as a settable range) that may be set as a search range when the processing target block is a non-specific block in the second motion search is used as a reference image. Remembered. The settable range includes the second search range used in the first motion search, and may be a size slightly larger than the second search range. For example, a range that is several pixels outside the second search range may be set as the settable range. However, the settable range is smaller than the first search range.

  When the switching timing arrives, the control unit 41 of the reference image memory circuit 27 stores in the reference image memory 42 a reference image in a settable range corresponding to a block to be processed for the second motion search at least immediately after the switching. It is assumed that update control is performed so as to be in a state. As an update control of the reference image memory 42, the control unit 41 outputs a read request to the memory controller 14, acquires a reference image in a necessary range, writes (stores) it in the reference image memory 42, and Control for deleting an unnecessary range of reference images from the reference image memory 42 is performed.

  Here, an example of update control of the reference image memory 42 will be described with reference to FIGS. 5 and 6.

  FIG. 5 is a diagram schematically showing the storage state of the reference image memory 42 immediately before the start of the second motion search process for the block B1 to be processed. FIG. 6 is a diagram schematically showing the storage state of the reference image memory 42 immediately before the start of the second motion search process for the block B2 to be processed next to the block B1.

  In FIG. 5, a region 80 indicates a block at the same position as the processing target block B1. An area 81 indicates a settable range of the block B1. The area 82 is an area stored in the reference image memory 42. The area 82 overlaps any one of the settable ranges of the blocks to be processed after the block B1 and the block B1 among the combined ranges obtained by combining the settable ranges of the blocks to be processed before the block B1. It is also an area.

  The area 83 is an area added to the reference image memory 42 immediately after the processing of the processing target block B1 is finished and immediately before the processing of the next block B2 is started. The area 84 is an area that is deleted from the reference image memory 42 simultaneously with the addition of the area 83.

  Next, in FIG. 6, a region 85 indicates a block at the same position as the processing target block B2. An area 86 indicates a settable range of the block B2. The area 87 is an area stored in the reference image memory 42. The area 88 overlaps any one of the settable ranges of the blocks to be processed after the block B2 and the block B2 out of the combined range obtained by combining the settable ranges of the blocks to be processed before the block B2. .

  The area 88 is an area added to the reference image memory 42 immediately after the processing of the processing target block B2 is completed and immediately before the processing of the next block B3 is started. The area 89 is an area that is deleted from the reference image memory 42 at the same time that the area 88 is added.

  As is apparent from FIGS. 5 and 6, when the block to be processed in the second motion search is switched to the next block, it is included in any of the settable ranges of each block to be processed after the switching. Reference images that are not stored are deleted from the reference image memory 42. A reference image that is included in the settable range of the next block but is not yet stored in the reference image memory 42 is read from the frame memory 18 and stored in the reference image memory 42. For example, the reference image can be erased by overwriting the newly added reference image in the area where the reference image to be erased is stored.

  In the update control method described with reference to FIGS. 5 and 6 above, the reference image area used as the search range of a certain block remains in the reference image memory 42 until it is not used in the search range of any block. It is said. Therefore, the access amount (read data amount) to the frame memory 18 can be suppressed.

  Next, another example of update control to the reference image memory 42 will be described with reference to FIGS.

  FIG. 7 is a diagram schematically showing the storage state of the reference image memory 42 immediately before the start of the second motion search process for the block B1 to be processed. FIG. 8 is a diagram schematically showing the storage state of the reference image memory 42 immediately before the start of the second motion search process for the block B2 to be processed next to the block B1.

  In FIG. 7, an area 90 indicates a block at the same position as the processing target block B1. An area 91 indicates a settable range of the block B1. The area 92 is an area stored in the reference image memory 42. The area 92 is equal to the settable range of the block B1.

  The area 93 is an area added to the reference image memory 42 immediately after the processing of the processing target block B1 is finished and immediately before the processing of the next block B2 is started. The area 94 is an area that is deleted from the reference image memory 42 at the same time as the addition of the area 93.

  Next, in FIG. 6, an area 95 indicates a block at the same position as the processing target block B2. An area 96 indicates a settable range of the block B2. An area 97 is an area stored in the reference image memory 42. The area 96 is equal to the settable range of the block B2.

  The area 98 is an area added to the reference image memory 42 immediately after the processing of the processing target block B2 is finished and immediately before the processing of the next block B3 is started. The area 99 is an area that is deleted from the reference image memory 42 simultaneously with the addition of the area 98.

  As is apparent from FIGS. 7 and 8, when the block to be processed in the second motion search is switched to the next block, the settable range of the next block is deleted from the reference image memory 42. A reference image that is included in the settable range of the next block but is not yet stored in the reference image memory 42 is read from the frame memory 18 and stored in the reference image memory 42. For example, the reference image can be erased by overwriting the newly added reference image in the area where the reference image to be erased is stored.

  In the update control method described with reference to FIGS. 7 and 8 above, when the block to be processed is switched (when the reference image memory 42 is updated), it is included in the settable range of the block to be processed next. A range of reference images is left in the reference image memory 42. Since the reference image not included in the settable range of the block to be processed next is deleted, the capacity of the reference image memory 42 can be reduced.

  The second motion search unit 25 performs a second motion search within a search range corresponding to the determination motion vector input from the determination unit 12 for each block that is a processing unit of the second motion search. However, since the determination motion vector is a motion vector searched from the reduced image, it is used in a state where the original magnification is restored. For example, when the first motion search is performed from a reduced image multiplied by 1 / d in each of the horizontal direction and the vertical direction, each value in the horizontal direction and the vertical direction indicated by the determination motion vector is multiplied by d and used.

  In the second motion search, the decoded image is partially used as the reference image as described above. In the present embodiment, when the reference image to be referred to is stored in the reference image memory 42, the second motion search unit 25 refers to the reference image read from the reference image memory 42. Further, when the reference image to be referred to is not stored in the reference image memory 42, the second motion search unit 25 refers to the reference image directly read from the frame memory 18 without using the reference image memory 42. .

  FIG. 4 shows an example of a schematic configuration diagram of the second motion search unit 25. The second motion search unit 25 includes a holding unit 44, a control unit 45, a calculation unit 46, a switching unit 47, an original image buffer 48, and a reference buffer 49.

  When the switching timing arrives, for example, by receiving a switching signal generated at a predetermined time interval from the signal generation unit, the control unit 45 controls each component constituting the second motion search unit 25 to control the second motion search. The processing target is switched to the next block.

  Further, when the switching timing comes, the control unit 45 outputs a read request for requesting an image of a block to be processed in the original image to the memory controller 14. The read request specifies a read range and a storage address of the read image. As a result, the image of the block to be processed is read from the original image memory 17 and stored in the original image buffer 48.

  Further, when the switching timing comes, the control unit 45 outputs a request for reading a reference image in the search range set based on the determination motion vector input from the determination unit 12. The read request specifies a read range and a storage address of the read image. In the present embodiment, the control unit 45 sets a predetermined size range including the point indicated by the determination motion vector as the search range of the second motion search according to the position and size of the block to be processed. It is configured. For example, a range of a size including ± X pixels in the horizontal direction and ± Y pixels in the vertical direction from the center of the determination motion vector can be set as the search range.

  The position and size of the block to be processed are known in advance by the control unit 45, so that the search range in the second motion search is determined according to the determination motion vector output from the determination unit 12. Become. Note that the search range in the second motion search is a range smaller than the range when the second search range used in the first motion search is returned to the magnification before reduction.

  The control unit 41 of the reference image memory circuit 27 acquires a search range read request output from the control unit 45 and performs read control according to the acquired read request. Specifically, for the reference image stored in the reference image memory 42 among the reference images in the search range requested to be read, the control unit 41 reads the reference image from the reference image memory 42 and stores it in the reference buffer 49. To do. Further, the control unit 41 outputs a read request to the memory controller 14 for reference images that are not stored in the reference image memory 42 among the reference images in the search range requested to be read, and the reference image is output from the frame memory 18. Are directly read out and stored in the reference buffer 49. As a result, the reference image of the search range is stored in the reference buffer 49.

  Since the reference image memory 42 stores reference images in the settable range, if the determination motion vector is the second motion vector, it is included in the search range set based on the determination motion vector. All the reference images are stored in the reference image memory 42. Therefore, the reference image is not acquired directly from the frame memory 18. When the determination motion vector is the first motion vector, a range including a range that is not included in the settable range is set as the search range. Therefore, at least one of the reference images included in the search range corresponding to the determination motion vector. Are not stored in the reference image memory 42. Accordingly, in this case, the frame memory 18 is directly accessed, and a reference image necessary for the reference buffer 49 is acquired and stored.

  Thus, immediately before processing the block to be processed, the reference image in the search range corresponding to the block to be processed is stored in the reference buffer 49.

  Note that the method of storing the reference image in the reference buffer 49 is not limited to this. For example, when at least a part of the reference image of the search range requested to be read is not stored in the reference image memory 42, the control unit 41 directly reads all the reference images requested to be read from the frame memory 18. You may make it store in the reference buffer 49. FIG. That is, in the present embodiment, at least an image that is not stored in the reference image memory 42 among images in the search range set based on the first motion vector is directly from the frame memory 18 without going through the reference image memory 42. The read image is referred to.

  The original image buffer 48 and the reference buffer 49 may be, for example, a FIFO memory (a first-in first-out memory).

  Further, the control unit 45 sequentially generates motion vectors to be evaluated by the calculation unit 46 and held by the holding unit 44, and the calculation unit 46, the holding unit 44, and the decimal vector prediction image generation To the unit 50 and the switching unit 47.

  For example, when searching for a motion vector with decimal pixel precision for each block in the second motion search, the decimal vector prediction image generation unit 50 performs reference image interpolation processing according to a method prescribed by the standard or another method. Then, a decimal prediction image having pixels with decimal coordinates is generated.

  When searching for a motion vector with decimal pixel accuracy in the second motion search, the switching unit 47 inputs the decimal prediction image generated by the decimal vector prediction image generation unit 50 to the calculation unit 46 as a reference image. When searching for a motion vector with integer pixel accuracy, the switching unit 47 inputs an image stored in the reference buffer 49 to the calculation unit 46 as a reference image.

  The calculation unit 46 is selected based on the motion vector transmitted from the control unit 45 from the image of the block to be processed stored in the original image buffer 39 and the reference image of the search range input from the switching unit 47. Block (hereinafter referred to as candidate block). Then, the calculation unit 46 calculates SAD (sum of absolute differences) as an evaluation value of the degree of similarity between each candidate block and the block to be processed. Here, an example in which the calculation unit 46 calculates SAD has been described, but other evaluation values such as SSD (Square Error Sum), SATD, and the like may be calculated. Further, an evaluation value considering the influence of the motion vector may be calculated. Other known techniques may be used.

  The holding unit 44 unconditionally holds the motion vector first sent from the control unit 45 in each block and the evaluation value calculated by the calculation unit 46 based on this motion vector as processing result candidates. Thereafter, each time an evaluation value is sent from the calculation unit 46, the evaluation value of the stored processing result candidate is compared. As a result of the comparison, if the newly sent evaluation value is better than the stored evaluation value, the processing result candidate is updated with the evaluation value and the motion vector corresponding to the evaluation value. By repeating this process for all the motion vectors sent from the control unit 45 and the evaluation values calculated by the calculation unit 46 for the motion vectors, the best evaluation among the motion vector candidates of this block is finally obtained. Since the motion vector indicating the value and the corresponding evaluation value are held as processing result candidates, this candidate is selected as the motion vector and evaluation value in this block.

  The inter-frame prediction unit 26 acquires a reference image from the reference buffer 49 included in the second motion search unit 25 based on the motion vector searched and input by the second motion search unit 25, and performs known motion compensation. Generate a predicted image. That is, in this embodiment, the inter-frame prediction unit 26 acquires the reference image from the reference image memory 42.

  Note that some or all of the reference images used by the inter-frame prediction unit 26 may be acquired by directly accessing the frame memory 18. In motion compensation, unlike the motion search in which each of the candidate blocks of the reference image is referred to, frequent readout is not performed. Therefore, even if the inter-frame prediction unit 26 directly accesses the frame memory 18 for motion compensation, the memory bandwidth The degree of compression is smaller than that of motion search.

  The switching unit 30 of the main encoding processing unit 13 selects one of the prediction image generated by the intra-frame prediction unit 29 and the prediction image generated by the inter-frame prediction unit 26 to select the prediction error image generation unit 20 and the decoded image. The data is output to the generation unit 23. The switching unit 30 is controlled and switched by, for example, an external control unit.

  Information of the prediction error image after quantization is input to the entropy encoding unit 31 from the orthogonal transform / quantization unit 21. Furthermore, the prediction mode determined by the intra-frame prediction mode determination unit 28 and the motion vector searched by the second motion search unit 25 are input to the entropy encoding unit 31 as side information. The entropy encoding unit 31 performs entropy encoding on the prediction error image information and the side information. Entropy coding is a process of assigning and coding a variable-length code according to the appearance frequency of a symbol. An encoded signal (bit stream) generated by entropy encoding is output to a predetermined output destination.

  Next, the determination process of the determination unit 12 will be described in detail.

  FIG. 9 is a flowchart illustrating an example of determination processing executed by the determination unit 12. The determination process shown in FIG. 9 is performed for each frame image.

  In step 100, the determination unit 12 sets the set Sc to be an empty set and sets 0 to the parameter m. Here, the parameter m is an identification number for identifying each block when the 1-frame image is divided into a plurality of blocks that are processing units of the second motion search, and from 0 to “total number of blocks of 1-frame image− It can be an integer value up to “1”.

  In step 102, the determination unit 12 acquires the first motion vector, the first index value, the second motion vector, and the second index value of the block Bm.

  In step 104, the determination unit 12 determines whether or not the first motion vector has higher encoding efficiency than the second motion vector. Here, the first index value and the second index value acquired in step 102 are used. In this embodiment, since both the first index value and the second index value are SAD, the smaller the evaluation index, the higher the encoding efficiency (the better the evaluation). Therefore, when the first index value is smaller than the second index value, it is determined that the first motion vector has higher encoding efficiency than the second motion vector. The same applies to the case where the first index value and the second index value are values indicating SSD and coding cost. If the determination unit 12 makes a positive determination in step 104, the determination unit 12 proceeds to step 106.

  In step 106, the determination unit 12 adds the block Bm to the set Sc. The determination unit 12 proceeds to step 108 after the process of step 106. If the determination unit 12 makes a negative determination in step 104, the determination unit 12 skips step 106 and proceeds to step 108.

  In step 108, the determination unit 12 adds 1 to the parameter m.

  In step 110, the determination unit 12 determines whether or not the parameter m is equal to the total number of blocks of one frame image. As described above, the parameter m can take an integer value from 0 to “the total number of blocks of one frame image−1”. Therefore, when the parameter m becomes equal to the total number of blocks in one frame image, it means that all the blocks included in the one frame image have been processed.

  If the determination unit 12 makes a negative determination in step 110, the determination unit 12 returns to step 102 and performs the same processing as described above for the next block Bm. On the other hand, if the determination unit 12 makes an affirmative determination in step 110, the determination unit 12 proceeds to step 112 and executes a specific block determination process.

  An example of the specific block determination process in step 112 is shown in FIG.

  In step 200, the determination unit 12 sorts the blocks included in the set Sc in descending order of coding efficiency of the first motion vector. Here, sorting is performed based on a value indicating the coding efficiency of the first motion vector.

  The x component of the first motion vector is vx1, the y component is vy1, the first evaluation value is v1, the x component of the second motion vector is vx2, the y component is vy2, the second evaluation value is v2, and the function An example of a function for deriving a value indicating the coding efficiency when the output is f is shown below.

f = v1 Formula 1
f = v1-v2 Expression 2
f = v1−v2 + α · g (vx1, vy1) Equation 3
f = v1−v2 + β · h (vx1, vy1, vx2, vy2) Equation 4

  Here, the function g and the function h in Expression 3 are the memory bandwidth of the main memory 15 that increases when the first motion vector is adopted for the block of interest (specifically, the amount of data directly read from the frame memory 18). This function returns a value proportional to. Α and β are predetermined positive constants.

  Here, the first evaluation value v1 and the second evaluation value v2 are assumed to have higher encoding efficiency (showing good results) as the values are smaller (for example, SAD), and the output f of the function is smaller as the values are smaller. Assume that encoding efficiency is high.

  The determination unit 12 sorts such that the smaller the output value f of the function, the higher the value.

  Here, although the case where the first evaluation value is SAD has been described as an example, SSD or encoding cost may be used. In this case as well, the SSD or encoding cost may be sorted as it is as a value indicating the encoding efficiency.

  In step 202, the determination unit 12 determines the top N blocks as specific blocks in the sorted state. N is a value set in advance so that the total value of the directly read data amount when the reference image memory 42 reads the reference image directly from the frame memory 18 and stores it in the reference buffer 49 does not exceed a preset upper limit value. It is. Here, for one specific block, “c × N = predetermined upper limit DN”, where c is the amount of direct read data when the reference image is directly read from the frame memory 18 without using the reference image memory 42. N was set so that As a result, the amount of direct read data by access to the frame memory 18 for obtaining the search range not held in the reference image memory 42 other than access to the frame memory 18 at the time of updating the reference image memory 42 is limited to the upper limit. It becomes possible to suppress it below the value DN.

  Depending on the updating method of the reference image memory 42, the search range set based on the first motion vector when determined as a specific block may include a range not held in the reference image memory 42. high. Therefore, if too many blocks are determined as specific blocks, the amount of read data directly increases and the memory bandwidth is compressed. Therefore, in this embodiment, an upper limit value for the number of specific blocks is provided.

  Hereinafter, reading the reference image directly from the frame memory 18 without passing through the reference image memory 42 and storing it in the reference buffer 49 is referred to as “direct reading”.

  As described above, a block in which the first motion vector has higher encoding efficiency than the second motion vector is determined as a specific block within a range where the total value of the directly read data amount does not exceed a preset upper limit value. Is done. When the specific block determination process exemplified with reference to FIG. 10 is adopted, the specific block can be determined with a simple algorithm.

  In FIG. 9, after step 112, the process of step 114 is executed.

  In step 114, the determination unit 12 determines all blocks other than the block determined as the specific block in step 112 among the blocks included in the one-frame image as non-specific blocks. Here, each of a block that is not determined as a specific block among blocks included in the set Sc and a block that is not included in the set Sc are determined as non-specific blocks.

  In step 116, the determination unit 12 outputs the determination result to the second motion search unit 25. That is, the determination unit 12 outputs, as a determination result, the first motion vector for the block determined to be a specific block to the main encoding processing unit 13 and the second motion vector for the block determined to be a non-specific block. The result is output to the main encoding processing unit 13.

  Here, FIG. 2 shows an example of a timing chart showing processing timing of each component of the encoding device 1 corresponding to H.264.

  In FIG. 11, the reduction unit 10 performs a reduction process on the original image of each frame image at the timing shown in the figure. In addition, the code | symbol I0, B1, B2, P3 ... shown in figure is an identification code showing each frame image. In FIG. 11, Ix represents a frame image to be encoded by the intra-frame predictive encoding method. In FIG. 11, Bx is a frame image encoded mainly by the inter-frame predictive encoding method, and mainly indicates a frame image of a B frame predicted from the two previously encoded frame images. In FIG. 11, Px is a frame image encoded mainly by the inter-frame predictive encoding method, and mainly indicates a frame image of a P frame predicted from one previously encoded frame image.

  Further, here, the illustration of the frame image encoded before the frame image I0 is omitted.

  The first motion search unit 11 performs a first motion search for each block of each frame image at the timing shown in FIG. As for the B frame, as shown in the forward reference image and the backward reference image, two reference images at different times are referred to. As for the P frame, as shown in the forward reference image, one reference image is referred to. For example, each block of the frame image P3 is subjected to the first motion search described above using the block included in the frame image I0 as a reference image.

  As indicated by t1 in the timing chart of FIG. 11, the determination unit 12 performs a specific block and a non-specific block for each block of the frame image after the first motion search for a certain frame image (P3 in t1) is completed. Judgment is made.

  The second motion search unit 25 performs a second motion search for each block of the frame image after the determination by the determination unit 12 is completed. As indicated by t2 in the timing chart of FIG. 11, the second motion search for the frame image P3 is performed after the determination by the determination unit 12 is completed. Note that, although depending on the determination method of the determination unit 12, the second motion search unit 25 normally performs the second motion search of the frame image after the determination of all the blocks of the one frame image is completed.

  The reference image memory 42 is updated every time a block to be processed is switched, but the reference image memory 42 is controlled so that a reference image of a frame image to be referred to during processing of the block to be processed is stored. Therefore, the reference image stored in the reference image memory 42 when encoding each block of the frame image P3 is a decoded image of the frame image I0.

  As described above, in this embodiment, a two-step motion search is performed, and in the search process in the non-specific block of the second-step motion search, a range narrower than the first search range of the first step is the search range. Is done. Therefore, the memory capacity of the reference image memory 42 can be reduced. Furthermore, since the second motion search using the first motion vector is performed for N blocks in which the first motion vector has higher encoding efficiency than the second motion vector, the encoding efficiency is improved. be able to. That is, the image quality is expected to be improved as compared with the case where no direct reading is performed from the frame memory 18. Further, N is a value set in advance so that the total value of the direct read data amount at the time of direct reading does not exceed the upper limit value, so that it is possible to suppress the read data amount from the frame memory 18. . Thus, in the above embodiment, each of the suppression of the capacity of the reference image memory 42, the improvement of the encoding efficiency, and the suppression of the read data amount of the frame memory 18 can be realized in a balanced manner.

  In addition, the content of the specific block determination process performed in the said step 112 is not limited to the content demonstrated above. For example, determination may be made by generating a frequency distribution or a histogram.

  FIG. 12 is a flowchart showing the flow of specific block determination processing using a histogram. The specific block determination process shown in FIG. 12 can be executed instead of the specific block determination process illustrated with reference to FIG.

  In step 210, the determination unit 12 generates a histogram of values indicating the coding efficiency of the first motion vector for the blocks included in the set Sc. The histogram includes a plurality of classes, but the class width is not particularly limited. As the value indicating the coding efficiency, the values of the functions exemplified in the above formulas 1 to 4 can be used. In step 210, for example, a histogram indicating the frequency (number) of blocks is generated for each class based on a value indicating the coding efficiency.

  In step 212, the determination unit 12 sets 0 to the parameter Dt and 0 to the parameter k. The parameter Dt is a parameter for measuring the cumulative value when the frequency of each class is added from the upper class in descending order of the value indicating the coding efficiency. The parameter k is an identification number for identifying the class. The parameter k can take a value from “0 to the total number of classes−1”. The lower the value indicating the encoding efficiency (higher encoding efficiency), the higher the class, and the smaller k.

  In step 214, the determination unit 12 acquires the frequency Dk of the class k of the histogram.

  In step 216, the determination unit 12 sets a value obtained by adding Dk to Dt to Dt.

  In step 218, the determination unit 12 determines whether or not Dt is equal to or less than a predetermined upper limit value N. As described above, N is a value set in advance so that the total value of the directly read data amount does not exceed a preset upper limit value in direct reading.

  If the determination unit 12 makes a positive determination in step 218, the process proceeds to step 220, 1 is added to the parameter k, and the process returns to step 214. As a result, the same processing as described above is performed for the next class k.

  If the determination unit 12 makes a negative determination in step 218, the determination unit 12 proceeds to step 222 and determines a block included in a class higher than the class k as a specific block.

  That is, when a negative determination is made in step 218, the parameter Dt indicating the cumulative value when the frequency Dk of the current class k is added exceeds the upper limit value N, but the class one higher than the class k. It indicates that the cumulative value up to k−1 is equal to or less than the upper limit value N.

  Accordingly, here, the blocks included in the classes higher than the class k, that is, the classes from the class 0 to the class k−1 are determined as the specific blocks.

  Although an example of generating a histogram has been described here, a frequency distribution may be generated and processed in the same manner as described above without generating a histogram. This is because if the width of the class is defined and the frequency for each class is counted, processing can be performed in the same manner as described above. As described above, when determining a specific block to be performed by generating a frequency distribution or a histogram, it is not necessary to perform the sort necessary for the specific block determination process described with reference to FIG. Can be done.

  In the specific block determination process, the specific block may be determined as follows. For example, as shown in FIG. 13, for the block Ba in which the adjacent blocks B1 and B2 are determined to be specific blocks, the reference image included in the search range of the blocks B1 and B2 is stored in the reference buffer 49 even when the block Ba is processed. It can be retained and reused. That is, depending on the search range when performing the second motion search for the block Ba, the search using the first motion vector can be performed without performing direct reading for the second motion search of the block Ba. In this case, even if the block Ba is treated as a specific block, the memory bandwidth is not compressed. Therefore, when the first motion vector of the block Ba determined to be a specific block on both sides has higher encoding efficiency than the second motion vector, the block Ba is also directly dependent on the search range based on the first motion vector for the block Ba. You may determine with the specific block which can be read. Compared with the case where the block Ba is not determined as a specific block, the coding efficiency is improved and the image quality is expected to be improved.

  FIG. 14 is a flowchart showing the flow of specific block determination processing for determining whether or not a block for which both adjacent blocks are determined to be specific blocks is also determined as a specific block. The specific block determination process shown in FIG. 14 can be performed instead of the specific block determination process illustrated with reference to FIG.

  In step 230, the determination unit 12 sets the set Sb as an empty set and sets 0 to the parameter n.

  In step 232, the determination unit 12 selects a block Bb having the highest coding efficiency from the set obtained by removing the set Sb from the set Sc. Here, the function f that derives a value indicating the coding efficiency exemplified by the above formulas 1 to 4 is used, and the block having the smallest value of f is selected. Note that the set Sc is a set of blocks determined in step 104 in FIG. 9 that the first motion vector is higher in encoding efficiency than the second motion vector.

  In step 234, the determination unit 12 determines whether or not n + 1> N. As described above, N is a value set in advance so that the total value of the directly read data amount at the time of direct reading does not exceed the upper limit value.

  When the determination unit 12 makes a negative determination in step 234, the process proceeds to step 236, and in step 236, the block Bb is added to the set Sb, and 1 is added to n.

  In step 238, the determination unit 12 determines whether or not the set Sc and the set Sb are equal. If the determination unit 12 makes a negative determination in step 238, the determination unit 12 proceeds to step 240.

  In step 240, the determination unit 12 determines that the block Ba is a specific block in the processing direction of the second motion search (that is, both adjacent blocks are included in the set Sb). Judge whether there is.

  If the determination unit 12 makes an affirmative determination in step 240, the determination unit 12 proceeds to step 242 and derives a synthesis range r for the block Ba. Here, the synthesis range r refers to a synthesis range obtained by synthesizing the following three ranges when the specific blocks on both sides of the block Ba are the block B1 and the block B2.

A search range in the second motion search of block B1 A search range in the second motion search of block B2 A range held in the reference image memory 42 during the second motion search of block Ba

  An example of the synthesis range r is shown in FIG. In FIG. 13, reference numeral 300 denotes a holding range stored in the reference image memory 42 during the second motion search process of the block Ba. Reference numeral 302 denotes a search range of the second motion search of the block B1. Reference numeral 304 denotes a search range of the second motion search of the block B2. Reference numeral 306 denotes a search range of the second motion search of the block Ba. Reference numeral 308 denotes a synthesis range r. Each of the search ranges 302, 304, and 306 is a search range set by the first motion vector.

  If there are a plurality of blocks whose specific blocks are on both sides, one of them is extracted, and a synthesis range r for the extracted block Ba is derived.

  In step 244, the determination unit 12 determines whether or not the synthesis range r includes a search range set based on the first motion vector when the block Ba is set as a second motion search processing target. .

  If the determination unit 12 makes an affirmative determination in step 244, in step 246, the block Ba is added to the set Sb (that is, the block Ba is added to the set of specific blocks), and the process returns to step 238. If the determination unit 12 makes a negative determination in step 238, the process proceeds to step 240, where blocks that have not yet been extracted from among the blocks whose both neighbors are specific blocks are extracted, and the processes after step 242 are repeated.

  On the other hand, if the determination unit 12 makes a negative determination in step 244, the process skips step 246 and returns to step 240, extracts blocks that have not been extracted from among the blocks whose both neighbors are specific blocks, and The processing after step 242 is repeated.

  If the determination unit 12 makes a negative determination in step 240, the determination unit 12 returns to step 232 and repeats the same processing as described above.

  If the determination unit 12 makes an affirmative determination in step 234, the process proceeds to step 248 without adding the block Bb to the set Sb.

  In step 248, the determination unit 12 determines a block included in the set Sb as a specific block.

  If the determination unit 12 makes a positive determination in step 238, it indicates that all blocks included in the set Sc have been added to the set Sb, which is a set of specific blocks. Therefore, the determination unit 12 proceeds to step 248, and determines the blocks included in the set Sb as specific blocks.

  Hereinafter, the flow of the specific block determination process described with reference to FIG. 14 will be described with reference to the schematic diagram of FIG. 15 and specific examples of the blocks included in the set Sc and the set Sb. Here, a case where N is 3 is taken as an example. Further, the processing from step 230 to step 246 prior to step 248 is referred to as “set operation processing” for determining a specific block. n is 2.

  As shown in FIG. 15A, the set Sc includes four blocks w, x, y, and z. In step 230, which is the first step of the specific block determination process, the set Sb is set as an empty set, and n is set to 0. The value of the function f of the block w is 100, the value of the function f of the block x is 1000, the value of the function f of the block y is 101, and the value of the function f of the block z is 200. Here, FIG. 16 shows an arrangement example of four blocks w, x, y, and z. As described above, the second motion search is performed from left to right, and after the motion search of the rightmost block is performed, the motion search of the leftmost block in the next lower row is performed. Note that the arrow extending from the upper left end of each block shown in FIG. 16 indicates the first motion vector of each block searched in the first motion search.

  As shown in FIG. 15 (2), in step 232, the block Bb is selected. As shown in FIG. 15 (2), since the block having the smallest value of f in the set excluding the set Sb from the set Sc is the block w, the block w is selected as the block Bb. At this stage, n is 0.

  As shown in FIG. 15 (3), in the next step 234, since n + 1> N is not satisfied, the set operation processing is continued.

  As shown in FIG. 15 (4), in step 235, the block Bb is added to the set Sb. Also, 1 is added to n, and n becomes 1.

  As shown in FIG. 15 (5), in step 238, the set Sc and the set Sb are not equal, so the set operation processing is continued. In step 240, it is determined that there is no block corresponding to the block Ba.

  Accordingly, as shown in FIG. 15 (6), since a negative determination is made in step 240, the process returns to step 232.

  As shown in FIG. 15 (7), in step 232 after FIG. 15 (6), the block Bb is selected. As shown in FIG. 15 (6), since the block having the smallest value of f in the set obtained by removing the set Sb from the set Sc is the block y, the block y is selected as the block Bb. At this stage, n is 1.

  As shown in FIG. 15 (8), in the next step 234, since n + 1> N is not satisfied, the set operation processing is continued.

  As shown in FIG. 15 (9), in step 235, the block Bb is added to the set Sb. Also, 1 is added to n, and n becomes 2.

  As shown in FIG. 15 (10), in step 238, the set operation is continued because the set Sc and the set Sb are not equal. In step 240, it is determined that there is a block corresponding to the block Ba. In the example shown in FIGS. 15 and 16, the block x is determined and selected as the block Ba. Thereafter, a synthesis range r is derived for the block Ba. For example, when the synthesis range r is derived as shown in FIG. 13 and the search range based on the first motion vector of the block Ba becomes the range shown in FIG. 13, in step 244, the search range of the block Ba is synthesized. It is determined to be included in the range r. Accordingly, in step 246, the block Ba is added to the set Sb.

  As shown in FIG. 15 (11), since there is no block corresponding to another Ba in the next step 240, a negative determination is made and the process returns to step 232.

  As shown in FIG. 15 (12), in step 232, the block Bb is selected. As shown in FIG. 15 (12), since the block having the smallest value of f in the set obtained by excluding the set Sb from the set Sc is the block z, the block z is selected as the block Bb. At this stage, n is 2.

  As shown in FIG. 15 (13), in the next step 234, since n + 1> N is satisfied, the set operation processing ends, and the processing proceeds to step 248.

  In the specific block determination process described with reference to FIGS. 13 to 15 above, every time the block Bb is added to the set Sb, it is determined whether or not the block Ba exists. If the block Ba exists, the block Ba is added to the set Sb.

  Incidentally, as described above, the motion search for one block includes a large number of SAD calculation processes by the calculation unit 38. The original image used in this SAD calculation process is not acquired from the frame memory 18 every time one SAD is calculated, but from the original image buffer 39 provided in the second motion search unit 25 to the calculation unit 38. Is output. Similarly, for a reference image, a reference image used in motion search for one block is stored in the reference buffer 49 in advance, and motion search for the block is started.

  Therefore, the content of the reference buffer 49 is updated after the motion search for a certain block is completed and before the motion search for the next block is started.

  In the configuration using the reference buffer 49 in this way, in the second motion search of the block Ba in which both neighbors are specified blocks, the function for diverting (referring to) the reference image in the search range of the neighbor blocks is the second motion search. The search unit 25 can be provided. Hereinafter, this function is referred to as “reference picture cache function” for convenience.

  In the reference image cache function, the reference image of the range referred to by the block previously processed as the second motion search is held in the reference buffer 49, and the range ( Is used as a reference image. With the reference image cache function, the reference image in the search range can be referred to by using the reference image held in the reference buffer 49 without acquiring the reference image from the frame memory 18.

  As described above, the control unit 45 of the second motion search unit 25 outputs a read request to the control unit 41 of the reference image memory circuit 27 at the timing of switching the block to be processed for the second motion search. The control unit 45 outputs the read request including the range to be read and the designation of the storage destination address for the reference buffer 49, controls the timing for storing and erasing data in the reference buffer 49, and controls the reference picture cache function. Realize.

  Here, the reference picture cache function performed by the control unit 45 of the second motion search unit 25 will be described with reference to FIGS. 17A and 17B. Here, a description will be given by taking, as an example, three consecutive blocks Bw, Bx, and By that are processing targets of the second motion search.

  In FIG. 17A, the search range of the block Bw includes a region a and a region b. The search range of the block By includes a region d and a region e. The search range of the block Bx includes a region c, a region b, and a region d. That is, the region b is a portion where the search range of the block Bw and the search range of the block Bx overlap, and the region d is a portion where the search range of the block By and the search range of the block Bx overlap.

  Hereinafter, in order to simplify the description, it is assumed that all of the search ranges shown in FIG. 17A are ranges that are not stored in the reference image memory 42 (a range that requires direct reading). .

  FIG. 17B is a diagram schematically showing an area held in the reference buffer 49 for each processing period of each block. (1) shows the case where the reference picture cache function is executed, and (2) shows the case where the reference picture cache function is not executed.

  First, (2) in which the reference image cache function is not executed will be described. First, the search range (region a and region b) of the block Bw is held in the reference buffer 49 before the second motion search of the block Bw (before t1). The original picture buffer 48 is also updated.

  After the second motion search of the block Bw is finished and before the second motion search of the block Bx (that is, at t2), the region a and the region b are deleted from the reference buffer 49, and the region b is again stored in the frame memory 18 And stored in the reference buffer 49. Thereby, the search range (region b, region c, region d) of the block Bx is held in the reference buffer 49 before the second motion search of the block Bx. The original picture buffer 48 is also updated.

  After the second motion search of the block Bx is finished and before the second motion search of the block By (ie, at t4), the region b, the region c, and the region d are deleted from the reference buffer 49, and the region d is It is read again from the frame memory 18 and stored in the reference buffer 49. Thereby, the search range (region d and region e) of the block By is held in the reference buffer 49 before the second motion search of the block By. The original picture buffer 48 is also updated.

  In the configuration illustrated in (2) of FIG. 17B, the amount of access to the frame memory 18 increases. Therefore, for example, it can be employed when a specific block is determined by the specific block determination process described with reference to FIGS. 10 and 12.

  Next, (1) for executing the reference image cache function will be described. First, as in the case of (2), the search range (region a and region b) of the block Bw is held in the reference buffer 49 before the second motion search of the block Bw (before t1). The original picture buffer 48 is also updated.

  After the second motion search of the block Bw ends, the original picture buffer 48 is updated before the second motion search of the block Bx (that is, at t2). Further, the area a that is not referred to in the processing of the block Bx is erased from the reference buffer 49. Furthermore, since the area b is also referred to in the processing of the block Bx, it is retained without being deleted from the reference buffer 49. Therefore, in the case of (1), it is not necessary to read the region b from the frame memory 18 again and store it in the reference buffer 49 before searching for the block Bx.

  After the second motion search of the block Bx ends, the original buffer 48 is updated before the second motion search of the block By (that is, at t4). In addition, the area b and the area c that are not referred to in the processing of the block By are deleted from the reference buffer 49. Furthermore, since the area d is also referred to in the processing of the block By, it is held as it is without being deleted from the reference buffer 49. Therefore, in the case of (1), it is not necessary to read the region d from the frame memory 18 again and store it in the reference buffer 49 before searching for the block By.

  That is, the control unit 45 of the second motion search unit 25 uses the next block of the reference image used in the second motion search of the next block for the region used in the second motion search of the next block. Control is performed so as to be held in the reference buffer 49 until the second motion search is completed. In the second motion search of the next block, the area held in the reference buffer 49 is reused so that the reuse area is not read from the frame memory 18.

  In the configuration illustrated in (2) of FIG. 17B, the access amount to the frame memory 18 is suppressed. Therefore, it can be adopted when a range read in an adjacent block is used as a reference image. The configuration illustrated in (2) can be employed even when a specific block is determined by the specific block determination process described with reference to FIGS. 10 and 12.

  In the above example, the original image buffer 48 and the reference buffer 49 are configured as a single buffer, and the motion search of each block and the update of the original image buffer 48 and the reference buffer 49 are alternately described as an example. It is not limited to this. For example, each of the original image buffer 48 and the reference buffer 49 may have a double buffer configuration, and may be configured to perform processing in a pipeline manner as shown in FIG.

  In each example described above, the value of N is set assuming that the amount of directly read data when the reference image in the search range is directly read from the frame memory 18 and stored in the reference buffer 49 is uniform, Although the example which determines a specific block was demonstrated, it is not limited to this. For example, the direct read data amount when the reference image in the direct search range is read from the frame memory 18 is derived for each block, and the total value of the derived direct read data amounts exceeds the upper limit DN of the direct read data amount. You may make it determine a specific block within the range which does not exist.

  Here, FIG. 19 shows a modification in which the specific block determination process exemplified with reference to FIG. 14 is changed to a process in which the direct read data amount is derived and determined for each block.

  The processing contents of step 260 and step 262 are the same as those of step 230 and step 232 of FIG.

  After step 262, in step 264, the determination unit 12 calculates and derives the direct read data amount dn of the block Bb. A method for calculating the direct read data amount dn is not particularly limited. For example, a function using the size of a range in which direct reading is performed as a parameter may be prepared in advance and calculated using the function.

  The above function can be a function whose value increases as the direct reading range increases. Note that the direct reading range may be obtained by excluding the range stored in the reference image memory 42 from the search range set according to the first motion vector of the block Bb.

  In step 266, the determination unit 12 determines whether or not n + dn> DN. DN is an upper limit value of the total value of the direct read data amount at the time of direct reading.

  If a negative determination is made in step 266, the determination unit 12 adds the block Bb to the set Sb in step 268 and adds dn to n. If the determination in step 266 is affirmative, the determination unit 12 proceeds to step 280. Note that the processing from step 270 to step 280 is the same as the processing from step 236 to step 248 described with reference to FIG.

  In this way, the total value of the direct read data amount is not calculated with the direct read data amount being uniformly c, but the determination of the specific block is advanced while actually calculating for each block, and the amount is suppressed to DN or less. I am doing so.

  When the direct read data amount is uniformly c, even if the actual direct read data amount fluctuates, there is a possibility that the total value of the direct read data amount may be set low so that the total value does not exceed the upper limit value. However, in this case, even if there is a margin in the memory bandwidth, the number of specific blocks is determined to be small. However, if the read data amount is directly calculated for each block, the total value can be calculated more accurately, so that DN can be set to the upper limit value itself or a value close to the upper limit value. Thereby, more blocks can be determined as specific blocks.

  Similarly, in the specific block determination process described with reference to FIG. 10, the direct read data amount may be calculated and determined for each block instead of being uniformly c. Specifically, instead of step 202, the following processing is performed. The determination unit 12 directly calculates and accumulates the read data amount in order from the block with the highest coding efficiency. Then, when the cumulative value of the directly read data amount exceeds the upper limit value, the determination unit 12 determines each block from the block with the highest coding efficiency to the block one higher than the block whose cumulative value exceeds the upper limit value. May be determined as a specific block.

  Similarly, in the specific block determination process described with reference to FIG. 12, the direct read data amount may be calculated and determined for each block instead of being uniformly c. For example, instead of the processing from step 214 to step 218 in FIG.

  Instead of the process of step 214, the determination unit 12 calculates the direct read data amount of each block included in the histogram class k and performs a process of calculating the total value Tk of the direct read data amount of the class k. . Next, the determination unit 12 performs a process of adding Tk to Dt instead of the process of step 216. Next, the determination unit 12 determines whether or not Dt ≦ DN, instead of the process of step 218. The determination unit 12 proceeds to step 220 when Dt ≦ DN, and proceeds to step 222 when Dt ≦ DN is not satisfied.

  The same effect as described above can be obtained by the modification of the specific block determination process described above.

  In the above embodiment, the size when the block of the first motion search processing unit (hereinafter referred to as the first block) is returned to the original magnification and the block of the processing unit of the second motion search (hereinafter referred to as the second block). Although the case where the sizes are the same has been described, they may be different. If the size of the second block is larger than the size of the first block before reduction, the following processing may be performed. First, an average value of each of the first motion vector, the first evaluation value, the second motion vector, and the second evaluation value of the plurality of first blocks corresponding to the second block is calculated. The determination unit 12 uses these four average values to determine a specific block. The processing method may be performed as described above. Further, if the size of the second block is smaller than the size of the first block before reduction, the search result of the first motion search of the first block corresponding to the second block may be used as it is.

  Furthermore, as shown in FIG. 20, the function of the determination unit 12 of the encoding device 1 is realized by a computer, and the functions of the first motion search unit 11 and the main encoding processing unit 13 are encoded by a computer. It may be realized by the unit 71.

  The computer 60 includes a CPU (Central Processing Unit) 61, a nonvolatile program storage unit 62 that stores programs, a main memory 15, an image processing unit 64, an external IF (Interface) 66, an image input IF 67, and an encoding unit 71. ing. The main memory 15 is connected to the memory controller 14. The CPU 61, the program storage unit 62, the memory controller 14, the image processing unit 64, the external IF 66, the image input IF 67, and the encoding unit 71 are connected to each other via a bus 70. The memory controller 14 and the main memory 15 shown in FIG. 20 correspond to the memory controller 14 and the main memory 15 described with reference to FIG.

  The program storage unit 62 stores a determination program for executing the determination process of the determination unit 12.

  The CPU 61 reads out the determination program from the program storage unit 62 and develops it in the main memory 15 to execute the determination program process. The function of the determination unit 12 illustrated in FIG. 1 is realized by the CPU 61 executing the determination program process. Note that the memory for developing the program executed by the CPU 61 may be a memory different from the main memory 15.

  The external IF 66 is an interface between the recording medium 68 and the operation panel controller 69. The image input IF 67 is an interface for inputting a moving image or the like.

  The encoding unit 71 realizes the functions of the first motion search unit 11 and the main encoding processing unit 13 by hardware such as ASIC.

  Such a configuration also acts in the same manner as described above.

  In the above description, the determination program is stored in the program storage unit 62. However, the determination program is recorded on a portable recording medium such as a CD-ROM or a flash memory connectable via USB. It is also possible to provide it in the form.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. A first motion search unit for each block that is a processing unit of the first motion search;
A reference image storage unit for storing a reference image partially read from a storage unit for storing a reference frame image referred to in a second motion search performed based on the processing result of the first motion search;
For each block that is a processing unit of the second motion search in the processing target frame image,
A control unit that stores the reference image including the search range of the second motion search set based on the second motion vector in the reference image storage unit before starting the second motion search of each block;
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed the upper limit value, using the first motion vector A determination unit that determines as a specific block that performs the second motion search, and determines a block other than the specific block as a non-specific block that performs the second motion search using the second motion vector;
The block determined as the specific block by the determination unit is set based on the first motion vector with reference to an image in a search range set based on the first motion vector in the reference frame image. When referring to at least an image that is not stored in the reference image storage unit among images in the search range, a motion vector is searched by referring to an image directly read from the storage unit without going through the reference image storage unit. The second motion search is performed, and the block determined as the non-specific block by the determination unit is set based on the second motion vector among the reference images stored in the reference image storage unit. A second motion search unit for performing the second motion search for searching for a motion vector with reference to an image of the search range to be performed;
An encoding device comprising:

(Appendix 2)
The determination unit selects a block having a higher encoding efficiency for the first motion vector than the second motion vector among the plurality of blocks, and selects the first motion vector of the selected blocks. The top N blocks (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) are determined as the specific block in descending order of coding efficiency, or Among the selected blocks, the code of the first motion vector is within a range where the cumulative value when the direct read data amount is accumulated in order from the highest coding efficiency of the first motion vector does not exceed the upper limit value. The encoding device according to appendix 1, wherein the specific blocks are determined in order from the block having the highest conversion efficiency.

(Appendix 3)
The determination unit selects a block in which the first motion vector has higher encoding efficiency than the second motion vector among the plurality of blocks, and the first motion vector of each block is selected for the selected block. A frequency distribution or histogram of values indicating the coding efficiency of the generated frequency distribution or histogram is generated, the frequencies are added from the higher rank in order of the coding efficiency indicated by the value of the generated frequency distribution or histogram, the cumulative value of the frequency is Whether the block included in each of the classes higher than the class exceeding N (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) is determined as the specific block Alternatively, the direct reading data amount of each block is added from the upper class in descending order of the coding efficiency indicated by the value, and the cumulative value exceeds the upper limit value. Encoding apparatus according to note 1 determines blocks included in each class of higher as the specific block.

(Appendix 4)
The determination unit
A block having a higher encoding efficiency of the first motion vector than the second motion vector is extracted from blocks excluding the block that has already been determined to be the specific block among the plurality of blocks, and the extracted block A first determination process for determining a block having the highest coding efficiency of the first motion vector as the specific block,
Each time the first determination process is executed once, the blocks whose both neighbors are the specific blocks are selected, and each of the selected blocks is the second motion search processing target. A third search range set based on one motion vector, each of the search ranges set based on the first motion vector when the adjacent blocks on both sides are set as the second motion search processing target; Determining whether or not the neighbor is included in a synthesis range with a range stored in the reference image storage unit during the second motion search process of the block determined as the specific block; A second determination process for determining, as the specific block, a block whose range is determined to be included in the synthesis range,
Repeatedly until N specific blocks are determined in the first determination process (N is an integer set in advance so that the total value of the directly read data amounts does not exceed the upper limit value), or The encoding apparatus according to claim 1, wherein the encoding is performed repeatedly within a range in which a total value of the direct read data amounts of the blocks determined as the specific block in the first determination process does not exceed the upper limit value.

(Appendix 5)
The control unit, when the block to be processed in the second motion search is switched to the next block, a reference image that may not be set as a search range when the next block is the non-specific block Is deleted from the reference image storage unit, and a reference image that is not yet stored in the reference image storage unit may be set as a search range when the next block is the non-specific block. The encoding apparatus according to any one of Supplementary Note 1 to Supplementary Note 4, wherein control is performed to read out from the storage unit and store the read-out in the reference image storage unit.

(Appendix 6)
The control unit is set as a search range when the block to be processed in the second motion search is switched to the next block, and each block to be processed after the switching is set as the non-specific block. The reference image that is not likely to be deleted is erased from the reference image storage unit and may be set as a search range when the next block is the non-specific block, but is still stored in the reference image storage unit The encoding apparatus according to claim 1, wherein a reference image that has not been read is read from the storage unit and stored in the reference image storage unit.

(Appendix 7)
A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. It is performed for each block that is a processing unit of the first motion search,
The reference image storage unit that stores the reference image partially read from the storage unit that stores the reference frame image referred to in the second motion search performed based on the processing result of the first motion search includes the process. The reference image including the search range of the second motion search that is set based on the second motion vector for each block that is a processing unit of the second motion search in the target frame image, Store in the reference image storage unit before the start of the second motion search,
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed the upper limit value, using the first motion vector Determining as a specific block for performing the second motion search, determining a block other than the specific block as a non-specific block for performing the second motion search using the second motion vector;
For the block determined to be the specific block, the image of the search range set based on the first motion vector is referenced with reference to the image of the search range set based on the first motion vector in the reference frame image. When referring to at least an image not stored in the reference image storage unit among the images, the second search is made for a motion vector by referring to an image directly read from the storage unit without going through the reference image storage unit. For a block determined as a non-specific block by performing a motion search, refer to an image in a search range set based on the second motion vector among the reference images stored in the reference image storage unit. And performing the second motion search for searching for a motion vector.
Encoding method.

(Appendix 8)
In the determination of the specific block, a block in which the first motion vector has higher encoding efficiency than the second motion vector is selected from the plurality of blocks, and the first motion is selected from the selected blocks. The top N blocks (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit) are determined as the specific blocks in descending order of vector encoding efficiency, or In the selected block, the first motion vector is within a range in which the cumulative value when the direct read data amount is accumulated in order from the highest coding efficiency of the first motion vector does not exceed the upper limit value. The encoding method according to appendix 7, wherein the specific block is determined in order from the block having the highest encoding efficiency.

(Appendix 9)
In the determination of the specific block, a block in which the first motion vector has a higher encoding efficiency than the second motion vector is selected from the plurality of blocks, and the first block of each block is selected for the selected block. Generate a frequency distribution or histogram of values indicating the coding efficiency of the motion vector, add the frequencies from the upper rank in descending order of the coding efficiency indicated by the value of the generated frequency distribution or histogram, and accumulate the frequencies A block included in each of the classes higher than the class whose value exceeds N (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) is determined as the specific block. Or the direct read data amount of each block is added from the upper class in descending order of the coding efficiency indicated by the value, and the cumulative value is Encoding method according to determining Appendix 7 blocks contained from class beyond a value to each of the class of higher as the specific block.

(Appendix 10)
In the determination of the specific block, out of the plurality of blocks excluding blocks that have already been determined as the specific block, the first motion vector has a higher encoding efficiency than the second motion vector. A first determination process of extracting and determining a block having the highest coding efficiency of the first motion vector among the extracted blocks as the specific block;
Each time the first determination process is executed once, the blocks whose both neighbors are the specific blocks are selected, and each of the selected blocks is the second motion search processing target. A third search range set based on one motion vector, each of the search ranges set based on the first motion vector when the adjacent blocks on both sides are set as the second motion search processing target; Determining whether or not the neighbor is included in a synthesis range with a range stored in the reference image storage unit during the second motion search process of the block determined as the specific block; A second determination process for determining, as the specific block, a block whose range is determined to be included in the synthesis range,
Repeatedly until N specific blocks are determined in the first determination process (N is an integer set in advance so that the total value of the directly read data amounts does not exceed the upper limit value), or The encoding method according to claim 7, wherein the encoding is performed repeatedly within a range in which a total value of the direct read data amounts of the blocks determined as the specific block in the first determination process does not exceed the upper limit value.

(Appendix 11)
A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. A first motion search unit that is performed for each block that is a processing unit of the first motion search, and a storage unit that stores a reference frame image that is referred to in the second motion search performed based on the processing result of the first motion search A reference image storage unit that stores a reference image partially read out from each of the blocks to be a processing unit of the second motion search in the processing target frame image,
A control unit that stores the reference image including the search range of the second motion search set based on the second motion vector in the reference image storage unit before starting the second motion search of each block; and For a specific block for performing the second motion search using the first motion vector, refer to an image in a search range set based on the first motion vector in the reference frame image, and the first motion vector When referring to at least an image that is not stored in the reference image storage unit among images in the search range set based on the image, the image directly read from the storage unit is referred to without going through the reference image storage unit. For the non-specific block that performs the second motion search for searching for a motion vector and performs the second motion search using the second motion vector, the reference image A second motion search unit for performing the second motion search for searching for a motion vector with reference to an image in a search range set based on the second motion vector among the reference images stored in a storage unit; A computer for obtaining a processing result of the first motion search unit of the apparatus comprising:
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has a higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed an upper limit value, A program for recording a program or a program for executing a process of determining a block other than a block as the non-specific block and supplying the determination result to the second motion search unit.

(Appendix 12)
In the processing, a block in which the first motion vector has higher encoding efficiency than the second motion vector is selected from the plurality of blocks, and the code of the first motion vector is selected from the selected blocks. The top N blocks (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) are determined as the specific blocks in descending order of the conversion efficiency, or the selection The first motion vector is encoded within a range in which the cumulative value when the amount of the directly read data is accumulated in order from the block having the highest coding efficiency of the first motion vector is within the range not exceeding the upper limit value. The program according to appendix 7, or a recording medium on which the program is recorded, is determined as the specific block in order from the block having the highest efficiency.

(Appendix 13)
In the processing, a block in which the first motion vector has higher encoding efficiency than the second motion vector is selected from the plurality of blocks, and the first motion vector of each block is selected for the selected block. A frequency distribution or histogram of values indicating the coding efficiency is generated, and the frequencies are added from the higher rank in order from the highest coding efficiency indicated by the value of the generated frequency distribution or histogram. (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value). Alternatively, the direct read data amount of each block is added from the upper class in the descending order of the coding efficiency indicated by the value, and the cumulative value exceeds the upper limit. Recording medium for recording a description of a program or programs to determine note 7 blocks included in each of the classes of the upper as the specific block.

(Appendix 14)
In the process,
A block having a higher encoding efficiency of the first motion vector than the second motion vector is extracted from blocks excluding the block that has already been determined to be the specific block among the plurality of blocks, and the extracted block A first determination process for determining a block having the highest coding efficiency of the first motion vector as the specific block,
Each time the first determination process is executed once, the blocks whose both neighbors are the specific blocks are selected, and each of the selected blocks is the second motion search processing target. A third search range set based on one motion vector, each of the search ranges set based on the first motion vector when the adjacent blocks on both sides are set as the second motion search processing target; Determining whether or not the neighbor is included in a synthesis range with a range stored in the reference image storage unit during the second motion search process of the block determined as the specific block; A second determination process for determining, as the specific block, a block whose range is determined to be included in the synthesis range,
Repeatedly until N specific blocks are determined in the first determination process (N is an integer set in advance so that the total value of the directly read data amounts does not exceed the upper limit value), or The program according to claim 7 or a recording medium on which the program is recorded is repeatedly performed within a range in which a total value of the direct read data amounts of the blocks determined as the specific block in the first determination process does not exceed the upper limit value.

DESCRIPTION OF SYMBOLS 1 Encoding apparatus 10 Reduction part 11 1st motion search part 12 Judgment part 13 Main encoding process part 14 Memory controller 15 Main memory 16 Reduced picture memory 17 Original picture memory 18 Frame memory 25 2nd motion search part 26 Inter frame prediction part 27 Reference image memory circuit 31 Entropy encoding unit 41 Control unit 42 Reference image memory 60 Computer 61 CPU
62 Program storage unit 71 Coding unit

Claims (8)

A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. A first motion search unit for each block that is a processing unit of the first motion search;
A reference image storage unit for storing a reference image partially read from a storage unit for storing a reference frame image referred to in a second motion search performed based on the processing result of the first motion search;
For each block that is a processing unit of the second motion search in the processing target frame image,
A control unit that stores the reference image including the search range of the second motion search set based on the second motion vector in the reference image storage unit before starting the second motion search of each block;
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed the upper limit value, using the first motion vector A determination unit that determines as a specific block that performs the second motion search, and determines a block other than the specific block as a non-specific block that performs the second motion search using the second motion vector;
The block determined as the specific block by the determination unit is set based on the first motion vector with reference to an image in a search range set based on the first motion vector in the reference frame image. When referring to at least an image that is not stored in the reference image storage unit among images in the search range, a motion vector is searched by referring to an image directly read from the storage unit without going through the reference image storage unit. The second motion search is performed, and the block determined as the non-specific block by the determination unit is set based on the second motion vector among the reference images stored in the reference image storage unit. A second motion search unit for performing the second motion search for searching for a motion vector with reference to an image of the search range to be performed;
An encoding device comprising: The determination unit selects a block having a higher encoding efficiency for the first motion vector than the second motion vector among the plurality of blocks, and selects the first motion vector of the selected blocks. The top N blocks (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) are determined as the specific block in descending order of coding efficiency, or Among the selected blocks, the code of the first motion vector is within a range where the cumulative value when the direct read data amount is accumulated in order from the highest coding efficiency of the first motion vector does not exceed the upper limit value. The encoding apparatus according to claim 1, wherein the specific blocks are determined in order from the block having the highest conversion efficiency. The determination unit selects a block in which the first motion vector has higher encoding efficiency than the second motion vector among the plurality of blocks, and the first motion vector of each block is selected for the selected block. A frequency distribution or histogram of values indicating the coding efficiency of the generated frequency distribution or histogram is generated, the frequencies are added from the higher rank in order of the coding efficiency indicated by the value of the generated frequency distribution or histogram, the cumulative value of the frequency is Whether the block included in each of the classes higher than the class exceeding N (N is an integer set in advance so that the total value of the directly read data amount does not exceed the upper limit value) is determined as the specific block Alternatively, the direct reading data amount of each block is added from the upper class in descending order of the coding efficiency indicated by the value, and the cumulative value exceeds the upper limit value. The encoding apparatus according to claim 1 determines blocks included in each class of higher as the specific block. The determination unit
A block having a higher encoding efficiency of the first motion vector than the second motion vector is extracted from blocks excluding the block that has already been determined to be the specific block among the plurality of blocks, and the extracted block A first determination process for determining a block having the highest coding efficiency of the first motion vector as the specific block,
Each time the first determination process is executed once, the blocks whose both neighbors are the specific blocks are selected, and each of the selected blocks is the second motion search processing target. A third search range set based on one motion vector, each of the search ranges set based on the first motion vector when the adjacent blocks on both sides are set as the second motion search processing target; Determining whether or not the neighbor is included in a synthesis range with a range stored in the reference image storage unit during the second motion search process of the block determined as the specific block; A second determination process for determining, as the specific block, a block whose range is determined to be included in the synthesis range,
Repeatedly until N specific blocks are determined in the first determination process (N is an integer set in advance so that the total value of the directly read data amounts does not exceed the upper limit value), or The encoding apparatus according to claim 1, wherein the encoding is performed repeatedly within a range in which a total value of the direct read data amounts of the blocks determined as the specific block in the first determination process does not exceed the upper limit value. The control unit, when the block to be processed in the second motion search is switched to the next block, a reference image that may not be set as a search range when the next block is the non-specific block Is deleted from the reference image storage unit, and a reference image that is not yet stored in the reference image storage unit may be set as a search range when the next block is the non-specific block. The encoding device according to any one of claims 1 to 4, wherein control is performed to read from the storage unit and store the read-out image in the reference image storage unit. The control unit is set as a search range when the block to be processed in the second motion search is switched to the next block, and each block to be processed after the switching is set as the non-specific block. The reference image that is not likely to be deleted is erased from the reference image storage unit and may be set as a search range when the next block is the non-specific block, but is still stored in the reference image storage unit The encoding device according to any one of claims 1 to 4, wherein a reference image that has not been read is read from the storage unit and stored in the reference image storage unit. A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. It is performed for each block that is a processing unit of the first motion search,
The reference image storage unit that stores the reference image partially read from the storage unit that stores the reference frame image referred to in the second motion search performed based on the processing result of the first motion search includes the process. The reference image including the search range of the second motion search that is set based on the second motion vector for each block that is a processing unit of the second motion search in the target frame image, Store in the reference image storage unit before the start of the second motion search,
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed the upper limit value, using the first motion vector Determining as a specific block for performing the second motion search, determining a block other than the specific block as a non-specific block for performing the second motion search using the second motion vector;
For the block determined to be the specific block, the image of the search range set based on the first motion vector is referenced with reference to the image of the search range set based on the first motion vector in the reference frame image. When referring to at least an image not stored in the reference image storage unit among the images, the second search is made for a motion vector by referring to an image directly read from the storage unit without going through the reference image storage unit. For a block determined as a non-specific block by performing a motion search, refer to an image in a search range set based on the second motion vector among the reference images stored in the reference image storage unit. And performing the second motion search for searching for a motion vector.
Encoding method. A first process for searching for a first motion vector with reference to a first search range in the reduced reference frame image to obtain an encoding efficiency, and a first process smaller than the first search range in the reduced reference frame image. A second motion vector is searched for a second motion vector with reference to a second search range included in one search range, and encoding efficiency is obtained. A first motion search unit that is performed for each block that is a processing unit of the first motion search, and a storage unit that stores a reference frame image that is referred to in the second motion search performed based on the processing result of the first motion search A reference image storage unit that stores a reference image partially read out from each of the blocks to be a processing unit of the second motion search in the processing target frame image,
A control unit that stores the reference image including the search range of the second motion search set based on the second motion vector in the reference image storage unit before starting the second motion search of each block; and For a specific block for performing the second motion search using the first motion vector, refer to an image in a search range set based on the first motion vector in the reference frame image, and the first motion vector When referring to at least an image that is not stored in the reference image storage unit among images in the search range set based on the image, the image directly read from the storage unit is referred to without going through the reference image storage unit. For the non-specific block that performs the second motion search for searching for a motion vector and performs the second motion search using the second motion vector, the reference image A second motion search unit for performing the second motion search for searching for a motion vector with reference to an image in a search range set based on the second motion vector among the reference images stored in a storage unit; A computer for obtaining a processing result of the first motion search unit of the apparatus comprising:
The reference frame image is directly read from the storage unit without going through the reference image storage unit among a plurality of blocks when the processing target frame image is divided into blocks which are processing units of the second motion search. A block in which the first motion vector has a higher encoding efficiency than the second motion vector within a range in which the total value of the directly read data amount does not exceed an upper limit value, A program for determining a block other than a block as the non-specific block and supplying the determination result to the second motion search unit.