JP2011114493A - Motion vector detection method and motion vector detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector detection method for implementing high-accuracy motion search even when a plurality of objects move in different directions. <P>SOLUTION: The motion vector detection method detects a motion vector by executing motion search aimed at a search range within a reference picture, and includes an area motion vector determining step (S101) for determining a plurality of motion vector extraction areas from a targeted picture, and determining area motion vectors, and a search range setting step (S102) for setting areas, as the search range, including the positions determined by the area motion vectors and the position in the reference picture corresponding to the position of a macro-block in the targeted picture. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a motion vector detection method and a motion vector for detecting a motion vector of a macroblock by executing a motion search for a search range in a reference picture for a macroblock in a target picture to be encoded. The present invention relates to a detection device.

  In recent years, moving picture compression coding techniques such as MPEG (Moving Picture Experts Group) that perform high-efficiency coding have been rapidly put into practical use and are widely used in video cameras and mobile phones.

  In an encoding technique such as MPEG, an image close to an image to be encoded is detected from frames that have already been encoded. An efficient encoding process is realized by an encoding method that encodes only the difference between an image to be encoded and an image close to the image to be encoded. Note that a frame referred to when encoding is referred to as a reference frame. Further, detecting an image close to the image to be encoded is called motion search.

  In addition to referring to a frame preceding the recording time as a reference frame (referring to the forward direction), a subsequent frame may be referred to as a reference frame (referring to the backward direction). Furthermore, MPEG-4 AVC (Advanced Video Coding) / H. In recent coding techniques represented by H.264, evolution in motion search has been attempted, such as referring to a plurality of frames in the forward direction and a plurality of frames in the backward direction. .

  As the accuracy of motion search increases, the encoding efficiency increases. More specifically, the compression rate increases. Therefore, it is possible to record a moving image for a longer time. In addition, it is possible to record an image with higher image quality.

  With regard to this motion search, for example, when the movement of an object in a frame is large, it is necessary to search a wide range to detect an optimal image. Therefore, a large-scale motion vector detection device is required. On the other hand, Patent Document 1 discloses a technique for detecting a wide range with high accuracy without requiring a very large-scale apparatus.

  The method described in Patent Literature 1 does not set a range centering on a macroblock that is a processing unit of encoding as a range of motion search for performing motion search, but rather a frame to be encoded (hereinafter, referred to as a frame to be encoded). A global vector that is a motion vector between the entire target frame and the entire reference frame is obtained, and a motion search range is set using the obtained global vector.

JP 2005-354528 A

  In the method described in Patent Document 1, when the object in the frame moves uniformly in the same direction as when the camera pans or tilts, for example, as shown in FIG. 21, the target frame 811 and the reference frame 812, the first object 813 and the second object 814 that exist in both directions move in the same direction (the motion vector 816 of the first object 813 and the motion vector 817 of the second object 814 are aligned), and obtained from them The global vector 815 to be used is effective when it has a similar value.

  However, as shown in FIG. 23, the first object 833 and the second object 834 existing in both the target frame 831 and the reference frame 832 move in different directions (the motion vector 836 of the first object 833 and the movement of the second object 834). In the case where the vectors 837 are not aligned) and the global vector 835 also has a value having no correlation, there has been a problem that the motion search cannot be performed normally.

  This will be specifically described below. An upper right image of the first object 813 in FIG. 21 is an encoding target macroblock (hereinafter also referred to as “target macroblock”) 821 (see FIG. 22A). The search range for searching the target macroblock 821 is not “range 823 centered on the same position 822 as the target macroblock 821”, but “search range 824 set by global vector 815” shifted by global vector 815. And As a result, an image having the same desired shape as that of the target macroblock 821 can be normally detected from the search range. The same applies when the image on the left side of the second object 814 in FIG. 21 is used as the target macroblock (see FIG. 22B).

  However, even if the search range shifted by the global vector is set for the object moving in a completely different direction shown in FIG. 23, the same desired as the target macroblock as shown in FIGS. 24A and 24B. Cannot be detected from within the search range.

  The same can be considered when a plurality of frames are referenced. This will be described with reference to FIG. A global vector 845 shown in FIG. 25 is a vector indicating a motion between one frame calculated from the target frame 841 and the second reference frame 843. In the first reference frame 842 located two frames before the target frame 841, the object is estimated to be at the position of the global vector × 2. Further, in the third reference frame 844 located one frame after the target frame 841, the object is estimated to be at the position of global vector × (−1). Therefore, the search range is set based on the position shifted by the vector (see the first search range 846, the second search range 847, and the third search range 848 in FIG. 25).

  Even in this case, if the object motion vector and the global vector have similar values (as shown in FIG. 21), the object can be detected normally. However, if the motion vector of the object and the global vector have values that are completely uncorrelated (as shown in FIG. 23), the motion search cannot be performed normally.

  As a result, the motion search accuracy is lowered and the compression rate is lowered, which hinders long-time recording.

  Accordingly, an object of the present invention is to provide a motion vector detection method that realizes a highly accurate motion search even when a plurality of objects move in different directions as shown in FIG.

  In order to solve the above-described problem, the motion vector detection method according to the present invention performs motion search on a search range in a reference picture for a macroblock in a target picture to be encoded, thereby performing the macroblock search. A motion vector detection method for detecting a motion vector of the motion vector, wherein a plurality of motion vector extraction regions are determined from the target picture, and an area motion vector indicating a motion vector of the motion vector extraction region is determined for each motion vector extraction region. An area motion vector determination step to be obtained; and a search area setting step for setting an area including a position determined by the position in the reference picture corresponding to the position of the macroblock in the target picture and the area motion vector as the search area; including.

  As a result, even when a plurality of objects in a picture to be encoded move in different directions, a high-precision motion search is realized based on motion characteristics obtained from a plurality of regions. Therefore, improvement of encoding efficiency and long-time recording of moving images are realized.

  Further, in the region motion vector determination step, the plurality of motion vector extraction regions are set such that at least one motion vector extraction region of the plurality of motion vector extraction regions includes a center position of the target picture. The region motion vector may be obtained by determination.

  As a result, it is possible to realize a highly accurate motion search for a region of interest of a photographer such as a camera that is likely to be located in the center of the picture. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  In addition, the motion vector detection method further includes a specific object detection step of detecting a position of a specific object existing in the target picture, and the region motion vector determination step includes, among the plurality of motion vector extraction regions, The plurality of motion vector extraction regions may be determined so that at least one motion vector extraction region includes the position detected by the specific object detection step, and the region motion vector may be obtained.

  Thereby, the precision of the motion search with respect to a specific thing becomes high. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  The specific object may be a human face.

  This increases the accuracy of motion search for the face area. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  In the region motion vector determination step, a pixel value in the motion vector extraction region of the target picture and a pixel value in the region of the reference picture corresponding to the motion vector extraction region of the target picture are used. The region motion vector may be obtained for each motion vector extraction region of the target picture.

  Thereby, a region motion vector indicating the motion of the entire motion vector extraction region in the picture to be encoded can be obtained with high accuracy.

  Further, in the region motion vector determination step, the region motion vector may be obtained for each motion vector extraction region from information on a motion vector of a macroblock that has already been encoded in the motion vector extraction region.

  Thereby, the area motion vector can be acquired without requiring a large amount of processing.

  In addition, the motion vector detection method further includes a camera shake correction step for reducing and correcting camera shake, which is a fine motion in the vertical and horizontal directions of the target picture, and the region motion vector determination step detects at the time of camera shake correction. The region motion vector may be obtained from information on the motion vector of the target picture.

  Thereby, camera shake information installed in an application such as a camera can be used for a motion vector detection method. Therefore, it is possible to reduce the processing amount for calculating the region motion vector and obtain a highly accurate region motion vector.

  The motion vector detection method may further include a search range adjustment step of adjusting the size of the search range depending on motion information in the motion vector extraction region.

  Thereby, the size of each search range is adaptively adjusted according to the motion information corresponding to each search range. Therefore, highly accurate motion search can be performed.

  The motion vector detection method may further include a search range adjustment step of adjusting the search range by changing the attribution of the search range to a reference picture depending on motion information in the motion vector extraction region. May be included.

  Thus, the pictures to which each search range belongs are adaptively adjusted according to the motion information corresponding to each search range. Therefore, a highly accurate motion search is executed.

  Further, the motion information may be a value indicating the size of the region motion vector of the motion vector extraction region.

  As a result, the search range is adjusted according to the size of the region motion vector (the amount of movement of the object) indicating the motion of the entire motion vector extraction region. Therefore, the accuracy of motion search is improved.

  The motion information may be a variance value of motion vectors of macroblocks that have already been encoded in the motion vector extraction region.

  As a result, the search range is adjusted according to the variation (dispersion) of the motion vector for each motion vector extraction region. Therefore, the accuracy of motion search is improved.

  The motion vector detection method may further include a difference between L area motion vectors respectively obtained from L (L is an integer of 2 or more) motion vector extraction areas among the plurality of motion vector extraction areas. Are within the range of a predetermined first threshold, the L motion vector extraction regions are set as one motion vector extraction region, and the L region motion vectors are set as one region. An area motion vector limiting step for limiting to motion vectors may be included.

  As a result, the search range can be minimized. Therefore, memory traffic is reduced and power consumption is reduced without reducing the encoding efficiency.

  In the search range setting step, when all the region motion vectors obtained from the target picture are limited to N (N is a natural number) region motion vectors in the region motion vector limiting step, the search range is The search range may be set so as to belong to N reference pictures at the maximum.

  Thereby, the reference picture used for encoding is limited to the target picture, and more efficient motion search is realized.

  In the search range setting step, if the average value of the plurality of region motion vectors before the limitation corresponding to the limited region motion vector is larger than a predetermined second threshold, the limited region Two search ranges included in two different reference pictures may be set for the motion vector.

  Thereby, when the movement amount is large, the processing can be performed with reference to both the front and rear reference pictures. Therefore, the encoding efficiency is improved by obtaining the effect of bidirectional prediction.

  The motion vector detection apparatus according to the present invention detects a motion vector of the macroblock by executing a motion search for a search range in a reference picture for a macroblock in a target picture to be encoded. A motion vector detection device, wherein a plurality of motion vector extraction regions are determined from the target picture, and a region motion vector determination unit that obtains a region motion vector indicating a motion vector of the motion vector extraction region for each motion vector extraction region And a search range setting unit that sets, as the search range, a region including a position defined by the position in the reference picture corresponding to the position of the macroblock in the target picture and the region motion vector. But you can.

  Thereby, even when a plurality of objects in a picture to be encoded move in different directions, high-precision motion search is realized. Therefore, improvement of encoding efficiency and long-time recording of moving images are realized.

  The region motion vector determination unit may determine the plurality of motion vector extraction regions such that at least one of the plurality of motion vector extraction regions includes a center position of the target picture. The region motion vector may be obtained by determination.

  As a result, it is possible to realize a highly accurate motion search for a region of interest of a photographer such as a camera that is likely to be located in the center of the picture. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  Further, the motion vector detection device further includes a specific object detection unit that detects a position of a specific object existing in the target picture, and the region motion vector determination unit includes the plurality of motion vector extraction regions, The plurality of motion vector extraction regions may be determined so that the region motion vector is obtained so that at least one motion vector extraction region includes a position detected by the specific object detection unit.

  Thereby, the precision of the motion search with respect to a specific thing becomes high. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  The specific object may be a human face.

  This increases the accuracy of motion search for the face area. Therefore, it is possible to realize recording of a moving image for a long time and recording of a high-quality image.

  The region motion vector determination unit uses a pixel value in the motion vector extraction region of the target picture and a pixel value in the region of the reference picture corresponding to the motion vector extraction region of the target picture. The region motion vector may be obtained for each motion vector extraction region of the target picture.

  Thereby, a region motion vector indicating the motion of the entire motion vector extraction region in the picture to be encoded can be obtained with high accuracy.

  In addition, the region motion vector determination unit may determine the region motion vector for each motion vector extraction region from information on a motion vector of a macroblock that has already been encoded in the motion vector extraction region.

  Thereby, the area motion vector can be acquired without requiring a large amount of processing.

  In addition, the motion vector detection device further includes a camera shake correction unit that reduces and corrects camera shake, which is a fine motion of the target picture in the vertical and horizontal directions, and the region motion vector determination unit is detected during camera shake correction. The region motion vector may be obtained from information on the motion vector of the target picture.

  Thereby, camera shake information installed in an application such as a camera can be used for a motion vector detection method. Therefore, it is possible to reduce the processing amount for calculating the region motion vector and obtain a highly accurate region motion vector.

  The motion vector detection device may further include a search range adjustment unit that adjusts the size of the search range depending on motion information in the motion vector extraction region.

  Thereby, the size of each search range is adaptively adjusted according to the motion information corresponding to each search range. Therefore, highly accurate motion search can be performed.

  Further, the motion vector detection device further includes a search range adjustment unit that adjusts the search range by changing the attribution of the search range to a reference picture depending on motion information in the motion vector extraction region. You may prepare.

  Thus, the pictures to which each search range belongs are adaptively adjusted according to the motion information corresponding to each search range. Therefore, a highly accurate motion search is executed.

  Further, the motion information may be a value indicating the size of the region motion vector of the motion vector extraction region.

  As a result, the search range is adjusted according to the size of the region motion vector (the amount of movement of the object) indicating the motion of the entire motion vector extraction region. Therefore, the accuracy of motion search is improved.

  The motion information may be a variance value of motion vectors of macroblocks that have already been encoded in the motion vector extraction region.

  As a result, the search range is adjusted according to the variation (dispersion) of the motion vector for each motion vector extraction region. Therefore, the accuracy of motion search is improved.

  Further, the motion vector detection device further includes a difference between L area motion vectors respectively obtained from L (L is an integer of 2 or more) motion vector extraction areas among the plurality of motion vector extraction areas. Are within the range of a predetermined first threshold, the L motion vector extraction regions are set as one motion vector extraction region, and the L region motion vectors are set as one region. You may provide the area | region motion vector limitation part limited to a motion vector.

  As a result, the search range can be minimized. Therefore, memory traffic is reduced and power consumption is reduced without reducing the encoding efficiency.

  The search range setting unit may be configured such that when the region motion vector limiting unit limits all the region motion vectors obtained from the target picture to N (N is a natural number) region motion vectors, The search range may be set so as to belong to N reference pictures at the maximum.

  Thereby, the reference picture used for encoding is limited with respect to the target picture, and more efficient motion search can be realized.

  In addition, the search range setting unit, when an average value of a plurality of previous area motion vectors corresponding to the limited area motion vector is larger than a predetermined second threshold, the limited area Two search ranges included in two different reference pictures may be set for the motion vector.

  Thereby, when the movement amount is large, the processing can be performed with reference to both the front and rear reference pictures. Therefore, the encoding efficiency is improved by obtaining the effect of bidirectional prediction.

  An imaging system according to the present invention includes the motion vector detection device, a sensor that converts image light into an image signal, an optical system that forms an image of incident image light on the sensor, and the image signal as digital data. An imaging system including an analog-to-digital converter that converts the signal into a motion vector detection device and outputs the motion vector detection device to the motion vector detection device.

  Thereby, memory traffic is reduced, and image processing with high image quality and low power can be realized.

  The signal processing system according to the present invention includes the motion vector detection device and an analog / digital converter that converts an input image signal of an analog value into digital data and outputs the digital data to the motion vector detection device. But you can.

  Thereby, it can utilize as a signal processing system which performs a highly accurate motion search by digital signal processing.

  In motion vector detection, which is the most important function in video compression, it is possible to individually search for a region that matches each motion of a moving object existing in a picture, so that the accuracy of motion search can be improved. As a result, encoding efficiency is improved, and high image quality and long-time recording can be realized.

  In addition, memory traffic can be reduced, and the effect of miniaturization of the system and power saving can be expected.

FIG. 1 is a configuration diagram of a motion vector detection apparatus according to the first embodiment. FIG. 2A is a diagram illustrating a motion vector extraction region for a frame. FIG. 2B is a diagram illustrating a target image in the first motion vector extraction region. FIG. 2C is a diagram illustrating a target image in the second motion vector extraction region. FIG. 2D is a diagram illustrating a target image in the third motion vector extraction region. FIG. 3 is a diagram showing each region motion vector of the target frame. FIG. 4 is a diagram illustrating each search range for the target macroblock. FIG. 5 is a diagram showing a modification of the motion vector extraction region. FIG. 6 is a flowchart showing the motion vector detection method according to the first embodiment. FIG. 7A is a flowchart showing a motion vector detection method in the first modification of the first embodiment. FIG. 7B is a flowchart showing a motion vector detection method according to the second modification of the first embodiment. FIG. 8 is a configuration diagram of the motion vector detection device according to the second embodiment. FIG. 9 is a conceptual diagram when the size of the search range is the same. FIG. 10 is a conceptual diagram when the sizes of the search ranges are different. FIG. 11 is a conceptual diagram when the object is moving. FIG. 12A is a diagram showing a first example of a search range in the second embodiment. FIG. 12B is a diagram illustrating a second example of the search range in the second embodiment. FIG. 13 is a diagram illustrating a third example of the search range in the second embodiment. FIG. 14 is a flowchart showing a motion vector detection method according to the second embodiment. FIG. 15 is a configuration diagram of the motion vector detection device according to the third embodiment. FIG. 16A is a diagram showing a motion vector extraction region in the third embodiment. FIG. 16B is a diagram illustrating threshold values used to limit the region motion vector. FIG. 17A is a diagram illustrating a first example of a region motion vector. FIG. 17B is a diagram illustrating a second example of the region motion vector. FIG. 17C is a diagram illustrating a third example of the region motion vector. FIG. 18A is a conceptual diagram when the movement is small. FIG. 18B is a conceptual diagram when the movement is large. FIG. 19 is a flowchart showing a motion vector detection method according to the third embodiment. FIG. 20 is a configuration diagram of an imaging system according to the fourth embodiment. FIG. 21 is a conceptual diagram when the overall movement is common. FIG. 22A is a diagram illustrating a first example of a search range when the overall movement is common. FIG. 22B is a diagram illustrating a second example of the search range when the overall movement is common. FIG. 23 is a conceptual diagram when the overall movement is not common. FIG. 24A is a diagram illustrating a first example of a search range when the overall movement is not common. FIG. 24B is a diagram illustrating a second example of the search range when the overall movement is not common. FIG. 25 is a diagram illustrating an example of a search range when a plurality of reference pictures are used.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description of each embodiment, constituent elements that have been described once and constituent elements that have similar functions are denoted by the same reference numerals and description thereof is omitted.

(Embodiment 1)
FIG. 1 is a configuration diagram of a motion vector detection device according to Embodiment 1 of the present invention.

  The motion vector detection device 100 shown in FIG. 1 includes a region motion vector determination unit 110, a search range setting unit 111, and a motion search execution unit 112. Furthermore, the motion vector detection apparatus 100 may include some or all of the target image temporary storage unit 113, the reference image temporary storage unit 114, the specific object detection unit 115, and the camera shake correction unit 116.

  The region motion vector determination unit 110 receives setting information 121 of motion vector extraction regions (hereinafter also referred to as “extraction regions”) that are a plurality of regions included in a frame, and determines a plurality of extraction regions in the target frame. Then, an image belonging to the extraction area in the target frame (hereinafter also referred to as “target image”) and an image in the reference frame (hereinafter also referred to as “reference image”) are extracted for each extraction area. A region motion vector indicating the region motion vector is obtained.

  For example, as shown in FIG. 2A, the extraction area includes a total of three areas, that is, a left portion (first extraction region 132), a central portion (second extraction region 133), and a right portion (third extraction region 134) of the frame 131. It is. The extraction area may be a predetermined area or an area determined by the setting information 121. Further, it may be an area determined adaptively according to the target frame.

  A region motion vector of the first extraction region 132 is obtained using the target image 136 of the first extraction region 132 in the target frame 141 and the reference image of the first extraction region 132 in the reference frame as shown in FIG. 2B. . Similarly, the region motion vector of the second extraction region 133 is obtained using the target image 137 and the reference image of the second extraction region 133 as shown in FIG. 2C. Furthermore, the region motion vector of the third extraction region 134 is obtained using the target image 138 and the reference image in the third extraction region 134 as shown in FIG. 2D.

  This region motion vector is determined, for example, by performing block matching between the target image and the reference image and calculating a point where the difference between the two images (difference absolute value sum) is small. At this time, the target image is divided into areas of m × n pixels (m and n are natural numbers), block matching is performed for each area, a local vector is obtained, and an average value of the plurality of local vectors is calculated and extracted. The region motion vector of the region may be used. Alternatively, the target image and the reference image may be reduced, and block matching processing may be performed on the reduced images to obtain a region motion vector. The region motion vector determination unit 110 may acquire the region motion vector from the camera shake correction unit 116.

  In this way, the region motion vectors in the first extraction region 132, the second extraction region 133, and the third extraction region 134 are obtained.

  The search range setting unit 111 receives the region motion vector of each extraction region obtained by the region motion vector determination unit 110. The search range setting unit 111 determines a search range, which is a range of a reference image used for motion search, using the region motion vector.

  For example, in each extraction region, a first region motion vector 145, a second region motion vector 146, and a third region motion vector 147 as shown in FIG. 3 are calculated. Using these area motion vectors, each search range is set as shown in FIG. FIG. 4 shows a case in which processing is performed with reference to three frames of the first reference frame 152, the second reference frame 153, and the third reference frame 154.

  The search range setting unit 111 sets the first search range 155 composed of pixels in the first reference frame 152 using the first region motion vector 145 obtained in the first extraction region 142.

  When each region motion vector is a vector indicating a motion between one frame, the target frame 141 to the first reference frame 152 are separated by 2 frames (hereinafter, p frames apart in time in the previous direction). When the frame distance is p frames, and when the frame distance is q frames away in the later direction, the frame distance is used as -q frame in the description). 145 is doubled and the position shifted by that vector is set as the first search range 155.

  Similarly, the second search range 156 configured with pixels in the second reference frame 153 is set using the second region motion vector 146 obtained in the second extraction region 143. Since the frame distance in this case is one frame, the search range setting unit 111 sets a position shifted by the value of the second region motion vector 146 as the second search range 156.

  Further, the third search range 157 configured by the pixels in the third reference frame 154 is set using the third region motion vector 147 obtained in the third extraction region 144. Since the frame distance in this case is −1 frame, the search range setting unit 111 multiplies the third region motion vector 147 by −1 and sets a position shifted by that vector as the third search range 157.

  When the setting is completed, the search range setting unit 111 transfers the target image 122 necessary for encoding from the external buffer to the target image temporary storage unit 113 for each macroblock unit. In addition, the search range setting unit 111 transfers the reference image 123 in the determined search range from the external buffer to the reference image temporary storage unit 114.

  The reference image takes a large area with respect to the target image. Even when processing different macroblocks, the motion search execution unit 112 may perform motion search using the same reference image. For this reason, if the search range setting unit 111 transfers all reference images from the reference image external buffer to the reference image temporary storage unit 114 for each macroblock, the transfer efficiency decreases. Therefore, the search range setting unit 111 stores the reference image in the reference image temporary storage unit 114 while the reference image is required for future motion search, and deletes it from the reference image temporary storage unit 114 when it is no longer needed. By doing so, the transfer efficiency may be increased.

  The target image temporary storage unit 113 and the reference image temporary storage unit 114 are used for motion search by the motion search execution unit 112. However, when the motion search execution unit 112 directly refers to the external buffer, the target image temporary storage unit 113 and the reference image temporary storage unit 114 are not necessary.

  Next, the search range setting unit 111 also sends information on the determined search range to the motion search execution unit 112.

  The motion search execution unit 112 uses information on the search range set by the search range setting unit 111 and images necessary for motion search obtained from the target image temporary storage unit 113 and the reference image temporary storage unit 114. Perform motion search. When the encoding target macroblock 151 in the first extraction area 142 is processed, the motion search is executed using all areas of the first search range 155, the second search range 156, and the third search range 157.

  Similarly, the motion search execution unit 112 also includes the first search range 155, the second search range 156, and the third search range 157 even when the encoding target macroblock 151 is in the second extraction region 143 or the third extraction region 144. The motion search is performed using all the regions.

  Alternatively, when processing the encoding target macroblock 151 in the first extraction region 142, the motion search execution unit 112 may execute a motion search only for the first search range 155. Similarly, for the target macroblock in the second extraction area 143, only the second search range 156 is targeted, and for the target macroblock in the third extraction area 144, only the third search range 157 is targeted. A search may be performed.

  Thereby, a reference image similar to the target image can be detected from the first search range 155 shifted based on the first region motion vector 145 of the first extraction region 142. Similarly, the target macroblock in the second extraction region 143 can be detected from the second search range 156, and the target macroblock in the third extraction region 144 can be detected from the third search range 157. Then, the motion search execution unit 112 outputs a motion vector 124.

  Conventionally, as shown in FIG. 23, when a plurality of objects in a frame have moved in different directions, the accuracy of the region motion vector is not high, and the search range set based on this vector is also low. In some cases, the image is not appropriate (an image similar to the target macroblock to be detected is not found in the search range).

  However, if the method described in the present embodiment is used, even when there are objects that move differently in the frame, it is possible to set a finer search range by calculating a region motion vector for each extraction region. It becomes possible. Therefore, the rate at which an image similar to the target macroblock to be detected within the search range can be detected can be improved. Therefore, the image is efficiently encoded and a high-quality image is recorded.

  Further, as shown in FIG. 5, the region motion vector determination unit 110 may determine an extraction region so that one extraction region always includes the center of the frame among a plurality of extraction regions in the frame. The determined extraction area may have an overlapping area or a gap. In the example shown in FIG. 5, the target frame 161 includes five extraction regions, a first extraction region 162, a second extraction region 163, a third extraction region 164, a fourth extraction region 165, and a fifth extraction region 166. Is set. The third extraction region 164 includes the center of the frame.

  For example, in an application such as a camera, the target image being photographed often exists in the center of the frame, and the search range shifted using the region motion vector of the third extraction region 164 is similar to the target image. There is a high possibility that there is a reference image. By doing in this way, since a suitable reference image can be detected, it leads also to the image quality improvement in an attention image.

  In addition, for applications where the position in the frame of the target image is known in advance, it is also effective to set an extraction area that includes the position and set the search range using the area motion vector. is there.

  Furthermore, by providing a detection device that recognizes a specific object, for example, a face detection device that detects a person's face, the position of the person's face can be specified in advance and an extraction region including the position can be set. Good. By determining the region motion vector from such an extraction region, the search range can be effectively set according to the motion of the specific object.

  Moreover, the size of each extraction area may be equal or may not be equal.

  The region motion vector determination unit 110 may determine an extraction region for a partial region of the frame. That is, there may be a region not included in the extraction region in the frame. Alternatively, the region motion vector determination unit 110 may determine the extraction region for all regions of the frame.

  Further, the size of the extraction area may be the same as the size of the macroblock. Even in such a case, it is possible to set a finer search range by calculating a region motion vector from a plurality of extraction regions.

  The area motion vector is calculated by using the target image and the reference image of the extraction area in the encoding target frame before encoding the encoding target frame.

  That is, the region motion vector determination unit 110 divides the target image into one or more blocks smaller than the extraction region, and calculates a local vector that is one or more motion vectors by block matching in units of the blocks. . An area motion vector can be obtained by calculating, for example, an average of these local vectors. At that time, the processing amount is reduced by performing block matching after performing the zoom processing for reducing each of the target image and the reference image, or reducing the information amount by thinning out each pixel. It can also be executed.

  Further, the region motion vector may be calculated from a macroblock that has already been encoded. That is, the region motion vector determination unit 110 acquires a plurality of motion vectors (local vectors) from the encoded macroblock in the extraction region in the encoded frame, and calculates a value calculated from the acquired local vectors. The region motion vector of the extraction region may be used.

  In addition, since a moving image has high frame correlation, the region motion vector determination unit 110 can also use information on a region motion vector at a position where the frame distance is short. Thereby, it is not necessary to perform the area motion vector calculation in advance using the encoded frame and the reference frame as described above, and the area motion vector can be efficiently calculated while reducing the processing amount.

  FIG. 6 is a flowchart showing the motion vector detection method according to the first embodiment.

  First, the region motion vector determination unit 110 determines a plurality of extraction regions from the target picture, and obtains a region motion vector indicating the motion of the extraction region for each extraction region (S101).

  Next, the search range setting unit 111 sets each region defined by the position of the target macroblock and the region motion vector determined for each extraction region as a search range (S102).

  Next, the motion search execution unit 112 detects a motion vector of the target macroblock by executing a motion search for each set search range (S103).

  This improves the accuracy of motion search even for target frames that show different motion for each extraction region.

  FIG. 7A is a flowchart showing a motion vector detection method in the first modification of the first embodiment.

  First, the specific object detection unit 115 detects the position of the specific object in the target frame (S111).

  Next, the region motion vector determination unit 110 determines a plurality of extraction regions so that the detected position is included in at least one extraction region, and obtains a region motion vector for each extraction region (S101).

  The subsequent processing is the same as in FIG.

  Thereby, the precision of motion search improves about the part which moves similarly to a specific thing.

  FIG. 7B is a flowchart showing a motion vector detection method according to the second modification of the first embodiment.

  First, the camera shake correction unit 116 detects the movement of the target frame (S112).

  Next, the region motion vector determination unit 110 obtains a region motion vector for each extraction region using the detected motion (S101).

  The subsequent processing is the same as in FIG.

  Accordingly, the region motion vector for each extraction region is obtained by using the motion vector information of the newer picture (motion vector information of the target picture itself) rather than the motion vector information of the previously encoded picture (reference vector motion vector information). Therefore, the accuracy of motion search is improved.

  As described above, the motion vector detection method and the motion vector detection device according to Embodiment 1 execute a motion search for a search range obtained from a plurality of region motion vectors. Thereby, even when a plurality of objects move in different directions, highly accurate motion search is realized.

(Embodiment 2)
FIG. 8 is a configuration diagram of a motion vector detection device according to Embodiment 2 of the present invention.

  The motion vector detection device 200 shown in FIG. 8 includes a region motion vector determination unit 110, a search range setting unit 111, a search range adjustment unit 231 and a motion search execution unit 112. Furthermore, the motion vector detection device 200 may include some or all of the target image temporary storage unit 113 and the reference image temporary storage unit 114.

  The search range adjustment unit 231 adjusts the search range based on the search range information set by the search range setting unit 111 and the region motion vector information determined by the region motion vector determination unit 110.

  The motion search execution unit 112 executes a motion search for the search range adjusted by the search range adjustment unit 231.

  Other parts are the same as those of the first embodiment.

  Next, a specific operation of the search range adjustment unit 231 will be described.

  The reference image temporary storage unit 114 stores a reference image necessary for executing motion search for one macroblock. For example, when there are three search ranges as shown in FIG. 4, as shown in FIG. 9, the reference image is divided into three regions equally divided into the storage region 245 in the reference image temporary storage unit 114 and stored in each of them. Has been. The motion search execution unit 112 reads out a reference image in each search range from there and executes a motion search.

  That is, when there are three reference frames of the first reference frame 152, the second reference frame 153, and the third reference frame 154 for the target frame 141, the first search range 155, The reference images of the search range 156 and the third search range 157 are stored in the first area 246, the second area 247, and the third area 248, respectively.

  Here, when a certain region motion vector is calculated from a plurality of local vectors, the variance of the plurality of local vectors (hereinafter also referred to as “vector variance”) can be obtained. When the vector variance is small, there is a high possibility that a plurality of objects existing in the corresponding extraction region are moving in the same direction. In this case, it is not necessary to search for a wide range. On the other hand, when the vector variance is large, there is a high possibility that a plurality of objects existing in the corresponding extraction region are moving in different directions. In this case, it is necessary to search a wide range.

  That is, the search range adjustment unit 231 uses the first region motion vector 145, the second region motion vector 146, and the third region used in setting the first search range 155, the second search range 156, and the third search range 157. The first vector variance, the second vector variance, and the third vector variance corresponding to the region motion vector 147 are compared. For example, when the comparison result is as shown in Expression 1, the search range adjustment unit 231 adjusts the size of the search range as shown in Expression 2.

First vector variance <second vector variance <third vector variance (Equation 1)
First search range 155 <second search range 156 <third search range 157 (Formula 2)

  FIG. 10 is a conceptual diagram showing the relationship between the search range and the storage area when the sizes of the search range are different. The third area 248 that stores the reference image of the third search range 157 is large, and the first area 246 that stores the reference image of the first search range 155 is small.

  As a result, the first search range 155 that does not need to search a very wide area is narrowed, and the third search range 157 that needs to search a wide area is widened. Further, by enlarging the third area 248 in the storage area 245 in the reference image temporary storage unit 114, the search range of the third search range 157 can be expanded, and the motion search detection rate increases.

  In addition, a plurality of objects that are moving in different directions even though they are in the same extraction region are present at more distant positions as the reference frame is further away from the target frame. Referring to FIG. 11, in FIG. 11, the first object 263 and the second object 264 in the extraction area 262 are moving in different directions. The distance D 2 between the objects in the second reference frame 266 is greater than the distance D 0 between the first object 263 and the second object 264 in the target frame 261. Further, the distance D1 between the objects in the first reference frame 265 is more distant than the distance D2 (D0 <D2 <D1). When moving in different directions as described above, the detection rate is improved by searching from a frame with a short frame distance.

  Whether or not an object in the same extraction region is moving in a different direction can be determined using the vector dispersion described above. Then, as shown in FIG. 12A and FIG. 12B, the search performance can be improved by assigning the region motion vector corresponding to the one having the larger vector variance to the reference frame having a short frame distance.

  FIG. 12A shows the region motion vector and the reference when the first vector variance corresponding to the first region motion vector 274 is smaller than or equal to the second vector variance corresponding to the second region motion vector 275. It is a figure which shows the relationship with a flame | frame.

  The corresponding first region motion vector 274 having a small vector variance is used for setting a search range of the first reference frame 272 farther from the target frame 271. The corresponding second region motion vector 275 having a large vector variance is used for setting the search range of the second reference frame 273 closer to the target frame 271.

  FIG. 12B is a diagram showing the relationship between the region motion vector and the reference frame when the first vector variance corresponding to the first region motion vector 274 is larger than the second vector variance corresponding to the second region motion vector 275. . In this case, the relationship between the region motion vector and the reference frame shown in FIG. 12A is reversed.

  When the vector variance is very large, for example, when the vector variance value is larger than the threshold (variance threshold), the search range set from the vector variance is set from the nearest reference frame as shown in FIG. It is also possible to perform processing with setting.

  FIG. 13 shows the relationship between the region motion vector and the reference frame when the first vector variance corresponding to the first region motion vector 274 and the second vector variance corresponding to the second region motion vector 275 are larger than the variance threshold. FIG.

  Thus, each search range is not necessarily limited to being set in a different reference frame.

  In the present embodiment, the search range adjustment unit 231 uses the vector variance to switch the search range region and the reference frame. However, the motion vector size (motion vector) It is also possible to switch by absolute value). The magnitude of the motion vector represents the amount of movement of the object in the extraction area, and may be the sum of the magnitudes of the local vectors or the magnitude of the area motion vector.

  The larger the value of the motion vector that represents the amount of movement of the object, the more the object moves from the target frame to the reference frame, and thus the motion search becomes difficult. Therefore, the search range adjustment unit 231 can also switch the search range region and the reference frame according to the magnitude of the motion vector, similarly to the vector dispersion.

  The search range adjustment unit 231 adjusts the search range by switching the search range area and switching the reference frame.

  Regarding a plurality of reference frames described in this embodiment, MPEG-4 AVC / H. It goes without saying that it can be used not only for reference to a plurality of frames at the time of P slice encoding such as H.264 but also for reference relations in B slices.

  Regarding the extraction region, it is possible to set a part of the same region for a plurality of extraction regions, or it is possible to set a completely different region.

  FIG. 14 is a flowchart showing a motion vector detection method according to the second embodiment.

  First, the region motion vector determination unit 110 determines a plurality of extraction regions from the target picture, and obtains a region motion vector indicating the motion of the extraction region for each extraction region (S201).

  Next, the search range setting unit 111 sets each region defined by the position of the target macroblock and the region motion vector determined for each extraction region as a search range (S202).

  Next, the search range adjustment unit 231 adjusts the search range based on the motion information for each extraction region (S203). For example, when the variance of local vectors used when obtaining the region motion vector is large or the region motion vector is large, the search range adjustment unit 231 expands the search range. Alternatively, when the variance of local vectors used when obtaining the region motion vector is large, the search range adjustment unit 231 adjusts the search range so that the search range belongs to a reference frame that is temporally close.

  Next, the motion search execution unit 112 detects a motion vector of the target macroblock by executing a motion search for each adjusted search range (S204).

  Accordingly, since the motion search is performed on the adaptively adjusted search range, the accuracy of the motion search is improved.

(Embodiment 3)
FIG. 15 is a configuration diagram of a motion vector detection device according to Embodiment 3 of the present invention.

  The motion vector detection device 300 shown in FIG. 15 includes a region motion vector determination unit 110, a region motion vector limitation unit 332, a search range setting unit 111, and a motion search execution unit 112. Furthermore, the motion vector detection device 300 may include some or all of the target image temporary storage unit 113 and the reference image temporary storage unit 114.

  The region motion vector limiting unit 332 has a function of limiting the region motion vector based on the information on the region motion vector determined by the region motion vector determination unit 110. Thereby, the extraction area and the search range are summarized.

  The search range setting unit 111 sets a search range using a limited region motion vector.

  Other parts are the same as those of the first embodiment.

  Next, a specific operation of the region motion vector limiting unit 332 will be described.

  In the third embodiment, as shown in FIG. 16A, the extraction areas are set on the upper side (first extraction area 342), the center (second extraction area 343), and the lower side (third extraction area 344) of the frame 341. . Further, in the third embodiment, as shown in FIG. 16B, the extraction area generalization threshold A is set to 20 in the horizontal direction and 10 in the vertical direction. Further, the threshold value B for adding the search range is set to 120 for the horizontal direction and 60 for the vertical direction.

  When the region motion vectors are in the state of FIG. 17A, all the region motion vectors of the first extraction region 342, the second extraction region 343, and the third extraction region 344 are within the range of the threshold A in both the horizontal direction and the vertical direction. Absent. Therefore, the region motion vector is not limited, and the search range is not summarized.

  When the region motion vector is in the state of FIG. 17B, the region motion vector is within the range of the threshold A in both the horizontal direction and the vertical direction. Therefore, the extraction region is grouped into one, and the region motion vector is limited to one. At this time, the value of the region motion vector of the extracted region that is generalized may be an average value of each region motion vector. In this case, the region motion vectors are horizontal direction = 30 and vertical direction = 20. As a result, the search range is summarized as one.

  If there are three extraction regions, the motion vector detection apparatus according to Embodiment 1 performs motion search for a maximum of three pictures.

  However, in this embodiment, the motion search target picture can be reduced to one by broadening the extraction area. Thereby, the memory traffic for accessing the reference image can be reduced, and the power consumption of the apparatus can be reduced.

  Also, for example, H. In H.264, when there is one reference frame, num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 of the slice header become 0, so that ref_idx_l0 [mbPartIdx] and ref1x can be coded in mb_pred (). Therefore, encoding efficiency can also be improved.

  When the region motion vector is in the state of FIG. 17C, the region motion vector is within the range of the threshold value A in both the horizontal direction and the vertical direction, so that the extraction region can be broadened into one. However, the search range is 2 because it exceeds the range of the threshold B for adding the search range in both the horizontal and vertical directions.

  As described above, when the region motion vector is larger than the threshold value B, it is assumed that panning or tilting is performed. Therefore, even if each region motion vector has the same value, encoding efficiency is improved by performing motion search in two or more search ranges included in two or more reference pictures. For example, in the case of a B picture, by referring to the forward reference frame and backward reference frame, it is not necessary to reduce the coding efficiency even in an image when panning or tilting.

  Specifically, this will be described with reference to FIGS. 18A and 18B.

  In FIG. 18A, the region motion vectors of the first extraction region 342, the second extraction region 343, and the third extraction region 344 of the target frame 351 have the same value, and the scalar value of each region motion vector is small. Therefore, even if encoding is performed with reference to only the forward reference frame 352 without referring to the backward reference frame 353, the encoding efficiency is not reduced.

  On the other hand, in FIG. 18B, the region motion vectors of the first extraction region 342, the second extraction region 343, and the third extraction region 344 of the target frame 354 have the same value, and the scalar value of each region motion vector is large. That is, in FIG. 18B, the entire frame is panned from right to left.

  In the case of the figure, an object (house) that does not exist in the front reference frame 355 exists in the rear reference frame 356, and an object (automobile) that exists in the front reference frame 355 exists in the rear reference frame 356. Absent. In such a case, it is necessary to perform encoding using both frames because the encoding efficiency is deteriorated only by referring to only the front or rear frame.

  FIG. 19 is a flowchart showing a motion vector detection method according to the third embodiment.

  First, the region motion vector determination unit 110 determines a plurality of extraction regions from the target picture, and obtains a region motion vector indicating the motion of the extraction region for each extraction region (S301).

  Next, if the value indicating the difference between the region motion vectors derived from the plurality of extraction regions is within the threshold range, the region motion vector limiting unit 332 outputs each region motion derived from the plurality of extraction regions. The vector is limited to one area motion vector (S302).

  Next, the search range setting unit 111 sets a region defined by the position of the target macroblock and each limited region motion vector as a search range (S303).

  Next, the motion search execution unit 112 detects a motion vector of the target macroblock by executing a motion search for each set search range (S304).

  This improves the accuracy of motion search even for target frames that show different motion for each extraction region. Furthermore, the search range is effectively limited.

  As described above, in the motion vector detection method according to the first embodiment, a plurality of search ranges are set even when the region motion vectors have the same value in a plurality of extraction regions. Therefore, an unnecessary motion search may be performed.

  However, if the method described in this embodiment is used, unnecessary motion search can be suppressed when the region motion vectors have the same value, so that the power consumption of the motion vector detection device can be suppressed. Further, the encoding efficiency can be improved.

  Thus, in the present embodiment, it is possible to achieve both high-efficiency encoding and power consumption reduction by adaptively limiting the search range.

(Embodiment 4)
FIG. 20 is a block diagram showing a configuration of an imaging system 601 according to Embodiment 4 of the present invention, for example, a digital still camera (DSC). A motion vector detection device 606 in FIG. 20 is any of the motion vector detection devices according to the first to third embodiments of the present invention.

  According to FIG. 20, the image light incident through the optical system 602 forms an image on the sensor 603. The sensor 603 is driven by the timing control circuit 609 to accumulate the imaged image light and photoelectrically convert it into an electrical signal. An electrical signal read from the sensor 603 is converted into a digital signal by an analog-digital converter (ADC) 604 and then input to an image processing circuit 605 including the motion vector detection device 606. In the image processing circuit 605, image processing such as Y / C processing, edge processing, image enlargement / reduction, and motion vector detection using the present invention is performed. The image-processed signal is recorded or transferred to a medium in a recording / transfer circuit 607. The recorded or transferred signal is reproduced by the reproduction circuit 608. The entire imaging system 601 is controlled by a system control circuit 610.

  Note that image processing in the motion vector detection device 606 according to the present invention is not necessarily applied only to a signal based on image light imaged on the sensor 603 via the optical system 602. For example, it goes without saying that the present invention can also be applied when processing an image signal input as an electrical signal from an external device.

  Moreover, the combination of the arbitrary component shown by this Embodiment and the motion vector detection apparatus 606 can be utilized as a signal processing system which performs a motion search.

  Although the motion vector detection method and the motion vector detection device according to the present invention have been described based on the first to fourth embodiments, the present invention is not limited to these embodiments. Forms obtained by subjecting these embodiments to modifications conceived by those skilled in the art, and other forms realized by arbitrarily combining components in these embodiments are also included in the present invention.

  Further, the present invention can be realized as a program for causing a computer to execute the steps included in the motion vector detection method. Furthermore, the present invention can be realized as a computer-readable storage medium storing the program.

  The frame described above is a concept including a field. Frames and fields are also called pictures.

  Since the motion vector method according to the present invention can realize a high-performance motion search, it is a very useful technique for an imaging system that requires high image quality and long-time recording. In addition, by limiting the reference frame, it is possible to reduce memory traffic and save power, which is a useful technique for a mobile imaging system.

DESCRIPTION OF SYMBOLS 100, 200, 300, 606 Motion vector detection apparatus 110 Area | region motion vector determination part 111 Search range setting part 112 Motion search execution part 113 Target image temporary storage part 114 Reference image temporary storage part 115 Specific object detection part 116 Camera shake correction part 121 Setting information 122, 136, 137, 138 Target image 123 Reference image 124, 816, 817, 836, 837 Motion vector 131, 341 Frame 132, 142, 162, 342 First motion vector extraction region (first extraction region)
133, 143, 163, 343 Second motion vector extraction region (second extraction region)
134, 144, 164, 344 Third motion vector extraction region (third extraction region)
141, 161, 261, 271, 351, 354, 811, 831, 841 Target frame 145, 274 First region motion vector 146, 275 Second region motion vector 147 Third region motion vector 151, 821 Encoding target macroblock ( Target macro block)
152, 265, 272, 842 First reference frame 153, 266, 273, 843 Second reference frame 154, 844 Third reference frame 155, 846 First search range 156, 847 Second search range 157, 848 Third search range 165 Fourth motion vector extraction region (fourth extraction region)
166 Fifth motion vector extraction region (fifth extraction region)
231 Search range adjustment unit 245 Storage area 246 First area 247 Second area 248 Third area 262 Motion vector extraction area (extraction area)
263, 813, 833 First object 264, 814, 834 Second object 332 Area motion vector limiting unit 352, 355 Forward reference frame 353, 356 Back reference frame 601 Imaging system 602 Optical system 603 Sensor 604 Analog to digital converter (ADC)
605 Image processing circuit 607 Recording transfer circuit 608 Reproduction circuit 609 Timing control circuit 610 System control circuit 812, 832 Reference frame 815, 835, 845 Global vector 822 Position 823 Range 824 Search range

Claims (30)

A motion vector detection method for detecting a motion vector of a macroblock by executing a motion search for a search range in a reference picture for a macroblock in a target picture to be encoded,
A region motion vector determination step for determining a plurality of motion vector extraction regions from the target picture and obtaining a region motion vector indicating a motion vector of the motion vector extraction region for each of the motion vector extraction regions;
A search range setting step of setting, as the search range, a region including a position defined by a position in the reference picture corresponding to a position of the macroblock in the target picture and the region motion vector. In the region motion vector determination step, the plurality of motion vector extraction regions are determined such that at least one of the plurality of motion vector extraction regions includes a center position of the target picture. The motion vector detection method according to claim 1, wherein the region motion vector is obtained. The motion vector detection method further includes a specific object detection step of detecting a position of a specific object existing in the target picture,
In the region motion vector determination step, the plurality of motion vector extraction regions so that at least one motion vector extraction region among the plurality of motion vector extraction regions includes a position detected by the specific object detection step. The motion vector detection method according to claim 1, wherein the region motion vector is determined. The motion vector detection method according to claim 3, wherein the specific object is a human face. In the region motion vector determination step, the pixel value in the motion vector extraction region of the target picture and the pixel value in the region of the reference picture corresponding to the motion vector extraction region of the target picture are used. The motion vector detection method according to claim 1, wherein the region motion vector is obtained for each motion vector extraction region of a target picture. The region motion vector determination step obtains the region motion vector for each motion vector extraction region from information on motion vectors of macroblocks already encoded in the motion vector extraction region. The motion vector detection method according to any one of the above. The motion vector detection method further includes a camera shake correction step for reducing and correcting camera shake, which is a fine motion in the vertical and horizontal directions of the target picture,
The motion vector detection method according to any one of claims 1 to 4, wherein in the region motion vector determination step, the region motion vector is obtained from information on a motion vector of the target picture detected during camera shake correction. The said motion vector detection method further includes the search range adjustment step which adjusts the magnitude | size of the said search range depending on the motion information in the said motion vector extraction area | region. Motion vector detection method. The motion vector detection method further includes a search range adjustment step of adjusting the search range by changing attribution of the search range to a reference picture depending on motion information in the motion vector extraction region. The motion vector detection method according to any one of Items 1 to 7. The motion vector detection method according to claim 8 or 9, wherein the motion information is a value indicating a size of the region motion vector of the motion vector extraction region. The motion vector detection method according to claim 8 or 9, wherein the motion information is a variance value of motion vectors of macroblocks that have already been encoded in the motion vector extraction region. The motion vector detection method further shows a difference between L area motion vectors respectively obtained from L (L is an integer of 2 or more) motion vector extraction areas among the plurality of motion vector extraction areas. When all values are within a predetermined first threshold range, the L motion vector extraction regions are used as one motion vector extraction region, and the L region motion vectors are used as one region motion vector. The motion vector detection method according to claim 1, further comprising a region motion vector limiting step limited to. In the search range setting step, when all the region motion vectors obtained from the target picture are limited to N (N is a natural number) region motion vectors in the region motion vector limiting step, the search range is maximum. The motion vector detection method according to claim 12, wherein the search range is set so as to belong to N reference pictures. In the search range setting step, when the average value of the plurality of region motion vectors before the limitation corresponding to the limited region motion vector is larger than a predetermined second threshold, the limited region motion vector The motion vector detection method according to claim 12, wherein two search ranges included in two different reference pictures are set. A motion vector detection device that detects a motion vector of a macroblock by executing a motion search for a search range in a reference picture for a macroblock in a target picture to be encoded,
A plurality of motion vector extraction regions from the target picture, and a region motion vector determination unit for determining a region motion vector indicating a motion vector of the motion vector extraction region for each motion vector extraction region;
A motion vector detection device comprising: a search range setting unit configured to set, as the search range, an area including a position defined by a position in the reference picture corresponding to a position of the macroblock in the target picture and the area motion vector. The region motion vector determination unit determines the plurality of motion vector extraction regions so that at least one motion vector extraction region of the plurality of motion vector extraction regions includes a center position of the target picture. The motion vector detection device according to claim 15, wherein the region motion vector is obtained. The motion vector detection apparatus further includes a specific object detection unit that detects a position of a specific object existing in the target picture,
The region motion vector determination unit includes the plurality of motion vector extraction regions such that at least one of the plurality of motion vector extraction regions includes a position detected by the specific object detection unit. The motion vector detection device according to claim 15 or 16, wherein the motion vector detection device determines the region motion vector. The motion vector detection device according to claim 17, wherein the specific object is a human face. The region motion vector determination unit uses the pixel value in the motion vector extraction region of the target picture and the pixel value in the region of the reference picture corresponding to the motion vector extraction region of the target picture, The motion vector detection device according to claim 15, wherein the region motion vector is obtained for each motion vector extraction region of the target picture. The region motion vector determination unit obtains the region motion vector for each motion vector extraction region from information on a motion vector of a macroblock that has already been encoded in the motion vector extraction region. The motion vector detection device according to any one of the above. The motion vector detection device further includes a camera shake correction unit that reduces and corrects camera shake that is a fine motion of the target picture in the vertical and horizontal directions
The motion vector detection device according to any one of claims 15 to 18, wherein the region motion vector determination unit obtains the region motion vector from information on a motion vector of the target picture detected during camera shake correction. The said motion vector detection apparatus is further provided with the search range adjustment part which adjusts the magnitude | size of the said search range depending on the motion information in the said motion vector extraction area | region. Motion vector detection device. The motion vector detection device further includes a search range adjustment unit that adjusts the search range by changing the attribution of the search range to a reference picture depending on motion information in the motion vector extraction region. Item 22. The motion vector detection device according to any one of Items 15 to 21. The motion vector detection device according to claim 22 or 23, wherein the motion information is a value indicating a size of the region motion vector of the motion vector extraction region. The motion vector detection device according to claim 22 or 23, wherein the motion information is a variance value of motion vectors of a macroblock that has already been encoded in the motion vector extraction region. The motion vector detection device further shows a difference between L area motion vectors respectively obtained from L (L is an integer of 2 or more) motion vector extraction areas among the plurality of motion vector extraction areas. When all values are within a predetermined first threshold range, the L motion vector extraction regions are used as one motion vector extraction region, and the L region motion vectors are used as one region motion vector. The motion vector detection device according to any one of claims 15 to 25, further comprising a region motion vector limiting unit limited to The search range setting unit has a maximum search range when all the region motion vectors obtained from the target picture are limited to N (N is a natural number) region motion vectors by the region motion vector limiting unit. 27. The motion vector detection device according to claim 26, wherein the search range is set so as to belong to N reference pictures. The search range setting unit, when an average value of a plurality of previous region motion vectors corresponding to the limited region motion vector is larger than a predetermined second threshold value, the limited region motion vector 27. The motion vector detection device according to claim 26, wherein two search ranges included in two different reference pictures are set. A motion vector detection device according to any one of claims 15 to 28;
A sensor that converts image light into an image signal;
An optical system that forms an image of incident image light on the sensor;
An imaging system comprising: an analog-digital converter that converts the image signal into digital data and outputs the digital data to the motion vector detection device. A motion vector detection device according to any one of claims 15 to 28;
An analog-to-digital converter that converts an input image signal of an analog value into digital data and outputs the digital data to the motion vector detection device.

Images