JP6394216B2 - Motion detection method, motion detection apparatus, and motion detection program - Google Patents

Description

  The present disclosure relates to a motion detection method, a motion detection device, and a motion detection program.

  In recent years, attention has been focused on technologies that enhance driving safety by supporting driver's driving due to increasing awareness of automobile safety and advances in car electronics technology. In particular, when a vehicle starts, there is an increasing demand for a vehicle surrounding approach detection system that automatically detects approaching objects such as pedestrians and other vehicles approaching the vehicle and alerts the driver. . Various methods are conceivable for detecting an approaching object, but a method using an on-vehicle camera image is low-cost and effective from the viewpoint of cooperation with other driving support systems.

  In a vehicle peripheral approaching object detection system using an in-vehicle camera, an optical flow is calculated for a moving image of a camera image, and when an optical flow directed in a specific direction is detected, it is determined that there is an approaching object. As a method for calculating an optical flow, a local correlation calculation method using SAD is often used.

  In the vehicle periphery approaching object detection system, detecting a moving object while the moving object is far away gives a driver a lot of time, and is extremely important from the viewpoint of improving safety. When the moving object is far away, the moving amount in the moving image is also small, so it is necessary to detect a small optical flow without missing it. However, when trying to detect a small optical flow, the risk of misdetecting a fine error flow increases. The error flow may not only occur due to a fine moving body such as a plant or the like in a moving image or noise of an image, but may also occur only when a luminance change occurs without any moving object.

  The phenomenon in which motion is detected due to only the luminance change is particularly likely to occur when the luminance change of the image is gradation. That the luminance change of the image is gradation means that the luminance gradually increases or decreases monotonously from the first luminance at the first pixel position to the second luminance at the second pixel position. In particular, when the luminance change is linear (that is, linear), erroneous movement is likely to be detected. This is because, when there is a spatially linear brightness change, the brightness change caused by spatial movement of an object having a linear brightness change and the overall brightness uniformly increase (or decrease). This is because a change in luminance caused by the above cannot be distinguished from only local information. That is, paying attention to a certain pixel position and a local area in the vicinity of the pixel position, when an increase (or decrease) in luminance occurs in that area, it is a luminance change caused by actual movement of the object, or the entire luminance It is indistinguishable whether the brightness change is caused by uniform change.

  The erroneous motion detection due to the luminance change as described above is not limited to the vehicle periphery approaching object detection system, but may be a problem that may occur in all systems that detect optical flows.

JP 2006-53692 A JP 2013-186668 A JP2013-84140A JP 2013-101419 A JP 2008-257626 A JP 2012-68842 A

  In view of the above, a motion detection apparatus and method that can avoid detection of an erroneous optical flow due to a change in luminance is desired.

  In the motion detection method, an optimal optical flow solution is obtained by performing motion search using a first evaluation function that is affected by a luminance change, and the optimal solution is obtained using a second evaluation function that is not affected by the luminance change. Each step of selecting the optimum solution of the stationary solution and the stationary solution where the object is stationary is executed by the computer.

  The motion detection device includes a processor and a memory, and when the processor executes a program stored in the memory, a motion search is performed by using the first evaluation function that is affected by a change in luminance, thereby performing an optical flow. And executing each step of selecting the optimum solution of the optimum solution and the stationary solution in which the object is stationary using the second evaluation function that is not affected by the luminance change. To do.

  According to at least one embodiment, detection of an erroneous optical flow due to a luminance change can be avoided.

It is a figure which shows an example of a structure of a motion detection apparatus. It is a figure which shows an example of a software structure of the motion detection apparatus of FIG. It is a figure for demonstrating a motion search process. It is a figure for demonstrating the correction process using the evaluation function which is not influenced by a luminance change. It is a flowchart which shows the procedure of a motion search and correction process. It is a flowchart which shows the procedure which determines the presence or absence of the possibility of misdetection by a luminance change. It is a table | surface which shows the effect of a correction process. It is a flowchart which shows the procedure of the process which removes the influence of a background motion. It is a figure for demonstrating the process which removes the influence of a motion of a background. It is a figure for demonstrating the pattern from which a background motion differs. It is a figure for demonstrating the pattern from which a background motion differs. It is a figure for demonstrating the pattern from which a background motion differs.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following drawings, the same or corresponding components are referred to by the same or corresponding numerals, and the description thereof will be omitted as appropriate.

  FIG. 1 is a diagram illustrating an example of a configuration of a motion detection device. 1 includes a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a GPU (Graphics Processing Unit) 13, a display interface 14, and a video input interface 15. including. The motion detection apparatus shown in FIG. 1 may be a computer that executes control processing and arithmetic processing, for example, a computer built in an on-vehicle peripheral object detection system or connected to a monitoring camera or the like. It may be a personal computer or the like.

  The CPU 10 executes the program based on the program and data stored in the ROM 11 and RAM 12 to control the entire motion detection device and performs arithmetic processing on the image data stored in the RAM 12. The ROM 11 stores a basic program necessary for controlling the motion detection device. The RAM 12 stores an application program executed on the motion detection device, an optical flow calculation processing program, an image processing basic software program, and the like. The basic image processing software program may be OpenGL, for example, and may be executed by the GPU 13.

  FIG. 2 is a diagram illustrating an example of a software configuration of the motion detection apparatus in FIG. In FIG. 2, hardware of the motion detection device illustrated in FIG. 1 is illustrated as hardware 104. For example, when the CPU 10 executes a program of the application 101 that detects and displays a motion in a video, for example, the optical flow calculation process 102 is called from the program, and the CPU 10 executes the program of the optical flow calculation process 102. When the optical flow is calculated from the video data (a plurality of image data arranged in time series), the image processing basic software 103 is executed by the GPU 13 to perform drawing on the frame buffer area (RAM 12) in the hardware 104. Note that instead of the CPU 10 executing the optical flow calculation process 102, the GPU 13 may execute the optical flow calculation process 102.

  By executing the optical flow calculation process 102, the motion detection apparatus shown in FIG. 1 first obtains an optimal optical flow solution by performing a motion search using the first evaluation function that is affected by a change in luminance. Next, the motion detection device selects the optimum solution of the optimum solution and the stationary solution in which the object is stationary using the second evaluation function that is not affected by the luminance change. Here, in motion search, the similarity or dissimilarity between pixel blocks is calculated using an evaluation function as described later, so that pixel blocks having the highest similarity (low dissimilarity) are temporally continuous 2 Identifies between two image frames.

  The first evaluation function that is affected by the luminance change is that the similarity or difference value calculated by using the first evaluation function changes due to a change in the overall luminance such that the luminance of the entire image changes uniformly. This is an evaluation function. The second evaluation function that is not affected by the luminance change is a value of the similarity or difference calculated by using the second evaluation function due to a change in the overall luminance such that the luminance of the entire image changes uniformly. An evaluation function that does not change. In addition, the change in overall brightness in which the brightness of the entire image changes uniformly, more specifically, a pixel value in which the pixel value of each pixel in the pixel block increases or decreases by the same value. It is a change of.

  The use of the second evaluation function that is not affected by the change in luminance means that the absolute pixel value information for each pixel in the pixel block is discarded, and only the relative pixel value information between the pixels is discarded. This is equivalent to calculating the degree of similarity or difference using. When such a second evaluation function is used, the amount of information used in the calculation of similarity or dissimilarity is small as compared with the case where the first evaluation function that is affected by a change in luminance is used. Therefore, when the optimal solution of the optical flow is obtained using the second evaluation function that is not affected by the luminance change, the possibility that an incorrect solution is obtained, that is, an erroneous motion different from the actual motion is detected. The possibility that the first evaluation function is larger than that in the case where the first evaluation function is used.

  Therefore, as described above, first, an appropriate solution is obtained by executing a motion search process for obtaining an optimal solution using the first evaluation function, and a reliable solution is obtained in a situation where there is no luminance change. Thereby, it can be ensured that the accuracy of optical flow detection does not drop. There is no problem with this optimal solution in a situation where there is no luminance change, but there is a possibility that this optimal solution is erroneous in a situation where there is a luminance change. Therefore, using the second evaluation function that is not affected by the luminance change, it is determined which of the optimal solution and the static solution is optimal, and correction processing for selecting the optimal solution is executed. . This makes it possible to obtain a highly reliable solution in motion detection regardless of whether or not there is a luminance change.

  Returning to FIG. 1, the video input interface 15 may be connected to an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) imaging device or a video playback device such as a DVD (Digital Versatile Disk) player. Video data received by the video input interface 15 from a connection destination device may be stored in the RAM 12. As the data storage device, for example, a hard disk device may be provided in addition to the RAM 12, and when storing video data as nonvolatile data, the hard disk device may be used as a storage destination.

  For example, an image display device such as an LCD (Liquid Crystal Display) is connected to the display interface 14. For example, image data obtained by superimposing and displaying the calculated optical flow as a vector on the original image may be read from the RAM 12 and displayed on the image display device via the display interface 14.

  FIG. 3 is a diagram for explaining the motion search process. The frame image data 20 and the frame image data 21 are data of frames that are continuous in time series (on the time axis). The frame image data 21 is the image data of the current frame, and the frame image data 20 is the image data of the frame immediately before the current frame. In FIG. 3, a point of interest on the frame image data 20, that is, a point for motion detection is indicated as a pixel position 30. A pixel position on the frame image data 21 that is the same as the pixel position 30 on the frame image data 20, that is, a pixel position on the frame image data 21 that has the same horizontal direction coordinate value and vertical direction coordinate value as the pixel position 30, Shown as pixel location 32.

  In the motion search process, the motion generated at the pixel position 30 with the passage of time from the previous frame image data 20 to the current frame image data 20 is detected. In order to detect the movement that has occurred at the pixel position 30, a pixel block 31 that is a small image area centered on the pixel position 30 in the frame image data 20 is considered. In the following description, it is assumed that the pixel blocks at different positions all have the same size.

  The most similar pixel block 35 in the frame image data 21 is searched for the pixel block 31 in the frame image data 20. Specifically, the candidate pixel positions are sequentially moved according to the similarity or dissimilarity between the pixel block that is a small image region centered on the candidate pixel position and the pixel block 31 around the pixel position 30 to be detected. The pixel position with the highest similarity is specified by using the evaluation function while calculating. The pixel position 34 on the frame image data 21 obtained in this way is considered to be the destination of the pixel position 30 on the frame image data 20. Accordingly, the optical flow vector found by the motion search process is a vector from the pixel position 30, that is, the pixel position 32 to the pixel position 34.

  When searching for a pixel block having the highest degree of similarity, if the entire frame image data 21 is used as a search range, it takes time to search. Therefore, a search range is set near the pixel position 30 of the motion detection target, that is, the pixel position 32. It is common to limit. Specifically, a search range 33 that is a rectangular area centered on the pixel position 32 is set, and the similarity or dissimilarity is calculated while sequentially moving candidate pixel positions within the search range 33, and the most similar Specify a high pixel position.

  Note that the first evaluation function affected by the luminance change may be any one of SSD (Sum of Squared Difference) and SAD (Sum of Absolute Difference), for example. The pixel value of each pixel of the pixel block 31 of the previous frame image data 20 is I (i, j), and the pixel value of each pixel of the comparison target pixel block of the current frame image data 21 is T (i, j). ), The SSD is calculated by the following equation.

またＳＡＤは以下の式により計算される。

SAD is calculated by the following equation.

  FIG. 4 is a diagram for explaining a correction process using an evaluation function that is not affected by the luminance change. 4, the same or corresponding elements as those in FIG. 3 are referred to by the same or corresponding numerals, and a description thereof will be omitted as appropriate.

  The pixel position on the frame image data 21 corresponding to the pixel position 30 on the frame image data 20, that is, the pixel position on the frame image data 21 having the same horizontal and vertical coordinates as the pixel position 30 is the pixel position. 40. A pixel block centered on the pixel position 40 is shown as a pixel block 41.

  In the example shown in FIG. 4, the pixel block 31 that is the optimal solution in the frame image data 21 (i.e., the pixel block 31 within the search range is the most) with respect to the pixel block 31 centered on the pixel position 30 in the frame image data 20. A similar pixel block) is a pixel block 43. The pixel block 43 is centered on the pixel position 42. At this time, the pixel position 42 which is the optimum solution obtained by the motion search process is not used as the final motion detection result as it is, but the correction process is executed to correct the motion detection result as necessary.

  Specifically, using the second evaluation function that is not affected by the luminance change, the degree of similarity (or the degree of difference) between the pixel block 31 and the pixel block 43 of the optimal solution is calculated. The similarity (or dissimilarity) between the pixel block 41 of the static solution is calculated. These two similarities (or dissimilarities) are compared with each other, and a solution having a higher similarity (that is, a smaller dissimilarity) is selected.

  The second evaluation function not affected by the luminance change is, for example, any one of ZSAD (Zero-mean SAD), NCC (Normalized Cross-Correlation), and ZNCC (Zero-mean NCC). It's okay. The average pixel value of the pixel block 31 in which the pixel value of each pixel is I (i, j) is / I ("/" is an overline), and the comparison target is that the pixel value of each pixel is T (i, j) If the average pixel value of the pixel block is / T ("/" is an overline), ZSAD is calculated by the following equation.

またＮＣＣは以下の式で計算される。

NCC is calculated by the following equation.

またＺＮＣＣは以下の式で計算される。

ZNCC is calculated by the following formula.

  FIG. 5 is a flowchart showing a procedure of motion search and correction processing. The CPU 10 shown in FIG. 1 may execute each step of the flowchart of FIG. 5 by executing a program of the optical flow calculation process 102 stored in the RAM 12, for example. However, as described above, not the CPU 10 but the GPU 13 may be provided with a function of executing the optical flow calculation process 102. In FIG. 5, SAD is used as the first evaluation function and ZSAD is used as the second evaluation function. However, this is only an example, and the above-described other evaluation functions or other evaluation functions may be used. Good.

  In FIG. 5 and subsequent figures, the execution order of each step described in the flowchart is merely an example, and the technical scope intended by the present application is not limited to the execution order described. For example, even if it is described in the present application that the B step is executed after the A step, it is not only possible to execute the B step after the A step, but also the A step after the B step. It may be physically and logically possible to perform. In this case, if all the results affecting the processing of the flowchart are the same regardless of the order in which the steps are executed, for the purpose of the technique disclosed in the present application, the A step is followed by the B step. It is obvious that may be executed. Even if it is described in the present application that the B step is executed after the A step, it is not intended to exclude the obvious case as described above from the technical scope intended by the present application. The obvious case naturally falls within the technical scope intended by the present application.

  In step S11 of FIG. 5, the CPU 10 determines that both SAD_MIN and XY_OPT are undecided. Here, SAD_MIN is a parameter for storing a calculated value of SAD, and XY_OPT is a parameter for storing coordinate data of a pixel position. In step S12, the CPU 10 determines whether or not the SAD calculation has been completed for the entire search area. If the calculation has not ended, the process proceeds to step S13.

  In step S13, the CPU 10 sets XY_CUR to an uncalculated point in the search area. Here, XY_CUR is a parameter for storing coordinate data indicating candidate pixel positions in the current frame image. In step S14, the CPU 10 obtains SAD between the pixel block centered on XY_PREV and the pixel block centered on XY_CUR. That is, the degree of difference using absolute pixel values is calculated according to the SAD calculation formula described above. Here, XY_PREV is coordinate data indicating the pixel position of the motion detection target in the previous frame image. When calculating the value of SAD, the CPU 10 stores the calculation result in SAD_CUR.

  In step S15, the CPU 10 determines whether or not a condition that SAD_MIN is undecided or SAD_CUR is smaller than SAD_MIN is satisfied. When this condition is satisfied, in step S16, the CPU 10 substitutes the value of SAD_CUR for SAD_MIN, and further substitutes the value of XY_CUR for XY_OPT. Thereafter, the process returns to step S12, and the subsequent processes are repeated. If the above condition is not satisfied in the determination in step S15, the process returns to step S12 without executing step S16, and the subsequent processes are repeated.

  If it is determined in step S12 that the calculation of SAD for the entire search region has been completed, in step S17, the CPU 10 determines whether the coordinate data of the optimal solution stored in XY_OPT may be an error due to a luminance change. In step S18, the CPU 10 changes the subsequent processing depending on whether or not there is a possibility of an error due to a luminance change. If it is determined that there is no possibility of an error due to a change in luminance, the process ends.

  If it is determined in step S18 that there is a possibility of an error due to a luminance change, in step S19, the CPU 10 obtains ZSAD between the pixel block centered on XY_PREV and the pixel block centered on XY_OPT. That is, the dissimilarity using relative pixel values is calculated according to the aforementioned ZSAD calculation formula. When calculating the value of ZSAD, the CPU 10 stores the calculation result in SAD_REL_OPT.

  In step S20, the CPU 10 obtains ZSAD between the pixel block centered on XY_PREV and the pixel block centered on XY_CENTER. Here, XY_CENTER is coordinate data indicating a pixel position in the current frame image, which is equal to XY_PREV, which is coordinate data indicating a pixel position of a motion detection target in the previous frame image. When calculating the value of ZSAD, the CPU 10 stores the calculation result in SAD_REL_CENTER.

  In step S21, the CPU 10 determines whether or not the condition that SAD_REL_CENTER is smaller than SAD_REL_OPT is satisfied. If the condition is not satisfied, the process returns to step S18 and the subsequent processes are repeated. If the condition is satisfied, in step S22, the CPU 10 substitutes the value of XY_CENTER for XY_OPT. The process ends here. The coordinate data stored in XY_OPT is the final optical flow detection result.

  In the process of FIG. 5, the possibility of an error due to a luminance change is determined in step S17, and the correction process using ZSAD is executed only when there is a possibility of an error in the optimal solution obtained by the motion search process. doing. This procedure is only an example, and correction processing using ZSAD may be executed unconditionally for all optimal solutions. When correction processing is executed for all optimal solutions, although it takes time to calculate, an appropriate final motion detection result can be obtained as in the case of the procedure shown in FIG. Even when correction processing is executed for all optimal solutions, the ZSAD calculation needs to be executed only for two points (XY_OPT and XY_CENTER) of the optimal solution and the static solution, so the correction processing is executed. As a result, the additional calculation time is very limited. In the procedure shown in FIG. 5, the correction process using the ZSAD is executed only when there is a possibility of an error in the optimal solution, so that the calculation time required for the correction process can be further shortened.

  FIG. 6 is a flowchart illustrating a procedure for determining whether or not there is a possibility of erroneous detection due to a change in luminance. This process is executed in step S17 of the flowchart of FIG.

  In step S21, the CPU 10 determines whether or not XY_OPT is XY_CENTER. That is, it is determined whether or not the result detected by the motion search process is no motion, that is, whether or not it is a static solution. If it is determined that there is no movement, in step S25, the CPU 10 determines that there is no possibility of an error due to a luminance change.

  If it is determined in step S21 that there is motion, in step S22, the CPU 10 determines whether the distance between XY_OPT and XY_CENTER is equal to or greater than a predetermined threshold. Generally, when a motion is erroneously detected due to a luminance change, the detected motion amount (movement amount) is a relatively small value. Therefore, when the detected amount of motion is equal to or greater than a predetermined threshold, it is appropriate to determine that there is a high possibility that the motion actually exists. Therefore, if the distance is greater than or equal to the predetermined threshold in the determination in step S22, the CPU 10 determines in step S25 that there is no possibility of an error due to a luminance change.

  If the distance is smaller than the predetermined threshold in the determination in step S22, in step S23, the CPU 10 determines whether or not the surrounding pixel values of XY_OPT and XY_CENTER are changing in gradation. In this determination, the pixel value of the pixel existing on or near the straight line connecting XY_OPT and XY_CENTER may be compared with the pixel value obtained by linear interpolation between the XY_OPT pixel value and the XY_CENTER pixel value. . For example, the absolute values of the differences between the pixel values are summed, and when the sum is equal to or less than a predetermined threshold value, it may be determined that the pixel values are changing in a gradation. As described above, since erroneous motion detection due to a luminance change occurs in an area where the pixel value is gradation, the possibility of error detection can be determined according to the determination result. If no gradation pixel value change is detected in the determination in step S23, in step S25, the CPU 10 determines that there is no possibility of an error due to a luminance change.

  When a gradation-like pixel value change is detected in the determination in step S23, in step S24, the CPU 10 determines that there is a possibility of an error due to a luminance change. The process ends here.

  FIG. 7 is a table showing the effect of the correction process. For the 231 test videos, correct answer data is created by manually specifying an object to be detected in advance, and the degree of coincidence between the correct answer data and the detection result data obtained by the process shown in FIG. 5 is calculated. did. In this calculation, the number of correct data that matches the detection result data is TPC, and the number of correct data that does not match the detection result data is FN. The number of detection result data that matches the correct answer data is TPR, and the number of detection result data that does not match the correct answer data is FP. Here, “match” means that there is an overlapping portion (overlapping region) between the correct answer data and the detection result data when they appear.

  As evaluation viewpoints of recognition accuracy, the following two performance indexes are used. The coverage ratio, which is the first performance index, is a correct answer rate when the correct answer data is used as a population, and indicates what percentage of the entire correct answer data the detection result data is correctly output. This coverage is calculated by TPC / (TPC + FN). The accuracy rate, which is the second performance index, is the accuracy rate when the detection result data is a population, and indicates what percentage of the entire detection result data matches the correct data. This accuracy rate is calculated by TPR / (TPR + FP).

  FIG. 7 shows a detection result before applying the correction process (detection result of only the motion search process in steps S11 to S16 in FIG. 5) and a detection result after applying the correction process (motion search process and correction in all the steps in FIG. 5). The detection rate by the processing) and the correct report rate are shown. The coverage rate hardly changes between before and after the application of the correction process. The correct report rate is improved to 78.1% after applying the correction process, compared to 55.6% before applying the correction process.

  FIG. 8 is a flowchart showing a processing procedure for removing the influence of the background motion. In the processing procedure shown in the flowchart of FIG. 5 described above, it is not assumed that the vehicle on which the motion detection device is mounted is moving and the background is moving uniformly due to the movement on the imaging side. However, by modifying the flowchart shown in FIG. 5 by considering the background movement due to the movement on the imaging side, it is possible to execute the correction process even when the imaging side is moving.

  In step S31 of FIG. 8, the CPU 10 acquires the optical flow of the entire screen. Specifically, for example, the optical flow may be calculated by the CPU 10 executing steps S11 to S16 in the procedure shown in FIG.

  FIG. 9 is a diagram for explaining processing for removing the influence of background motion. In the frame image 51 shown in FIG. 9A, the object 50 shows the movement indicated by the motion vector 52 and the like, but the background is the same direction as indicated by the motion vector 53 due to the movement on the imaging side. Are moving at the same speed. The optical flow as shown in FIG. 9A is detected by the process of step S31 of FIG.

  In step S32 of FIG. 8, the CPU 10 acquires the average amount of movement of the background. Specifically, the average of the motion vectors of all the pixels obtained in step S31 may be taken and used as the average amount of movement of the background. Alternatively, the motion vectors of all the pixels obtained in step S31 are averaged, and motion vectors having a magnitude or direction different from the average motion vector by a predetermined threshold or larger are excluded, and the remaining motion vectors are averaged again. Thus, the average movement amount of the background may be obtained. Alternatively, for example, the motion of the vehicle is estimated based on a speedometer, a steering wheel angle, an acceleration sensor, an angular velocity sensor, etc. mounted on the vehicle, and the average of the background is calculated from the estimation result and the motion vector obtained in step S31. The amount of movement may be obtained. FIG. 9B shows an example of the average movement amount of the background calculated in the frame image 51.

  In step S33, the CPU 10 subtracts the optical flow corresponding to the average background movement amount from the optical flow of the entire screen. That is, the motion vector indicating the average amount of movement of the background is subtracted from the motion vector obtained for each pixel. FIG. 9C shows an example of the optical flow remaining after subtracting the optical flow corresponding to the average movement amount of the background in the frame image 51.

  In step S <b> 34, the CPU 10 determines the possibility of an error due to a luminance change based on the optical flow after subtracting the average amount of background movement. Specifically, based on the optical flow after subtracting the average amount of movement of the background, the CPU 10 may execute the process shown in step S17 of FIG. Thereafter, by executing the processes shown in steps S18 to S22 of FIG. 5, an appropriate correction process can be realized.

  As described above, when the background is moving uniformly, the average movement amount of the background is obtained, and the obtained average movement amount is subtracted from the optical flow of each pixel to eliminate the influence of the background movement. Can do. Alternatively, the calculation of steps S20 and S21 may be executed using the coordinate position of “XY_PREV + average movement amount” instead of the coordinate position indicated by XY_CENTER. In this way, by removing the influence of the background movement according to the movement on the imaging side, using the second evaluation function not affected by the luminance change, the optimum solution and the stationary solution where the object is stationary You can choose the best solution.

  10 to 12 are diagrams for explaining patterns with different background movements. In the example illustrated in FIG. 10A, the vehicle 60 is traveling straight in the direction indicated by the arrow 61, and the imaging device connected to the motion detection device captures an area indicated by the imaging field of view 62. The imaging field of view 62 is oriented in a direction perpendicular to the direction in which the vehicle 60 travels straight. In this case, as shown in FIG. 10B, a movement occurs in which the entire screen moves in the same direction at substantially the same speed. By calculating the average movement amount of the background in the same manner as described with reference to FIG. 9, it is possible to remove the influence of the movement of the background according to the movement on the imaging side. However, since the moving speed varies depending on the distance to the object in the imaging field of view 62, it is desirable to devise methods such as obtaining a local average moving amount in a small area obtained by dividing the entire screen.

  In the example illustrated in FIG. 11A, the vehicle 60 is traveling straight in the direction indicated by the arrow 61, and the imaging device connected to the motion detection device captures an area indicated by the imaging field 63. The imaging visual field 63 faces in the same direction as the direction in which the vehicle 60 goes straight. In this case, as shown in FIG. 11B, motion vectors that radiate from the center of the screen are generated. For example, the motion of the vehicle is estimated based on a speedometer, a handle angle, an acceleration sensor, an angular velocity sensor, etc. mounted on the vehicle, and the imaging device in each part of the background is estimated from the estimation result and the motion vector obtained from the image You may obtain | require the movement amount derived from a motion. By calculating the movement amount of the background thus obtained, it is possible to remove the influence of the background movement according to the movement on the imaging side. Also in this case, since the moving speed varies depending on the distance to the object in the imaging field of view 63, it is desirable to devise such as obtaining a local average moving amount within a small area obtained by dividing the entire screen.

  In the example shown in FIG. 12A, the vehicle 60 is traveling so as to turn right as indicated by an arrow 64, and the imaging device connected to the motion detection device captures an area indicated by the imaging visual field 65. ing. The imaging visual field 65 faces the left side from the direction in which the vehicle 60 travels. In this case, as shown in FIG. 12B, the entire screen moves to the left side, but the moving direction and moving distance depend on the position on the screen. For example, the motion of the vehicle is estimated based on a speedometer, a handle angle, an acceleration sensor, an angular velocity sensor, etc. mounted on the vehicle, and the imaging device in each part of the background is estimated from the estimation result and the motion vector obtained from the image. You may obtain | require the movement amount derived from a motion. By calculating the movement amount of the background thus obtained, it is possible to remove the influence of the background movement according to the movement on the imaging side. Also in this case, since the moving speed varies depending on the distance to the object in the imaging field of view 65, it is desirable to devise such as obtaining a local average moving amount in a small area obtained by dividing the entire screen.

  As mentioned above, although this invention was demonstrated based on the Example, this invention is not limited to the said Example, A various deformation | transformation is possible within the range as described in a claim.

10 CPU
11 ROM
12 RAM
13 GPU
14 Display Interface 15 Video Input Interface 20 Previous Frame Image Data 21 Current Frame Image Data 33 Search Range

Claims (7)

The optimal solution of the optical flow is obtained by performing motion search using the first evaluation function affected by the luminance change,
A motion detection method for selecting the optimum solution of the optimum solution and the stationary solution in which the object is stationary by using a second evaluation function that is not affected by a change in luminance. .   The method further includes the step of determining whether or not there is a possibility that an erroneous optical flow has been detected due to a change in luminance, and the step of selecting includes selecting the optimum determined to be possible by the step of determining. The motion detection method according to claim 1, wherein only the solution is selected using the second evaluation function to select the optimum solution.   The motion detection method according to claim 2, wherein the determining step determines that the possibility does not exist when a distance between the optimal solution and the stationary solution is equal to or greater than a threshold value.   4. The motion detection method according to claim 2, wherein the determining step determines that there is no possibility when a pixel value change is not gradation around the optimum solution and the stationary solution. 5.   The first evaluation function is one of SSD (Sum of Squared Difference) and SAD (Sum of Absolute Difference), and the second evaluation function is ZSAD (Zero-mean SAD), NCC (Normalized). The motion detection method according to any one of claims 1 to 4, wherein the motion detection method is one of Cross-Correlation) and ZNCC (Zero-mean NCC). A processor;
Memory,
And when the processor executes a program stored in the memory,
The optimal solution of the optical flow is obtained by performing motion search using the first evaluation function affected by the luminance change,
A motion detection apparatus that executes each step of selecting an optimum solution of the optimum solution and a stationary solution in which an object is stationary by using a second evaluation function that is not affected by a luminance change. The optimal solution of the optical flow is obtained by performing motion search using the first evaluation function affected by the luminance change,
A motion detection program including each step of selecting an optimum solution of the optimum solution and a stationary solution in which an object is stationary by using a second evaluation function that is not affected by a luminance change.

Images