JP5762250B2 - Image signal processing apparatus and image signal processing method - Google Patents

Description

The present invention relates to an image signal processing apparatus and an image signal processing method for detecting motion of an object using a plurality of images acquired at different times.

Conventionally, a monitoring system that detects intruders by analyzing captured images has been used, and the detection of objects in this monitoring system is based on changes in the brightness of the acquired image and other data Is used.
Specifically, the captured image frame is divided into a plurality of blocks, the average luminance value of each block and the average luminance value of the entire frame are calculated, and the average luminance value of each block and the entire frame of the current frame and the previous frame are calculated. A motion detection device that detects a block including motion by performing a difference calculation using an average luminance value and comparing a value obtained by the difference calculation with a preset threshold value is disclosed (see Patent Document 1). .

  In addition, Patent Document 2 calculates a luminance average value for each block of a background frame that is a past frame with respect to the motion detection target frame and the motion detection target frame, and calculates a luminance average value of the background frame for each block. And a motion detection device that determines whether or not there is motion in the motion detection target frame based on the difference between the average luminance value for each block of the motion detection target frame and a value obtained by subtracting the correction value.

JP 2004-157799 A JP 2008-257626 A

  In the prior art of Patent Document 1, when two photographed image frames are divided into a plurality of blocks, and the luminance change of the block at the same position is equal to or greater than a threshold, the block is a block including motion. Judgment. For this reason, there is a problem that even if the object does not move due to only a change in illumination in the two image frames, it may be erroneously determined as a block including movement.

  Further, in the prior art of Patent Document 2, for example, when an object is moving in the entire area of the screen, that is, when the entire screen is moving, or a block in which motion is detected in a motion detection target frame, a luminance change occurs. In this case, there is a problem in that the luminance change in the block in which the motion is detected is not reflected in the correction value and causes erroneous determination. In addition, there is a problem that erroneous determination is caused even when the correction value does not represent the luminance difference between the motion detection target frame and the background frame.

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an image signal processing device and an image signal processing method that suppress erroneous determination of object motion detection.

Image signal processing apparatus according to the present invention, reference storage means for storing the image data to be input to the current frame, the image data before the current frame stored predetermined time in the storage means the input image data Obtained as a frame, divides the current frame and the reference frame into a plurality of image blocks, and extracts the luminance average value for each divided image block, and refers to the luminance average value extracted by the image information extraction unit If the difference between the average luminance value of the image block of the current frame and the average luminance value of the image block of the reference frame located at the same point as the image block of the current frame is equal to or greater than the threshold value , the image block of the current frame And the amount of change in luminance due to the change in the brightness of the image data. An offset calculation means for calculating an offset value for subtraction, and a pixel value of an image block of the current frame and an image block of a reference frame at the same position as the image block of the current frame in consideration of the offset value calculated by the offset calculation means of a judgment whether the difference value is greater than or equal to the threshold of the pixel value, if the difference value of the pixel values of the image block of the reference frame and the current frame is greater than or equal to the threshold, the object having a motion in the image block Motion detection determination means for detecting the presence of the.

  According to the present invention, it is possible to suppress erroneous determination of motion detection of an object.

1 is a block diagram illustrating a configuration of an image signal processing device according to Embodiment 1. FIG. 6 is a diagram showing image frame division processing of the image signal processing apparatus according to Embodiment 1. FIG. 3 is a flowchart showing the operation of the image signal processing apparatus according to the first embodiment. 6 is an explanatory diagram illustrating a display result of the image signal processing device according to Embodiment 1. FIG. FIG. 10 is an explanatory diagram showing a display result of the image signal processing device according to the second embodiment. FIG. 10 is a block diagram illustrating a configuration of an image signal processing device according to a third embodiment. 10 is a flowchart illustrating an operation of the image signal processing device according to the third embodiment. FIG. 10 is an explanatory diagram illustrating a display result of an image signal processing device according to a third embodiment. FIG. 10 is a block diagram illustrating a configuration of an image signal processing device according to a fourth embodiment. 10 is a flowchart showing an operation of the image signal processing apparatus according to the fourth embodiment. FIG. 10 is an explanatory diagram illustrating a display result of an image signal processing device according to a fourth embodiment.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of an image signal processing apparatus according to Embodiment 1 of the present invention.
The image signal processing apparatus 10 according to the first embodiment includes a memory unit 1, an image information extraction unit 2, an offset calculation unit 3, a motion detection determination unit 4, and a display control unit 5. Further, a camera 20 is connected to the image signal processing apparatus 10, and image data captured by the camera 20 is input to the image signal processing apparatus 10. In the first embodiment, assuming that the image signal processing apparatus 10 is applied to a monitoring system or the like, detection of a moving object will be described as an example. Therefore, the image signal processing apparatus 10 is connected to a camera 20 that captures a predetermined area, and the captured image data is input to the image processing apparatus 10. In the following description, real-time image frame data input from the camera 20 is referred to as a current frame, and image frame data input from the camera 20 at different times before the current frame is referred to as a reference frame.

  The memory unit 1 is a storage area for accumulating image data input from the camera 20, and accumulates the above-described current frame and a reference frame for a period that is a predetermined time back from the current frame. When a current frame as image data is input from the camera 20, the image information extraction unit 2 acquires a reference frame from the memory unit 1 and divides the image frame of the current frame and the reference frame into image blocks of a predetermined size. Further, image feature information such as a luminance average value and contour information is extracted for each divided image block. This is a process for detecting a change in luminance of an object that causes erroneous motion detection, that is, a luminance of the object and a feature of the image of the object, and suppressing erroneous motion detection. Here, the contour information is extracted by applying an edge detection filter to each image block constituting the current frame and the reference frame. The edge detection filter can be appropriately configured with a Sobel filter or other filters, and does not limit the present invention.

The division of the image frame of the image information extraction unit 2 will be described with reference to FIG. FIG. 2 is an explanatory diagram showing image frame division processing by the image information extraction unit of the image signal processing apparatus according to Embodiment 1 of the present invention.
The image information extraction unit 2 divides the image frame in the horizontal direction and the vertical direction based on a preset number of divisions. In the example of FIG. 2, the image frame 30 is divided so as to have the horizontal block number bh in the horizontal direction and the vertical block number bv in the vertical direction. Each of the divided blocks constitutes an image block 40.

  The offset calculation unit 3 refers to the luminance average value for each image block extracted by the image information extraction unit 2, and calculates the luminance average value difference between the same block in the current frame and the reference frame. The calculated brightness average value difference is compared with a preset threshold value, and it is determined whether or not the brightness of the image has changed. When it is determined that there is no change in the luminance of all the image blocks, the luminance value offset for motion detection is set to “0”, and when it is determined that there is a change in the luminance of any of the image blocks, A luminance value offset for detection is calculated. The motion detection luminance value offset is a value for subtracting the amount of change in the luminance value due to the change in screen brightness. By calculating the motion detection luminance value offset, the luminance value based only on the motion within the screen. Can be calculated. The luminance value offset for motion detection can be variously set such as the average value or the median value of the luminance average value differences of blocks in one frame between the current frame and the reference frame.

The motion detection determination unit 4 calculates the pixel difference of the image block at the same position in the current frame that is the image data input from the camera 20 and the reference frame acquired from the memory unit 1 and compares it with a threshold value. Perform motion detection. Specifically, after adding the motion detection luminance value offset calculated by the offset calculation unit 3 to one of the current frame and the reference frame having a smaller average luminance value, the same position in the current frame and the reference frame is added. The pixel difference of the image block is calculated, and when the calculated pixel difference value is equal to or greater than the threshold value, it is determined that there is a moving object in the image block. Note that the motion detection luminance value offset may be subtracted from one frame having a larger average luminance value.
For the image block whose pixel difference value is determined to be less than the threshold, the motion detection determination unit 4 further refers to the contour information extracted by the image information extraction unit 2 for the image block whose change in contour information is equal to or greater than the threshold. It is also determined that there is an object having movement.

  The display control unit 5 refers to the detection result input from the motion detection determination unit 4, and when there is an object having motion in the current frame, the display control unit 5 moves to each image block including the object. A detection mark or the like is displayed, and display control is performed so that a user can recognize a moving object.

Next, the operation of the image signal processing apparatus 10 will be described.
FIG. 3 is a flowchart showing the operation of the image signal processing apparatus according to Embodiment 1 of the present invention.
When image data is input from the camera 20 (step ST1), the image information extraction unit 2 divides the current frame that is the input image data and the reference frame stored in the memory unit 1 by a preset number of divisions. Then, image feature information such as a luminance average value and contour information is calculated for each image block of each frame as a plurality of image blocks (step ST2). The offset calculation unit 3 refers to the luminance average value of the image feature information calculated in step ST2, calculates the luminance average value difference of the image block at the same position in the current frame and the reference frame, and the calculated luminance average value difference is It is determined whether the threshold value is equal to or greater than a preset threshold value (step ST3). Here, the image block at the same position is an image block having the same horizontal position and vertical position in the current frame and the reference frame.

  When it is determined that the luminance average value difference is less than the threshold value in all the image blocks (step ST3; NO), the offset calculation unit 3 determines that there is no change in luminance between the current frame and the reference frame, and moves The luminance value offset for detection is set to “0” (step ST4), and the process proceeds to step ST6 described later. On the other hand, when it is determined that there is an image block whose luminance average value difference is equal to or greater than the threshold (step ST3; YES), the offset calculation unit 4 determines that there is a change in luminance between the current frame and the reference frame, A motion detection luminance value offset is calculated (step ST5).

  The motion detection determination unit 4 adds the motion detection luminance value offset calculated in step ST5 to the image of either the current frame or the reference frame, and the pixel of the image block at the same position in the current frame and the reference frame The difference is calculated (step ST6). Further, it is determined whether the pixel difference value calculated in step ST6 is equal to or greater than a preset threshold value (step ST7). When it is determined that the pixel difference value is equal to or greater than the threshold value (step ST7; YES), it is determined that an object having motion between the current frame and the reference frame is detected (step ST8).

  On the other hand, when it is determined that the pixel difference value is less than the threshold value (step ST7; NO), the contour extracted by the image information extraction unit 2 in step ST2 for the image block whose pixel difference value is determined to be less than the threshold value. The information is acquired, the difference sum between the current frame and the reference frame of the acquired contour information is calculated, and it is determined whether or not it is equal to or greater than a preset threshold value (step ST9). For example, when the contour information is “contour pixel (1)” or “non-contour pixel (0)” for each pixel, the pixel difference absolute value is obtained at the same pixel position in the current frame and the reference frame. Calculate the sum of absolute differences. If it is determined that the difference sum of the contour information is greater than or equal to the threshold (step ST9; YES), it is determined that there is a change in the contour information, and the process proceeds to step ST8, where the object is between the current frame and the reference frame. It is determined that the motion detection has been performed. In this embodiment, the contour information is “contour pixel (1)” or “non-contour pixel (0)” for each pixel. However, the contour information is not limited to binary values, but has 5 levels, 10 levels, and the like. It may be multi-value information and does not limit the present invention.

  On the other hand, when it is determined that the difference sum of the contour information is less than the threshold value (step ST9; NO), the contour information does not change, that is, the motion of the object is not detected between the current frame and the reference frame. Is determined (step ST10). The motion detection determination unit 4 determines whether or not processing has been performed for all image blocks constituting the current frame (step ST11), and when it is determined that processing has not been performed for all image blocks (step ST11; If NO, the process returns to step ST6.

  On the other hand, when it is determined that all image blocks have been processed (step ST11; YES), the detection result is output to the display control unit 5 (step ST12). The display control unit 5 refers to the input detection result, performs display control to display a motion detection mark on the image block of the current frame determined to be motion detection (step ST13), and ends the process.

  Note that in step ST5, it is determined that there is a change in luminance when there is no movement of the object between the current frame and the reference frame, but the brightness of the frame changes (case 1), and depending on the movement of the object. There is a case (Case 2) where the luminance average value difference of the image block at the same position of the current frame and the reference frame is equal to or greater than a threshold value.

  However, Case 1 and Case 2 have different processing results in Step ST6. In case 1, the pixel difference value calculated in step ST6 is small. This is because a pixel change between image blocks at the same position in the current frame and the reference frame corresponds to an offset which is a difference in screen brightness between frames. On the other hand, in case 2, the pixel difference value calculated in step ST6 is generally not a small value. This is different from the case where the screen brightness between frames changes, and the pixel change between image blocks at the same position in the same position of the current frame and the reference frame is not necessarily the difference in screen brightness between frames. Because. Therefore, by calculating the pixel difference value in step ST6 and determining whether the pixel difference value is greater than or equal to a predetermined threshold value in step ST7, case 1 and case 2 can be accurately distinguished.

FIG. 4 is an explanatory view showing a display result of the image signal processing apparatus according to Embodiment 1 of the present invention. 4A shows the reference frame, and FIG. 4B shows the current frame.
The reference frame 200 shown in FIG. 4 (a) and the current frame 100 shown in FIG. 4 (b) are both composed of a plurality of image blocks divided by the number of horizontal blocks bh and the number of vertical blocks bv. 50 is reflected. As shown in FIG. 4, when it is determined that the position of the target object 50 is different between the reference frame 200 and the current frame 100 and the target object 50 is moving, each block including the target object 50 in the current frame 100 is assigned to each block. On the other hand, the motion detection mark 60 is displayed. Note that there are cases where the object 50 is not shown in the reference frame 200 and / or the current frame 100, and the presence or absence of the object 50 does not affect the present invention.

  As described above, according to the first embodiment, the occurrence of a luminance change is determined based on the difference between the average luminance values of the image blocks at the same position in the current frame and the reference frame. An offset calculation unit that calculates a motion detection luminance value offset for subtracting the amount of change in luminance due to a change in data brightness, and an image block at the same position in the current frame and the reference frame in consideration of the calculated offset value And a motion detection determination unit that detects that there is an object having motion in the image block when the difference between the pixel values is equal to or greater than a threshold value. Thus, motion detection for each image block can be performed, and erroneous determination of motion detection of the object can be suppressed. In addition, it is possible to determine the movement of the object by subtracting the luminance change due to the change in external light or illumination, and it is possible to suppress erroneous determination of the movement detection. Specifically, it is possible to suppress erroneous determination that an object is moving when the object is not moving only due to changes in external light or illumination.

  Further, according to the first embodiment, when the motion detection determination unit determines that the difference between the pixel values of the image blocks at the same position in the current frame and the reference frame is less than the threshold, the contour information is further changed. Since motion detection is determined based on this, motion detection for each image block can be performed by finer condition division in consideration of contour information, and erroneous determination of motion detection can be suppressed. Specifically, even when the subject is moving but the luminance change is small, it is possible to perform motion detection in consideration of the contour information, and to detect an object having motion more accurately.

Embodiment 2. FIG.
In the above-described first embodiment, the configuration in which the image frame data input from the camera 20 at different times before the current frame is used as the reference frame has been described. However, in this second embodiment, the background in which the object is not shown. The structure which uses only image frame data as a background reference frame is shown.
The constituent elements of the image signal processing apparatus 10 according to the second embodiment are the same as those in the first embodiment, and therefore, the block diagram of the image signal processing apparatus 10 is omitted and the reference numerals used in the first embodiment are the same. The description will be made using the same reference numerals.

  The camera 20 captures only the background in which the object is not captured, and stores the image frame data of only the background in the memory unit 1 as a background reference frame. The background reference data is not limited to one type, and a plurality of different background reference frames may be prepared and stored in the memory unit 1 depending on the shooting conditions.

The image signal processing apparatus 10 detects a moving object based on the current frame and the background reference frame. Note that the motion detection process is the same as that of the first embodiment except that the reference frame is changed to the background reference frame, and thus the description thereof is omitted.
FIG. 5 is an explanatory diagram showing a display result of the image signal processing apparatus 10 according to the second embodiment of the present invention. FIG. 5A shows the background reference frame, and FIG. 5B shows the current frame.
As shown in FIG. 5, by using the background reference frame 300 in which the object is not shown, the object 50 can be detected more easily than in the case of using the reference frame in which the object is shown. This is because, in the current frame 100 and the background reference frame 300, when a change in outside light or illumination is excluded, by taking a pixel difference between the current frame 100 and the background reference frame 300, an area with a small difference value is a background area, This is because a region having a large difference value is easily determined as a region where the object 50 exists.

  As described above, according to the second embodiment, whether or not the difference between the pixel values of the image block at the same position as the current frame is greater than or equal to the threshold using the background reference frame obtained by capturing the background where the object is not captured. Since it comprised so that it might determine, the image block in which a target object exists can be detected easily.

Embodiment 3 FIG.
In the first embodiment described above, it is determined whether or not there is a luminance change based on the luminance average value difference between the image blocks at the same position in the current frame and the reference frame. High-precision motion detection is performed by detecting a moving object in consideration of a luminance value offset, and by detecting a moving object in consideration of changes in contour information when motion is not detected. Realized. On the other hand, in the configuration of the first embodiment, when the moving object is stationary, it is difficult for the user to recognize that the moving object has existed on the screen. Therefore, in the third embodiment, a configuration is shown in which a monitoring target is clarified by attaching a marker to an image block whose motion has been detected for a predetermined time.

FIG. 6 is a block diagram showing the configuration of the image signal processing apparatus according to the third embodiment.
The motion detection determination unit 4 includes a flag setting unit 6 and a flag determination unit 7 in addition to the configuration shown in the first embodiment. In addition to the configuration shown in the first embodiment, the memory unit 1 stores information necessary when the flag determination unit 7 performs the determination process.

  The flag setting unit 6 sets the motion detection flag of the detected image block to “ON” when the motion detection is determined as the processing of the motion detection determination unit 4, and sets the motion detection flag and flag determination time thus set. , And the position of the image block are output to the memory unit 1 and stored. Here, the flag determination time indicates the time after the motion detection flag is “ON”, and the flag determination time of each image block is initialized in advance (for example, the initialization time is set to “0”). The elapsed time from when the motion detection flag is “ON” is added and stored on the memory unit 1 side.

  The memory unit 1 stores a motion detection flag, a flag determination time, and an image block position input from the flag setting unit 6. Further, the memory unit 1 has a frame storage time indicating how long an image frame is accumulated. When image data is input from the camera 20, the memory unit 1 assigns a frame storage time to the input image frame.

  The flag determination unit 7 refers to the memory unit 1 when the motion non-detection is determined as the process of the motion detection determination unit 4, and refers to the image block at the same position as the image block determined to be the non-motion detection. It is determined whether or not the motion detection flag is “ON”. If the motion detection flag is “ON”, the image block is in a static state or the normal motion non-detection information described in the first embodiment is used. Determine if there is any. The flag determination unit 7 has a flag determination threshold value in advance, and corresponds by comparing the difference between the frame recording time stored in the memory unit 1 and the flag determination time with the stored flag determination threshold value. It is determined whether the object exists in the image block but is in a stationary state, or whether the object does not exist and is in a normal motion non-detection state.

  The display control unit 5 refers to the detection results input from the motion detection determination unit 4 and the flag determination unit 7 and displays a motion detection mark when an object having motion is detected in the current frame. When a stationary object is detected in the frame, a stop detection mark is displayed, and display control is performed so that the user can recognize the movement or stop of the object.

FIG. 7 is a flowchart showing the operation of the image signal processing apparatus according to the third embodiment. The same steps as those in the image signal processing apparatus according to the first embodiment are denoted by the same reference numerals as those used in FIG. 3, and the description thereof is omitted or simplified.
First, from step ST1 to step ST10, processing similar to that of the first embodiment is performed. Thereafter, when it is determined in step ST8 that motion detection has been performed between the current frame and the reference frame, the flag setting unit 6 sets the motion detection flag of the detected image block to “ON” (step ST21). . Further, the set motion detection flag, the initialized flag determination time, and the position information of the image block are output to the memory unit 1 and stored (step ST22). Thereafter, the flow proceeds to the process of step ST11.

  On the other hand, when it is determined in step ST10 that motion is not detected between the current frame and the reference frame, the flag determination unit 7 refers to the memory unit 1 and determines the current frame determined to be motion non-detected. It is determined whether the motion detection flag of the image block of the reference frame at the same position as the image block is “ON” (step ST23).

  When it is determined that the motion detection flag is “ON” (step ST23; YES), the flag determination unit 7 further refers to the memory unit 1, and a flag determination in which the difference between the frame storage time and the flag determination time is set in advance. It is determined whether it is below the time threshold (step ST24). For example, when the frame recording time of the current frame is set to “100”, the flag determination time of the corresponding image block is “90”, and the flag determination time threshold is “20”, the frame storage time and the flag determination time Is less than the threshold value (100−90 ≦ 20) and satisfies the determination condition (step ST24; YES), it is determined that the object is stationary (step ST25), and the process proceeds to step T11.

  On the other hand, when it is determined that the motion detection flag is not “ON” (step ST23; NO) and in the determination process of step ST24, for example, the frame storage time of the current frame is set to “100”, and the corresponding image block When the flag determination time is “70” and the flag determination time threshold is “20”, the difference between the frame recording time and the flag determination time is larger than the threshold (100−70> 20), and the determination condition is not satisfied. (Step ST24; NO), it is determined that the normal motion is not detected (Step ST26), and the process proceeds to Step ST11.

  The motion detection determination unit 4 determines whether or not processing has been performed for all image blocks constituting the current frame (step ST11), and when it is determined that processing has not been performed for all image blocks (step ST11; If NO, the process returns to step ST6. On the other hand, when it is determined that processing has been performed for all image blocks (step ST11; YES), the detection result is output to the display control unit 5 (step ST27). The output detection result differs for each image block, and is a corresponding determination result among the motion detection determination result in step ST8, the determination result of the stationary state in step ST25, and the determination result of normal motion non-detection in step ST26.

  The display control unit 5 refers to the input detection result, displays a motion detection mark on the image block determined to be motion detection, displays a stop detection mark on the image block determined to be stationary, Any display control is performed so as not to display the mark on the image block determined to be detected (step ST28), and the process ends.

  FIG. 8 is an explanatory diagram showing a display result of the image signal processing apparatus according to the third embodiment. FIG. 8A shows an example in which a motion detection mark 60 is displayed in each image block where the object 50 is located in the current frame 100, and FIG. In this example, the stop detection mark 70 is displayed for a predetermined time in each image block where the target object 50 is located. The stop detection mark 70 is displayed and distinguished from a mark different in shape or color from the motion detection mark 60.

  As described above, according to the third embodiment, the flag setting unit 6 that sets the motion detection flag of an image block for which motion detection has been determined to “ON”, and the image of the current frame that has been determined to have no motion detection It is determined whether the motion detection flag of the image block of the reference frame at the same position as the block is “ON”, and the difference between the frame storage time and the flag determination time is compared with a preset flag determination time threshold value. The flag determining unit 7 that determines that the object is in a stationary state, the image block that stops moving, that is, the image block in which the object is in a stationary state is clearly indicated. The object which is not moving can be clarified.

Embodiment 4 FIG.
In the fourth embodiment, in addition to the configuration of the first embodiment described above, a configuration is shown in which image blocks having the same motion among the detected image blocks are grouped and displayed.
FIG. 9 is a block diagram showing the configuration of the image signal processing apparatus according to the fourth embodiment.
A grouping processing unit 8 is additionally provided in the display control unit 5.

  The motion detection determination unit 4 compares the current frame with the reference frame for the motion-detected image block, calculates a motion vector, combines the motion detection result and the motion vector, and outputs the result to the display control unit 5. . The grouping processing unit 8 refers to the detection result input from the motion detection determination unit 4 and groups image blocks having the same motion vector among a plurality of image blocks whose motion has been detected. Note that grouping of image blocks may be performed on image blocks having the same motion vector, or may be performed on image blocks having a motion vector within a preset range, and motion vectors used for grouping. The range can be set as appropriate. The display control unit 5 displays grouping blocks and the like on image blocks belonging to the same group based on the grouping result of the grouping processing unit 8.

FIG. 10 is a flowchart showing the operation of the display control unit of the image signal processing apparatus according to the fourth embodiment. In the flowchart of FIG. 10, an example in which image blocks having the same motion vector are grouped will be described.
The grouping processing unit 8 of the display control unit 5 refers to the detection result input from the motion detection determination unit 4 and determines whether there are a plurality of motion detected image blocks (step ST31). If it is determined that there are a plurality of images (step ST31; YES), it is determined whether there are image blocks having the same motion vector among the plurality of image blocks (step ST32).

  When it is determined that there are image blocks having the same motion vector (step ST32; YES), the image blocks having the same motion vector are grouped (step ST33). The display control unit 5 refers to the grouping result in step ST33, performs display control for enclosing image blocks belonging to the same group with grouping blocks (step ST34), and ends the process. On the other hand, when it is determined that there are not a plurality of motion-detected image blocks (step ST31; NO), or when it is determined that there is no image block having the same motion vector (step ST32; NO), grouping processing Exit without doing.

  FIG. 11 is an explanatory diagram illustrating a display result of the image signal processing device according to the fourth embodiment. FIG. 11A shows a plurality of objects 50a and 50b in the current frame 100, a motion detection mark 60a is displayed on each image block where the object 50a is located, and each image block where the object 50b is located. The case where the motion detection mark 60b is displayed is shown as an example. FIG. 11B shows the current frame 100 when the objects 50a and 50b shown in FIG. 11A move, and a plurality of image blocks in which the object 50a is located and have the same motion vector 80a. Is displayed in a grouping block 90a, and a plurality of image blocks having the same motion vector 80b and an object 50b are displayed in a grouping block 90b. In the example of FIG. 11B, movement vectors 80a and 80b are also displayed. By performing the display of FIG. 11B, the object and the moving direction of the object can be clarified.

  As described above, in the fourth embodiment, the motion detection determination unit 4 that calculates the motion vector by comparing the current frame with the reference frame, and the image having the same motion vector by referring to the motion vector of the image block Since it is configured to include a grouping processing unit 8 for grouping blocks, each object can be clarified even when there are a plurality of objects in the frame, and the moving direction of each object can be determined. Can be recognized.

  Further, in the above-described fourth embodiment, the description has been given of adding the configuration for performing the grouping process to the image signal processing apparatus of the first embodiment. However, the fourth embodiment is additionally provided in the image signal processing apparatus of the third embodiment. You may comprise as follows.

  In Embodiments 1 to 4 described above, the configuration in which the motion detection mark or the stop detection mark is displayed on all the image blocks where the object is located is shown. However, the four corners of the object and the outer frame of the object are shown. The display may be displayed only in the center of the object, and the display method can be changed as appropriate. Moreover, although the example which displays a motion detection mark and a stop detection mark with a square or a circle was shown, a display shape and further a display color can be freely set.

  In the first to fourth embodiments described above, the image signal processing apparatus has been described as an example, but the present invention may be applied to an image signal processing method and a computer program.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Memory part, 2 Image information extraction part, 3 Offset calculation part, 4 Motion detection determination part, 5 Display control part, 6 Flag setting part, 7 Flag determination part, 8 Grouping process part, 10 Image signal processing apparatus, 20 Camera, 30 image frame, 40 image block, 50, 50a, 50b object, 60 motion detection mark, 70 stop detection mark, 80a, 80b motion vector, 90a, 90b grouping block, 100 current frame, 200 reference frame, 300 background reference frame .

Claims (8)

Storage means for storing input image data;
The input image data is the current frame, the image data of a predetermined time before the current frame stored in the storage means is acquired as a reference frame, the current frame and the reference frame are divided into a plurality of image blocks, Image information extracting means for extracting a luminance average value for each divided image block;
With reference to the average brightness value extracted by the image information extraction means, the average brightness value of the image block of the current frame, and the average brightness value of the image block of the reference frame located at the same point as the image block of the current frame, If the difference value is equal to or greater than a threshold value, it is determined that a luminance change has occurred between the image block of the current frame and the image block of the reference frame, and a luminance change due to a change in the brightness of the image data Offset calculating means for calculating an offset value for subtracting the change amount of
In consideration of the offset value calculated by the offset calculating means, a difference value between the pixel value of the image block of the current frame and the pixel value of the image block of the reference frame at the same position as the image block of the current frame is a threshold value. The motion detection is performed to determine whether or not there is an object having motion in the image block when a difference value between pixel values of the image block of the current frame and the reference frame is equal to or greater than a threshold value. An image signal processing apparatus comprising a determination unit. The image information extraction means extracts contour information for each of the divided image blocks,
The motion detection determination means refers to the contour information extracted by the image information extraction means when the difference value between the pixel values of the image blocks of the current frame and the reference frame is less than a threshold, and It is determined whether the amount of change in the contour information of the image block of the reference frame is greater than or equal to a threshold, and when the amount of change in the contour information of the image block in the current frame and the reference frame is greater than or equal to the threshold, the image block The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus detects that there is an object having movement in the object. A detection flag is set in the image block of the current frame that is detected as an object having motion in the motion detection determination means, and the set detection flag and the position information of the image block in which the detection flag is set are stored in the storage means. Flag setting means to be stored in
The detection flag of the image block at the same position as the image block of the current frame is set with reference to the storage unit for the image block of the current frame detected by the motion detection determination unit that there is no moving object. Flag determining means for determining whether or not a stationary object exists in the image block of the current frame when the detection flag is set. The image signal processing apparatus according to claim 1 or 2, wherein The storage means stores an image frame storage time and an elapsed time since the detection flag is set,
The flag determination means applies an image block of the current frame detected that there is no object having the motion when a difference value between the storage time of the frame and the elapsed time is equal to or less than a preset threshold value. The image signal processing apparatus according to claim 3, wherein it is determined that there is a stationary object.   5. The display control unit according to claim 1, further comprising: a display control unit configured to perform display indicating the presence of the target object in an image block detected by the motion detection determination unit as having a target object having a motion. The image signal processing apparatus of any one of them. The motion detection determination means displays an image block indicating that there is a moving object, and the flag determination means determines that there is a stationary object in the image block. 5. The image signal processing apparatus according to claim 3, further comprising display control means for performing display indicating that the object is stopped on the image block. The display control means includes grouping processing means for grouping a plurality of image blocks,
The motion detection determination means calculates a motion vector of an image block determined by comparing the current frame with the reference frame and determining that an object having the motion exists.
The grouping processing unit refers to the motion vector calculated by the motion detection determination unit, groups a plurality of image blocks having motion vectors in substantially the same vector direction,
7. The image signal processing apparatus according to claim 5, wherein the display control means displays that the image blocks grouped by the grouping processing means constitute the same object. A storage step for accumulating the input image data;
Input image data is a current frame, image data a predetermined time before the current frame accumulated in the storing step is acquired as a reference frame, and the current frame and the reference frame are divided into a plurality of image blocks. An image information extraction step for extracting a luminance average value for each divided image block;
With reference to the luminance average value extracted in the image information extraction step, the luminance average value of the image block of the current frame and the luminance average value of the image block of the reference frame located at the same point as the image block of the current frame If the difference value between the current frame image block and the reference frame image block is determined to have changed in brightness, the brightness due to the brightness change in the image data is determined. An offset calculating step for calculating an offset value for subtracting the change amount of the change;
In consideration of the offset value calculated in the offset calculating step, the difference value between the pixel value of the image block of the current frame and the pixel value of the image block of the reference frame at the same position as the image block of the current frame is A motion that determines whether or not there is an object having motion in the image block when a difference value between pixel values of the image block of the current frame and the reference frame is equal to or greater than the threshold value. An image signal processing method comprising a detection determination step.

Images