JP6083695B2 - Motion detection system - Google Patents

Description

  The present invention relates to an operation detection system. More specifically, the present invention relates to an operation detection system used for detecting the operation of a driver while driving an automobile or the like.

  As a cause of accidents such as automobiles, driving aside sees a large percentage. Causes of driving aside are those related to driving, such as those that are distracted by the surroundings (watching the surroundings), loading and unloading the dashboard, browsing the map, operating audio equipment, etc. There are things that are caused by not working. In particular, in recent years, the act of operating mobile phones has become a major cause of driving aside.

  As a method of preventing accidents caused by such aside-looking driving, when the driver's motion is analyzed and a dangerous motion is detected, a warning is given to the driver and the operation of the steering wheel, brake, etc. is controlled R & D has been done on technology to avoid dangers.

  In order to realize the above-described technology, it is important to accurately detect and grasp the driver's motion, so various technologies for detecting the driver's motion have been developed (for example, Patent Documents 1 and 2). .

  In the techniques of Patent Documents 1 and 2, the inside of a driving vehicle is photographed, and the operation of the driver is determined based on the photographed image. In the techniques of Patent Documents 1 and 2, an optical flow of the driver's motion is obtained, and the driver's motion is evaluated based on the optical flow.

  Here, as a method for obtaining an optical flow from an image, there are roughly a matching method (collation method) and a spatiotemporal gradient method. The matching method is a method of finding an optical flow by searching for a corresponding point between successive images and comparing the relative positions of the corresponding points. On the other hand, the spatiotemporal gradient method is a method for obtaining an optical flow based on the spatial and temporal gradients of the brightness of each point in an image. In the above-described techniques of Patent Documents 1 and 2, the optical flow is obtained using the matching method.

Japanese Patent No. 425271 JP 2011-159214 A

However, in the case of the matching method, a portion corresponding to the object must be detected from one image, and a portion corresponding to this portion must be detected (matched) from another image. Since this matching takes a long time, there is a problem that it takes a long time to calculate the optical flow.
Also, if the image quality of the captured image is poor, the corresponding portion cannot be detected properly, so it is very difficult to calculate the optical flow. On the other hand, if the image is captured with high image quality, one image data There is also a problem that the capacity increases and a large capacity storage device is required.

  On the other hand, when the spatio-temporal gradient method is employed when obtaining the optical flow, the following problems occur.

First, in the spatiotemporal gradient method, an apparent movement (movement in an image) is calculated using a luminance value. Assuming that the luminance value of the pixel (x, y) at time t is I (x, y, t), between the horizontal motion u (x, y) and the vertical motion v (x, y), The relationship is established. If u (x, y) and v (x, y) satisfying the relationship are obtained, an optical flow can be obtained. Note that I _x (x, y, t), I _y (x, y, t), and I _t (x, y, t) are the x-direction derivative of the luminance value, the y-direction derivative of the luminance value, and the luminance value, respectively. In the case of a digital image, it is obtained by the difference in luminance value in each direction.

I _x (x, y, t) u (x, y) + I _y (x, y, t) v (x, y) + I _t (x, y, t) = 0

Here, in the above _formula, I _x, I y, but _{I t} can be determined from the photographed image, two of u (x, y) and v (x, y) is unknown. Then, since there are two unknowns for one relational expression, u (x, y) and v (x, y) cannot be grasped, and an optical flow cannot be obtained.
If the constraint condition is set separately, the values of u (x, y) and v (x, y) can be obtained, but the values of u (x, y) and v (x, y) are quickly calculated. It is very difficult.

  In view of the above circumstances, an object of the present invention is to provide an operation detection system that can detect a driver's movement quickly and with a certain degree of accuracy while using a spatiotemporal gradient method.

A motion detection system according to a first aspect of the present invention is a system used for evaluating a driver's motion, and is based on an image capturing unit that captures an image of the driver, and an image captured by the image capturing unit. Evaluation means for evaluating the driver's movement, and the evaluation means calculates a difference in luminance between two directions orthogonal to each other and a temporal difference value of the luminance in each part of the image. And an evaluation unit that evaluates the operation of the driver based on the spatial difference value of the luminance in the two directions calculated by the difference calculation unit and the temporal difference value of the luminance. The evaluation unit estimates the driver's movement based on the speed in the two orthogonal directions of each pixel of the captured image obtained on the assumption that there is no apparent movement in one direction. Specially To.
The motion detection system according to a second aspect of the present invention is the motion detection system according to the first aspect, wherein the evaluation unit evaluates the driver's motion based on the amount of movement obtained from the spatial difference value of the luminance in the two directions. It is characterized by that.
A motion detection system according to a third aspect of the present invention is the motion detection system according to the first or second aspect, wherein the evaluation unit includes a detection unit that detects an inspection region in which the driver's face is photographed in an image. In the image, the operation of the driver is evaluated based on the spatial difference value of the luminance in the two directions and the temporal difference value of the luminance of the inspection area detected by the detection unit.

According to the first invention, the driver's movement is estimated on the assumption that there is no apparent movement in one direction based on the spatial difference value of the luminance in two orthogonal directions, so that the driver can be quickly and with a certain degree of accuracy. Can be evaluated. For example, a large movement of the driver can be sufficiently detected. In addition, the spatial difference value of the luminance and the temporal difference value of the luminance can be obtained with such an accuracy that a large movement of the driver can be estimated even if the image is rough. Then, since it is not necessary to improve the quality of the image to be captured, an inexpensive image capturing means or storage device can be used, and the system can be configured at a low cost.
According to the second aspect of the present invention, when the driver makes a large movement, the calculated movement amount increases, so that it is possible to grasp a dangerous operation among the operations of the driver.
According to the third aspect, since the area where the driver's face is photographed is used as the inspection area, the influence of disturbance can be reduced. And since it becomes easy to detect a side look that moves a face greatly, it becomes easy to grasp | ascertain dangerous action among a driver | operator's operation | movement.

It is a schematic explanatory drawing of the operation | movement detection system 1 of this embodiment, (A) is an example arrange | positioned at the vehicle, (B) is a block diagram of the operation | movement detection system 1. FIG. (A) is explanatory drawing of the time difference of a brightness | luminance, (B), (C) is explanatory drawing of the spatial difference of a brightness | luminance. (A) is an example in which the movement of each pixel is represented by a scatter diagram, and (B) is an example in which time fluctuation of the average movement amount L1 is represented. It is a schematic explanatory drawing of the test | inspection area | region set in an image. There are actually captured images, (A) is a continuous image during normal driving, and (B) is a continuous image in a state where the driver has moved greatly. (A) is a scatter diagram created based on the image of FIG. 5 (A), and (B) is a scatter diagram created based on the image of FIG. 5 (B). There are actually captured images, (A) is a continuous image during normal driving, and (B) is a continuous image in a state where the driver has moved greatly. (A) is a scatter diagram created based on the image of FIG. 7 (A), and (B) is a scatter diagram created based on the image of FIG. 7 (B).

  The motion detection system of the present invention is a system that detects the motion of a driver who is driving an automobile, and can detect the motion when the driver performs a motion larger than a normal motion. It has the feature in doing so.

In the present invention, “larger operation than normal operation” means an operation in which the driver searches for or takes an article (for example, a mobile phone or a map) on the passenger seat or dashboard, It means an action that looks back on. In other words, it means a situation where the user does not look forward for a relatively long period of time, that is, a dangerous operation that results in a so-called side-view driving state.
Of course, it is possible to detect not only the side-viewing operation but also an operation such as wearing a seat belt or an operation such as releasing a hand from the handle, and this operation can also be detected by the present invention.

First, a schematic configuration of the motion detection system 1 of the present embodiment will be described.
As shown in FIG. 1, the motion detection system 1 of the present embodiment includes an image photographing unit 2 and an evaluation unit 10.

(Image shooting means 2)
The image capturing means 2 captures an image of the driver by capturing the interior of the car. The image photographing means 2 may be provided at any position in the car, but is preferably arranged so that the driver can be photographed from the front. For example, if it is provided at the position of the rearview mirror or in the vicinity of the indicator, the driver can be photographed from a position close to the front to some extent.

  Further, the image photographing means 2 only needs to have a function capable of continuously photographing the interior of the vehicle for a certain period and obtaining a plurality of continuously photographed image data. For example, a CCD camera or a CMOS camera can be employed. In particular, it is preferable that images can be taken at intervals of 0.1 to 1 second, preferably at intervals of 0.1 second.

  The image photographing unit 2 has a storage function for temporarily storing the image, but the photographed image may be transmitted to the evaluation unit 10 immediately.

(Evaluation means 10)
The evaluation means 10 evaluates the driver's movement based on the image photographed by the image photographing means 2. As illustrated in FIG. 1, the evaluation unit 10 includes a storage unit 11, a difference calculation unit 12, and an evaluation unit 13.

(Description of storage unit 11)
The storage unit 11 has a function of storing an image captured by the image capturing unit 2. The storage unit 11 has a function of storing all image data taken during a period in which the vehicle on which the motion detection system 1 of the present embodiment is mounted is continuously driven. .
For example, if the motion detection system of the present embodiment is mounted on a taxi, bus, truck, or the like, it is preferable that the image data for about 5 hours can be stored.

  Here, when all the image data photographed during the driving period is stored, the storage unit 11 usually requires a large storage capacity. However, the motion detection system 1 of the present embodiment can detect the motion of the intended driver even if the image is somewhat rough. That is, since one image data capacity can be reduced, the storage unit 11 can be used even if the storage capacity is not so large. For example, it is possible to accommodate all data for about 5 hours on an SD card having a storage capacity of about 8G.

  Of course, the storage unit 11 may sequentially delete old data, and in that case, the storage capacity can be further reduced. Even in this case, if the image data before and after the accident is detected, the image data can be used for investigation of the cause of the accident.

(Description of the difference calculation unit 12)
The difference calculation unit 12 has a function of creating data serving as a basis for evaluating the driver's movement in the evaluation unit 13 based on the image data.
Specifically, the difference calculation unit 12 has a function of calculating a luminance time difference value and a luminance spatial difference value of each part of the image using the luminance data for the captured image.

First, the luminance time difference value is calculated using an image (target image) for which the time difference value is obtained and an image (comparison image) taken at a timing different from the target image. That is, in both images, the difference between the brightness values of the areas where the same position of the shooting target is shot is calculated as the time difference value.
For example, as shown in FIG. 2A, the time difference value I _t (x, y, t) of the luminance of the pixel A (x, y) of the image taken at time t is taken at time t−1. The obtained image is used to obtain the following equation. Note that I (x, y, t) is the luminance of the pixel A (x, y) of the image taken at time t, and I (x, y, t−1) is the image taken at time t−1. The luminance of each pixel A (x, y) is shown.

I _t (x, y, t) = I (x, y, t) −I (x, y, t−1)

Next, the spatial difference value of the luminance is calculated using the difference between the luminance values of different areas in the target image. That is, the spatial difference value of the brightness is calculated from the difference in brightness value between the areas where the different positions of the shooting target are shot in the target image. Specifically, in the target image, spatial difference values in two directions orthogonal to each other are calculated separately. Note that the target image is the same image as the image for obtaining the time difference value of the luminance.
For example, as shown in FIGS. 2B and 2C, it is assumed that the image has a plurality of pixels in the x direction and the y direction, respectively. Then, for the pixel A (x, y) of the target image, the spatial difference value I _x (x, y, t) in the x direction and the spatial difference value I _y (x, y, t) in the y direction are as follows: It is calculated by the formula.
Note that I (x, y, t) indicates the luminance of the pixel A (x, y) in FIGS. 2 (B) and 2 (C). Further, I (x−1, y, t) indicates the luminance of the pixel B (x−1, y) in FIG. 2B, and I (x, y−1, t) indicates FIG. The luminance of the pixel C (x, y−1) in (C) is shown.

I _x (x, y, t) = I (x, y, t) −I (x−1, y, t)
I _y (x, y, t) = I (x, y, t) −I (x, y−1, t)

  Note that the difference calculation unit 12 may read out data once stored in the storage unit 11 and use it for calculation of the difference value, or may use image data directly transmitted from the image photographing unit 2. . Then, the obtained difference value may be directly transmitted to the evaluation unit 13 (or the detection unit 14), or may be transmitted to the storage unit 11 and stored in association with the image data.

(Description of evaluation unit 13)
The evaluation unit 13 has a function of evaluating the action of the driver photographed in the target image using the spatiotemporal gradient method. In the evaluation unit 13, the time difference value I _t (x, y, t) calculated by the difference calculation unit 12 described above, the space difference value I _x (x, y, t) in the x direction, and the space in the y direction. Based on the difference value I _y (x, y, t), the operation of the driver photographed in the target image is evaluated.

Here, in the spatiotemporal gradient method, when calculating the apparent movement (movement in the captured image) of the object to be imaged (in this case, the driver corresponds) using the luminance value of the image, Assuming that the luminance value of the pixel (x, y) at time t is I (x, y, t), two orthogonal directions of movement, that is, horizontal movement u (x, y) and vertical movement v (x, y The following relationship is established with y).

_{I x (x, y, t} ) u (x, y) + I y (x, y, t) v (x, y) + I t (x, y, t) = 0

In this equation, I _x (x, y, t), I _y (x, y, t) and I _t (x, y, t) can be obtained from the image, but u (x, y) and v Two of (x, y) are unknown. That is, since there are two unknowns in one equation, u (x, y) and v (x, y) cannot be obtained. In other words, it is impossible to grasp the apparent movement of the subject to be photographed (in this case, the driver corresponds).

However, when the evaluation unit 13 calculates the spatial difference value of the luminance in one direction in order to calculate u (x, y) and v (x, y), there is no apparent movement in the other direction. Is assumed. That is, as shown in FIGS. 2B and 2C, the luminance spatial difference value is obtained using only the unmasked pixels in the x direction and the y direction. To establish.

I _x (x, y, t) u (x, y) + I _t (x, y, t) = 0
I _y (x, y, t) v (x, y) + I _t (x, y, t) = 0

In the above two formulas, both are one unknown in one formula, and therefore u (x, y) and v (x, y) can be calculated. That is, if I _x (x, y, t), I _y (x, y, t) and I _t (x, y, t) can be calculated, u (x, y) and v (x, y) are calculated. can do. That is, it is possible to calculate the moving speed in each direction of the photographing target photographed at the pixel (x, y), and to grasp the movement of the photographing target.

Thus, the region where the driver in the image is captured, the time difference calculating portion 12 of the luminance difference value I _t and the x direction, spatial difference value I _x of y-direction _luminance, by calculating the I _y, evaluation The unit 13 can estimate the driver's movement by the above method.

  Here, when the driver's motion is estimated on the assumption that there is no apparent movement in one direction, it is assumed that the accuracy of the calculated u (x, y) and v (x, y) is lowered. The However, even with u (x, y) and v (x, y) calculated by the above method, it is possible to obtain sufficient accuracy to distinguish a driver's large movement from normal driving.

  In addition, the spatial difference value of luminance and the temporal difference value of luminance can be calculated with such an accuracy that a large movement of the driver can be estimated even if the image is slightly rough (even if the image quality of the image is slightly poor). Then, since it is not necessary to make the image to be photographed high in image quality, an inexpensive device can be used as the storage device 11 of the image photographing means 2 or the evaluation means 10, and the system can be configured at a low cost.

(Evaluation method of driver's movement)
The evaluation unit 13 can obtain the movement amount L of each pixel (x, y) based on the movement speed in the x direction and the y direction by the following expression.

L = (u (x, y) ² + v (x, y) ² ) ^1/2

When the movement amount L is calculated for each pixel (x, y), based on the movement amount L, it is possible to determine the driver's action and evaluate whether or not the action is a dangerous action. it can.
For example, when a driver's movement is determined based on an average value (average movement amount L1) of movement amounts L of all pixels (x, y) for a certain image, the average movement amount L1 has a predetermined threshold value. When it exceeds, it can be evaluated that the driver has performed a dangerous motion. This threshold value can be determined based on the average movement amount L1 when the driver is performing a normal driving operation.

Moreover, for each image, a scatter diagram of u (x, y) and v (x, y) for each pixel is created, and it is determined whether or not the driver is performing a dangerous motion depending on the distribution status. You can also.
For example, as shown in FIG. 3A, when the horizontal axis is u (x, y) (the horizontal direction of the image) and the vertical axis is v (x, y) (the vertical direction of the image), each image If a scatter diagram is created by hitting points at the intersection of the average value of u (x, y) and v (x, y), the tendency of the driver's movement can be determined from the distribution of the points Can do. In addition, in the scatter diagram, the distance from the intersection (origin) of the horizontal axis and the vertical axis to the point is the average movement amount L1 of each image. Therefore, if the scatter diagram is used, the magnitude of the driver's movement and the tendency of the movement can be determined at the same time. For example, as shown in FIG. 3B, when the average movement amount L1 fluctuates over time, the driver's operation can be confirmed including the direction of the movement. Then, as shown in FIG. 3A, if a concentric risk level line is shown according to the degree of motion risk, it can be determined whether or not a dangerous motion has occurred.

  In addition, without calculating the average movement amount L1, it is evaluated that the driver has performed a dangerous action when the number of pixels exceeding a predetermined threshold is greater than or equal to a predetermined threshold among the movement amounts L of all the pixels (x, y). You can also

Furthermore, a scatter diagram may be created for each image. In this case, the operation of the driver at the timing when each image is captured can be grasped in more detail than the average movement amount L1.
For example, if a scatter diagram is created with the u (x, y) value and v (x, y) value at each pixel of the image, the movement of the driver when the image is captured is dangerous due to the distribution of the points. It is possible to verify in more detail whether or not it is an operation. In the scatter diagram, the distance from the intersection (origin) of the horizontal axis and the vertical axis to the point is the movement amount L of each pixel. That is, when there is a point in the vicinity of the origin, the driver can determine that there is no significant movement corresponding to the dangerous action. On the other hand, when there are a number of points away from the origin and the points are fixed in a specific direction as viewed from the origin, it can be determined that the driver has made a large movement. If the number of points away from the origin is small with respect to all pixels, it can be determined that there is a possibility of an error in the point. Therefore, if the scatter diagram is used, there is a possibility that the movement of the driver at the timing when each image is taken can be more accurately evaluated.

As described above, when a scatter diagram based on the u (x, y) value and v (x, y) value of each pixel of each image is created, a pitching of a person due to the vibration of the vehicle occurs. In such a case, the influence of the pitching is also reflected in the u (x, y) value and v (x, y) value. That is, the change in luminance (error factor) not caused by the movement of the driver is also reflected in the u (x, y) value and the v (x, y) value. However, if a scatter diagram is created by displaying the u (x, y) value and v (x, y) value at each pixel of a plurality of images taken at different times on a single graph, the above is obtained. The driver's movement can be evaluated by eliminating the error factor.
For example, the effects of vehicle vibrations appear as periodic fluctuations in the u (x, y) value and v (x, y) value. In other words, if the u (x, y) value or v (x, y) value is constant in the same pixel when the influence of vibration or the like is removed (that is, when the driver is not making a large movement). The u (x, y) value and the v (x, y) value vary with time around the correct u (x, y) value and v (x, y) value. That is, the u (x, y) value and the v (x, y) value periodically change in accordance with the timing of capturing an image. For this reason, if a scatter diagram is created by displaying the u (x, y) value and v (x, y) value of each pixel of a plurality of images taken at different times on one graph, the correct u The points are distributed around the point of (x, y) value or v (x, y) value. Then, since the presence / absence of the influence of the vibration of the vehicle can be determined from the point distribution, it is possible to prevent erroneous evaluation due to the influence of the vibration of the vehicle. Therefore, there is a possibility that the movement of the driver can be more accurately evaluated. In particular, this method is suitable for evaluating the movement of the driver in a vehicle such as a truck that is subject to large vibrations.

  Note that the evaluation unit 13 may read the image data or difference value data once stored in the storage unit 11 to calculate the movement amount L or the like, or use data directly transmitted from the difference calculation unit 12. Then, the movement amount L or the like may be calculated. The movement amount L or the like is preferably transmitted to the storage unit 11 and stored in association with the image data.

(Regarding limitation of inspection area)
In the photographed image, surrounding objects other than the driver who is the subject whose movement should be grasped are also photographed. If the surrounding object is a scenery outside the vehicle, the surrounding object also moves with the movement of the vehicle. Therefore, the u (x, y) value and the v (x, y) value are also obtained in the pixels that photograph the surrounding object. It will be calculated. For example, in the image, the driver's arm and the like are also photographed, but this part is a part where a large movement occurs even in a normal driving operation (steering operation, etc.). It will be a factor to cause.

  In addition, since the image photographing means 2 is fixed to the vehicle body, the vehicle body or a non-moving object in the vehicle should not move relative to the image photographing means 2. For this reason, the u (x, y) value and the v (x, y) value should normally be 0 in the portion where these are photographed. However, if the image photographing means 2 is shaken by the vibration of the vehicle body or the like, even the above-mentioned one moves relative to the image photographing means 2, and the amount of movement is calculated even in the pixel that photographed that portion. End up. Even if the u (x, y) value or the v (x, y) value shows a large value due to such factors, it becomes an error factor when determining the driver's movement.

  Therefore, if only the movement of the driver's face, which is considered to be most relevant to the driver's dangerous driving, is extracted, the above error factors can be eliminated, and the driver's movement can be judged more accurately. . In particular, it becomes easier to detect a side look that moves the face greatly, so it becomes easier to grasp dangerous behavior.

  Therefore, as shown in FIG. 4, the difference calculation unit 12 may calculate the luminance time difference value and the luminance spatial difference value only for the pixels in the region (inspection region) assumed to be a face. preferable. In this case, the method for determining an area assumed to be a face is not particularly limited.

For example, if analysis is to be performed after the end of driving, a person inspects on the image a region assumed to have a face in a reference image (for example, an image when the vehicle starts driving). It is only necessary to specify an area as an area, set an inspection area at the same position as the reference image for all images, and perform analysis only within the inspection area.
Alternatively, it is possible to confirm in advance the region where the driver's face is located when sitting in the driver's seat, set the region as the inspection region, and analyze only this inspection region. In this case, only the data in the area may be stored in the storage unit 11 described above. Then, the storage capacity of the storage unit 11 can be further reduced.

  Of course, the inspection unit 14 is provided in the evaluation unit 10, and this inspection unit 14 uses the image immediately after the start of photographing to detect the area of the driver's face by a known method such as pattern matching. An inspection area may be automatically set around the periphery. In this case, once the inspection area is set, it is only necessary to perform photographing and processing based on the inspection area. That is, it is not necessary to perform pattern matching or the like for each image, so that problems such as a long processing time and an increase in the capacity of the storage device can be prevented.

In the case of analyzing only the inspection region designated in advance, as described above, the difference calculation unit 12 calculates the time difference in luminance and the time difference in space for only the pixels in the inspection region in the image. be able to.
On the other hand, when the difference calculation unit 12 calculates the luminance time difference or the space time difference for all the pixels of the image, the evaluation unit 13 evaluates the driver's movement using only the value of the inspection region. It is also possible to make it.

(Other embodiments)
In the above example, the case where both the image photographing unit 2 and the evaluation unit 10 are mounted on the vehicle has been described. However, when the image photographing means 2 has a storage capacity comparable to that of the storage unit 11 of the evaluation means 10, only the image photographing means 2 may be provided in the vehicle so that the evaluation means 10 is not mounted on the vehicle. For example, when used in a truck or taxi for the purpose of confirming the driving situation of the driver, only the image photographing means 2 may be provided in the vehicle. In this case, after returning to the office, data may be acquired from the image capturing unit 2 and the data may be analyzed by the evaluation unit 10.
Of course, if the image photographing means 2 does not have a sufficient storage capacity, a storage device may be provided separately from the image photographing means 2. For example, it is possible to mount only the storage unit 11 of the evaluation unit 10 on the vehicle.

It was confirmed that the motion of the driver can be detected by the motion detection system of the present invention.
In the experiment, the inside of the vehicle was photographed at 0.1 second intervals by a camera (manufactured by NP System Development, model number: NDR200P) disposed on the top of the windshield of the vehicle. Using the captured image, the motion (speed) in two directions orthogonal to each pixel of the image was calculated. And a scatter diagram was created with two orthogonal directions as two axes.

  The camera is capable of taking an image of 480 pixels by 640 pixels and identifies the upper body region from the result of detecting the driver's face and analyzes it within that region.

The results are shown in FIG. 5 and FIG.
5A and 5B are images actually taken, FIG. 5A is a continuous image during normal driving, and FIG. 5B is a continuous image in a state where the driver has moved greatly.
As shown in FIG. 6A, it can be confirmed that during normal operation, the movement of each pixel is grouped to some extent and the size (distance from the origin) is not so large.
On the other hand, as shown in FIG. 6B, when the driver moves greatly, the points are distributed to a position farther from the origin than in the normal operation, and it can be confirmed that a large movement can be detected.

Further, FIG. 7 and FIG. 8 show the results of setting the inspection area around the face and creating a scatter diagram of only the pixels in the inspection area.
7A and 7B are images actually taken. FIG. 7A is a continuous image during normal driving, and FIG. 7B is a continuous image in a state where the driver has moved greatly.
As shown in FIG. 8A, when only the periphery of the face is set as the inspection region, it can be confirmed that the movement of each pixel is concentrated almost in the vicinity of the origin during normal operation.
On the other hand, as shown in FIG. 8 (B), even when only the periphery of the face is set as the inspection area, when the driver moves greatly, the points are distributed to a position farther from the origin than in the normal driving. It can be confirmed.
Then, it was confirmed that the difference between the normal driving and the case where the driver moved greatly appears more clearly when only the periphery of the face is set as the inspection region.

From the above results, it has been confirmed that when the driver makes a large movement, the movement detection system of the present invention can recognize and recognize the movement from the movement during normal driving.
In the case of the motion detection system of the present invention, it was also confirmed that the motion can be identified even when the image is rough to some extent.

  The motion detection system of the present invention is suitable as a driving monitoring system that detects a dangerous motion of a driver.

DESCRIPTION OF SYMBOLS 1 Action detection system 2 Image photographing means 10 Evaluation means 11 Storage part 12 Difference calculation part 13 Evaluation part 14 Detection part

Claims (3)

A system used to evaluate a driver's movement,
Image capturing means for capturing an image of the driver;
Evaluation means for evaluating the operation of the driver based on an image photographed by the image photographing means,
The evaluation means is:
In each part of the image, a difference calculating unit that calculates a spatial difference value of luminance and a temporal difference value of luminance in two orthogonal directions;
An evaluation unit that evaluates the movement of the driver based on the spatial difference value of the luminance in the two directions calculated by the difference calculation unit and the temporal difference value of the luminance;
The evaluation unit is
The operation of the driver is estimated based on the speed in the two orthogonal directions of each pixel of the captured image obtained on the assumption that there is no apparent movement in one direction. Motion detection system. The evaluation unit is
The motion detection system according to claim 1, wherein the motion of the driver is evaluated based on a movement amount obtained from a spatial difference value of luminance in the two directions. The evaluation means includes
A detection unit that detects an inspection area in which the driver's face is photographed in an image;
The evaluation unit is
3. The operation of the driver is evaluated based on a spatial difference value in the two directions of the inspection region detected by the detection unit and a temporal difference value of the luminance in the image. Motion detection system.