JP5471310B2 - Operation analysis system - Google Patents

Description

  The present invention is based on speed information calculated based on three-dimensional information obtained using a stereo camera, a driving recording device that accumulates inter-vehicle distance information, driving information that is accumulated in the driving recording device, and the like. The present invention relates to an operation analysis system for analyzing a state.

  Reducing traffic accidents is an important issue in the land transportation industry. The driver is required to drive safely, but the manager and operation manager are required to educate and manage the driver for safe driving.

  The former request is considered to be dealt with by the development of a warning device and an automatic danger avoidance device. For example, in Patent Document 1, the driver's braking operation is analyzed to estimate the driver's braking tendency, It is disclosed that it is used to issue a warning for preventing a rear-end collision to the driver and to perform appropriate brake assist.

  Moreover, in patent document 2, the moving image image | photographed with the vehicle-mounted camera is sent to a management center, the tendency regarding driving | running | working, such as an inter-vehicle distance with a preceding vehicle, is analyzed in a management center. A system is disclosed in which a simulation image that two-dimensionally represents this relationship is transmitted to an in-vehicle navigation system and displayed on the in-vehicle display device to inform the driver of the current state.

  On the other hand, in the latter requirement, many people rely on human resources for education and management, leading to an increase in cost. For this reason, managers are required to save labor by omitting the part that relies on manpower as much as possible.

  For example, Patent Document 3 analyzes image information of an in-vehicle camera to detect a positional relationship between a vehicle and a lane, and gives a warning to the driver when the vehicle deviates from the lane, and also provides information to the vehicle manager. And a system in which the vehicle manager can instruct the driver to take a break or change the operation plan is disclosed.

JP 2007-8327 A JP 2004-302902 A JP 2009-99062 A

  As explained above, not only a system for assisting safe driving for drivers, but also a system for educating and managing drivers for safe driving by managers and operation managers is required. In any case, no effective proposals have been made for educational systems.

  The present invention has been made to solve the above-described problems, and is a driving recording device that provides a driving record for safe driving education for a driver, and a driving that analyzes a driving state based on the driving record. The purpose is to provide an analysis system.

A driving analysis system according to a first aspect of the present invention includes a stereo camera mounted on a vehicle, an image recording unit that records image data captured by the stereo camera, and image data recorded by the image recording unit. A three-dimensional image information calculation unit that calculates three-dimensional image information from a moving object specifying unit that specifies a moving object-corresponding image portion in the image based on the three-dimensional image information calculated by the three-dimensional image information calculation unit; A speed calculation unit that detects a host vehicle speed based on the three-dimensional image information calculated by the three-dimensional image information calculation unit, and an inter-vehicle distance detection unit that detects an inter-vehicle distance between the preceding vehicle specified by the moving object specification unit A recording unit that records the inter-vehicle distance detected by the inter-vehicle distance detection unit as one of the environmental information affecting the driving of the own vehicle and the own vehicle speed calculated by the speed calculation unit as driving information; It takes in the operation information acquired by the operation recording apparatus equipped calculates the driver's driving pattern based on the environment information, analyzes the driving style, in the analysis of the driving style is determined inter-vehicle distance in advance The average speed in time units from the start of acceleration to the end of acceleration when the acceleration is greater than the predetermined value or when the inter-vehicle distance is not recorded at the start of acceleration. Classification is made for each speed, and the driving tendency is analyzed from the standard acceleration pattern obtained based on the average speed in the time unit from the acceleration start to the acceleration end for the classified result .

A driving analysis system according to a second aspect of the present invention includes a stereo camera mounted on a vehicle, an image recording unit that records image data captured by the stereo camera, and image data recorded by the image recording unit. A three-dimensional image information calculation unit that calculates three-dimensional image information from a moving object specifying unit that specifies a moving object-corresponding image portion in the image based on the three-dimensional image information calculated by the three-dimensional image information calculation unit; A speed calculation unit that detects a host vehicle speed based on the three-dimensional image information calculated by the three-dimensional image information calculation unit, and an inter-vehicle distance detection unit that detects an inter-vehicle distance between the preceding vehicle specified by the moving object specification unit A recording unit that records the inter-vehicle distance detected by the inter-vehicle distance detection unit as one of the environmental information affecting the driving of the own vehicle and the own vehicle speed calculated by the speed calculation unit as driving information; It takes in the operation information acquired by the operation recording apparatus equipped calculates the driver's driving pattern based on the environment information, analyzes the driving style, in the analysis of the driving style is determined inter-vehicle distance in advance The average speed in units of time from the start of deceleration to the end of deceleration when the vehicle distance is greater than the predetermined value or when the inter-vehicle distance is not recorded at the start of deceleration is calculated for each steady speed in the steady running state before the deceleration start of the host vehicle. The driving tendency is analyzed from the standard deceleration pattern obtained based on the average speed of the time unit from the deceleration start to the deceleration end for the classified results .

According to the first and second aspects of the driving analysis system according to the present invention, the driving information acquired by the driving recording device is taken in, the driving pattern of the driver based on the environmental information is calculated, and the driving tendency is analyzed. Thus, the driving tendency and improvement points of the driver are clarified, and safe driving education for the driver can be performed , and the driving tendency when there is no preceding vehicle can be analyzed.

It is a block diagram which shows the structure of the operation recording apparatus which concerns on this invention. It is a figure for demonstrating the corresponding point search in a time series stereo image. It is a figure which shows the relationship between the square of a parallax, and the vertical component of an optical flow. It is a figure which shows the straight line obtained from the relationship between the square of a parallax, and the vertical component of an optical flow. It is a figure which shows the speed change at the time of normal driving | running | working, and the change of the distance between vehicles. It is a block diagram which shows the structure of the driving | running analysis system which concerns on this invention. It is a figure explaining the process of sampling of distance between vehicles. It is a figure which shows the relationship between a steady speed and the distance between vehicles. It is a figure explaining the process of sampling of the time required for the deceleration start. It is a figure explaining the process of the sampling under the acceleration state until it reaches steady speed from the acceleration start. It is a figure which shows the relationship between the speed at the time of acceleration, and the distance between vehicles.

<Embodiment>
<Configuration of operation recording device>
FIG. 1 is a block diagram showing a configuration of an operation recording apparatus 100 according to the present invention. As shown in FIG. 1, a driving recording apparatus 100 includes an image recording unit 2 that records image data captured by a stereo camera 1 mounted on a vehicle, and a three-dimensional image from the image data recorded in the image recording unit 2. A three-dimensional image information calculating unit 3 that calculates information, a moving object specifying unit 4 that specifies a moving object-corresponding image portion in the image based on the three-dimensional image information calculated by the three-dimensional image information calculating unit 3, and a three-dimensional Based on the three-dimensional image information calculated by the image information calculation unit, the inter-vehicle distance between the speed calculation unit 5 that detects the speed of the host vehicle and the moving object (here, the preceding vehicle) specified by the moving object specifying unit 4 is detected. An inter-vehicle distance detection unit 6 and a recording unit 7 that records driving information such as the inter-vehicle distance detected by the inter-vehicle distance detection unit 6 and the speed of the host vehicle detected by the speed calculation unit 5 are provided. Next, each configuration will be described.

  The stereo camera 1 is mounted on a moving body such as a vehicle and acquires a stereo time-series image. The stereo camera 1 is a camera having an image sensor such as a CCD (Charge-Coupled Devices), for example, and is configured to include two cameras that are installed on the left and right sides at an appropriate distance. The left and right cameras in the stereo camera 1 capture the subject at the same timing to obtain a pair of left and right images. In addition, it is preferable that the aberrations of the left and right cameras are well corrected, and they are installed in parallel to each other. In this way, in the stereo camera, each camera is installed in parallel, so that a parallelized image is obtained, and three-dimensional image information can be easily obtained from these images. Note that the three-dimensional image information refers to three-dimensional coordinates, two-dimensional and three-dimensional motion vectors, etc., which can be obtained from stereo time-series images, with reference to the camera position. The stereo camera 1 repeats imaging at any time with a constant period. In addition, the stereo image that the stereo camera 1 captures and captures the subject includes stereoscopic information.

  For example, when a monocular camera is used instead of a stereo camera, it is configured to include a device capable of three-dimensional measurement, for example, a measuring instrument using a laser or a millimeter wave, for obtaining three-dimensional information. What is necessary is just to acquire three-dimensional information.

  The image recording unit 2 stores images captured by the stereo camera 1 at regular intervals and transmitted as signals to the operation recording apparatus 100. Note that the image recording unit 2 does not store more than a predetermined amount of image, and erases the recorded image from the old image at any time while storing the newly generated image.

  The three-dimensional image information calculation unit 3 calculates the three-dimensional image information of the image stored in the image recording unit 2. Specifically, the three-dimensional image information calculation unit 3 obtains a three-dimensional coordinate, a motion vector, and the like based on the imaging position of the point on the image. A known method may be used as a method for obtaining three-dimensional image information (such as the three-dimensional coordinates and motion vectors) based on the time-series stereo image.

  Specifically, the three-dimensional image information in the stereo image is obtained by searching for a point corresponding to a point on a certain image from the image corresponding to the image (corresponding point search). For example, by performing corresponding point search between a pair of stereo images, three-dimensional coordinates at that time can be obtained.

  Further, for example, by searching for corresponding points between images of subjects with different imaging times captured by the same camera, a motion vector at that point is obtained. When the stereo camera 1 is not a stereo camera but a monocular camera, a three-dimensional measuring device using a laser or millimeter wave is provided. Then, the three-dimensional image information calculation unit 3 obtains three-dimensional image information by associating the measurement values obtained by these measuring instruments with the time-series images generated by capturing the subject with the monocular camera. For example, three-dimensional information obtained by three-dimensional measurement with a laser or millimeter wave emitted in the same direction as the optical axis of the monocular camera may be associated with the subject image captured by the monocular camera.

  As a corresponding point search method, there is a correlation method in which a point (corresponding point) on a reference image corresponding to an arbitrary point of interest on a standard image is searched for and obtained. The reference image is an image corresponding to the standard image.

  Specifically, in a stereo image, one of a pair of images taken at the same time is a standard image, and the other is a reference image. In a time-series image, among images captured by the same camera, the temporally previous image is a reference image, and the temporally subsequent image is a reference image. A template is set for the attention point on the reference image, a window on the reference image corresponding to the template is searched, and a corresponding point is obtained from the searched window.

  A specific corresponding point search performed by the three-dimensional image information calculation unit 3 will be described below. One of the stereo images generated by the stereo camera 1 is set as a reference image, a point of interest is set in the reference image, and a template including the point of interest is set on the reference image. Here, the template is a range divided by a certain area in the reference image, and has information (image pattern) such as a luminance value of each pixel in the range. Then, a correlation value (similarity) between the template and a plurality of windows set in the reference image (the other image in the stereo image) corresponding to the reference image is calculated. Based on the correlation value, these templates and It is determined whether or not the window corresponds.

  Note that a window is an area in the range of the same size as the template generated in the reference image, and has information (image pattern) such as a luminance value of each pixel in the range. As described above, the correlation value is obtained from the image pattern of the template and the window. For example, the correlation value between the template and one of the windows is obtained, and if these correlation values are low, if it is determined that they do not correspond, for example, it is generated at a position shifted in one direction of one pixel. A correlation value between the determined window and the template is obtained. In this way, the correlation value is obtained while the windows are sequentially changed, and a window in which the correlation value takes a peak value is searched. A window having an image pattern having a peak correlation value with the template image pattern is determined to be a window corresponding to the template.

  Next, a specific method for calculating the correlation value will be described. Specifically, a correlation value is obtained using a function. For example, SAD (Sum of Absolute Difference) method, SSD (Sum of Squared Difference) method (square residual method), NCC (Normalize cross Correlation) method (normalized cross correlation method), etc. are known. ing.

  The SAD method is a method using a function for obtaining a sum of absolute values of luminance values of a template and a window, and a correlation value for each template and window is obtained using this function. There is also a correlation value calculation method having robustness compared to the SAD method and the like. Specifically, this method is a method of performing similarity calculation using a signal having only a phase component in which an amplitude component is suppressed from a frequency decomposition signal of an image pattern. By using this method, it is possible to realize a robust correlation value calculation that is not easily affected by differences in shooting conditions of the left and right cameras in a stereo image, noise, and the like.

  Note that the method of calculating the frequency-resolved signal of the image pattern is, for example, fast Fourier transform (FFT), discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete sine transform (DST), wavelet transform, Hadamard transform, etc. It has been known. Here, the phase-only correlation method (hereinafter referred to as the POC method) in the correlation value calculation having such robustness will be briefly described.

  Also in the POC method, a template is set on the standard image and a window having the same size is set on the reference image. Then, a correlation value (POC value) between the template and each window is calculated, and a window corresponding to the template is obtained from the correlation value. First, the template of the standard image and the window of the reference image are each subjected to two-dimensional discrete Fourier transform, normalized, synthesized, and then subjected to two-dimensional inverse discrete Fourier transform. In this way, a POC value that is a correlation value is obtained. Further, since the POC value is obtained discretely for each pixel in the window, the correlation value for each pixel can be obtained. This is different from the above-described SAD method or the like that obtains a correlation value for each window. Thus, in the POC method, since the correlation value can be obtained for each pixel in the window, it is easy to narrow the window setting range, and there is an effect that the processing for obtaining the corresponding points can be performed at high speed. In addition, in the correlation value calculation method having robustness such as the POC method, the correlation value can be obtained for each pixel in the window. Therefore, the correlation value is obtained by shifting the window by one pixel as in the SAD method. Even without it, the corresponding window can be searched.

  In the POC method, when calculating a correlation value with a template, the correlation value may be calculated while shifting the window by a plurality of pixels. Specifically, how much can be shifted depends on the searchable range of the corresponding points, but is generally said to be about half the window size. In other words, for example, the shifted window and the window before being shifted may be set so as to overlap in about half of the window size. For example, assuming that the maximum parallax between the standard image and the reference image is 128 pixels, the window size is 31 × 31, and the range that can be searched by the POC method is ± 8 pixels with respect to the center of gravity of the window, this parallax is searched. In order to do this, the windows need only be shifted by 16 pixels, so eight windows need only be set. In the POC method, a search method using a multi-resolution strategy can be used. In the multi-resolution strategy, once the base image and the reference image are reduced in resolution, the correlation value calculation is performed in a state where the number of pixels is reduced, and the coordinates at which the correlation value reaches the peak for the attention point are obtained. Thereafter, the resolution is restored to the original, and the corresponding point search is performed by narrowing the window setting range around the coordinates obtained at the low resolution. In a state where the resolution of the reference image and the reference image is low, the information of the image pattern is reduced, so that the correlation value can be obtained in a short time. In addition, there should be coordinates where the correlation value at the original resolution has a peak in the vicinity of the coordinates at which the correlation value at the low resolution has a peak. Therefore, by using this method, since the range in which the window corresponding to the template exists is determined in a short time, the corresponding window can also be searched in a short time. In this method, a plurality of low-resolution images divided into several stages may be created, and the search range may be narrowed down gradually. In the above-described example, it is only necessary to set eight windows. However, if the resolution is reduced to 1/16, for example, by using a search method based on a multi-resolution strategy, only one window may be set. This makes it possible to search for corresponding points more easily.

  Here, the corresponding point search in the time-series stereo image will be briefly described with reference to the drawings. FIG. 2 is a diagram for explaining the corresponding point search in the time-series stereo image. In FIG. 2, an image L1 and an image R1, which are stereo images taken at time T1, are shown. In order to simplify the description, it is assumed that the cameras are arranged in parallel in a stereo camera having a pair of left and right cameras that generate these images. Further, an image L2 and an image R2 taken at time T2, which is a time later than time T1, are shown. In the images L1, R1, L2, and R2, each square indicates one pixel.

  First, it is assumed that the point 15a in the image L1 at time T1 is input as a point of interest (start point). A point 15b on the image R1, which is a point corresponding to the point 15a, is obtained by a corresponding point search. Further, when the point 15a is set as an attention point, a point 16a corresponding to the point 15a on the image L2 at time T2 is obtained by the corresponding point search. Then, with this point 16a as a point of interest, a point 16b corresponding to the point 16b in the image R2 at time T2 is obtained by the corresponding point search. Note that the position of the point 16a is deviated from the center of gravity of the pixel. In such a case, since a window whose pixel is a structural unit cannot be set, a search template having a size of, for example, 3 pixels × 3 pixels is generated without using the pixel as a structural unit with the point 16a as the center of gravity. A method is adopted in which the corresponding point 16b is obtained based on the degree of similarity with the window (3 pixels × 3 pixels) on the image R2. This method is disclosed in Japanese Patent Application No. 2008-023323 by the present applicant.

  Note that each point 15a, 15b, 16a, 16b is actually a point, but is shown in the same size as the pixel in FIG. Note that, for example, the corresponding point at the time when the time series image does not exist, such as the time between T1 and T2, is interpolated using the corresponding points at T1 and T2, which are the time before and after the time series image exists. What is necessary is just to obtain | require by etc. In addition, the corresponding point search is not limited to being performed on a subject image captured at a later time, but can also be performed on a subject image captured at a previous time.

  Next, a method for calculating three-dimensional image information using the corresponding points obtained by the corresponding point search will be described. The coordinates of the point 15a are (p1x, p1y), the coordinates of the point 15b are (q1x, q1y), the coordinates of the point 16a are (p2x, p2y), and the coordinates of the point 16b are (q2x, q2y). Note that the vertical direction in the drawing is the Y direction of each image, and the horizontal direction is the X direction of each image. As described above, since the cameras are arranged in parallel, the Y coordinates of the points 15a and 15b are the same, and the Y coordinates of the points 16a and 16b are also the same.

  First, Δd1 which is a vector indicating the parallax in the images L1 and R1 is obtained from the coordinates of the point 15b obtained from the points 15a and 15a. Specifically, Δd1 is (q1x−p1x, 0). Further, Δf1, which is a vector indicating the motion in the images L1 and L2, is obtained from the coordinates of the point 16a obtained from the points 15a and 15a. Specifically, Δf1 is (p2x−p1x, p2y−p1y). Further, Δd2, which is a vector indicating parallax in the image at time T2, is obtained from the coordinates of the point 16b obtained from the points 16a and 16a. Specifically, Δd2 is (q2x−p2x, 0).

  Note that the depth distance D1 of the image obtained from the image at time T1 is obtained based on Δd1. Here, the distance D1 is a coordinate in the direction perpendicular to the paper surface in FIG. 2, and this coordinate is a Z coordinate. Moreover, in the stereo camera which produced | generated the image L1, R1, L2, R2, if the focal length of each camera is set to f and the base line length of each camera is set to B, D1 is represented by following Numerical formula (1). In Equation (1), Δd1 is the magnitude of the vector.

D1 = fB / Δd1 (1)
Similarly, the distance D2 of the depth (Z coordinate direction) of the image obtained from the image at time T2 is expressed by Equation (2) using Δd2. In the following formula (2), Δd2 is the magnitude of the vector.

D2 = fB / Δd2 (2)
From these, the three-dimensional coordinates (X1, Y1, Z1) at the points 15a and 15b at the time T1 can be expressed as (p1x · D1 / f, p1y · D1 / f, D1), and the points 16a and 16 at the time T2 The three-dimensional coordinates (X2, Y2, Z2) in 16b can be expressed as (p2x · D2 / f, p2y · D2 / f, D2).

  A three-dimensional motion vector is obtained from these three-dimensional coordinates (X1, Y1, Z1) and (X2, Y2, Z2). Specifically, the three-dimensional motion vector is a vector represented by (X2-X1, Y2-Y1, Z2-Z1).

  The moving object specifying unit 4 specifies a moving object corresponding image portion in the image. Here, the moving body refers to an object that is actually moving with respect to the ground, such as a vehicle such as an automobile or a motorcycle, a bicycle, or a pedestrian. The moving object corresponding image portion refers to a portion corresponding to the moving object displayed in the image. In the embodiment of the present invention, since the stereo camera 1 is mounted on a moving body such as a vehicle and performs imaging, even if the stereo camera 1 moves relative to the moving body, it is not necessarily a moving body. . Therefore, a method for specifying the moving object corresponding image portion in the time-series image generated by the stereo camera 1 will be described below. In specifying the moving object corresponding image portion, the moving object specifying unit 4 uses three-dimensional image information such as the three-dimensional coordinates, the two-dimensional motion vector, and the three-dimensional motion vector obtained by the three-dimensional image information calculating unit 3. Note that specifying the moving object-corresponding image portion on the image specifically refers to specifying the portion where the moving object is displayed among the objects represented as the image, and acquiring the three-dimensional image information. .

  First, there is a method of specifying a moving object corresponding image portion using a vanishing point of movement. Here, the vanishing point of the motion is a point where a straight line obtained by extending the motion vector at each point on the image along the direction intersects. This vanishing point is determined according to the moving direction of the object on the image. That is, when the camera is moving in the same direction, if it is the same object, it has moved in the same direction, so there is a vanishing point for that object. In addition, when the object on the image is a stationary object, it has been reported that the same vanishing point exists for all the objects that are stationary objects (“Examination of moving object recognition method using principal component analysis”). Information Processing Society of Japan Research Report-Computer Vision and Image Media Vol. 1996, No. 31, 1995-CVIM-099, literature number: IPSJ-CVIM 9509008).

  Note that most of the images of the subject captured by the stereo camera 1 are considered to be occupied by a stationary object corresponding image portion corresponding to a stationary object such as a traffic light, a road surface, a pedestrian crossing, and a wall. Here, the stationary object-corresponding image portion refers to a portion displayed in the image and corresponding to the stationary object. Then, assuming that, the vanishing point for the most motion vectors is estimated to be the vanishing point of the stationary object corresponding to the stationary object corresponding image portion. Therefore, it can be estimated that each vanishing point that exists after removing vanishing points for the most motion vectors among vanishing points existing in the image is the vanishing point of the moving object corresponding to the moving object corresponding image portion.

  Therefore, the moving object specifying unit 4 extends the motion vector calculated in the time-series image calculated by the three-dimensional image information calculating unit 3 along the direction, and determines a vanishing point on the image where they intersect. . Of these vanishing points, each vanishing point other than the vanishing points for the most motion vectors is estimated to be a vanishing point corresponding to the moving object corresponding image portion.

  Thus, based on the vanishing point of the estimated moving object corresponding image part, the moving object corresponding image part on the image is specified, and the three-dimensional image information is acquired. In this way, the moving object corresponding image portion in each time-series image can be specified. Since the motion vector is calculated by the three-dimensional image information calculation unit 3, it is not necessary to calculate a new motion vector in order to obtain the vanishing point, and the vanishing point can be easily calculated.

  Further, the three-dimensional motion vector obtained from the stereo time-series image is corrected by the moving speed of the stereo camera 1 that generated the stereo time-series image, so that the still-body-corresponding image portion and the moving-object-corresponding image on the image are corrected. There is also a method for discriminating the portion (see, for example, JP-A-2006-134035). When this method is used, the moving object specifying unit 4 receives speed information of the moving object on which the stereo camera 1 is mounted, and uses the three-dimensional motion vector calculated by the three-dimensional image information calculating unit 3 to move the moving object on the image. The corresponding image portion can be specified and the three-dimensional image information can be acquired.

  The speed calculation unit 5 can calculate the traveling speed of the host vehicle by sequentially executing the following steps (a-1) to (a-5).

(a-1) From the stereo images taken at time t ₀ and time t ₁ , the square of the parallax d and the optical flow vertical component Δy related to each point of interest are obtained.

  (a-2) Numerical values related to each attention point are plotted in a two-dimensional coordinate space having the square of the parallax d and the vertical component Δy of the optical flow as coordinate axes. FIG. 3 is a diagram illustrating a relationship between the square of the parallax d relating to each attention point and the vertical component Δy of the optical flow. FIG. 3 shows a state in which the numerical values relating to each point of interest are plotted in a two-dimensional coordinate space in which the horizontal axis indicates the square of the parallax d and the vertical axis indicates the vertical component Δy of the optical flow.

  (a-3) A straight line calculation method such as a least square method or a Hough transform is used to detect a straight line L1 drawn by each plotted point as shown in FIG.

  (a-4) The slope and intercept of the straight line L1 are detected.

(a-5) The traveling speed (forward speed) v of the vehicle 100 is calculated from the slope of the straight line L1 detected in the step (a-4), and the straight line L1 detected in the step (a-4). The angular velocity ω in the pitch direction of the vehicle 100 is calculated from the intercept. Here, as shown in FIG. 4, the straight line L1 has Δy and d ² as variables, (1 / α) · (v · Δt / 1) · (h / B ² ) as an inclination, and −α · It is a straight line with ω · Δt as an intercept. The camera parameter α, the base line length B, and the frame period Δt are constants determined by the design of the stereo camera 1. Therefore, if the slope and intercept of the straight line L1 are detected, the constants α, B, Δt are used to calculate the value of h · v and the value of the angular velocity ω. And if the preset initial value is employ | adopted as the height h of the stereo camera 1 on the basis of a road surface, the running speed v of the own vehicle will be calculated | required.

  In addition to the method of detecting the speed from the stereo image as described above, information from an in-vehicle speedometer may be used.

  In the inter-vehicle distance detection unit 6, since the distance D1 (or D2) is calculated when the motion vector is calculated in the three-dimensional image information calculation unit 3, the distance information is received from the three-dimensional image information calculation unit 3, Moreover, the distance with a moving body is calculated by receiving the information of the moving body specified by the moving body specific | specification part 4. FIG.

  In addition, as a method for obtaining the inter-vehicle distance, a method disclosed in Japanese Patent No. 3522317 can also be used. In addition, a configuration in which a measuring device that performs distance measurement using a millimeter wave radar or a laser radar is provided as the inter-vehicle distance detection unit 6 may be used.

  The recording unit 7 uses a detachable recording medium such as a memory card, for example. After the data has been accumulated, the recording unit 7 can be easily removed by attaching a new memory card. Data can be transferred to Note that the recording unit 7 may be configured by a substrate-fixed memory chip or a hard disk, and only data may be read out via an input / output interface.

<Recording operation>
Next, a recording operation in the operation recording apparatus 100 will be described.

  The recording unit 7 records the inter-vehicle distance and speed data detected by the inter-vehicle distance detection unit 6 and the speed calculation unit 5, respectively. However, it is not necessary to record all data during driving of the host vehicle.

  FIG. 5 shows a graph of changes in speed and distance between vehicles during normal driving. In FIG. 5, the horizontal axis represents time, the vertical axis represents speed (km / h) and inter-vehicle distance (m), the speed change is indicated by a thin line, and the inter-vehicle distance change is indicated by a thick line.

  Taking a car that has stopped waiting for a signal as an example, when the preceding vehicle starts to travel, the following vehicle accelerates while gradually increasing the inter-vehicle distance. Thereafter, when the preceding vehicle reaches a steady speed due to a speed limit or the like, the vehicle is in a steady running state while maintaining the inter-vehicle distance in that speed range. The inter-vehicle distance in this state is related to the braking distance and the like, but generally represents the driving characteristics of the driver. That is, the inter-vehicle distance is preferably kept at least equal to the stop distance, and the stop distance is expressed as follows.

Stop distance = square of free running distance / braking distance Here, the free running distance is expressed as reaction time (seconds) x vehicle speed (second speed), and the braking distance is square of speed (km / hour) / (254 x friction) (Coefficient).

  For this reason, it is necessary to keep the distance between the vehicles, that is, the presence of the preceding vehicle, so the driver is forced to drive with a driving pattern different from his own driving pattern, so the driving characteristics of the driver there Appears. Since the presence of the preceding vehicle is also an external factor that causes a change in the driving pattern, the information on the inter-vehicle distance can also be included in the environmental information.

  When the preceding vehicle starts to decelerate by a signal or the like, the following vehicle decelerates while gradually reducing the inter-vehicle distance from the preceding vehicle, and stops in a state where the inter-vehicle distance from the preceding vehicle is maintained to some extent.

  Assuming such driving conditions, the vehicle speed and inter-vehicle distance data in the steady driving condition do not reflect the driving characteristics of the driver so much, and it can be said that the utility is low in analyzing driving characteristics. . Therefore, it is possible to reduce the recorded data by recording only the data at the time of acceleration and deceleration without recording the data of the speed and the distance between the vehicles in such a steady running state.

  More specifically, data for a predetermined time Δt in an area during acceleration and deceleration and a steady running state before and after the area are recorded. In FIG. 5, the data of the section of time t1 during acceleration and the section of time t2 during deceleration are the data to be recorded. In FIG. 5, since the acceleration is from the stopped state, the predetermined time Δt is not described.

  Note that the data to be recorded in the recording unit 7 is recorded at unit time intervals. However, if the data is obtained based on a stereo image, data can usually be acquired at a cycle of 30 frames / second. The data may be recorded as it is, or may be recorded as data every second by performing averaging processing or thinning processing from this data.

  In the above description, the preceding vehicle is specified based on the stereo image, the inter-vehicle distance detection unit 6 and the speed calculation unit 5 detect the inter-vehicle distance and the speed, respectively, and record them in the recording unit 7. As described above, instead of specifying the preceding vehicle, it is possible to record the distance information with respect to the stationary bodies around the host vehicle and calculate the speed in a later process using the distance information.

  Further, when detecting the inter-vehicle distance and speed based on the stereo image, the recording unit 7 records parallax information and two pairs of image information, and acquires three-dimensional image information from the information. It is good also as a structure which calculates a speed and the distance between vehicles by subsequent process using.

  Moreover, you may make it record the environmental information at the time of driving | operation other than the distance between vehicles in the recording part 7. FIG. That is, information such as the linearity of the course of the vehicle, road surface information such as road conditions, positional relationship with traffic lights, information based on traffic rules such as signs, visibility information due to weather such as daytime, nighttime, rainy weather, etc. The environmental information is also considered to contribute to the driving pattern, and these are recorded in the recording unit 7, and when the driving of the day is completed, these data are read and analyzed. This analysis will be described later.

  Here, the linearity information of the course of the host vehicle can be acquired based on the three-dimensional information obtained from the stereo image, and the linearity of the road can be obtained from GPS (Global Positioning System) information. It is also possible to record.

  The road surface condition is measured based on the three-dimensional image information calculated by the three-dimensional image information calculation unit 3 to measure the position, height, depth, and shape of obstacles that obstruct travel such as road surface unevenness. By measuring the slope of the road surface, it can be acquired as road surface shape data.

  In addition, with regard to the positional relationship with traffic lights and signs, a technology for discriminating traffic lights and signs by image recognition has been disclosed, and it is possible to obtain information on where the vehicle is located relative to the traffic lights and signs It is. In addition, it is possible to detect and record a positional relationship with a traffic light or a sign from GPS information.

  Also, for daytime and nighttime information, it becomes information by recording time, and for visibility due to weather such as rainy weather, an image is recorded and the white line used for the center line etc. and the road surface Visibility can be detected and recorded from the contrast.

<Configuration of operation analysis system>
Next, the structure of the driving | running analysis system 200 which analyzes a driving | running state using the driving | operation information etc. which were recorded on the recording part 7 of the driving | operation recording apparatus 100 is demonstrated.

  FIG. 6 is a block diagram showing the configuration of the driving analysis system 200 according to the present invention. As shown in FIG. 6, the driving analysis system 200 includes a data reading unit 21, a database system 22, and an analysis unit 23.

  The driving analysis system 200 is built in a computer system that performs operation management disposed in a taxi or truck office. In addition, if there is a facility such as an analysis center that receives data by wireless communication from a computer system that manages the operation of each company or the in-vehicle operation recording device 100 and processes the data of each company collectively, You may build.

  When the recording unit 7 of the operation recording apparatus 100 is a detachable recording medium such as a memory card, the data reading unit 21 has a slot into which the recording medium is inserted, and reads data from the recording medium to read out the database system. 22 In this case, if data to be read out is determined in advance, information about all the driving patterns of the driver of the day, for example, speed information at the time of acceleration and deceleration, and information on the distance between vehicles are given to the database system 22 while organizing. be able to.

  The database system 22 is a system that records and manages daily driving information and the like for each driver, and uses a storage device in a computer. Further, in response to a request from the analysis unit 23, the recorded data of each traveling pattern (data of vehicle speed, distance data, etc. every moment) is sent to the analysis unit 23, and the analysis result in the analysis unit 23 is also recorded. Do.

  The analysis unit 23 analyzes the driving tendency of each driver by performing statistical processing on the driving pattern data obtained from the database system 22 (data of vehicle speed, distance between vehicles, etc. every moment).

<Analysis operation>
Hereinafter, the analysis operation of the driving tendency in the analysis unit 23 will be described by taking the first to sixth analysis operations as examples.

<First analysis operation: inter-vehicle distance under steady speed>
The inter-vehicle distance under driving at a steady speed is sampled for each speed at an interval of 10 km / h, and an average value and variation thereof are calculated and used for analysis of driving tendency.

  FIG. 7 is a diagram for explaining the sampling process of the inter-vehicle distance, and similarly to FIG. 5, shows a graph of the speed change during normal driving and the change of the inter-vehicle distance, with the horizontal axis representing time and the vertical axis representing speed ( km / h) and inter-vehicle distance (m), speed changes are indicated by thin lines, and inter-vehicle distance changes are indicated by bold lines.

  From the acceleration and deceleration patterns of the graph shown in FIG. 7, the region in the steady operation state is determined, the speed in the region of the steady operation state is set as the steady speed S, and the inter-vehicle distance in the steady operation state is set as Δd.

  Such a graph is created for a plurality of acceleration and deceleration patterns, the steady speed in each pattern and the inter-vehicle distance at that time are obtained, and the obtained speed data is 10 km / h. Sampling is performed for each speed at intervals, and a standard deviation, maximum value, minimum value, etc. are obtained for the sample data and a graph is created.

  FIG. 8 is a graph showing the relationship between the steady speed and the inter-vehicle distance, where the vertical axis represents the steady speed S (km / h), and the horizontal axis represents the inter-vehicle distance Δd (m).

  In FIG. 8, the data plotted with white circles is the average value of the inter-vehicle distance at each steady speed of a certain driver, and the arrows extending left and right starting from the average value are the steady speeds. The maximum value and minimum value of the inter-vehicle distance are shown.

  It can be seen from FIG. 8 that the inter-vehicle distance increases as the steady speed increases, and that the maximum value and the minimum value of the inter-vehicle distance increase as the steady speed increases. The characteristics of the excellent driver are plotted on the top of FIG.

  Thus, by comparing with others, it becomes easier to understand the driving tendency and improvement points of the driver who is the analysis target.

  In addition, the comparison target is not limited to the characteristics of excellent drivers, but by comparing with the average characteristics of all drivers by company or the average characteristics of all drivers of all companies using the analysis center, Objective analysis is possible.

  Further, the same analysis can be performed not by the inter-vehicle distance but also by a graph showing a change in the braking distance at each steady speed.

  The above-described analysis is based on the case where there is no preceding vehicle ahead, that is, the case where no inter-vehicle distance is recorded at the start of acceleration, or the inter-vehicle distance is equal to or greater than a predetermined value (for example, a braking distance at a reached steady speed). Is larger, it is desirable to exclude it from the processing target, and the same applies to the second to fourth analysis operations described below.

  Here, although it is a method of excluding from the processing target, it can also be realized by setting a weight for the data. For example, when the inter-vehicle distance is extremely large or when data is not recorded, the first to fourth analysis operations are processed with a weight 0 and the fifth and sixth analysis operations are performed. Is processed with a weight of 1. As a result, the data with the weight 0 can be collectively handled as data that does not contribute to the analysis.

  In addition, in the acceleration and deceleration data, by changing the weight according to the initial inter-vehicle distance at the start of acceleration, processing similar to grouping can be performed.

<Second analysis operation: Deceleration timing by speed>
When driving at steady speed, the time from when the inter-vehicle distance starts to approach and the vehicle starts to decelerate is sampled by speed at intervals of 10 km / h, and the average value and variation are calculated. Used for trend analysis.

  FIG. 9 is a diagram for explaining the sampling process of the time required to start deceleration. Similar to FIG. 5, a graph of speed change and inter-vehicle distance change during normal driving is shown, with time on the horizontal axis and time on the vertical axis. Represents the speed (km / h) and the inter-vehicle distance (m), the speed change is indicated by a thin line, and the change of the inter-vehicle distance is indicated by a thick line.

  The area of the steady operation state is determined from the acceleration and deceleration patterns of the graph shown in FIG. 9, the speed in the area of the steady operation state is set as the steady speed S, and the steady running before and after the acceleration and deceleration areas. At a predetermined time Δt in the state, the time from the start of approaching the vehicle distance to the start of deceleration is defined as the time ΔT until the start of deceleration.

  Such a graph is created for a plurality of acceleration and deceleration patterns, the steady speed in each pattern and the time required to start deceleration at that time are obtained, and from the obtained steady speed data, the hourly speed is obtained. Sampling is performed for each speed at intervals of 10 km / h, and a standard deviation, a maximum value, a minimum value, etc. are obtained for the relationship between the steady speed and the time required to start deceleration, and a graph is created.

  The graph showing the relationship between the steady speed and the time until the start of deceleration serves as an index for analysis of the driver's attention and judgment, and represents the driving characteristics of the driver.

<Third analysis operation: Distance between vehicles in acceleration state>
When driving in an accelerated state, for example, the inter-vehicle distance when reaching 10 km / h, the inter-vehicle distance when reaching 20 km / h, the inter-vehicle distance when reaching 30 km / h, and when reaching 40 km / h Are sampled, and the average value and variation thereof are calculated and used to analyze driving tendency.

  FIG. 10 is a diagram for explaining the sampling process under the acceleration state from the start of acceleration until the steady speed is reached. The horizontal axis indicates time, and the vertical axis indicates speed (km / h) and inter-vehicle distance (m). The change in speed is indicated by a thin line, and the change in the inter-vehicle distance is indicated by a thick line.

  From FIG. 10, the inter-vehicle distance Δd10 at 10 km / h, the inter-vehicle distance Δd20 at 20 km / h, the inter-vehicle distance Δd30 at 30 km / h and the inter-vehicle distance Δd40 at 40 km / h can be obtained.

  All the inter-vehicle distances at each speed at the time of acceleration are extracted from the recorded acceleration pattern and subjected to statistical processing, and an average, standard deviation, maximum value, minimum value, etc. are obtained to create a graph.

  FIG. 11 is a graph showing the relationship between the acceleration speed and the inter-vehicle distance, where the vertical axis indicates the acceleration speed S (km / h), and the horizontal axis indicates the inter-vehicle distance Δd (m). Yes.

  In FIG. 11, the data plotted with white circles is the average value of the inter-vehicle distance at each speed at the time of acceleration of a certain driver, and arrows extending left and right starting from the average value The maximum and minimum values of the inter-vehicle distance at steady speed are shown.

  As can be seen from FIG. 11, the distance between the vehicles increases as the speed increases, but the average distance between the vehicles is narrower than the characteristics of the excellent driver (× mark plot).

  Thus, by comparing with others, it becomes easier to understand the driving tendency and improvement points of the driver who is the analysis target.

  The comparison target is not limited to the characteristics of excellent drivers, but by comparing with the average characteristics of all drivers for each company or the average characteristics of all drivers of all companies using the analysis center, Objective analysis is possible.

  Similar analysis can be performed not by the inter-vehicle distance but also by a graph showing a change in the braking distance at each speed.

<Fourth analysis operation: inter-vehicle distance in a deceleration state>
Similar to the data of the inter-vehicle distance under the driving in the acceleration state described with reference to FIGS. 10 and 11, the driving tendency of the driver is also analyzed by acquiring the data of the inter-vehicle distance under the driving in the deceleration state. Can do. The method for sampling and analyzing data is the same as in the acceleration state, and therefore illustration and description thereof are omitted.

<Fifth analysis operation: acceleration pattern>
Acceleration from the average speed and variation in time units from the start of acceleration to the end of acceleration when the inter-vehicle distance is greater than a predetermined value or when the inter-vehicle distance is not recorded at the start of acceleration (when there is no preceding vehicle ahead) A pattern is calculated and used for analysis of driving tendency.

  First, the data at the time of acceleration when the inter-vehicle distance is not recorded or when the inter-vehicle distance more than the braking distance at the reached steady speed is recorded before the start of acceleration is extracted.

  For these acceleration data, for example, the final steady speed is classified by speed at 10 km / h intervals (10 km / h, 20 km / h, 30 km / h, 40 km / h).

  Then, for the classified data, the average speed in time units from the start of acceleration to the end of acceleration is obtained, and the standard acceleration pattern of the individual driver is obtained.

  By showing the acceleration pattern possessed by the individual driver and the acceleration pattern of an excellent driver, for example, on the same graph, it becomes easier to understand the driving tendency and improvement points of the driver to be analyzed.

<Sixth analysis operation: deceleration pattern>
When the inter-vehicle distance is greater than a predetermined value or when the inter-vehicle distance is not recorded at the start of deceleration (when there is no preceding vehicle ahead), the vehicle decelerates from the average speed and variation in time units from the start of deceleration to the end of deceleration. A pattern is calculated and used for analysis of driving tendency.

  First, the data at the time of deceleration when the inter-vehicle distance is not recorded or the inter-vehicle distance more than the braking distance at the ultimate steady speed is recorded before the start of deceleration is extracted.

  For example, classification (10 km / h, 20 km / h, 30 km / h, 40 km / h) is performed on the data at the time of deceleration at a constant speed before deceleration at intervals of 10 km / h.

  Then, an average speed in time units from the start of deceleration to the end of deceleration is obtained for the classified data, and a standard acceleration pattern of the driver is obtained.

  By showing the deceleration pattern of the individual driver and, for example, the deceleration pattern of an excellent driver on the same graph, it becomes easier to understand the driving tendency and improvement points of the driver to be analyzed.

  By performing the analysis described above, the driving tendency and improvement points of the driver are clarified, and safe driving education for the driver can be performed.

<Modification>
In the embodiment described above, the example of analyzing the driving tendency of the driver using the preceding vehicle information as the environmental information at the time of driving, that is, the inter-vehicle distance has been described. However, the stereo image is displayed on the recording unit 7 of the driving recording device 100. It is also possible to detect the situation of the curve by detecting the white line used for the center line etc. from the stereo image, classify it in the straight section and the curve section, and analyze the driving tendency is there.

  Similarly, by detecting a traffic signal from a stereo image and treating the distance to the traffic signal in the same manner as the inter-vehicle distance, the distance to the traffic signal can be used as environmental information.

  Based on the analysis using these environmental information, the difference between the driver's individual driving tendency analysis result and the driving tendency of a good driver is detected, and the total in each of the dangerous direction and the safe direction is obtained. It is also possible to calculate driving tendency.

  This comprehensive driving tendency can be reflected in the treatment of salary and commendation using objective indicators by outputting and linking the driver's personnel evaluation system.

DESCRIPTION OF SYMBOLS 1 Stereo camera 2 Image recording part 3 Three-dimensional image information calculation part 4 Moving body specific part 5 Speed calculation part 6 Inter-vehicle distance detection part 7 Recording part

Claims (2)

A stereo camera mounted on the vehicle,
An image recording unit for recording image data captured by the stereo camera;
A three-dimensional image information calculation unit for calculating three-dimensional image information from the image data recorded by the image recording unit;
Based on the 3D image information calculated by the 3D image information calculation unit, a moving object specifying unit for specifying a moving object corresponding image part in the image;
A speed calculation unit that detects the vehicle speed based on the three-dimensional image information calculated by the three-dimensional image information calculation unit;
An inter-vehicle distance detection unit that detects an inter-vehicle distance from a preceding vehicle specified by the moving object specifying unit;
A recording unit that records the inter-vehicle distance detected by the inter-vehicle distance detection unit and the own vehicle speed calculated by the speed calculation unit as driving information as one of environmental information that affects driving of the own vehicle. Capture the driving information acquired by the driving recording device, calculate the driving pattern of the driver based on the environmental information, analyze the driving tendency ,
In the analysis of the driving tendency,
When the inter-vehicle distance is greater than a predetermined value or when the inter-vehicle distance is not recorded at the start of acceleration, the host vehicle accelerates the average speed in units of time from the start of acceleration to the end of acceleration, resulting in a steady running state. A driving analysis system that classifies each reached steady speed after that and analyzes the driving tendency from the standard acceleration pattern obtained based on the average speed in the time unit from the start of acceleration to the end of acceleration for the classified results . A stereo camera mounted on the vehicle,
An image recording unit for recording image data captured by the stereo camera;
A three-dimensional image information calculation unit for calculating three-dimensional image information from the image data recorded by the image recording unit;
Based on the 3D image information calculated by the 3D image information calculation unit, a moving object specifying unit for specifying a moving object corresponding image part in the image;
A speed calculation unit that detects the vehicle speed based on the three-dimensional image information calculated by the three-dimensional image information calculation unit;
An inter-vehicle distance detection unit that detects an inter-vehicle distance from a preceding vehicle specified by the moving object specifying unit;
A recording unit that records the inter-vehicle distance detected by the inter-vehicle distance detection unit and the own vehicle speed calculated by the speed calculation unit as driving information as one of environmental information that affects driving of the own vehicle. Capture the driving information acquired by the driving recording device, calculate the driving pattern of the driver based on the environmental information, analyze the driving tendency,
In the analysis of the driving tendency,
When the inter-vehicle distance is greater than a predetermined value or when the inter-vehicle distance is not recorded at the start of deceleration, the average speed in time units from the start of deceleration to the end of deceleration is calculated in the steady running state before the start of deceleration A driving analysis system that classifies each steady speed and analyzes the driving tendency from the standard deceleration pattern obtained based on the average speed in the time unit from the start of deceleration to the end of deceleration for the classified results .

Images