JP5062091B2 - Moving object identification device, computer program, and optical axis direction specifying method - Google Patents

Description

The present invention, an area including the road to extract a feature from the captured image obtained by imaging, mobile identification apparatus for identifying moving body using the extracted feature quantity, the mobile identification equipment in the computer optical axis direction identification method by a computer program and the mobile identification equipment for implementing related.

  A system in which a roadside infrastructure device and a vehicle (on-vehicle device) exchange information by wired or wireless road-to-vehicle communication and each device realizes a function that cannot be realized independently has been studied. In particular, there is a study on a road-vehicle cooperative safe driving support system that suppresses traffic accidents by providing information on vehicles or pedestrians in a blind spot from the vehicle side to the vehicle side from the infrastructure side. It is being advanced.

  As main technologies for realizing such a system, for example, there are sensors that detect information such as vehicle position information, vehicle type information (for example, automobiles, two-wheeled vehicles, pedestrians, etc.), vehicle movement speed or direction, and the like. is necessary. Examples of such sensors include those that use lasers or those that use images captured by a video camera. The range of information that can be detected, product life, cost, performance, etc. Considering this balance, the image processing method is effective.

As an example of using an image processing type sensor (image sensor), for example, traffic flow measurement is performed by detecting the presence of a vehicle, a vehicle type such as a small or medium-sized vehicle, a vehicle speed, etc. from image information captured by a video camera. An apparatus for performing this is disclosed (see Patent Document 1).
JP-A-5-307695

  However, one of the problems of the sensor of the image processing method disclosed in Patent Document 1 is that the imaging range captured by the video camera according to the width of the road to be imaged, the position of the lane, the traveling direction of the vehicle, etc., the video camera It is necessary to adjust the mounting angle of the video camera (the optical axis direction of the lens of the video camera) such as the installation height, the depression angle, or the rotation angle, and to set the lane position on the captured image. In particular, in order to accurately determine the vehicle speed measurement and position, it is necessary to match the position on the captured image with the position on the actual road, and it is necessary to set the video camera mounting angle accurately. It was.

  Conventionally, the mounting angle has been adjusted at the installation location of the video camera, but the adjustment work has required a great deal of labor. In addition, during the adjustment work, it is necessary to regulate road lanes, which may cause traffic congestion and traffic accidents. For this reason, it has been desired to set the mounting angle of a video camera or the like by a simple method.

The present invention has been made in view of such circumstances, mobile identification apparatus capable of setting the mounting angle of such a video camera (an optical axis direction of the lens) conveniently, the mobile identification equipment computer in a computer program, and the optical axis direction identification method by the mobile identification equipment for realizing.

  A mobile object identification device according to a first aspect of the present invention extracts a feature amount from a captured image obtained by imaging an area including a road with an imaging device, and identifies the mobile object using the extracted feature amount. A position information acquisition unit that acquires position information of a moving body, a partial image specification unit that specifies a partial image on a captured image, and a feature that extracts a feature amount of the partial image specified by the partial image specification unit An amount extracting unit; an identifying unit that identifies a moving object using the feature amount extracted by the feature amount extracting unit; and a position information acquired by the position information acquiring unit when the moving unit is identified by the identifying unit, Based on the known optical axis direction of the lens of the imaging device and the position on the captured image of the partial image specified by the partial image specifying unit, the moving body that acquired the position information by the position information acquiring unit captures the partial image. Whether or not A determination means for determining, a position calculation means for calculating a position of the moving body on a captured image when the determination means determines that the moving body is imaged, and a plurality of imaging calculated by the position calculation means And an optical axis direction specifying means for specifying the optical axis direction to correct the optical axis direction using each position of the one or more moving bodies at the time and the imaging parameters of the imaging device. .

  According to a second aspect of the present invention, there is provided a mobile unit identification apparatus comprising: an environment identification unit for identifying an environmental situation including weather or day and night using the feature amount extracted by the feature amount extraction unit. Is configured to identify the moving object using the environmental situation identified by the environment identification means and the feature amount.

  A computer program according to a fourth invention functions as a means for extracting a feature amount from a captured image obtained by capturing an image of an area including a road with an imaging device, and identifying a moving object using the extracted feature amount. A computer program for causing a computer to use a partial image specifying unit for specifying a partial image on a captured image, a feature amount extracting unit for extracting a feature amount of the specified partial image, and the extracted feature amount An identification means for identifying a moving body and, when the moving body is identified, the movement based on positional information of the moving body, a known optical axis direction of the lens of the imaging device, and a position of the specified partial image on the captured image. A determination unit that determines whether or not a body is captured in the partial image, and a position that calculates a position on the captured image of the mobile body when it is determined that the mobile body is captured An optical axis direction specification for specifying the optical axis direction to correct the optical axis direction by using the output means, the calculated positions of one or a plurality of moving bodies at a plurality of imaging time points, and the imaging parameters of the imaging device It is made to function as a means.

  According to a sixth aspect of the present invention, there is provided an optical axis direction identification method that extracts a feature amount from a captured image obtained by capturing an area including a road with an imaging device, and identifies the moving body using the extracted feature amount. A method for specifying an optical axis direction by an apparatus, which acquires position information of a moving body, specifies a partial image on a captured image, extracts a feature amount of the specified partial image, and uses the extracted feature amount to move the moving body When the moving body is identified, the moving body that has acquired the position information based on the acquired position information, the known optical axis direction of the lens of the imaging device, and the position of the specified partial image on the captured image Is determined to be captured in the partial image, and when it is determined that the moving body is captured, the position of the moving body on the captured image is calculated, and 1 at a plurality of calculated imaging time points is calculated. Alternatively, each position of a plurality of moving bodies and the imaging device Using the parameters, and identifies the optical axis direction so as to correct the optical axis direction.

In the first invention, the third invention, and the fourth invention, the partial image on the captured image is specified, and the feature amount of the specified partial image is extracted. The partial image is, for example, a detection range of 24 × 32 pixels, and the detection range is scanned on the captured image to extract the feature amount of the detection range. For example, HOG (Histograms of Oriented Gradients) features can be used as the feature amount, which is a feature vector in which the gradient intensity of the luminance in the detection range is histogrammed for each gradient direction. Note that the feature amount is not limited to the HOG feature, and may be extracted by edge detection. When a moving body (for example, a vehicle) is identified by an identification unit (identifier) using the extracted feature amount, the position information of the moving body, the known optical axis direction of the lens of the imaging device (for example, the depression angle of the optical axis, and the like) The moving body that has acquired the vehicle position information based on the position on the captured image of the rotation angle) and the identified partial image (the detection range from which the feature amount is extracted when the moving body is identified) is the partial image (detection range). It is determined whether or not it is inside. When a feature amount (feature vector) is input, the discriminator outputs a discrimination result (determination information, for example, there is a vehicle, no vehicle, etc.) based on the input feature amount and the discriminator parameter. Further, the known optical axis direction is, for example, roughly set and is the one before specifying with high accuracy.

  When it is determined that the moving body is imaged (within the detection range), the position of the moving body on the captured image is calculated. The position of the moving body is calculated in the same way at a plurality of imaging time points. Thereby, a virtual straight line representing the traveling locus of the moving body can be obtained on the captured image. Similarly, a plurality of positions on the captured image are calculated for a plurality of moving objects. Thereby, a plurality of virtual straight lines can be obtained. Using each calculated position, that is, an intersection (a point at infinity) of a plurality of virtual straight lines and imaging parameters of the imaging device (for example, the size and resolution of the imaging surface of the imaging device, the focal length of the lens, etc.) The optical axis direction (for example, the depression angle and the rotation angle of the optical axis) is specified to correct the known optical axis direction of the lens. Thereby, the optical axis direction (the depression angle and the rotation angle) of the lens can be automatically obtained with high accuracy. In addition, adjustment work related to the mounting angle of the imaging device can be performed automatically instead of manual work, lane regulation required for the adjustment work becomes unnecessary, and work labor and cost can be reduced. , Leading to prevention of traffic jams and traffic accidents.

  In the second invention, there is provided environment identification means (environment identifier) for identifying an environmental situation including weather or day and night using the extracted feature quantity. When a feature quantity (feature vector) is input, the environment discriminator determines an identification result (determination information, for example, environment such as day, evening, night, clear sky, rainy weather, etc.) based on the input feature quantity and the environment discriminator parameter. When the vehicle is identified, the vehicle (moving object) is identified based on the extracted feature amount and the required environmental condition (for example, daytime, evening, sunny night, rainy night). Thus, the moving object can be accurately identified regardless of the environmental situation.

  According to the present invention, adjustment work related to the mounting angle of the imaging apparatus can be automatically performed instead of manual work, lane regulation required for the adjustment work becomes unnecessary, and work labor and work cost are reduced. Can also prevent traffic jams and traffic accidents.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments. FIG. 1 is a block diagram showing an example of a configuration of a vehicle identification device 100 that is a moving body identification device according to the present invention. The vehicle identification device 100 includes an image input unit 11, an A / D conversion unit 12, an image memory 13, a communication unit 14, a storage unit 15, a control unit 20 that controls the entire device, a partial image specification unit 30, and a feature amount extraction unit 40. , An identification unit 50 as identification means (identifier), a vehicle presence position determination unit 60, an optical axis direction identification unit 70 that identifies the optical axis direction of the lens of the video camera 10, and the like. The identification unit 50 includes a vehicle presence / absence identification unit 51, a vehicle type identification unit 52, an environment identification unit 53, an environment-specific vehicle presence / absence identification unit 54, and the like. A video camera 10 is connected to the image input unit 11. Note that the video camera 10 may be a separate device from the vehicle identification device 100, or may be configured such that both are integrated.

  The video camera 10 captures a required area including a road, and a predetermined point near the road in a state where imaging conditions such as a predetermined height and a lens optical axis direction (for example, depression angle and rotation angle) are set. It is installed in. The video camera 10 outputs a captured image obtained by imaging to the image input unit 11 as a video signal (analog signal).

  First, a method for setting approximate values (initial values) in the optical axis direction of the lens of the video camera 10 will be described. FIG. 2 is an explanatory diagram showing a road coordinate system. As shown in FIG. 2, a road coordinate system (X, Y, Z) is defined with the lens center of the video camera 10 as the origin. In the road coordinate system, the traveling direction of the road (the direction away from the video camera 10) is the Y axis (the forward direction is positive), and the direction on the road surface perpendicular to the road direction is the X axis (the right direction toward the front is positive). The direction perpendicular to the road surface is taken as the Z-axis (upward is positive). The camera coordinate system (X ′, Y ′, Z ′) is defined with the lens center of the video camera 10 as the origin. In the camera coordinate system, the optical axis of the camera lens is the Y ′ axis, the horizontal axis perpendicular to the optical axis is the X ′ axis, and the upward direction of the camera is the Z ′ axis. Furthermore, the rotation angle of each axis of the camera coordinate system with respect to each axis of the road coordinate system is α (pitch angle), β (roll angle), and γ (yaw angle), respectively, and the direction in which the right screw advances is positive (α : Positive upward from the horizontal plane, β: positive clockwise, γ: positive counterclockwise). In this case, the conversion equation from the road coordinate system to the camera coordinate system can be expressed by equation (1).

  Each of the coefficients P11 to P33 of the transformation matrix can be expressed by Expression (2). Further, the coordinates (x, y) on the captured image can be expressed by Expression (3), where F is the focal length of the lens.

  FIG. 3 is an explanatory view showing a mounting state of the video camera 10. The optical axis direction of the lens of the video camera 10 can be specified by the depression angle and the rotation angle. As shown in FIG. 3A, the depression angle is an angle formed by a plane parallel to the road surface and the optical axis of the lens. When the optical axis of the lens is parallel to the road surface, the depression angle is 0 degree, and the direction in which the optical axis of the lens faces downward is defined as a positive depression angle. In this case, the depression angle is opposite to the direction of the pitch angle described above and can be represented by -α.

  As shown in FIG. 3B, the rotation angle is an angle formed by the traveling direction of the road and the optical axis of the lens. When the optical axis of the lens is parallel to the traveling direction of the road, the rotation angle is 0 degree, and the direction in which the optical axis of the lens faces left is defined as a positive rotation angle. In this case, the rotation angle can be expressed by the above-mentioned yaw angle (γ). The roll angle β is set to 0 using a leveling device. Setting approximate values in the optical axis direction of the lens of the video camera 10 is to tentatively obtain the pitch angle (α) and yaw angle (γ) described above.

  FIG. 4 is an explanatory diagram showing an example of initial setting of the lens of the video camera 10 in the optical axis direction. The initial setting in the optical axis direction is to set approximate values in the optical axis direction (the depression angle and the rotation angle) of the lens when the video camera 10 is installed. Thus, the depression angle and the rotation angle are determined in advance. As an approximate setting method in the optical axis direction, for example, two boundary lines are set by setting the boundary positions (E1, E2, E3, E4) of the left and right ends (for example, the end of the road) of the measurement range on the captured image. To decide. This operation can be easily performed manually. Then, the depression angle and the rotation angle are calculated from the intersection E0 of the two boundary lines, and the coordinates on the captured image are matched with the actual position on the road.

  The imaging parameters of the video camera 10 are as follows. That is, the installation height of the video camera is H, the horizontal size of the imaging surface is SH, the vertical size is SV, the horizontal resolution (number of pixels) of the imaging surface is RH, the vertical resolution (number of lines) is RV, and the focal length of the lens is F.

  As shown in FIG. 4, the coordinates of the points E1 to E4 on the captured image are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively. Let E0 be the intersection of a virtual straight line E1E2 connecting points E1 and E2 and a virtual straight line E3E4 connecting points E3 and E4. The virtual straight line E1E2 can be expressed by Expression (4), and the virtual straight line E3E4 can be expressed by Expression (5).

  Further, when the coordinates of the intersection point E0 are (x0, y0), the intersection point E0 can be obtained by Expression (6). The coordinates (s0, t0) of the intersection point E1 on the imaging surface can be obtained by Expression (7), and the depression angle α and the rotation angle γ can be calculated by Expression (8). The depression angle and the rotation angle calculated by Expression (8) are approximate values, and are provisional known values before being obtained by the optical axis direction identification unit 70.

  The image input unit 11 outputs the acquired video signal to the A / D conversion unit 12.

  The A / D converter 12 converts the input video signal into a digital signal, and stores the converted digital signal in the image memory 13 as image data. The captured image input from the video camera 10 via the image input unit 11 is synchronized with the frame rate of the video camera 10 (interval at the time of imaging, for example, 30 frames per second) in units of one frame (for example, 240 Is stored in the image memory 13 as (× 320 pixels) image data.

  FIG. 5 is an explanatory diagram illustrating an example of a captured image. As shown in FIG. 5, by installing the video camera 10 in the vicinity of the road under predetermined imaging conditions, it is possible to image a vehicle traveling on the required road. When the video camera 10 is installed, the optical axis direction of the lens (for example, the depression angle and the rotation angle) is generally set, and the optical axis direction of the lens is specified as the optical axis direction as described later. The part 70 is automatically specified (set) with high accuracy. In FIG. 5, the size of the captured image is 240 × 320 pixels, but the size of the captured image is not limited to this.

  The communication unit 14 includes a narrow-area communication function, a mid-range communication function such as a wireless LAN such as a UHF band or a VHF band, and a wide-area communication function such as a mobile phone, PHS, multiple FM broadcasting, or Internet communication. The communication unit 14 obtains predetermined vehicle information (for example, vehicle position information, vehicle speed information, vehicle type information, operation statuses of vehicle lights such as wipers and headlights) from a vehicle existing in or near the imaging region of the video camera 10. Information). Moreover, the communication part 14 transmits the identification information obtained as a result of identifying a vehicle to the vehicle which exists in the vicinity. Moreover, the communication part 14 transmits / receives predetermined information between the server apparatus installed in the traffic control center etc., and another vehicle identification apparatus.

  The storage unit 15 stores data obtained by processing of the vehicle identification device 100, data received via the communication unit 14, and the like under the control of the control unit 20.

  The partial image specifying unit 30 specifies a partial image having a required size on the captured image, and scans the specified partial image on the captured image. The partial image defines a detection range for extracting a feature amount described later.

  FIG. 6 is an explanatory diagram showing an example of a detection range. As shown in FIG. 6, the size of the detection range is, for example, 24 × 32 pixels. When the detection range is scanned on the captured image, the detection range can be moved by 16 pixels in the horizontal direction in one scan as shown in FIG.

  The feature quantity extraction unit 40 extracts feature quantities in the detection range. The feature quantity can be a HOG (Histograms of Oriented Gradients) feature, and is a feature vector in which the gradient intensity of the luminance in the detection range is histogrammed for each gradient direction. Note that the feature amount is not limited to the HOG feature, and may be extracted by edge detection.

  Hereinafter, a method for calculating a feature vector as a feature amount will be described. FIG. 7 is an explanatory diagram showing an example of the local region. The local area (block) is extracted so as to partially overlap from the detection range of 24 × 32 pixels, and has a size of 16 × 16 pixels, for example. Accordingly, six local regions can be extracted from the detection range. The local area is composed of four cells (area of 8 × 8 pixels).

  FIG. 8 is an explanatory view showing an example of the gradient intensity of the luminance and the gradient direction. The position of the pixel in the cell is represented by (x, y). It is assumed that the luminance gradient intensity at pixel (x, y) is m (x, y) and the gradient direction is θ (x, y). The gradient strength m (x, y) and the gradient direction θ (x, y) can be obtained from Equations (9) to (11). Here, fx (x, y) is a luminance gradient in the horizontal direction (x direction), and fy (x, y) is a luminance gradient in the vertical direction (y direction). , And the Sobel filter. L (x, y) is the luminance of the pixel (x, y).

  Then, a histogram in eight directions is created for each cell. FIG. 9 is an explanatory diagram illustrating an example of a gradient direction, and FIG. 10 is an explanatory diagram illustrating an example of a histogram of gradient strength. As shown in FIG. 9, the gradient direction is divided every 45 °, and eight (j = 1 to 8) gradient directions are set. Also, as shown in FIG. 10, a histogram is created by distributing the calculated intensity gradients for each gradient direction. When calculating the gradient strength, the edge strength can be smoothed by applying a Gaussian filter of σ = 1.6 inversely proportional to the distance from the center of the local region (block) to the local region.

  Histograms obtained for each cell are collected in a local region, and normalized by dividing each histogram element (gradient strength) by the maximum element (gradient strength maximum) of the histogram for each local region. As a result, the maximum component of the 32-dimensional feature vector extracted in one local region (block) is 1. Since there are six local regions in one detection range, 192-dimensional (32 dimensions × 6 = 192 dimensions) feature vector V can be calculated for one detection range as shown in Expression (12).

  Since the gradient of adjacent pixels is normalized by creating a histogram for each detection range, local geometrical changes that are less affected by noise or brightness changes and the presence or absence of shadows on captured images obtained by imaging roads ( Robust against translation, rotation, etc.).

  The identification unit 50 uses the feature quantity (feature vector) extracted by the feature quantity extraction unit 40 as input information, performs a predetermined identification determination including the presence or absence of a vehicle, and outputs an identification result (determination information). The discriminating unit 50 includes a vehicle presence / absence discriminating unit 51 having a vehicle presence / absence discriminator, a vehicle type discriminating unit 52 having a vehicle type discriminator, an environment discriminating unit 53 having an environment discriminator, and an environment-specific vehicle presence / absence discriminating unit having environment-specific vehicle presence / absence discriminators. Part 54 and the like.

  FIG. 11 is an explanatory diagram showing an outline of each classifier of the classifier 50, and FIG. 12 is an explanatory diagram showing identification contents of each classifier of the classifier 50. When the feature vector V is input, the vehicle presence / absence discriminator outputs identification information Dc = Ac · V. Here, Ac is a discriminator parameter of the vehicle presence / absence discriminator. The vehicle presence / absence discriminator performs a predetermined calculation based on the discriminator parameter Ac on the input feature vector V, and determines whether there is a vehicle or no vehicle as an identification result (class). For example, if the output identification information Dc is Dc> 0, it is determined that there is a vehicle, and if Dc <0, it can be determined that there is no vehicle. Note that the determination condition is an example, and the present invention is not limited to this.

  In addition, when the feature vector V is input, the vehicle type identifier outputs identification information Dt = At · V. Here, At is a discriminator parameter of the vehicle type discriminator. The vehicle type discriminator performs a predetermined calculation based on the discriminator parameter At on the input feature vector V, and distinguishes the classification result (class), for example, an ordinary vehicle, a large vehicle, a two-wheeled vehicle or a pedestrian. judge. For example, if the output identification information Dt is 0 <Dt <Tt1, it is determined as a normal vehicle, if Tt1 <Dt <Tt2, it is determined as a large vehicle, and if Tt2 <Dt <Tt3, it is determined as a motorcycle. If Dt <0, it can be determined as a pedestrian. Note that the determination condition is an example, and the present invention is not limited to this.

  Further, when the feature vector V is input, the environment classifier outputs the identification information Dw = Aw · V. Here, Aw is a classifier parameter of the environment classifier. The environment classifier performs a predetermined calculation based on the classifier parameter Aw on the input feature vector V, and the classification result (class) is, for example, different from daytime, evening, sunny night, rainy night, etc. Determine. For example, if the output identification information Dw is 0 <Dw <Tw1, it is determined as daytime, if Tw1 <Dw <Tw2, it is determined as evening, and if Tw2 <Dw <Tw3, it is determined as a clear night. If Tw3 <Dw <Tw4, it can be determined that the rainy night. Note that the determination condition is an example, and the present invention is not limited to this.

  Further, when the feature vector V is input, the environment-specific vehicle presence / absence discriminator outputs identification information Dcw = Acw · V. Here, Acw is a discriminator parameter of the vehicle presence / absence discriminator by environment. The environment-specific vehicle presence / absence discriminator performs a predetermined calculation based on the discriminator parameter Acw on the input feature vector V, and the discrimination result (class) is, for example, a day vehicle, an evening vehicle, or a sunny vehicle. It is determined whether there is a rainy night vehicle, no daytime vehicle, no evening vehicle, no clear night vehicle, no rainy night vehicle. Note that the determination contents can be distinguished according to the value of the identification information Dcw, as with the above-described classifiers.

  When the vehicle presence / absence discriminating unit 51 identifies the vehicle in the detection range (partial image) (when it is determined that there is a vehicle), the vehicle presence position determination unit 60 uses the vehicle information received from the vehicle via the communication unit 14. Based on the position information of the included vehicle, it is determined whether or not the vehicle exists within the detection range. That is, when the vehicle presence position determination unit 60 receives the vehicle information, whether the vehicle that transmitted the vehicle information is the same vehicle as the vehicle in the detection range in which the feature vector is extracted and determined to be a vehicle. Determine whether or not. The determination of whether or not they are the same vehicle can include a required error range according to the position detection accuracy of the vehicle. That is, the determination may be made based on whether or not the position where the error range is added to the vehicle position information is within the detection range. Further, instead of the vehicle presence / absence identifying unit 51, an environment-specific vehicle presence / absence identifying unit 54 may be used. As a result, the vehicle can be accurately identified regardless of the environmental situation.

  More specifically, whether or not the vehicle exists within the detection range can be performed as follows. The position of the vehicle included in the vehicle information received from the vehicle is defined as Pcar, and the center position of the detection range in which the feature vector is extracted and determined to be present is defined as Pdet. However, Pcar and Pdet are positions on the actual road with the camera installation position as a reference. In this case, when the distance between the two positions is equal to or smaller than a certain error (Err) range, that is, when | Pcar−Pdet | <Err, it is determined that the vehicle exists within the detection range.

  The optical axis direction specifying unit 70 automatically and accurately specifies a predetermined approximate optical axis direction of the lens of the video camera 10 without performing manual adjustment. Hereinafter, a method for specifying the optical axis direction (the depression angle and the rotation angle) will be described.

  FIG. 13 is an explanatory diagram showing an application example of the optical axis direction identification mode of the vehicle identification device 100. As illustrated in FIG. 13, the vehicle identification device 100 captures an image of the vehicle C1 traveling in the imaging range, and receives vehicle information including position information of the vehicle C1 from the vehicle C1. The vehicle identification device 100 identifies a vehicle by the identification unit 50 (the vehicle presence / absence identification unit 51 or the environment-specific vehicle presence / absence identification unit 54) using the feature amount (feature vector) extracted from the captured image.

  When the vehicle presence position determination unit 60 determines that the vehicle is within the detection range, the optical axis direction identification unit 70 determines the vehicle position identified by the identification unit 50 (for example, the center position of the detection range that identifies the vehicle). Plotting is performed on the captured image, and similarly, vehicle positions identified at a plurality of imaging time points are plotted. As a result, a virtual straight line representing the traveling locus of the vehicle can be obtained on the captured image. In order to track the same vehicle, for example, a template matching method or the like can be used. Further, as the position for tracking the vehicle, for example, the center of gravity position of the detection range, the head position of the vehicle, or the like can be used.

  Next, the same processing is performed for a plurality of vehicles. Thereby, the virtual straight line showing the traveling locus of each of a plurality of vehicles can be obtained.

  FIG. 14 is an explanatory diagram showing an example of a method of specifying the optical axis direction of the lens of the video camera 10. The positions plotted on the captured image of one vehicle are A and B, and the respective coordinates are A (xa, ya) and B (xb, yb). Also, the positions plotted on the captured images of other vehicles are C and D, and the coordinates are C (xc, yc) and D (xd, yd). The plotted positions are not limited to two points for each vehicle, and a larger number of positions can be plotted.

  Next, a virtual straight line AB is obtained by linearly approximating the traveling locus of the vehicle passing through the positions A and B. Similarly, the virtual straight line CD is obtained by linearly approximating the traveling locus of the vehicle passing through the positions C and D. The virtual straight line AB can be expressed by Expression (13), and the virtual straight line CD can be expressed by Expression (14).

  Assuming that the intersection (infinite point) of the virtual straight line AB and CD is M and the coordinates are M (mx, my), the infinite point M (mx, my) can be calculated by the equation (15). Then, the coordinates (ms, mt) of the infinity point M on the imaging surface can be obtained by Expression (16), and the depression angle α ′ and the rotation angle γ ′ can be calculated by Expression (17). .

  In Expressions (16) and (17), the imaging parameters of the video camera 10 are as follows. That is, the horizontal size of the imaging surface of the video camera 10 is SH, the vertical size is SV, the horizontal resolution (number of pixels) of the imaging surface is RH, the vertical resolution (number of lines) is RV, and the focal length of the lens is F.

  The depression angle and the rotation angle calculated by the equation (17) are for correcting the provisional known values obtained from FIG. 4 and can be obtained automatically and with high accuracy.

  The control unit 20, the partial image specifying unit 30, the feature amount extracting unit 40, the identifying unit 50, the vehicle presence position determining unit 60, the optical axis direction specifying unit 70, and the like can be realized by a dedicated hardware circuit. It can also be realized by causing a CPU to execute a computer program that defines the function of each unit.

  In the conventional method using integration of the time difference result, noise components other than the vehicle are included in the vehicle presence range due to changes in the surrounding environment, and it is difficult to specify an accurate vehicle presence range. It was. According to the present invention, by using both the vehicle position obtained from the captured image captured by the video camera 10 and the position information acquired from the vehicle, it is possible to reliably determine the range in which the vehicle exists.

  FIG. 15 is an explanatory diagram illustrating an application example of the information providing mode of the vehicle identification device 100. As shown in FIG. 15, the vehicle identification device 100 is installed in the vicinity of an intersection, and a required imaging range is imaged with a video camera. The vehicle identification device 100 identifies an out-of-sight vehicle or an out-of-sight pedestrian with respect to the vehicles C2, C3 and the like whose imaging range is out of sight, and provides information based on the identification result. Thereby, it is possible to provide information on a vehicle or a pedestrian at a spot that becomes a blind spot from the driver of the vehicle, and it is possible to prevent a traffic accident.

  Next, the operation of the vehicle identification device 100 will be described. FIG. 16 is a flowchart showing a processing procedure for specifying the optical axis direction. The control unit 20 acquires a captured image (S11), scans after specifying a detection range as a partial image (S12). The control unit 20 calculates a feature vector as a feature amount of the detection range (S13), and performs an identification process (for example, identification of the presence / absence of a vehicle) by performing an operation based on the classifier parameter on the calculated feature vector ( S14). In this case, a vehicle presence / absence discriminator or an environment-specific vehicle presence / absence discriminator can be used as the discriminator.

  The control unit 20 determines whether or not there is a vehicle (S15). If there is a vehicle (YES in S15), the control unit 20 determines whether or not vehicle information including vehicle position information has been received (S16). The vehicle information may be only position information. When the vehicle information is received (YES in S16), the control unit 20 determines whether or not the vehicle position is within the detection range based on the received position information (S17).

  When the vehicle position is within the detection range (YES in S17), that is, when the vehicle information is received, the vehicle that has transmitted the vehicle information extracts the feature vector and is within the detection range determined to have a vehicle. When it is the same vehicle as the vehicle, the control unit 20 plots the vehicle position on the captured image (S18).

  The control unit 20 calculates a trajectory of the vehicle based on plot positions at a plurality of imaging points (S19), and calculates an infinity point based on the trajectories of the plurality of vehicles (S20). The control unit 20 specifies the optical axis direction of the lens (calculates the depression angle and the rotation angle) (S21), and ends the process.

  When there is no vehicle (NO in S15), when vehicle information is not received (NO in S16), or when the vehicle position is not within the detection range (NO in S17), the control unit 20 performs step S11 and subsequent steps. Continue processing.

  As a result, adjustment work related to the mounting angle of the video camera 10 can be automatically performed instead of manual work, lane regulation required for the adjustment work becomes unnecessary, and work labor and work costs can be reduced. It also leads to prevention of traffic jams and traffic accidents.

  The initial value (provisional value) in the optical axis direction of the lens of the video camera 10 can be obtained by manually setting a plurality of points on the road on the captured image as in the example of FIG. However, the method of obtaining the optical axis direction is not limited to this, and can be obtained by image processing.

  FIG. 17 is a block diagram showing an example of the configuration of the image processing apparatus 200 according to the present invention. The image processing device 200 does not necessarily have to be a device for identifying a vehicle, and may be any device that performs other predetermined image processing.

  The image processing apparatus 200 includes an image input unit 11, an A / D conversion unit 12, an image memory 13, a communication unit 14, a storage unit 15, a control unit 201 that controls the entire apparatus, an image processing unit 202 that performs predetermined image processing, A moving body specifying unit 203, an optical axis direction specifying unit 204, and the like are provided. A video camera 10 is connected to the image input unit 11. Note that the video camera 10 may be a separate device from the image processing apparatus 200, or may be configured such that both are integrated. In addition, the same components as those of the vehicle identification device 100 are denoted by the same reference numerals and description thereof is omitted.

  The moving body specifying unit 203 extracts the feature amount of the captured image, and calculates the position of the moving body (vehicle) on the captured image using the extracted feature amount. The feature amount may be extracted by edge detection, for example, or a HOG (Histograms of Oriented Gradients) feature may be used.

  The optical axis direction identification unit 204 plots the vehicle position calculated by the moving body identification unit 203 on the captured image, and similarly plots the vehicle position calculated at a plurality of imaging points. As a result, a virtual straight line representing the traveling locus of the vehicle can be obtained on the captured image. In order to track the same vehicle, for example, a template matching method or the like can be used. In addition, as the position for tracking the vehicle, for example, the head position of the vehicle can be used.

  The optical axis direction identification unit 204 performs the same process on a plurality of vehicles. Thereby, the virtual straight line showing the traveling locus of each of a plurality of vehicles can be obtained. The optical axis direction specifying unit 204 uses the intersection (infinite point) of a plurality of virtual straight lines and the imaging parameters of the video camera 10 (for example, the size and resolution of the imaging surface, the focal length of the lens, etc.) to The optical axis direction of the lens (the depression angle and the rotation angle of the optical axis) is specified.

  As described above, according to the image processing apparatus 200 of the present invention, the adjustment work related to the mounting angle of the video camera 10 can be automatically performed instead of the manual work, and the lane regulation required for the adjustment work is unnecessary. Therefore, it is possible to reduce work labor and work cost, and to prevent traffic congestion and traffic accidents.

  In addition, according to the vehicle identification device 100 of the present invention, the range in which the vehicle exists is reliably determined by using both the vehicle position obtained from the captured image captured by the video camera 10 and the position information acquired from the vehicle. can do. For this reason, the optical axis direction (the depression angle and the rotation angle) of the lens can be automatically obtained with high accuracy. In addition, adjustment work related to the mounting angle of the video camera 10 can be automatically performed instead of manual work, lane regulation required for the adjustment work becomes unnecessary, and work labor and work cost can be reduced, It also helps prevent traffic jams and traffic accidents.

  In the embodiment described above, the depression angle and the rotation angle of the optical axis of the lens of the video camera 10 are obtained from the intersection of the virtual straight lines obtained by linearly approximating the traveling locus of the vehicle. However, the present invention is not limited to this. It is not a thing. For example, the internal parameters of the video camera such as the focal length of the video camera and the pixel spacing on the imaging surface, external parameters determined by the position and orientation of the video camera, and parameters that combine these internal parameters and external parameters, etc. A perspective projection matrix can be used. In this case, the depression angle and rotation angle of the optical axis of the lens can be calculated by a combination of six or more positions on the road surface and positions on the captured image.

  In the above-described embodiment, the number of lanes, lane positions, and the like on the road can also be detected based on the traveling trajectories of a plurality of vehicles and the vehicle existence ranges. That is, the lane boundary can be obtained as the center position between the travel tracks (passage paths of the vehicle) in each lane. Further, the lane position can be detected by detecting a white line along the tracking direction in a range where the vehicle distribution is small.

  The embodiments and examples disclosed above should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above embodiments and examples but by the scope of claims, and is intended to include all modifications and variations within the meaning and scope equivalent to the scope of claims. .

It is a block diagram which shows an example of a structure of the vehicle identification device which is a moving body identification device which concerns on this invention. It is explanatory drawing which shows a road coordinate system. It is explanatory drawing which shows the attachment state of a video camera. It is explanatory drawing which shows the example of the initial setting of the optical axis direction of the lens of a video camera. It is explanatory drawing which shows an example of a captured image. It is explanatory drawing which shows an example of a detection range. It is explanatory drawing which shows an example of a local area | region. It is explanatory drawing which shows an example of the gradient intensity | strength of a brightness | luminance, and a gradient direction. It is explanatory drawing which shows an example of a gradient direction. It is explanatory drawing which shows an example of the histogram of gradient intensity | strength. It is explanatory drawing which shows the outline | summary of each discriminator of an identification part. It is explanatory drawing which shows the identification content of each discriminator of an identification part. It is explanatory drawing which shows the example of application of the optical axis direction specific mode of a vehicle identification device. It is explanatory drawing which shows an example of the identification method of the optical axis direction of the lens of a video camera. It is explanatory drawing which shows the example of application of the information provision mode of a vehicle identification device. It is a flowchart which shows the optical axis direction specific process sequence. It is a block diagram which shows an example of a structure of the image processing apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video camera 11 Image input part 12 A / D conversion part 13 Image memory 14 Communication part 15 Storage part 20 Control part 30 Partial image specific part 40 Feature-value extraction part 50 Identification part 51 Vehicle presence / absence identification part 52 Vehicle type identification part 53 Environment identification 54 Vehicle presence / absence identifying unit according to environment 60 Vehicle presence position determining unit 70 Optical axis direction specifying unit

Claims (4)

A mobile object identification device that extracts a feature amount from a captured image obtained by imaging an area including a road with an imaging device, and identifies the mobile object using the extracted feature amount,
Position information acquisition means for acquiring position information of the moving body;
A partial image specifying means for specifying a partial image on the captured image;
Feature quantity extraction means for extracting the feature quantity of the partial image identified by the partial image identification means;
Identification means for identifying a moving object using the feature amount extracted by the feature amount extraction means;
When the moving unit is identified by the identifying unit, the position information acquired by the position information acquiring unit, the known optical axis direction of the lens of the imaging device, and the position on the captured image of the partial image specified by the partial image specifying unit Determination means for determining whether or not the moving body that has acquired the position information by the position information acquisition means is captured in the partial image,
A position calculating unit that calculates a position of the moving body on a captured image when the determining unit determines that the moving body is captured;
An optical axis direction that specifies the optical axis direction to correct the optical axis direction by using each position of one or a plurality of moving bodies at a plurality of imaging time points calculated by the position calculation means and imaging parameters of the imaging device A moving body identification device comprising: a specifying unit. Environment identifying means for identifying an environmental situation including weather or day and night using the feature quantity extracted by the feature quantity extraction means,
The identification means includes
The mobile object identification device according to claim 1, wherein the mobile object identification device is configured to identify a mobile object using the environmental situation and the feature amount identified by the environment identification unit. A computer program for causing a computer to extract a feature amount from a captured image obtained by capturing an area including a road with an imaging device, and to function as a means for identifying a moving object using the extracted feature amount,
Computer
A partial image specifying means for specifying a partial image on the captured image;
Feature amount extraction means for extracting the feature amount of the identified partial image;
An identification means for identifying the moving object using the extracted feature amount;
When the moving body is identified, the moving body is captured in the partial image based on the position information of the moving body, the known optical axis direction of the lens of the imaging device, and the position of the identified partial image on the captured image. Determination means for determining whether or not,
If it is determined that the moving body is being imaged, position calculating means for calculating the position of the moving body on the captured image;
Using the calculated positions of one or a plurality of moving bodies at a plurality of imaging time points and the imaging parameters of the imaging device, the optical axis direction specifying means for specifying the optical axis direction to correct the optical axis direction is caused to function. A computer program characterized by the above. An optical axis direction specifying method by a mobile object identification device that extracts a feature amount from a captured image obtained by imaging an area including a road with an imaging device and identifies a mobile object using the extracted feature amount,
Get the location information of the moving object,
Identify the partial image on the captured image,
Extract the feature quantity of the identified partial image,
Identify moving objects using the extracted features,
When the moving object is identified, the moving object that has acquired the position information is based on the acquired position information, the known optical axis direction of the lens of the imaging device, and the position of the specified partial image on the captured image. To determine whether or not
When it is determined that the moving body is imaged, the position of the moving body on the captured image is calculated,
A moving body identification characterized by specifying an optical axis direction to correct the optical axis direction by using the calculated positions of one or a plurality of moving bodies at a plurality of imaging time points and imaging parameters of the imaging apparatus. An optical axis direction specifying method by a device.

Images