JP5115390B2 - Mobile object identification device, computer program, and learning method of mobile object identification device - Google Patents

Description

  The present invention extracts a feature quantity from a captured image obtained by imaging a region including a road, identifies a mobile object using the extracted feature quantity, and realizes the mobile object identification apparatus with a computer The present invention relates to a computer program and a learning method for a mobile object identification device.

  A system in which a roadside infrastructure device and a vehicle (on-vehicle device) exchange information by wired or wireless road-to-vehicle communication and each device realizes a function that cannot be realized independently has been studied. In particular, there is a study on a road-vehicle cooperative safe driving support system that suppresses traffic accidents by providing information on vehicles or pedestrians in a blind spot from the vehicle side to the vehicle side from the infrastructure side. It is being advanced.

  As main technologies for realizing such a system, for example, there are sensors that detect information such as vehicle position information, vehicle type information (for example, automobiles, two-wheeled vehicles, pedestrians, etc.), vehicle movement speed or direction, and the like. is necessary. Examples of such sensors include those that use lasers or those that use images captured by a video camera. The range of information that can be detected, product life, cost, performance, etc. Considering this balance, the image processing method is effective.

However, as one of the problems of an image processing system sensor (image sensor), there are various environmental changes such as weather changes and the influence of surrounding objects on the road, and it is difficult to realize an image processing system that can cope with all of them. On the other hand, a technique is disclosed in which the effects of various disturbances such as tree fluctuations due to changes in sunlight conditions and wind are included in the machine learning process, and these disturbances are excluded from detection targets (Non-Patent Document 1). reference).
"Development of software that automatically detects abnormal behavior in real time from camera images", [online], announced on October 16, 2007, National Institute of Advanced Industrial Science and Technology (AIST) search [August 1, 2008 search], Internet (URL: http://www.aist.go.jp/aist_j/press_release/pr2007/pr20071016/pr20071016.html)

  If the technique of Non-Patent Document 1 is applied, it is possible to collect all the data of a video scene that may occur such as an environmental change in advance, and to learn an identification device that identifies the vehicle using the collected data as learning data. However, if the location where the camera is installed is different, the road and traffic conditions will change, and even at the same location, there are various environmental change factors such as time of day, day of the week, and weather. Collecting all the data requires a great deal of labor and expense. For this reason, the conventional method is not so feasible and has a limited application range, and it has been desired to efficiently collect learning data necessary for learning of an identification device for identifying a vehicle or the like.

  The present invention has been made in view of such circumstances, and can identify a moving body such as a vehicle with high accuracy and can efficiently collect learning data, and the moving body. It is an object of the present invention to provide a computer program for realizing an identification device by a computer and a learning method for a mobile object identification device.

  A mobile object identification device according to a first aspect of the present invention is a mobile object identification device that extracts a feature amount from a captured image obtained by imaging an area including a road and identifies the mobile object using the extracted feature amount. Mobile body information acquisition means for acquiring mobile body information including position information of the mobile body, partial image specification means for specifying a partial image on the captured image, and feature amounts of the partial images specified by the partial image specification means Feature amount extracting means to be extracted; identification means for identifying a moving body using the feature amount extracted by the feature amount extracting means; and when the moving body is identified by the identifying means, acquired by the moving body information acquisition means Based on the position information and the position on the captured image of the partial image specified by the partial image specifying means, it is determined whether or not the mobile object whose moving body information is acquired by the moving body information acquisition means is captured in the partial image. Determination means for determining, and If it is determined that the moving body is imaged by the means, the identification unit uses the feature amount extracted by the feature amount extraction unit and the moving body information acquired by the mobile body information acquisition unit to use the information used for identification. And learning means for learning.

  According to a second aspect of the present invention, there is provided a moving body identification apparatus according to the first aspect, further comprising a moving body identification means for determining the presence or absence of a moving body, wherein the learning means includes the feature amount extracted by the feature amount extraction means and the determination The mobile object identifying means is configured to learn information used for identification using the determination result determined by the means.

  According to a third aspect of the present invention, in the first or second aspect, the mobile unit identification device includes an environment identification unit that identifies an environmental situation including weather, and the mobile unit information acquisition unit includes a wiper operation information. Information is acquired, and the learning unit uses the feature amount extracted by the feature amount extraction unit and the moving body information acquired by the moving body information acquisition unit to be used for identification by the environment identification unit. It is configured to learn information.

  According to a fourth aspect of the present invention, in the first or second aspect of the present invention, the mobile unit identification device further includes an environment identification unit that identifies environmental conditions including day and night, and the mobile unit information acquisition unit includes a vehicle light. Information is acquired, and the learning unit uses the feature amount extracted by the feature amount extraction unit and the moving body information acquired by the moving body information acquisition unit to be used for identification by the environment identification unit. It is configured to learn information.

  According to a fifth aspect of the present invention, there is provided a mobile unit identification apparatus according to the third or fourth aspect, further comprising environment-specific mobile unit identification means for determining the presence or absence of a mobile unit for each different environment, wherein the learning unit includes the feature quantity. The environment-based mobile object identification means is configured to learn information used for identification using the feature amount extracted by the extraction means, the determination result determined by the determination means, and the mobile object information acquired by the mobile object information acquisition means. It is characterized by being.

  According to a sixth aspect of the present invention, there is provided a mobile unit identification apparatus according to any one of the second to fifth aspects of the present invention, comprising a type identification unit for determining the type of the mobile unit, It is configured to acquire moving body information including whether the vehicle is a motorcycle or a pedestrian, and the learning means uses the feature amount extracted by the feature amount extraction means and the moving body information acquired by the moving body information acquisition means. The type identifying means is used to learn information used for identification.

  According to a seventh aspect of the present invention, in any one of the first to sixth aspects, the mobile object identification device acquires learning data for causing the identification means to learn information used for identification from another mobile object identification device. Learning data acquisition means is provided, and the learning means is configured to cause the identification means to learn information used for identification using the learning data acquired by the learning data acquisition means.

  According to an eighth aspect of the present invention, in the seventh aspect, the mobile object identification device includes: an imaging condition acquisition unit that acquires an imaging condition of another mobile body identification device; an imaging condition acquired by the imaging condition acquisition unit; And determining means for determining whether or not learning is performed using the learning data acquired by the learning data acquiring means.

  A computer program according to a ninth aspect is a computer program for causing a computer to function as a means for extracting a feature amount from a captured image obtained by capturing an area including a road and identifying a moving object using the extracted feature amount. A computer program comprising: a computer, a partial image specifying unit that specifies a partial image on a captured image; a feature amount extracting unit that extracts a feature amount of the specified partial image; and a moving object using the extracted feature amount. When identifying the identifying means and the moving body, it is determined whether the moving body is imaged in the partial image based on the position information of the moving body and the position of the specified partial image on the captured image. When it is determined that the moving unit is picked up by the determining unit, the identifying unit is made to learn information used for identification using the extracted feature amount and the moving unit information of the moving unit. Characterized in that to function as learning means.

  According to a tenth aspect of the present invention, there is provided a learning method for a mobile object identification device including an identification unit that extracts a feature amount from a captured image obtained by imaging a region including a road and identifies the mobile object using the extracted feature amount. The mobile object identification device learning method, wherein the mobile object identification device acquires mobile object information including position information of a mobile object, specifies a partial image on a captured image, and features of the specified partial image When the moving body is identified using the extracted feature amount, the moving body information is acquired based on the acquired position information and the position on the captured image of the specified partial image. It is determined whether or not the moving body is captured in the partial image, and when it is determined that the moving body is captured, the extracted means and the acquired moving body information are used for identification in the identification unit. Characterized by learning information

  In the first invention, the ninth invention, and the tenth invention, the partial image on the captured image is specified, and the feature amount of the specified partial image is extracted. The partial image is, for example, a detection range of 24 × 32 pixels, and the detection range is scanned on the captured image to extract the feature amount of the detection range. The feature amount may be, for example, a HOG (Histograms of Oriented Gradients) feature, and is a feature vector in which the intensity gradient intensity of the detection range is histogrammed for each gradient direction. Note that the feature amount is not limited to the HOG feature, and may be extracted by edge detection. When a moving object (for example, a vehicle) is identified using the extracted feature value, the position information of the moving object and the specified partial image (detection range from which the feature value is extracted when the moving object is identified) on the captured image Based on the position, the moving body that has acquired the moving body information (for example, information on the vehicle speed, information on the vehicle type, information indicating the operating state of the headlight, wiper, etc. in addition to the vehicle position information) is included in the partial image (detection range). It is determined whether or not there is. When it is determined that the moving body is imaged (within the detection range), information used for identification (for example, identification of the classifier) using the extracted feature amount and the acquired moving body information Parameter).

  When a feature amount (feature vector) is input, the discriminator outputs a discrimination result (determination information, for example, there is a vehicle, no vehicle, etc.) based on the input feature amount and the discriminator parameter. When it is determined that the mobile body (vehicle) is within the detection range, the mobile body identification device can extract the feature amount (input information of the identifier) when the vehicle is present in the detection range, The vehicle information (output information of the discriminator) of the vehicle imaged within the detection range can be acquired. That is, it is possible to obtain the feature amount when the vehicle exists as the input information of the discriminator and the determination information with the vehicle as the output information, and the discriminator parameters can be obtained by using these information, Can learn the vessel. Further, the obtained discriminator parameter and the extracted feature amount can be stored as learning data. Thereby, many video scenes can be collected efficiently, learning of the discriminator can be performed efficiently, and discrimination accuracy of the discriminator can be improved.

  In the second invention, using the extracted feature amount and the determination result, the mobile body identifying means (vehicle presence / absence classifier) that determines the presence / absence of the mobile body (vehicle) is caused to learn information used for identification. For example, a vehicle presence / absence discriminator parameter can be obtained using a feature amount and a determination result (for example, determination information with a vehicle) when there is a vehicle as input information of the vehicle presence / absence discriminator. You can learn. In addition, the vehicle presence / absence discriminator parameters can be obtained using the feature amount and the determination result (for example, determination information indicating no vehicle) when no vehicle is present as input information of the vehicle presence / absence discriminator. You can learn. Thereby, many video scenes can be efficiently collected, the vehicle presence / absence discriminator can be efficiently learned, and the identification accuracy of the vehicle presence / absence discriminator can be improved.

  In the third aspect of the invention, information used for identification is learned by an environment identification unit (environment identifier) that identifies an environmental situation including weather using the extracted feature amount and the acquired moving body information. The environmental situation is, for example, fine weather, rainy weather, and the like. The environment classifier parameters for rainy weather can be obtained using the feature quantity as input information of the environment classifier and the moving body information (for example, determination information with wiper operation), and the environment classifier can be learned. . As a result, many video scenes can be efficiently collected, the environment identifier can be learned efficiently, and the identification accuracy of the environment identifier can be improved.

  In the fourth invention, using the extracted feature quantity and the acquired moving body information, the environment identification means (environment identifier) for identifying the environmental situation including day and night is learned. The environmental situation is, for example, daytime, evening, night. It is possible to obtain an environment classifier parameter for night using feature values and moving body information (for example, determination information with headlight operation) as input information of the environment classifier, and to train the environment classifier. it can. As a result, many video scenes can be efficiently collected, the environment identifier can be learned efficiently, and the identification accuracy of the environment identifier can be improved.

  In the fifth invention, using the extracted feature amount, the determination result (for example, determination information with or without a vehicle) and the acquired moving body information, the presence or absence of the moving body is determined for each different environment. Information used for identification is learned by the environment-specific moving body identification means (environment-specific vehicle presence / absence classifier). For example, the presence / absence of a vehicle for rainy weather using the feature amount as input information of the vehicle presence / absence discriminator according to the environment, the determination result (for example, determination information with a vehicle) and the moving body information (for example, determination information with a wiper operation) A discriminator parameter can be obtained, and an environment-specific vehicle presence / absence discriminator can be learned. Further, the presence / absence of a vehicle for nighttime using the feature amount as input information of the vehicle presence / absence discriminator according to the environment, and the determination result (for example, determination information with a vehicle) and moving body information (for example, determination information with headlight operation). A discriminator parameter can be obtained, and an environment-specific vehicle presence / absence discriminator can be learned. As a result, many video scenes can be efficiently collected, the environment-specific vehicle presence / absence classifier can be efficiently learned, and the identification accuracy of the environment-specific vehicle presence / absence classifier can be improved. .

  In the sixth invention, using the extracted feature quantity and the acquired mobile object information, the type identification means (vehicle type identifier) for determining the type of the mobile object is made to learn information used for identification. The type of the moving body is, for example, vehicle type information such as a normal vehicle, a large vehicle, a two-wheeled vehicle, and a pedestrian. The vehicle type discriminator parameter can be obtained by using the feature quantity as the input information of the vehicle type discriminator and the moving body information (for example, car type information), and the vehicle type discriminator can be learned. As a result, many video scenes can be efficiently collected, the vehicle type discriminator can be learned efficiently, and the discrimination accuracy of the car type discriminator can be improved.

  In the seventh aspect of the invention, learning data for learning the information used for identification by the identification means (identifier) is acquired from another mobile object identification device, and the identification means is used for identification using the acquired learning data. Learn the information to use. It is possible to collect video scenes at other points where other mobile unit identification devices are installed, collect more video scenes, and learn classifiers more efficiently. The identification accuracy of the classifier can be further improved.

  In the eighth invention, whether or not to acquire the imaging condition of the other mobile object identification device and learn using the acquired learning data using the comparison result between the acquired imaging condition and its own imaging condition To decide. Imaging conditions are conditions, such as a depression angle or a rotation angle, of an imaging device (video camera etc.), for example. When the imaging conditions are compared and are equal or approximate, the learning data acquired by another moving body identification device is used to learn information used for identification in its own classifier. As a result, it is possible to efficiently learn the discriminator using only valid learning data, and it is possible to improve the discrimination accuracy of the discriminator.

  In the present invention, many video scenes can be collected efficiently, learning of the discriminator can be performed efficiently, and discrimination accuracy of the discriminator can be improved.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments. FIG. 1 is a block diagram showing an example of a configuration of a vehicle identification device 100 that is a moving body identification device according to the present invention. The vehicle identification device 100 includes an image input unit 11, an A / D conversion unit 12, an image memory 13, a communication unit 14, a storage unit 15, a control unit 20 that controls the entire device, a partial image specification unit 30, and a feature amount extraction unit 40. , An identification unit 50 as an identification unit (identifier), a vehicle presence position determination unit 60, a learning unit 70, and the like. The identification unit 50 includes a vehicle presence / absence identification unit 51, a vehicle type identification unit 52, an environment identification unit 53, an environment-specific vehicle presence / absence identification unit 54, and the like. A video camera 10 is connected to the image input unit 11. Note that the video camera 10 may be a separate device from the vehicle identification device 100, or may be configured such that both are integrated.

  The video camera 10 is installed at a required point near the road in a state where imaging conditions such as a predetermined height, depression angle, and rotation angle are set in order to capture a required area including the road. The video camera 10 outputs a captured image obtained by imaging to the image input unit 11 as a video signal (analog signal).

  The image input unit 11 outputs the acquired video signal to the A / D conversion unit 12.

  The A / D converter 12 converts the input video signal into a digital signal, and stores the converted digital signal in the image memory 13 as image data. The captured image input from the video camera 10 via the image input unit 11 is synchronized with the frame rate of the video camera 10 (interval at the time of imaging, for example, 30 frames per second) in units of one frame (for example, 240 Is stored in the image memory 13 as (× 320 pixels) image data.

  FIG. 2 is an explanatory diagram illustrating an example of a captured image. As shown in FIG. 2, by installing the video camera 10 in the vicinity of a road under a predetermined imaging condition, it is possible to image a vehicle traveling on a required road. In FIG. 2, the size of the captured image is 240 × 320 pixels. Note that the size of the captured image is not limited to this.

  The communication unit 14 includes a narrow-area communication function, a mid-range communication function such as a wireless LAN such as a UHF band or a VHF band, and a wide-area communication function such as a mobile phone, PHS, multiple FM broadcasting, or Internet communication. The communication unit 14 obtains predetermined vehicle information (for example, vehicle position information, vehicle speed information, vehicle type information, operation statuses of vehicle lights such as wipers and headlights) from a vehicle existing in or near the imaging region of the video camera 10. Information). In addition, the communication unit 14 transmits the identification information obtained as a result of identifying the vehicle to a vehicle in the vicinity to provide the identification information. Moreover, the communication part 14 transmits / receives predetermined information between the server apparatus installed in the traffic control center etc., and another vehicle identification apparatus.

  The storage unit 15 stores data obtained by processing of the vehicle identification device 100, data received via the communication unit 14, and the like under the control of the control unit 20.

  The partial image specifying unit 30 specifies a partial image having a required size on the captured image, and scans the specified partial image on the captured image. The partial image defines a detection range for extracting a feature amount described later.

  FIG. 3 is an explanatory diagram showing an example of a detection range. As shown in FIG. 3, the size of the detection range is, for example, 24 × 32 pixels. When scanning the detection range on the captured image, as shown in FIG. 3, the detection range can be moved by 16 pixels in the horizontal direction in one scan.

  The feature quantity extraction unit 40 extracts feature quantities in the detection range. The feature quantity can be a HOG (Histograms of Oriented Gradients) feature, and is a feature vector in which the gradient intensity of the luminance in the detection range is histogrammed for each gradient direction. Note that the feature amount is not limited to the HOG feature, and may be extracted by edge detection.

  Hereinafter, a method for calculating a feature vector as a feature amount will be described. FIG. 4 is an explanatory diagram showing an example of the local region. The local area (block) is extracted so as to partially overlap from the detection range of 24 × 32 pixels, and has a size of 16 × 16 pixels, for example. Accordingly, six local regions can be extracted from the detection range. The local area is composed of four cells (area of 8 × 8 pixels).

  FIG. 5 is an explanatory diagram showing an example of the luminance gradient strength and gradient direction. The position of the pixel in the cell is represented by (x, y). It is assumed that the luminance gradient intensity at pixel (x, y) is m (x, y) and the gradient direction is θ (x, y). The gradient strength m (x, y) and the gradient direction θ (x, y) can be obtained by Equations (1) to (3). Here, fx (x, y) is a luminance gradient in the horizontal direction (x direction), fy (x, y) is a luminance gradient in the vertical direction (y direction), and in addition to Expression (3), for example, It can be obtained by a Sobel filter. L (x, y) is the luminance of the pixel (x, y).

  Then, a histogram in eight directions is created for each cell. FIG. 6 is an explanatory diagram showing an example of a gradient direction, and FIG. 7 is an explanatory diagram showing an example of a histogram of gradient strength. As shown in FIG. 6, the gradient direction is divided every 45 °, and eight (j = 1 to 8) gradient directions are set. Further, as shown in FIG. 7, a histogram is created by assigning the calculated intensity gradient intensity for each gradient direction. When calculating the gradient strength, the edge strength can be smoothed by applying a Gaussian filter of σ = 1.6 inversely proportional to the distance from the center of the local region (block) to the local region.

  Histograms obtained for each cell are collected in a local region, and normalized by dividing each histogram element (gradient strength) by the maximum element (gradient strength maximum) of the histogram for each local region. As a result, the maximum component of the 32-dimensional feature vector extracted in one local region (block) is 1. Since there are six local regions in one detection range, a feature vector V of 192 dimensions (32 dimensions × 6 = 192 dimensions) can be calculated for one detection range as shown in Expression (4).

  Since the gradient of adjacent pixels is normalized by creating a histogram for each detection range, local geometrical changes that are less affected by noise or brightness changes and the presence or absence of shadows on captured images obtained by imaging roads ( Robust against translation, rotation, etc.).

  The identification unit 50 uses the feature quantity (feature vector) extracted by the feature quantity extraction unit 40 as input information, performs a predetermined identification determination including the presence or absence of a vehicle, and outputs an identification result (determination information). The discriminating unit 50 includes a vehicle presence / absence discriminating unit 51 having a vehicle presence / absence discriminator, a vehicle type discriminating unit 52 having a vehicle type discriminator, an environment discriminating unit 53 having an environment discriminator, and an environment-specific vehicle presence / absence discriminating unit having environment-specific vehicle presence / absence discriminators. Part 54 and the like.

  FIG. 8 is an explanatory diagram showing an outline of each discriminator of the discriminating unit 50, and FIG. 9 is an explanatory diagram showing identification contents of each discriminator of the discriminating unit 50. When the feature vector V is input, the vehicle presence / absence discriminator outputs identification information Dc = Ac · V. Here, Ac is a discriminator parameter of the vehicle presence / absence discriminator. The vehicle presence / absence discriminator performs a predetermined calculation based on the discriminator parameter Ac on the input feature vector V, and determines whether there is a vehicle or no vehicle as an identification result (class). For example, if the output identification information Dc is Dc> 0, it is determined that there is a vehicle, and if Dc <0, it can be determined that there is no vehicle. Note that the determination condition is an example, and the present invention is not limited to this.

  In addition, when the feature vector V is input, the vehicle type identifier outputs identification information Dt = At · V. Here, At is a discriminator parameter of the vehicle type discriminator. The vehicle type discriminator performs a predetermined calculation based on the discriminator parameter At on the input feature vector V, and distinguishes the classification result (class), for example, an ordinary vehicle, a large vehicle, a two-wheeled vehicle or a pedestrian. judge. For example, if the output identification information Dt is 0 <Dt <Tt1, it is determined as a normal vehicle, if Tt1 <Dt <Tt2, it is determined as a large vehicle, and if Tt2 <Dt <Tt3, it is determined as a motorcycle. If Dt <0, it can be determined as a pedestrian. Note that the determination condition is an example, and the present invention is not limited to this.

  Further, when the feature vector V is input, the environment classifier outputs the identification information Dw = Aw · V. Here, Aw is a classifier parameter of the environment classifier. The environment classifier performs a predetermined calculation based on the classifier parameter Aw on the input feature vector V, and the classification result (class) is, for example, different from daytime, evening, sunny night, rainy night, etc. Determine. For example, if the output identification information Dw is 0 <Dw <Tw1, it is determined as daytime, if Tw1 <Dw <Tw2, it is determined as evening, and if Tw2 <Dw <Tw3, it is determined as a clear night. If Tw3 <Dw <Tw4, it can be determined that the rainy night. Note that the determination condition is an example, and the present invention is not limited to this.

  Further, when the feature vector V is input, the environment-specific vehicle presence / absence discriminator outputs identification information Dcw = Acw · V. Here, Acw is a discriminator parameter of the vehicle presence / absence discriminator by environment. The environment-specific vehicle presence / absence discriminator performs a predetermined calculation based on the discriminator parameter Acw on the input feature vector V, and the discrimination result (class) is, for example, a day vehicle, an evening vehicle, or a sunny vehicle. It is determined whether there is a rainy night vehicle, no daytime vehicle, no evening vehicle, no clear night vehicle, no rainy night vehicle. Note that the determination contents can be distinguished according to the value of the identification information Dcw, as with the above-described classifiers.

  When the vehicle presence / absence discriminating unit 51 identifies the vehicle in the detection range (partial image) (when it is determined that there is a vehicle), the vehicle presence position determination unit 60 uses the vehicle information received from the vehicle via the communication unit 14. Based on the position information of the included vehicle, it is determined whether or not the vehicle exists within the detection range. That is, when the vehicle presence position determination unit 60 receives the vehicle information, whether the vehicle that transmitted the vehicle information is the same vehicle as the vehicle in the detection range in which the feature vector is extracted and determined to be a vehicle. Determine whether or not. The determination of whether or not they are the same vehicle can include a required error range according to the position detection accuracy of the vehicle. That is, the determination may be made based on whether or not the position where the error range is added to the vehicle position information is within the detection range.

  More specifically, whether or not the vehicle exists within the detection range can be performed as follows. The position of the vehicle included in the vehicle information received from the vehicle is defined as Pcar, and the center position of the detection range in which the feature vector is extracted and determined to be present is defined as Pdet. However, Pcar and Pdet are positions on the actual road with the camera installation position as a reference. In this case, when the distance between the two positions is equal to or smaller than a certain error (Err) range, that is, when | Pcar−Pdet | <Err, it is determined that the vehicle exists within the detection range.

  The learning unit 70 obtains classifier parameters of each classifier such as a vehicle presence / absence classifier, a vehicle type classifier, an environment classifier, an environment-specific vehicle presence / absence classifier, and the like (information used for classification) for each classifier To learn. Hereinafter, a learning method of the classifier will be described.

  FIG. 10 is an explanatory diagram illustrating an example of a learning method for a classifier. In FIG. 10, a vehicle presence / absence discriminator is described as an example, but the learning method of other discriminators is the same. As shown in FIG. 10A, when the feature vector V is input, the vehicle presence / absence discriminator performs a predetermined calculation based on the discriminator parameter Ac on the feature vector V to obtain the identification information Dc = Ac · V. Is output. For simplicity, the feature vector V will be described as a two-dimensional vector, but the learning method can be considered in the same manner regardless of the number of dimensions.

  When it is determined by the vehicle presence position determination unit 60 that the vehicle that has transmitted the vehicle information (including the position information) is the same vehicle as the vehicle within the detection range in which the feature vector is extracted and determined to be a vehicle, Let the extracted feature vector be V1. In this case, as shown in FIG. 10B, the learning data includes a feature vector V1 when the vehicle is identified in the detection range and a vehicle in the detection range based on the vehicle information transmitted by the vehicle. Then, the determination information d determined by the vehicle presence position determination unit 60 (for example, d = 1, with a vehicle) exists. By repeating the same processing for other vehicles imaged at different points in time, it is possible to obtain feature vectors V2,... And corresponding determination information d,. Further, feature vectors Vk when the vehicle does not exist within the detection range,... And determination information (d = 0),.

  As shown in FIG. 10C, the learning unit 70 learns the vehicle presence / absence discriminator by calculating the discriminator parameter Ac using the collected learning data.

  When learning the vehicle presence / absence discriminator, when only one vehicle is captured on the captured image, or when a plurality of vehicles are captured, the vehicles are separated by at least about one. Since learning is easy when the user is doing, learning may be performed when such a condition is satisfied.

  The position information transmitted by the vehicle can be obtained, for example, by performing position detection using a GPS mounted on the vehicle. Further, if the position detection with higher accuracy becomes possible as the technology develops, more accurate learning can be performed by setting the error range to be smaller. Furthermore, if the ratio of vehicles that can transmit vehicle information including position information increases, learning opportunities increase, and learning can proceed more quickly.

  If learning unit 70 can receive vehicle type information in addition to vehicle position information when learning a vehicle presence / absence discriminator, it can also learn a vehicle type discriminator. In this case, the learning data includes the feature vector V when the vehicle is identified in the detection range and the determination information d determined based on the vehicle type information transmitted by the vehicle (for example, a normal vehicle, a large vehicle, a two-wheeled vehicle, etc. ) And exist.

  The learning unit 70 can learn the environment discriminator using information indicating the operation state of the wiper and the vehicle lamp received from the vehicle as the determination information. Thereby, by learning the brightness around the road, the rainfall state, and the road surface state, it is possible to learn for vehicle presence and vehicle type determination according to the environment. For example, when 20% or less of the vehicles that have transmitted information have their headlights turned on, it is possible to determine that it is daytime and use the classifier learned at that time for daytime. Further, when 20% or more and 80% or less of the vehicles that transmitted the information have the headlights on, it is determined that the vehicle is in the evening, and the classifier learned at that time can be used for the evening. In addition, when 80% or more of the vehicles that have transmitted information turn on the headlights and 20% or less are in the wiper operation, it is determined that the night is sunny, and the classifier learned at that time is for sunny night Can do. Furthermore, when 80% or more of the vehicles that have transmitted information turn on the headlights and more than 80% are in the wiper operation, it is determined that it is rainy night, and the classifier learned at that time is used for rainy night. it can. When learning the environment discriminator, the vehicle is not necessarily present in the detection range.

  The learning unit 70 can also learn an environment-specific vehicle presence / absence discriminator by learning a vehicle presence / absence discriminator and an environment discriminator.

  FIG. 11 is an explanatory diagram illustrating an application example of the information providing mode of the vehicle identification device 100. As shown in FIG. 11, the vehicle identification device 100 is installed near an intersection, and a required imaging range is imaged with a video camera. The vehicle identification device 100 identifies an out-of-sight vehicle or an out-of-sight pedestrian with respect to the vehicles C1, C2, and the like outside the imaging range, and provides information based on the identification result. Thereby, it is possible to provide information on a vehicle or a pedestrian at a spot that becomes a blind spot from the driver of the vehicle, and it is possible to prevent a traffic accident.

  FIG. 12 is an explanatory diagram illustrating an application example of the learning mode of the vehicle identification device 100. As shown in FIG. 12, the vehicle identification device 100 is installed in the vicinity of an intersection, and a required imaging range is imaged with a video camera. The vehicle identification device 100 captures the vehicle C3 traveling in the imaging range and receives vehicle information including position information of the vehicle C3 from the vehicle C3. The vehicle identification device 100 learns the classifier by using the feature amount (feature vector) extracted from the captured image and / or the vehicle information received from the vehicle as determination information. As a result, it is not necessary to take a lot of manpower, and data for learning the discriminator can be efficiently collected simply and inexpensively.

  The control unit 20, the partial image specifying unit 30, the feature amount extraction unit 40, the identification unit 50, the vehicle presence position determination unit 60, the learning unit 70, and the like can be realized by dedicated hardware circuits. It can also be realized by causing a CPU to execute a computer program that defines the function of each unit.

  Next, the operation of the vehicle identification device 100 will be described. FIG. 13 is a flowchart showing a processing procedure of the vehicle identification device 100. The control unit 20 acquires a captured image (S11), scans after specifying a detection range as a partial image (S12). The control unit 20 calculates a feature vector as a feature amount of the detection range (S13), and performs an identification process (for example, identification of the presence / absence of a vehicle) by performing an operation based on the classifier parameter on the calculated feature vector ( S14).

  The control unit 20 determines whether or not there is a vehicle (S15). If there is a vehicle (YES in S15), the control unit 20 determines whether or not vehicle information including vehicle position information has been received (S16). When the vehicle information is received (YES in S16), the control unit 20 determines whether or not the vehicle position is within the detection range based on the received position information (S17).

  When the vehicle position is within the detection range (YES in S17), that is, when the vehicle information is received, the vehicle that has transmitted the vehicle information extracts the feature vector and is within the detection range determined to have a vehicle. When it is the same vehicle as the vehicle, the control unit 20 performs a learning process using the extracted feature vector and vehicle information (S18). Thereby, the discriminator parameter of the discriminator is calculated.

  The control unit 20 stores the feature vector and the calculated discriminator parameter as learning data (S19). The control unit 20 determines whether or not the entire captured image has been scanned (S20). If not all has been scanned (NO in S20), the processing from step S12 is continued.

  When all scanning is performed (YES in S20), the control unit 20 stores the identification result (S21), determines whether or not to end the process (S22), and does not end the process (NO in S22), step When the processing after S11 is continued and the processing is terminated (YES in S22), the processing is terminated.

  Further, when there is no vehicle (NO in S15), when vehicle information is not received (NO in S16), or when the vehicle position is not within the detection range (NO in S17), the control unit 20 performs step S21. Process.

  Note that, when the vehicle position is within the detection range in the above-described processing, the learning process is performed, whereby the vehicle presence / absence discriminator and the vehicle type discriminator can be learned. The environment classifier can be learned whether the vehicle position is within the detection range or not.

  In practical use, the classifier is trained in advance using the collected data set. If different information from the classifier results is obtained through road-to-vehicle communication, this is added to the training data. Then, the discriminator can be re-learned, and adjustment according to the point can be performed.

  Moreover, the learning result obtained by the vehicle identification device 100 can be transmitted to the server device and used for the learning data of the vehicle identification device at another point. FIG. 14 is a block diagram illustrating a configuration of the server device 200. The server device 200 includes a control unit 201 that controls the entire device, a communication unit 202, a storage unit 203, a learning unit 204, a classification unit 205, and the like.

  The communication unit 202 has the same function as the communication unit 14 and receives learning data and imaging conditions at each point (each vehicle identification device). Imaging conditions are conditions, such as a depression angle or a rotation angle, of an imaging device (video camera etc.), for example.

  The classification unit 205 classifies the learning data collected based on the acquired imaging conditions. That is, it classifies for every learning data of the point where imaging conditions are equal or approximate. The communication unit 202 transmits learning data classified according to the imaging conditions at each point (each vehicle identification device). Each vehicle identification device learns its own classifier using learning data including learning data obtained at other points. Thereby, learning using more video scenes can be performed efficiently, and the discrimination accuracy of the discriminator can be improved. The learning unit 204 has the same function as that of the learning unit 70, and uses the learning data classified based on the imaging condition to cause the server device to learn the classifier, and uses the result as one or more other vehicles. It can also be transmitted to the identification device.

  As described above, according to the present invention, the cost required for data collection can be reduced by efficiently collecting data of possible changes. Furthermore, by feeding back the collected data, the identification accuracy of each vehicle identification device is improved, more accurate information is provided to the vehicle, and traffic accidents can be further suppressed. Further, the present invention can be applied to an abnormal traffic monitoring system, a parking lot system, a general road traffic control system, etc. in a long distance tunnel.

  The embodiments and examples disclosed above should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above embodiments and examples but by the scope of claims, and is intended to include all modifications and variations within the meaning and scope equivalent to the scope of claims. .

It is a block diagram which shows an example of a structure of the vehicle identification device which is a moving body identification device which concerns on this invention. It is explanatory drawing which shows an example of a captured image. It is explanatory drawing which shows an example of a detection range. It is explanatory drawing which shows an example of a local area | region. It is explanatory drawing which shows an example of the gradient intensity | strength of a brightness | luminance, and a gradient direction. It is explanatory drawing which shows an example of a gradient direction. It is explanatory drawing which shows an example of the histogram of gradient intensity | strength. It is explanatory drawing which shows the outline | summary of each discriminator of an identification part. It is explanatory drawing which shows the identification content of each discriminator of an identification part. It is explanatory drawing which shows an example of the learning method of a discriminator. It is explanatory drawing which shows the example of application of the information provision mode of a vehicle identification device. It is explanatory drawing which shows the example of application of the learning mode of a vehicle identification device. It is a flowchart which shows the process sequence of a vehicle identification device. It is a block diagram which shows the structure of a server apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video camera 11 Image input part 12 A / D conversion part 13 Image memory 14 Communication part 15 Storage part 20 Control part 30 Partial image specific part 40 Feature-value extraction part 50 Identification part 51 Vehicle presence / absence identification part 52 Vehicle type identification part 53 Environment identification 54 Vehicle presence / absence identification unit according to environment 60 Vehicle presence position determination unit 70 Learning unit

Claims (10)

A mobile object identification device that extracts a feature amount from a captured image obtained by imaging a region including a road and identifies a mobile object using the extracted feature amount,
Mobile body information acquisition means for acquiring mobile body information including position information of the mobile body;
A partial image specifying means for specifying a partial image on the captured image;
Feature quantity extraction means for extracting the feature quantity of the partial image identified by the partial image identification means;
Identification means for identifying a moving object using the feature amount extracted by the feature amount extraction means;
When the moving body is identified by the identifying means, the moving body information acquiring means uses the positional information acquired by the moving body information acquiring means and the position on the captured image of the partial image specified by the partial image specifying means. Determining means for determining whether or not the moving body that has acquired the moving body information is captured in the partial image;
When the determination unit determines that the moving body is imaged, the identification unit uses the feature amount extracted by the feature amount extraction unit and the mobile unit information acquired by the mobile unit information acquisition unit for identification. A moving body identification device comprising: learning means for learning information. A moving body identification means for determining the presence or absence of a moving body is provided,
The learning means includes
2. The apparatus according to claim 1, wherein the moving object identifying unit is made to learn information used for identification using the feature amount extracted by the feature amount extracting unit and the determination result determined by the determining unit. The moving body identification device described. Environmental identification means for identifying environmental conditions including the weather,
The mobile body information acquisition means includes
It is configured to acquire moving body information including movement information of the wiper,
The learning means includes
The environment identification unit is configured to learn information used for identification using the feature amount extracted by the feature amount extraction unit and the mobile unit information acquired by the mobile unit information acquisition unit. The mobile object identification device according to claim 1 or 2. An environmental identification means for identifying environmental conditions including day and night,
The mobile body information acquisition means includes
It is configured to acquire moving body information including car lights,
The learning means includes
The environment identification unit is configured to learn information used for identification using the feature amount extracted by the feature amount extraction unit and the mobile unit information acquired by the mobile unit information acquisition unit. The mobile object identification device according to claim 1 or 2. Provided with a moving body identification means for each environment for determining the presence or absence of a moving body for each different environment
The learning means includes
Using the feature amount extracted by the feature amount extraction unit, the determination result determined by the determination unit, and the mobile unit information acquired by the mobile unit information acquisition unit, the environment-specific mobile unit identification unit learns information used for identification. The mobile object identification device according to claim 3, wherein the mobile object identification device is configured as described above. Provided with a type identifying means for determining the type of the moving object,
The mobile body information acquisition means includes
It is configured to acquire moving body information including the type of vehicle or motorcycle or pedestrian,
The learning means includes
The feature identifying unit is configured to cause the type identifying unit to learn information used for identification using the feature amount extracted by the feature amount extracting unit and the moving body information acquired by the moving body information acquiring unit. The mobile object identification device according to any one of claims 2 to 5. Learning data acquisition means for acquiring learning data for causing the identification means to learn information used for identification from another mobile object identification device;
The learning means includes
7. The configuration according to claim 1, wherein learning information acquired by the learning data acquisition unit is used to cause the identification unit to learn information used for identification. 8. Mobile object identification device. Imaging condition acquisition means for acquiring imaging conditions of another mobile object identification device;
Determining means for determining whether or not to learn using the learning data acquired by the learning data acquisition means, using a comparison result between the imaging conditions acquired by the imaging condition acquisition means and the own imaging conditions; The mobile object identification device according to claim 7. A computer program for extracting a feature amount from a captured image obtained by imaging a region including a road, and causing the computer to function as a means for identifying a mobile object using the extracted feature amount,
Computer
A partial image specifying means for specifying a partial image on the captured image;
Feature amount extraction means for extracting the feature amount of the identified partial image;
An identification means for identifying the moving object using the extracted feature amount;
A determination unit that determines whether or not the moving body is captured in the partial image based on position information of the moving body and a position on the captured image of the identified partial image when the mobile body is identified;
When it is determined that the moving body is imaged, the identifying means uses the extracted feature amount and the moving body information of the moving body to function as learning means for learning information used for identification. Computer program. A method for learning a mobile object identification device including an identification unit that extracts a feature amount from a captured image obtained by imaging a region including a road and identifies the mobile object using the extracted feature amount,
The mobile object identification device includes:
Get the mobile object information including the position information of the mobile object,
Identify the partial image on the captured image,
Extract the feature quantity of the identified partial image,
Identify moving objects using the extracted features,
When identifying the moving body, based on the acquired position information and the position on the captured image of the identified partial image, it is determined whether the moving body that acquired the moving body information is captured in the partial image,
A learning method for a mobile object identification device, wherein when the mobile object is determined to be imaged, the identification unit is made to learn information used for identification using the extracted feature amount and the acquired mobile object information.

Images