JP2007058751A - Apparatus, method, and program for discriminating object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the reduction of accuracy in the case of discriminating an object in an image by an object discrimination apparatus. <P>SOLUTION: The object discrimination apparatus includes a synthetic discrimination table for leading out a final object discrimination result. In order to lead out the final object discrimination result by using the synthetic discrimination table, a candidate object to be discriminated is detected from an image at first. Then the candidate object detected from the image is discriminated. For instance, when both of discrimination results transmitted from a pattern matching means and a neural network means are "a prescribed object" (e.g. a pedestrian) as shown in Fig.1, a synthetic discrimination result that the object is finally discriminated as the "prescribed object" (e.g. pedestrian) is led out while collating it with the synthetic discrimination table. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an object discrimination device, an object discrimination method, and an object discrimination program.

  2. Description of the Related Art Conventionally, for example, a technique for detecting an object that can be an obstacle to vehicle travel and avoiding a collision with the object has been developed. Therefore, in order to detect such an object that is an obstacle around the vehicle, there is a technique for photographing the periphery of the vehicle using a camera or the like and processing the captured image. As a method for detecting an object displayed in the image, for example, there is a background subtraction method. In such a background difference method, each pixel of the current image currently input and the immediately preceding image input immediately before is compared, and an object that is not seen in the immediately preceding image is found in the region where the difference is extracted. It is assumed that the object exists, and the object is detected from the image as a candidate for a travel obstacle. If such a method is used, an object that is a candidate for a running obstacle is detected, but it is still unclear what the detected object is.

  Therefore, pattern matching and neural network (Neural Network) are typical image processing methods for determining what is an object that is a candidate for a running obstacle detected from within an image. (For example, refer to Patent Documents 1 and 2). In general, any one of these object discrimination methods is used to discriminate an object in an image.

  Among these, in this pattern matching method, all templates are assigned to objects that are candidates for running obstacles detected from within an image, and whether or not the object is a specific object (for example, a pedestrian) is used as a template. Image processing is performed based on the degree of coincidence with the candidate object.

  A neural network is an information processing mechanism created by imitating the structure of the human brain. It is a network that applies a complex combination of neurons, called neurons, that stretches around the human brain. By configuring, it is a method of implementing object discrimination that humans are good at by incorporating it into an information processing mechanism. Specifically, whether the object is a specific object (for example, a pedestrian) by comparing the object pattern that has been learned and stored in advance with an object that is a candidate for a running obstacle detected from within the image Image processing is performed to determine whether or not.

JP 59-226981 A JP-A-6-231258

  By the way, as described below, the above-described conventional technique has a problem that the accuracy of object discrimination may be reduced.

  In other words, when pattern matching is used as a method for discriminating objects displayed in an image, a template that has been held in advance is applied to the entire area of the image. As a result, there is a problem that the processing may not be in time, and as a result, the accuracy of object discrimination may be reduced.

  In addition, when using a neural network, the processing speed when performing object discrimination is faster than template matching, but the accuracy of object discrimination is likely to be affected by the weather, and the accuracy of object discrimination is reduced as with template matching. There was also a problem that there could be.

  Therefore, if an object in an image is determined using any one of these object determination methods, it is not always possible to accurately determine an object in the image.

  In addition, when performing object discrimination with an in-vehicle surveillance camera, etc., it is possible that the accuracy of the discrimination may be reduced, so it is also a problem in predicting danger while driving the vehicle and avoiding danger as soon as possible. There is.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and an object discrimination device, an object discrimination method, and an object that can prevent a reduction in accuracy when discriminating an object in an image. An object is to provide a discrimination program.

  In order to solve the above-described problems and achieve the object, the invention according to claim 1 is an object discriminating apparatus for discriminating an object in an image, and detects an object detected in the image using a plurality of object discriminating methods. First object discrimination means for discriminating each object discrimination method; and second object discrimination means for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means. It is characterized by that.

  In the invention according to claim 2, in the above invention, the second object discriminating unit may be configured such that all of the discrimination results by the first object discriminating unit are objects on the image as predetermined objects. In the case of discrimination, one discrimination result is derived that the object on the image is a predetermined object.

  In the invention according to claim 3, in the above invention, the second object discriminating unit may use at least one of the discrimination results by the first object discriminating unit as an object on the image. If there is something that is determined to be present, one determination result that the object on the image is a predetermined object is derived.

  According to a fourth aspect of the present invention, in the above invention, the first object discrimination means calculates a discrimination value of an object on the image, and the second object discrimination means After each discriminant value by the first object discriminating means is converted into a predetermined value, if the total sum of the discriminant values exceeds a predetermined threshold, the object on the image is It is characterized by deriving one discrimination result as.

  Further, in the invention according to claim 5, in the above invention, the second object discriminating unit may determine all the discrimination results by the first object discriminating unit for objects within a predetermined distance range on the image. When determining that the object on the image is a predetermined object, a determination result that the object on the image is a predetermined object is derived, and the predetermined object on the image As for an object outside the distance range, if there is one that determines that an object on the image is a predetermined object from among the determination results by the first object determination means, the object on the image is It is characterized by deriving one discrimination result indicating that the object is a predetermined object.

  The invention according to claim 6 is characterized in that, in the above invention, the second object discrimination means derives one discrimination result from each discrimination result by the first object discrimination means according to the weather. To do.

  The invention according to claim 7 is characterized in that, in the above invention, the second object discrimination means derives one discrimination result from each discrimination result by the first object discrimination means in accordance with a processing load. And

  Further, the invention according to claim 8 is characterized in that, in the above-mentioned invention, vehicle control means for controlling the vehicle is further provided according to each discrimination result by the second object discrimination means.

  The invention according to claim 9 is characterized in that, in the above-mentioned invention, the apparatus further comprises warning notifying means for notifying the vehicle according to each determination result by the second determining means. .

  According to a tenth aspect of the present invention, in the object discrimination method for discriminating an object in an image according to the above invention, an object detected in the image using a plurality of object discrimination methods is determined for each object discrimination method. A first object discrimination step for discrimination; and a second object discrimination step for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means. .

  According to an eleventh aspect of the present invention, in the above-described invention, in an object determination program for causing a computer to execute a method for determining an object in an image, an object detected in the image using a plurality of object determination methods is detected. A first object discrimination procedure for discriminating each object discrimination method, and a second object discrimination procedure for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means It is made to perform.

  According to the first, tenth, and eleventh aspects of the present invention, the object discriminating apparatus discriminates an object in the image by deriving one discrimination result from each object discrimination result by a plurality of object discrimination methods. It is possible to prevent the accuracy from being lowered as compared with the case where an object in an image is determined using an object determination method.

  According to the invention of claim 2, in the object discrimination device, when all of the object discrimination results by the plurality of object discrimination methods are such that the object in the image is a predetermined object, Since the object is determined by deriving the determination result that the object in the image is a predetermined object, the accuracy is prevented from being reduced compared to the case of determining the object in the image using one object determination method. It becomes possible to do.

  According to the invention of claim 3, the object discriminating apparatus assumes that any one of the object discrimination results obtained by the plurality of object discrimination methods is a predetermined object. Since the object is identified by deriving the determination result that the object in the image is the predetermined object, the object in the image is compared with the case where the object in the image is determined using one object determination method. Can be discriminated, and as a result, it is possible to prevent the accuracy of object discrimination from being lowered.

  According to the invention of claim 4, the object discriminating apparatus adds the discriminant values calculated by a plurality of object detection methods, and if the sum of discriminant values exceeds a predetermined threshold, Since the object discrimination is performed by deriving the discrimination result that the object is a predetermined object, it is possible to prevent the accuracy from being lowered compared to the case where the object in the image is discriminated using one object discrimination method. It becomes possible.

  According to the invention of claim 5, the object discriminating apparatus is configured such that, for an object within a predetermined distance range in the image, all the object discrimination results by the plurality of object discriminating methods are the objects in the image. If it is determined that the object is a predetermined object, a determination result that the object in the image is the predetermined object is derived, and a plurality of object determination methods are used for an object outside the predetermined distance range in the image. If any object in the image is determined to be a predetermined object, the object is obtained by deriving the determination result that the object in the image is a predetermined object. Since it is discriminated, the accuracy of discriminating an object located at a short distance, which is highly necessary for discriminating objects, is prevented compared to discriminating an object in an image using one object discriminating method. It becomes possible.

  According to the invention of claim 6, this object discrimination device performs object discrimination by deriving one discrimination result from each object discrimination result by a plurality of object detection methods according to the weather. Compared to discriminating an object in an image using a discrimination method, it is possible to prevent the accuracy from being lowered due to the weather.

  Further, according to the invention of claim 7, the object discrimination device performs object discrimination by deriving one discrimination result from each object discrimination result by a plurality of object detection methods according to the processing load. As compared with the case where an object in an image is discriminated using an object discriminating method, it is possible to prevent the accuracy from being lowered due to a processing load.

  In addition, according to the invention of claim 8, since the object discriminating apparatus controls the vehicle according to each object discriminating result by the plurality of object detecting methods, the object discriminating vehicle Compared with performing control, it becomes possible to perform vehicle control more appropriately.

  According to the invention of claim 9, since the object discriminating apparatus issues a warning in accordance with each object discrimination result obtained by a plurality of object detection methods, the object discrimination device warns according to the object discrimination result obtained by one object discrimination method. It is possible to more appropriately notify the warning as compared to the above notification.

  Exemplary embodiments of an object discrimination device, an object discrimination method, and an object discrimination program according to the present invention will be described below in detail with reference to the accompanying drawings. In the following embodiments, a case will be described in which an in-vehicle imaging device discriminates an object imaged in an image.

  In the following first embodiment, the outline and features of the object discriminating apparatus according to the first embodiment, the configuration of the object discriminating apparatus, and the flow of processing will be described in order, and finally the effects of the first embodiment will be described. In the following first embodiment, a case will be described in which an object in an image captured using an in-vehicle imaging device is determined.

[Outline and Features (Example 1)]
First, the outline and features of the object discrimination device according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining the outline of the object determination device according to the first embodiment. As shown in the figure, the object discriminating apparatus according to the first embodiment has an outline of discriminating an object in an image. However, an object in an image is discriminated using a plurality of object discriminating methods. There is a main feature, and this makes it possible to prevent a decrease in accuracy when an object in an image is discriminated.

  Briefly explaining the main features, the object discriminating apparatus possesses a comprehensive discrimination table for deriving a final object discrimination result. In order to derive a final object discrimination result using this comprehensive discrimination table, first, candidate objects to be discriminated are detected from the image. The candidate object is detected by image processing using a background difference method, a saliency calculation method, or the like.

  Next, the candidate object detected from within the image is determined. This discrimination is performed using a pattern matching method and a neural network method. Then, each object discrimination result by each method is checked against the comprehensive discrimination table to finally derive one object discrimination result. For example, as shown in FIG. 1, if each of the discrimination results by the pattern matching method and the neural network method is “pedestrian”, the final result is “walking” against the comprehensive discrimination table. The result of the determination of “person” is derived. On the other hand, if each of the discrimination results by the pattern matching method and the neural network method is “not a pedestrian”, it is finally judged as “not a pedestrian” in light of the comprehensive discrimination table. The discrimination result is derived.

  As described above, this object discrimination device discriminates an object in an image by deriving one discrimination result from each object discrimination result by a plurality of object discrimination methods. It is possible to prevent the accuracy from being lowered as compared with the case of discriminating an object.

[Configuration of Object Discriminating Device (Example 1)]
Next, the configuration of the object discrimination device 30 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the object discrimination device 30. As shown in the figure, the object determination device 30 is connected to a navigation unit 11, a radar unit 12, a preprocessing unit 13, an object detection device 14, and a collision risk determination unit 15. The object discrimination device 30 includes a storage unit 31 and a control unit 32.

  Among them, the navigation (navigation) unit 11 is a means for setting and guiding a travel route from the specified vehicle position and map data by communicating with a GPS (Global Positioning System) artificial satellite. The navigation unit 11 also supplies the driver with various types of information useful for vehicle driving operations such as vehicle position information, road shape, road width, and inclination.

  The radar unit 12 is means for irradiating a radar (for example, 76.5 GHz millimeter wave) and measuring a distance, speed, direction, and the like with a vehicle or an obstacle ahead. The preprocessing unit 13 is a processing unit that performs preprocessing on an image transmitted from the imaging device 10 (for example, a visible light camera, a monocular camera, or a stereo camera), and includes a filter unit 13a and a contour extraction unit 13b. The Among these, the filter unit 13a is means for performing preprocessing (for example, sharpness, contrast adjustment, and saturation adjustment) for enhancing the outline of an object displayed in the image. The contour extracting unit 13b is means for extracting the contour of the object in the image based on the filtering performed by the filter unit 13a.

  The object detection device 14 is a device that detects an object from an image received from the imaging device 10 via the preprocessing unit 13 (restricts an area in the image). For example, an area occupied by an object projected in an image is detected by image processing using a background difference method (interframe difference method) or a saliency calculation method (a region to which a template is applied is narrowed down). Then, the object discriminating apparatus 30 described later performs image processing using a template matching method and a neural network method on this region. The object detection device 14 transmits the object detection processing result to the object determination device 30 and the collision risk determination unit 15.

  The collision risk determination unit 15 is a processing unit that predicts and determines the collision risk with the object detected from the image based on the processing results received from the object detection device 14 and the general determination unit 32c of the object determination device 30. It is. For example, when the object detected by the object determination device 30 is a pedestrian and is moving near the vehicle from a position in the image, or when the determination result by the object determination device 30 is a child, the collision risk level Is determined to be high. Further, when the object detected from the image by the object detection device 14 is a pedestrian and is stationary at a position away from the vehicle, it is determined that the collision risk is low. The processing result by the collision risk determination unit 15 is transmitted to the vehicle control unit 16 and the notification control unit 17.

  Among these, the vehicle control unit 16 is a processing unit that performs vehicle control based on the processing result received from the collision risk determination processing unit 15. For example, when the determination result that the risk of collision with an object detected from within the image by the object detection device 14 is high is received from the collision risk determination processing unit 15, control for stopping the vehicle is performed (for example, Even if the driver does not step on the brake, he / she automatically intervenes in the vehicle control to stop the vehicle safely). In addition, when the determination result that the risk of collision with the object detected from the image by the object detection device 30 is low is received from the collision risk determination processing unit 15, control for decelerating the vehicle or assistance for vehicle control is performed. (For example, reduce brake play and prepare the vehicle for braking).

  The notification control unit 17 is a processing unit that notifies a warning based on the processing result received from the collision risk determination processing unit 15. For example, when a determination result that the risk of collision with an object detected from within the image by the object detection device 14 is high is received from the collision risk determination processing unit 15, an alarm sound or a voice is transmitted from the in-vehicle notification unit 18. A warning is notified by. In addition, when the determination result that the risk of collision with an object detected from within the image by the object detection device 30 is low is received from the collision risk determination unit 15, attention is given by voice or image from the in-vehicle notification unit 18. Prompt.

  The in-vehicle notification unit 18 is a means for notifying information from the navigation unit 11 and the notification control unit 17 and includes a display, a speaker, and the like. For example, a part of the display is blinked to alert the driver, or a warning or warning is given to the driver by sending a message or an alarm sound from a speaker.

  The storage unit 31 of the object discrimination device 30 is a storage unit (storage unit) that stores data and programs necessary for various processes performed by the control unit 32. For example, a template used in a pattern matching execution unit 32a described later is also stored. is doing. In particular, as a component closely related to the present invention, a comprehensive discrimination table 31a is provided.

  The comprehensive discrimination table 31a is a means for storing various types of information related to final object discrimination. Specifically, as illustrated in FIG. 3, all discrimination results obtained by pattern matching and a neural network are predetermined. If the object is an object, the final comprehensive discrimination result is defined as “predetermined object”. Otherwise, any one or all of the discrimination results are predetermined. The final comprehensive discrimination result is defined as “not a predetermined object”.

  The control unit 32 of the object discrimination device 30 is a processing unit that has a predetermined control program, a program that defines various processing procedures, and an internal memory for storing required data, and executes various processes using these. Particularly, those closely related to the present invention include a pattern matching execution unit 32a, a neural network execution unit 32b, and an overall determination unit 32c. The pattern matching execution unit 32a and the neural network execution unit 32b correspond to the “first object determination unit” described in the claims, and the comprehensive determination unit 32c also corresponds to the “second object determination unit”. To do.

  Among these, the pattern matching execution unit 32a is a processing unit that discriminates an object in the image. Specifically, the candidate object detected from the image by the object detection device 14 is located in an area located in the image. By applying the template, a determination result that the candidate object is “predetermined object” is derived from the degree of matching between each pixel of the candidate object and each pixel of the template. For example, if a predetermined degree of matching between each pixel of the candidate object and a pixel of the template for the pedestrian is obtained, a determination result that the candidate object is “pedestrian” is derived.

  In other words, in pattern matching, for a candidate object that appears in the image, several templates corresponding to the shape of the pedestrian are applied at the position on the image, and compared with the shape of the candidate object, the matching of the shape The degree of determination as to whether or not the person is a pedestrian is derived by looking at (degree). Then, as a result of the fitting, “pedestrian” may be determined based on complete matching, but may be determined as “pedestrian” with a degree of matching equal to or higher than a predetermined level by similarity comparison. The pattern matching execution unit 32a transmits the object determination result to the comprehensive determination unit 32c.

  Similar to the pattern matching execution unit 32a, the neural network execution unit 32b is a processing unit that discriminates objects in the image. Specifically, the neural network execution unit 32b previously learns candidate objects detected from the image by the object detection device 14. The discrimination result “determined to be a predetermined object” is derived in light of the stored object pattern. For example, a determination result that a candidate object is a “pedestrian” is derived from an object pattern that has been learned and stored in advance.

  That is, the neural network learns the shape and movement of a pedestrian, and derives a discrimination result while viewing the shape and movement of an actual candidate object in the image based on the learning result. The neural network execution unit 32b transmits the object determination result to the comprehensive determination unit 32c.

  The overall discrimination unit 32c is a processing unit that derives a final object discrimination result by comparing each object discrimination result transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b with the overall discrimination table 31a.

  Specifically, it is determined whether or not each determination result transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b is a determination result that makes the candidate object “predetermined object”. . As a result of the determination, if both of the determination results are “predetermined objects”, the comprehensive determination unit 32c checks the image in the image against the comprehensive determination table 31a (see FIG. 3). The final comprehensive discrimination result that the object is “predetermined object” is derived.

  On the other hand, as a result of the determination, if any one or all of the determination results indicate that the candidate object is not a “predetermined object”, the comprehensive determination unit 32c performs the comprehensive determination table 31a (FIG. 3), the final comprehensive discrimination result that the object in the image is “not a predetermined object” is derived.

  For example, when each of the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b is that the candidate object is “pedestrian”, the total determination unit 32c In light of the table 31a, a final comprehensive discrimination result is derived in which the object in the image is “pedestrian”. On the other hand, if any one or all of the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b respectively make the candidate object “not a pedestrian”, the comprehensive determination The unit 32c derives a final comprehensive determination result that makes the object in the image “not a pedestrian” in light of the comprehensive determination table 31a.

[Processing by Object Discriminating Device (Example 1)]
Next, a processing flow of the object discrimination device according to the first embodiment will be described with reference to FIG. FIG. 4 is a flowchart illustrating a process flow of the object determination device according to the first embodiment.

  As shown in the figure, when an image input is received (step S401), the object detection device 14 detects a candidate object to be determined from within the image (step S402). As a result, when a candidate object to be discriminated is detected from the image in the object detection device 14 (Yes in step S403), the pattern matching execution unit 32a and the neural network execution unit 32b discriminate the candidate object. The determination result is derived (step S404).

  Specifically, pattern matching applies a number of templates according to the shape of the pedestrian to the candidate object appearing in the image at the position on the image, and compares it with the shape of the candidate object. The result of determination as to whether or not the person is a pedestrian is derived by looking at the coincidence (degree). Then, as a result of the fitting, “pedestrian” may be determined based on complete matching, but may be determined as “pedestrian” with a degree of matching equal to or higher than a predetermined level by similarity comparison. The neural network learns the shape and movement of a pedestrian, and derives a discrimination result from the learning result while observing the shape and movement of an actual candidate object in the image.

  Then, the pattern matching execution unit 32a and the neural network execution unit 32b give each determination result (for example, a determination result indicating that the candidate object is “pedestrian” or “not pedestrian”) to the general determination unit 32c. Send. Here, returning to the description of step S403, if no candidate object is detected from the image in the object detection device 14 (No in step S403), the object determination device 30 waits for the next image input.

  Receiving the transmission of each discrimination result from the pattern matching execution unit 32a and the neural network execution unit 32b, the overall discrimination unit 32c derives a final overall discrimination result (steps S405 to S407). Specifically, first, it is determined whether or not each determination result transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b is a determination result in which the candidate object is “predetermined object”. This is performed (step S405).

  When each of the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b both sets the candidate object as “predetermined object” (for example, “is a pedestrian”). In step S405, the comprehensive determination unit 32c determines that the object in the image is “predetermined object” (for example, “is a pedestrian”) against the comprehensive determination table 31a. A comprehensive discrimination result is derived (step S406).

  Here, returning to the description of step S405, any one or all of the discrimination results respectively transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b are designated as “not a predetermined object”. (For example, “not a pedestrian”) (No in step S405), the comprehensive determination unit 32c compares the object in the image with the “predetermined object” according to the comprehensive determination table 31a. The final comprehensive discrimination result “No” (for example, “not a pedestrian”) is derived (step S407).

[Effect of Example 1]
As described above, according to the first embodiment, the object discrimination device 30 discriminates an object in an image by deriving one discrimination result from each object discrimination result by a plurality of object discrimination methods. It is possible to prevent the accuracy from deteriorating as compared with the case of discriminating an object in an image using two object discrimination methods.

  Further, according to the first embodiment, the object determination device 30 is configured to display an image when all of the object determination results obtained by the plurality of object determination methods indicate that the object in the image is a predetermined object. The object is discriminated by deriving the discrimination result that the inside object is a predetermined object, so that the accuracy is prevented from being lowered compared to the case where the object in the image is discriminated using one object discrimination method. It becomes possible.

  In addition, according to the first embodiment, the object determination device 30 controls the vehicle according to each object determination result obtained by a plurality of object detection methods, so that the vehicle control is performed according to the object determination result obtained by one object determination method. It is possible to perform vehicle control more appropriately than performing the above.

  In addition, according to the first embodiment, the object determination device 30 issues a warning according to each object determination result obtained by a plurality of object detection methods, so that a warning is generated according to the object determination result obtained by one object determination method. It is possible to notify the warning more appropriately than performing the notification.

  By the way, in the above-described embodiment, the case where all the object discrimination results by the plurality of object discrimination methods are pedestrians has been described, but the present invention is not limited to this. Alternatively, if any one of the object discrimination results obtained by the plurality of object discrimination methods is to be an object in the image as a pedestrian, it may be determined as a pedestrian. Therefore, in the second embodiment below, the flow of processing of the object discrimination device in this case will be described using FIGS. 5 and 6, and finally the effects of the second embodiment will be described. FIG. 5 is a flowchart illustrating a process flow of the object determination device according to the second embodiment. FIG. 6 is a diagram illustrating a configuration example of a comprehensive discrimination table according to the second embodiment.

  As shown in FIG. 5, when an image input is received (step S501), the object detection device 14 detects a candidate object to be determined from within the image (step S502). As a result, when the object detection device 14 detects a candidate object to be determined from the image (Yes in step S503), the pattern matching execution unit 32a and the neural network execution unit 32b determine the candidate object. The determination result is derived (step S504).

  Then, the pattern matching execution unit 32a and the neural network execution unit 32b transmit each determination result (for example, a determination result indicating that the candidate object is “pedestrian” or “not pedestrian”) to the general determination unit 32c. To do. Here, returning to the description of step S503, if no candidate object is detected from the image in the object detection device 14 (No in step S503), the object determination device waits for the next image input.

  Receiving the transmission of each discrimination result from the pattern matching execution unit 32a and the neural network execution unit 32b, the overall discrimination unit 32c derives a final overall discrimination result (steps S505 to S507). Specifically, first, any one or all of the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b are determination results that make the candidate object “predetermined object”. Is determined (step S505).

  Then, any one or all of the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b are “predetermined objects” as candidate objects (for example, “is a pedestrian”). (Yes in step S505), the comprehensive determination unit 32c refers to the comprehensive determination table 31a (see FIG. 6) to determine that the object in the image is “predetermined object” (for example, “ A final comprehensive discrimination result that indicates “is a pedestrian” is derived (step S506).

  Here, returning to the description of step S505, the determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b are both “not predetermined objects” (for example, “walking”). If it is determined that the object in the image is not a “predetermined object” (for example, “ A final comprehensive discrimination result “not a pedestrian”) is derived (step S507).

[Effect of Example 2]
As described above, according to the second embodiment, this object discriminating apparatus assumes that any one of the object discrimination results obtained by the plurality of object discrimination methods is a predetermined object. In some cases, the object is determined by deriving a determination result that the object in the image is a predetermined object, so that the object in the image is compared with the case where the object in the image is determined using one object determination method. Therefore, it is possible to prevent the object discrimination accuracy from being lowered as a result.

  Further, in the above-described embodiment, a case has been described in which an object in an image is determined by using each determination result obtained by a plurality of object determination methods as it is, but the present invention is not limited to this, and a plurality of object determinations are performed. A predetermined discriminant value may be obtained from each discriminant result by the technique, and an object in the image may be discriminated based on whether or not a value obtained by adding the discriminant values exceeds a predetermined threshold value.

  Therefore, in the following third embodiment, the configuration of the object discriminating apparatus and the flow of processing in this case will be described in order with reference to FIGS. FIG. 7 is a block diagram illustrating the configuration of the object determination device according to the third embodiment. FIG. 8 is a diagram illustrating a configuration example of a weight regulation table according to the third embodiment. FIG. 9 is a diagram illustrating a configuration example of a threshold value table according to the third embodiment. FIG. 10 is a flowchart illustrating a process flow of the object determination device according to the third embodiment.

[Configuration of Object Discriminating Device (Example 3)]
As shown in FIG. 7, the object determination device is connected to a navigation unit 11, a radar unit 12, a preprocessing unit 13, an object detection device 14, and a collision risk determination unit 15. The object discrimination device 40 includes a storage unit 41 and a control unit 42. The navigation unit 11, the radar unit 12, the preprocessing unit 13, the object detection device 14, the collision risk determination unit 15, the vehicle control unit 16, the notification control unit 17, and the in-vehicle notification unit 18 are the same as in the above-described embodiment. So I will omit the explanation.

  The storage unit 41 of the object detection device 40 is a storage unit (storage unit) that stores data and programs necessary for various processes performed by the control unit 42. For example, a template used in a pattern matching execution unit 42a described later is also stored. is doing. In particular, as closely related to the present invention, a weight defining table 41a and a threshold defining table 41b are provided.

  Among these, the weight defining table 41a is used for calculating a discriminant value in the general discriminating unit 42c from a predetermined value obtained when discriminating an object in the image in the pattern matching executing unit 42a and the neural network executing unit 42b described later. Specifically, as shown in FIG. 8, the storage means for storing information necessary for calculating the discriminant value for the object in the image (for example, pattern matching 0.8, neural network) Network 0.6) is defined for each object discrimination method. The weight may be appropriately changed in a user-friendly manner, for example, by defining it based on pattern matching and neural network discrimination experiment data.

  The threshold value definition table 41b is storage means for storing information used for finally determining an object in the image in the overall determination unit 42c described later. Specifically, as illustrated in FIG. A threshold value (for example, 0.7) necessary for finally discriminating an object in the image from the discriminant value calculated by the discriminating unit 42c is stored. Note that this threshold may be changed as appropriate.

  The control unit 42 of the object discrimination device 40 is a processing unit that has a predetermined control program, a program that defines various processing procedures, and an internal memory for storing required data, and executes various processes using these. Particularly, those closely related to the present invention include a pattern matching execution unit 42a, a neural network execution unit 42b, and an overall determination unit 42c.

  Among these, the pattern matching execution unit 42a and the neural network 42b are processing units that determine an object in the image, and calculate the determination result of the object in the image as a predetermined value. For example, the pattern matching execution unit 42a calculates a predetermined value called a so-called matching number from the matching number between the pixel value of the object displayed in the image and the pixel value of the template.

  The overall discrimination unit 42c is a processing unit that derives a final object discrimination result. Specifically, the predetermined value calculated by the pattern matching execution unit 42a and the neural network 42b is normalized to a value of 0 to 1, respectively. (For example, “1” for perfect match, “0” for mismatch, “0.5” for approximation, etc.) and then multiplying by the weight (0.8) defined in the weight definition table 41a. To calculate the discriminant value. Then, final object discrimination is performed based on whether the sum of these discrimination values exceeds the overall discrimination threshold (for example, 0.7) defined in the threshold definition table 41b, and exceeds this threshold. Then, it is finally determined that the object in the image is a predetermined object (for example, a pedestrian).

[Processing by Object Discriminating Device (Example 3)]
Next, the flow of processing by the object discrimination device 40 according to the third embodiment will be described in detail with reference to FIG.

  As shown in the figure, when the image receives an input (Yes at step S1001), the object detection device 14 detects a candidate object to be determined from within the image (step S1002). As a result, when a candidate object to be discriminated is detected from the image in the object detection device 14 (Yes in step S1003), the pattern matching execution unit 42a and the neural network execution unit 42b discriminate the candidate object. A discrimination result is derived (step S1004).

  Then, the pattern matching execution unit 32a and the neural network execution unit 32b respectively transmit predetermined values calculated as the determination results to the comprehensive determination unit 32c. Here, returning to the description of step S1003, if no candidate object is detected from the image in the object detection device 14 (No in step S1003), the object determination device 40 waits for the next image input.

  In response to the transmission of predetermined values calculated as the respective discrimination results from the pattern matching execution unit 42a and the neural network execution unit 42b, the total discrimination unit 32c calculates the discrimination value. Here, the discriminant values are defined in the weight definition table 41a (see FIG. 8) after normalizing the predetermined values calculated by the pattern matching execution unit 42a and the neural network 42b to values of 0 to 1, respectively. The discriminant value is calculated by multiplying the weights (pattern matching: 0.8, neural network: 0.6). Then, these discriminant values are added together (step S1005).

  Subsequently, the overall determination unit 32c makes a determination to derive a final determination result by comparing the sum of these determination values with the threshold value definition table 42b. Specifically, when the sum obtained by adding the discriminant values exceeds a predetermined threshold (see FIG. 9) (Yes in step S1006), the comprehensive discriminating unit 42c determines that the object in the image is “predetermined object”. Is the final determination result (for example, “is a pedestrian”) (step S1007).

  That is, for example, if the degree of coincidence in pattern matching is high, the overall discriminant value (accuracy) increases even if the discriminant value (accuracy) by the neural network is somewhat low, and it is determined as a pedestrian. Therefore, even if it is difficult to detect with a neural network, it can be complemented by pattern matching.

  Here, returning to the description of step S1006, when the sum of the respective discriminant values does not exceed a predetermined threshold (No in step S1006), the comprehensive discriminating unit 42c determines that the object in the image is “predetermined. It is determined that the object is not an object (for example, “not a pedestrian”) (step S1008).

[Effect of Example 3]
As described above, according to the third embodiment, the object discriminating apparatus 40 adds the discriminant values calculated by the plurality of object detection methods, and the sum of the discriminant values exceeds a predetermined threshold value. For example, object discrimination is performed by deriving a discrimination result that an object in the image is a predetermined object, so that the accuracy is reduced compared to discriminating an object in the image using one object discrimination method. Can be prevented.

  Further, in the above-described embodiment, a case has been described in which an object in an image is discriminated from each object discrimination result by a plurality of object detection methods regardless of the distance to the object in the image, but the present invention is not limited to this. Instead, the object in the image may be determined from each object determination result by a plurality of object detection methods in consideration of the distance to the object in the image. Therefore, in the following fourth embodiment, the flow of processing of the object discriminating apparatus in this case will be described with reference to FIG. 11, and finally the effects of the fourth embodiment will be described. FIG. 11 is a flowchart illustrating a process flow of the object determination device according to the fourth embodiment.

  As shown in the figure, upon receiving an input of an image (step S1101), the object detection device 14 detects a candidate object to be determined from within the image (step S1102). As a result, when the object detection device 14 detects a candidate object to be determined from the image (Yes in step S1103), the pattern matching execution unit 32a and the neural network execution unit 32b determine the candidate object. The determination result is derived (step S1104).

  And the pattern matching execution part 32a and the neural network execution part 32b transmit each discrimination | determination result with respect to the comprehensive discrimination | determination part 32c. Here, returning to the description of step S1103, when the object detection device 14 does not detect a candidate object from the image (No in step S1103), the object determination device waits for the next image input.

  In response to the transmission of each discrimination result from the pattern matching execution unit 32a and the neural network execution unit 32b, the total discrimination unit 32c first measures the distance to the candidate object in the image (step S1105). Here, the distance to the candidate object is measured, for example, from the vertical position (number of pixels) in the image of the candidate object detected by the object detection device 14.

  Then, the distance to the candidate object is within the predetermined distance range (Yes at step S1106), and the respective determination results transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b are both “predetermined” If the object is “object” (for example, “is a pedestrian”) (Yes in step S1107), the comprehensive determination unit 32c checks the image in the image against the comprehensive determination table 31a (see FIG. 12). The final comprehensive discrimination result that “is a predetermined object” (for example, “is a pedestrian”) is derived (step S1108).

  Here, returning to the description of step S1107, each of the determinations that the distance to the candidate object is within the predetermined distance range (Yes in step S1106) and is transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b, respectively. When any one or all of the results indicate that the candidate object is “not a predetermined object” (for example, “not a pedestrian”) (No in step S1107), the comprehensive determination unit 32c In the light of the discrimination table 31a, a final comprehensive discrimination result is derived in which the object in the image is “not a predetermined object” (for example, “not a pedestrian”) (step S1109).

  Here, returning to the description of step S1106, the distance to the candidate object is outside the predetermined distance range (No in step S1106), and transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b, respectively. If any one or all of the determination results are those for which the candidate object is “predetermined object” (eg, “is a pedestrian”) (Yes in step S1110), the comprehensive determination unit 32c Then, in light of the comprehensive determination table 31a, a final comprehensive determination result is derived in which the object in the image is “predetermined object” (for example, “is a pedestrian”) (step S1111).

  Here, returning to the description of step S1110, the distance to the candidate object is outside the predetermined distance range (No in step S1106) and has been transmitted from the pattern matching execution unit 32a and the neural network execution unit 32b, respectively. When both the determination results indicate that the candidate object is “not a predetermined object” (for example, “not a pedestrian”) (No in step S1110), the comprehensive determination unit 32c stores in the comprehensive determination table 31a. In comparison, a final comprehensive determination result that an object in the image is “not a predetermined object” (for example, “not a pedestrian”) is derived (step S1112).

[Effect of Example 4]
As described above, according to the fourth embodiment, this object detection device, for an object within a predetermined distance range in the image, all of the object discrimination results by the plurality of object discrimination methods are included in the image. If the object is a predetermined object, a determination result that the object in the image is the predetermined object is derived, and a plurality of objects are detected for objects outside the predetermined distance range in the image. If any one of the object discrimination results obtained by the discrimination method is to determine that the object in the image is a predetermined object, the determination result that the object in the image is a predetermined object is derived. Therefore, the accuracy of the identification of objects located at a short distance, which is highly necessary for object identification, is lower than that for identifying objects in an image using one object identification method. It becomes possible to prevent .

  The object discriminating apparatus according to the first to fourth embodiments has been described so far, but the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, in the following, as the fifth embodiment, various different embodiments will be described as divided into (1) to (4).

(1) Object discrimination according to the weather In the above-described embodiment, an object in the image may be discriminated from the object discrimination results obtained by a plurality of object discrimination methods in consideration of the weather. As a result, this object discrimination device performs object discrimination by deriving one discrimination result from each object discrimination result by a plurality of object detection methods according to the weather. Compared to determining an object, it is possible to prevent the accuracy from decreasing due to the weather.

(2) Object Discrimination According to Processing Load In the above embodiment, an object in an image may be discriminated from object discrimination results obtained by a plurality of object discrimination methods in consideration of the processing load. As a result, this object discriminating apparatus performs object discrimination by deriving one discrimination result from each object discrimination result by a plurality of object detection methods according to the processing load. As compared with the case of discriminating the object, it is possible to prevent the accuracy from being lowered due to the processing load.

(3) Device Configuration Each component of the object discriminating device shown in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of the object discriminating apparatus 30 is not limited to the one shown in the figure. Depending on the situation, it can be configured to be functionally or physically distributed and integrated in arbitrary units. Further, all or a part of each processing function performed in the object discrimination device 30 is realized by a CPU and a program analyzed and executed by the CPU, or is realized as hardware by wired logic. obtain.

(4) Object Discriminating Program Various processes described in the first embodiment can be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. In the following, an example of a computer that executes an object determination program having the same function as that of the first embodiment will be described with reference to FIG. FIG. 13 is a diagram illustrating a computer that executes an object determination program.

  As shown in the figure, a computer 50 as an object discrimination device is configured by connecting an HDD 51, a RAM 52, a ROM 53, and a CPU 54 via a bus 60.

  In the ROM 53, an object discrimination program that exhibits the same function as that of the object discrimination device 30 shown in the first embodiment, that is, as shown in FIG. 13, a pattern matching execution program 53a, a neural network execution program 53b, and A comprehensive discrimination program 53c is stored in advance. Note that these programs 53a, 53b, and 53c may be appropriately integrated or distributed in the same manner as each component of the object determination device 30 shown in FIG. The ROM 53 may be a nonvolatile “RAM”.

  The CPU 54 reads out these programs 53a, 53b and 53c from the ROM 53 and executes them. As shown in FIG. 13, these programs 53a, 53b and 53c are processed by a pattern matching execution process 54a and a neural network execution process. 54b and the overall discrimination process 54c. These processes 54a, 54b, and 54c correspond to the pattern matching execution unit 32a, the neural network execution unit 32b, and the overall determination unit 32c shown in FIG. 2, respectively.

  Further, as shown in FIG. 13, the HDD 51 is provided with a comprehensive discrimination table 51a. This comprehensive determination table 51a corresponds to the comprehensive determination table 31a shown in FIG. The CPU 54 reads the overall discrimination data 52 a from the overall discrimination table 51 a and stores it in the RAM 52, and executes the object detection process based on the overall discrimination data 52 a stored in the RAM 52.

  Note that the above-described programs 53a, 53b and 53c do not necessarily need to be stored in the ROM 53 from the beginning. For example, a flexible disk (FD), a CD-ROM, an MO disk, a DVD disk inserted into the computer 50, “Portable physical media” such as magneto-optical disks and IC cards, or “fixed physical media” such as HDDs provided inside and outside the computer 50, and further via public lines, the Internet, LAN, WAN, etc. Alternatively, each program may be stored in “another computer (or server)” connected to the computer 50, and the computer 50 may read and execute each program from these.

  In the above-described embodiment, the case where an object in an image captured by an in-vehicle imaging device is detected has been described. However, the present invention is not limited to this, and a fixed imaging device (for example, an intersection) An object imaged by a traffic flow monitoring camera attached to the camera may be detected from the image.

  As described above, the object discriminating apparatus, the object discriminating method, and the object discriminating program according to the present invention are useful for discriminating an object in an image, and in particular, prevent a decrease in accuracy when discriminating an object in an image. Suitable for doing.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining an overview of an object determination device according to a first embodiment. 1 is a block diagram illustrating a configuration of an object discrimination device according to Embodiment 1. FIG. It is a figure which shows the structural example of the comprehensive discrimination | determination table which concerns on Example 1. FIG. 3 is a flowchart illustrating a processing flow of the object determination device according to the first embodiment. 12 is a flowchart illustrating a process flow of the object determination device according to the second embodiment. FIG. 10 is a diagram illustrating a configuration example of a comprehensive discrimination table according to the second embodiment. FIG. 10 is a block diagram illustrating a configuration of an object determination device according to a third embodiment. It is a figure which shows the structural example of the weight prescription | regulation table concerning Example 3. FIG. It is a figure which shows the structural example of the threshold value table which concerns on Example 3. FIG. 12 is a flowchart illustrating a process flow of the object determination device according to the third embodiment. 12 is a flowchart illustrating a flow of processing of an object determination device according to a fourth embodiment. 10 is a flowchart illustrating a process flow of an object determination device according to a fifth embodiment. It is a figure which shows the computer which performs an object discrimination | determination program.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Navigation (navigation) part 12 Radar part 13 Preprocessing part 13a Filter part 13b Contour extraction part 14 Object discrimination device 15 Collision risk judgment part 16 Vehicle control part 17 Notification control part 18 In-car notification part 30 Object discrimination apparatus 31 Storage unit 31a General determination table 32 Control unit 32a Pattern matching execution unit 32b Neural network execution unit 32c General determination unit 40 Object determination device 41 Storage unit 41a Weight definition table 41b Threshold definition table 42 Control unit 42a Pattern matching execution unit 42b Neural network execution Part 42c Total discriminating part 50 Computer 51 HDD (Hard disk drive)
52 RAM (Random Access Memory)
53 ROM (Read Only Memory)
54 CPU (Central Processing Unit)
60 bus

Claims (11)

In an object discriminating apparatus that discriminates an object in an image,
First object discriminating means for discriminating, for each object discrimination method, an object detected in the image using a plurality of object discrimination methods;
Second object discrimination means for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means;
An object discrimination device comprising:   The second object discriminating unit determines that the object on the image is a predetermined object when all the discrimination results by the first object discriminating unit determine that the object on the image is a predetermined object. The object discrimination device according to claim 1, wherein one discrimination result indicating that the object is a predetermined object is derived.   The second object discriminating unit determines whether an object on the image is a predetermined object among at least one of the discrimination results by the first object discriminating unit. 2. The object discriminating apparatus according to claim 1, wherein one discrimination result indicating that the object is a predetermined object is derived. The first object discrimination means calculates a discrimination value of an object on the image,
The second object discriminating unit converts each discriminant value obtained by the first object discriminating unit into a predetermined value and then adds the discriminant value added to each other and exceeds a predetermined threshold value. 2. The object discriminating apparatus according to claim 1, wherein one discrimination result that the object on the image is a predetermined object is derived.   The second object discriminating unit discriminates that all the discrimination results by the first object discriminating unit are objects on the image as the predetermined object for objects within a predetermined distance range on the image. If the object is a thing, a determination result that the object on the image is a predetermined object is derived, and the object outside the predetermined distance range on the image is determined by the first object determination means. If at least one of the determination results determines that the object on the image is a predetermined object, a determination result that the object on the image is a predetermined object is derived. The object discrimination device according to claim 1, wherein   The object discriminating apparatus according to claim 1, wherein the second object discriminating unit derives one discrimination result from each discrimination result by the first object discriminating unit according to the weather.   The object discriminating apparatus according to claim 1, wherein the second object discriminating unit derives one discrimination result from each discrimination result by the first object discriminating unit according to a processing load.   The object discriminating apparatus according to claim 1, further comprising vehicle control means for controlling the vehicle according to each discrimination result by the second object discrimination means.   The object discriminating apparatus according to claim 1, further comprising warning notifying means for notifying the vehicle of a warning according to each discrimination result by the second discriminating means. In an object discrimination method for discriminating an object in an image,
A first object determination step of determining, for each object determination method, an object detected in the image using a plurality of object determination methods;
A second object discrimination step for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means;
The object discrimination method characterized by including. In an object discrimination program for causing a computer to execute a method for discriminating an object in an image,
A first object discrimination procedure for discriminating, for each object discrimination method, an object detected in the image using a plurality of object discrimination methods;
A second object discrimination procedure for deriving one discrimination result for the object on the image from each discrimination result by the first object discrimination means;
An object discrimination program for causing a computer to execute.

Images