JP2006331266A - Method and apparatus for detecting object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and an apparatus capable of more simply and efficiently detecting an object in a low misdetection state by constituting a plurality of images obtained by photographing the same object as a plurality of detection processes in the detection of the object in the images and effectively utilizing relation of these images. <P>SOLUTION: The object detection method and apparatus include a first detection process for detecting a first image, a second detection area setting process for setting a detection area of a second image, a second detection process for detecting the detection area, and a determination process for determining object detection on the basis of the detection results of the first and second detection processes. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a detection method and an object detection device for an object such as a face in an image.

  Conventionally, a processing technique for detecting an object such as a face from an image and extracting the area has been developed as an important issue. In particular, for human faces, more advanced identification and facial expression recognition issues are becoming important, and the accuracy and efficiency of the extraction process itself from the image as the base are further required.

  Human face detection has various problems. The target image is usually taken under uncontrollable illumination, the background is not constant, the image quality is not good, and the distance to the object cannot be predicted. Of course, the position and orientation of the face is not controlled either.

  However, even under such circumstances, there has been a demand for a detection method that has very few false detections and requires a fast processing time.

  As a conventional method for detecting an object, particularly a face, a template matching method is known in which matching processing is performed with a template prepared in advance using light and shade of an image to select an optimal template. There is also a method in which various feature amounts are extracted from the image of the detection region, and determination is made from the values of the feature amounts. It is possible to experimentally set or learn what value of what feature value is important for object determination. In addition, there are many detection methods for setting the discrimination space by positioning the image of the detection area in the vector space, and there are many variations on the space setting and discrimination indicators.

  However, with either method, the problem of accuracy (false detection) and the problem of speed (slow) have always been around. In particular, in most methods, it is necessary to set an appropriate rectangular area for detection. In practice, in order not to reduce the accuracy of detection, the entire target image is detected with an appropriate rectangular area. It was necessary to scan and search for a rectangular area to be detected. For this reason, much time is required.

  Therefore, it is required to simplify the detection process itself, not to lengthen the entire processing time, and not to increase false detections.

  In order to solve the problem of accuracy and speed of detection, a type of processing technique has been proposed in which a plurality of simple detection methods are combined for comprehensive detection determination (see, for example, Patent Document 1).

  Patent Document 1 proposes a detection process called a cascade detection method. One by one, a number of simple discriminators with weak detection power are executed in cascade, i.e., sequentially, and the evaluation values according to each execution result are accumulated. This is what sorts are done. As a result, the rectangular area that is easy to be discriminated is determined at an early stage, and those that are difficult to discriminate remain until later, so that the processing time is not so long.

  However, the problem of accuracy, that is, erroneous detection, depends on how many numbers are prepared because a simple classifier with weak detection power is used. Also, simply increasing the number of discriminators having similar detection power has a limit on the effect. It would be desirable to add another means to reduce false detections.

  In addition, a technique of using a stereo image to specify the size of an object to be detected and to save the trouble of exhaustively searching for a rectangular area has been proposed (for example, see Patent Document 2). .

  In Patent Literature 2, distance information is obtained using a stereo image. A stereo image is composed of two images with parallax that are photographed so as to be stereoscopically viewable. From this, distance information can be calculated for each specific location in the image. When the distance information is obtained over the entire image, the edge portion of the subject is obtained, and thereby the contour of the object candidate is derived. If the position and size of the object candidate can be almost estimated, the labor for searching the rectangular area can be greatly reduced, and the processing speed can be increased.

However, the detection process itself requires a conventional method. In addition, the accuracy of calculating the distance information becomes a problem in order to obtain the contour of the object candidate more accurately. It is desirable to use the stereo image itself for detection rather than distance information with poor accuracy.
JP 2004-252940 A JP 2002-216129 A

  An object of the present invention is to use a plurality of images obtained by photographing the same subject in the detection of an object in an image, configure it as a plurality of detection steps, and effectively use the relationship between the images, It is an object of the present invention to provide an object detection method and an object detection apparatus that solve the above-mentioned problems, are simpler, more efficient, and have fewer erroneous detections.

  The object of the present invention can be achieved by the following configurations.

(Claim 1)
A specific object in the first image and the second image is detected using the first image and the second image acquired by a photographing device that photographs a plurality of images of the same subject from different positions. In the object detection method, based on the first object detection step of detecting an object candidate in the first image and the object candidate detected in the first object detection step, a detection area in the second image is determined. A second detection area setting step for setting, a second object detection step for detecting an object candidate in the detection area set in the second detection area setting step, and an object candidate in the second object detection step An object determination step for determining object detection based on the detection result. Object detection method.

(Claim 2)
The first object detection step detects information related to the position and size of the object candidate in the first image, and the second detection region setting step is detected in the first object detection step. The position and size of the object candidate in the second image corresponding to the object candidate are estimated from information on the position and size of the object candidate in the first image, and The object detection method according to claim 1, wherein a region where an object candidate is to be detected is set.

(Claim 3)
The first object detection step includes a first detection region setting step for setting a plurality of different detection regions in the first image, and each set in the first detection region setting step. In the detection area, object candidates are detected, and the second detection area setting step is associated with each object candidate when the object candidate is detected in the first object detection step. The detection area in the second image is set, and the second object detection step detects an object candidate in each detection area set by the second detection area setting step, The object determination step includes a detection result of each object candidate detected in the first object detection step, Serial based on the detection result of the second each object candidates detected in association with the object detection process determines object detection, object detection method according to claim 1 or 2, characterized in that.

(Claim 4)
The first object detection step is to detect object candidates using a plurality of different object detection methods in the first image, and the object determination step is a detection set in the first object detection step. 4. The method according to claim 1, wherein when no object candidate is detected by at least one of the object detection methods for each area, it is determined that no object is detected in the detection area. The object detection method according to any one of the preceding claims.

(Claim 5)
The second object detection step is to detect object candidates by a plurality of different object detection methods in the second image, and the object determination step is a detection set in the second object detection step. 5. The method according to claim 1, wherein if no object candidate is detected in at least one of the object detection methods for each area, it is determined that no object is detected in the detection area. The object detection method according to any one of the preceding claims.

(Claim 6)
In the first object detection step, object candidates are detected by a plurality of different object detection methods in each detection region set in the first detection region setting step, and detected by one detection method. 6. The object detection method according to claim 1, wherein when the number of detected object candidates is equal to or less than a predetermined threshold value, the process proceeds to a second object detection step. .

(Claim 7)
The object detection method according to claim 1, wherein the object to be detected is a face.

(Claim 8)
A specific object in the first image and the second image is detected using the first image and the second image acquired by a photographing device that photographs a plurality of images of the same subject from different positions. In the object detection device, a detection area in the second image is determined based on a first object detection unit that detects an object candidate in the first image and an object candidate detected by the first object detection unit. Second detection area setting means for setting, second object detection means for detecting object candidates in the detection area set by the second detection area setting means, and object candidate in the second object detection means And an object determination unit that determines object detection based on the detection result. Object detection apparatus.

(Claim 9)
The first object detection means detects information regarding the position and size of the object candidate in the first image, and the second detection area setting means is detected by the first object detection means. The position and size of the object candidate in the second image corresponding to the object candidate are estimated from information on the position and size of the object candidate in the first image, and 9. The object detection apparatus according to claim 8, wherein an area where an object candidate is to be detected is set.

(Claim 10)
The first object detection means includes first detection area setting means for setting a plurality of different detection areas in the first image, and each set by the first detection area setting means. The second detection region setting means is associated with each object candidate when the object candidate is detected by the first object detection means. A detection area in the second image is set, and the second object detection means detects an object candidate in each detection area set by the second detection area setting means, The object determination means includes a detection result of each object candidate detected by the first object detection means, Serial based on the detection result of each object candidates detected in association with the second object detection means determines an object detection, object detection apparatus according to claim 8 or 9, characterized in that.

(Claim 11)
The first object detection means detects an object candidate by a plurality of different object detection methods in the first image, and the object determination means is a detection set by the first object detection means. 11. The method according to claim 8, wherein if no object candidate is detected by at least one of the object detection methods for each area, it is determined that no object is detected in the detection area. The object detection device according to any one of the preceding claims.

(Claim 12)
The second object detection means is for detecting object candidates by a plurality of different object detection methods in the second image, and the object determination means is a detection set by the second object detection means. The object according to any one of claims 8 to 11, wherein if no object candidate is detected by at least one of the respective object detection methods for an area, it is determined that no object is detected in the detection area. The object detection device according to any one of the preceding claims.

(Claim 13)
The first object detection means detects object candidates by a plurality of different object detection methods in each detection area set by the first detection area setting means, and is detected by one detection method. 13. The object detection device according to claim 8, wherein the second object detection unit is operated when the number of the object candidates is equal to or less than a predetermined threshold value. .

(Claim 14)
The object detection apparatus according to claim 8, wherein the object to be detected is a face.

According to invention of Claim 1 and 8,
A plurality of images obtained by photographing the same subject are used as a plurality of object detection processes, and object detection is determined only when an object candidate is detected in these detection processes. By stopping the subsequent processing as detection, it is possible to provide an object detection method that is simpler, more efficient, and less erroneously detected.

According to invention of Claim 2 and 9,
By using multiple images taken of the same subject and configuring them as multiple object detection steps, and by specifying the detection region in the next step according to the detection region set in the preceding step, the object The detection method can be made simpler and more efficient.

According to the inventions of claims 3, 4, 5, 10, 11 and 12,
By using multiple images of the same subject and configuring them as multiple object detection steps, combining them with different detection area settings and different detection method settings appropriate for each step, The detection method can be further reduced in erroneous detection.

According to invention of Claim 6 and 13,
Using multiple images of the same subject and configuring them as multiple object detection processes, combining them with appropriate detection area settings and detection method settings in each process, the object candidate is detected even once in the detection process. If the detection method is not detected, the subsequent processing is aborted, so that a plurality of object detection steps can be configured so that the number of object candidates decreases every time the detection method is changed. The object detection method can be made simpler and more efficient by changing the processing step depending on whether or not the value of the object becomes equal to or less than a predetermined threshold value.

According to invention of Claim 7 and 14,
Using multiple images taken of the same subject, it is configured as multiple face detection processes, combined with appropriate detection area settings and detection method settings in each process, and detection set in the preceding process Depending on the area, the detection area in the next process can be specified, and the face candidate detection results in all the detection processes are combined to determine face detection, so that detection in the image can be performed. It is possible to provide a face detection method that is simpler, more efficient, and has fewer false detections for difficult human faces.

  An object detection method and an object detection apparatus according to the present invention, taking as an example a case where a stereo camera is used as a photographing apparatus according to the present invention and an object is detected from first and second images acquired using the stereo camera. An example of the embodiment will be described.

  As an object to be detected, a human or its face is often required. In the following example, a human face is considered as an object to be detected.

  Faces are the simplest and most accurately available for humans to visually identify specific individuals. Therefore, many face detection methods for automatically extracting a human face from an image have been studied. In addition to detection, there is an increasing demand for a system that can mechanically authenticate a specific individual from a detected face. The use of three-dimensional face information to authenticate a specific person from the face has become popular, and the number of cases in which stereo images that are the basis of three-dimensional information are also used for face detection is increasing. Yes.

  FIG. 1 is an external view of a stereo camera 1 which is an embodiment of a photographing apparatus according to the present invention.

  The stereo camera 1 has two optical systems, ie, a first optical system A and a second optical system B, which are positioned in parallel, and both optical systems capture an object at a specific position with a certain parallax. It has a simple arrangement structure. Therefore, when an object to be detected, for example, a face is captured in a captured image, the first image captured by the two optical systems, that is, the first optical system A and the second optical system B, and the second image. Should appear in both images.

  FIG. 9 is a simplified diagram showing samples of the first image (image AX1) and the second image (image BX1) taken by the stereo camera. There is parallax between images, and the same subject P is shown, but its horizontal position is different. In the present embodiment, the special relationship between the first image and the second image is used to improve the efficiency of the detection process and improve the detection accuracy.

  Although a stereo camera is taken as an example, the photographing apparatus is not limited to this, and a combination of a plurality of photographing apparatuses may be used. Any arrangement may be used as long as it captures a plurality of images of the same subject from different positions and can identify the arrangement relationship.

  FIG. 2 is a block diagram showing a configuration of an object detection system including the stereo camera 1 shown in FIG. 1 and an image processing apparatus 2 which is an embodiment of the object detection apparatus according to the present invention. A flow in which the first image and the second image captured by the stereo camera 1 are acquired by the image processing unit 3 for object detection in the image processing apparatus 2 will be described with reference to FIG.

  The image processing device 2 functions as an object detection device. In the image processing apparatus 2, the image processing unit 3 executes an object detection process.

  In the stereo camera 1, A is a first optical system. B is a second optical system arranged so that the optical axis is parallel to the first optical system. Reference numeral 9 denotes a first imaging element that captures a first image formed by the first optical system A. Reference numeral 10 denotes a second imaging element that captures a second image formed by the second optical system B. The first image and the second image captured by the first image sensor 9 and the second image sensor 10 are converted into electric signals and sent to the output image generation unit 11.

  The first image and the second image formed as digital image data for output in the output image generation unit 11 are sent to the image input unit 5 through the communication path 13 by the image output unit 12, and the object detection process Is acquired by the image processing unit 3 that executes. The image processing unit 3 temporarily stores the acquired first image and second image in the memory 4 and prepares for object detection processing.

  In this embodiment, the object detection process is performed by the image processing apparatus 2 outside the stereo camera 1, but the stereo camera itself includes an image processing unit 3 that executes the object detection process. Also good. Further, the first image and the second image may be transferred to a removable recording medium and then read out instead of passing through the communication path 13. The image acquisition method of the image processing unit 3 can take various forms.

  The object detection result by the image processing unit 3 is displayed or printed out by the output unit 6 including a display and a printer as needed.

  The configuration of the image processing unit 3 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the image processing unit 3 and the memory 4 in the image processing apparatus 2 in FIG.

  In the image processing unit 3, reference numeral 31 denotes a control unit including a CPU, a ROM for storing a program, and a memory such as a RAM as a work area, and controls the activity of the entire image processing unit 3. An image switching unit 32 mainly functions to hold one of the first and second images according to the processing process. An object determination unit 33 functions as an object determination unit. That is, an object determination process described later is executed.

  Reference numeral 34 denotes a first detection area setting unit, which functions as a first detection area setting means. That is, a first detection area setting process described later is executed. Reference numeral 35 denotes a first detection method setting unit, which functions as first detection method setting means. That is, a first detection method setting step described later is executed. Reference numeral 36 denotes a first object detection unit that functions as the first object detection unit in cooperation with the first detection region setting unit and the first detection method setting unit. That is, a first object detection process described later is executed.

  Reference numeral 37 denotes a second detection area setting unit, which functions as second detection area setting means. That is, a second detection area setting process described later is executed. Reference numeral 38 denotes a second detection method setting unit, which functions as second detection method setting means. That is, a second detection method setting step described later is executed. 39 is a 2nd object detection part, and functions as a 2nd object detection means. That is, a second object detection process described later is executed.

  In the memory 4, reference numeral 41 denotes a plurality of detection information. For example, the position information of the detection area, the size information of the detection area set by the first detection area setting unit 34 and the second detection area setting unit 37, the first object Evaluation values of object candidates detected by the detection unit 36 and the second object detection unit 39 are included.

  In 43, specific data of a plurality of detection areas in which the object candidates are detected is stored as detection data. Index information is stored in the detection information 41, and the substance is detection data 43.

  FIG. 4A shows an example of the contents of the detection data 43 stored in the memory 4. Serial number 431, the start point coordinate 432 of the area where the object candidate for the first image is detected, the end point coordinate 433, if necessary, the evaluation value 434 at the time of detection, the object candidate was detected in the second image corresponding to them The start point coordinates 435 of the region, the end point coordinates 436, and the evaluation value 437 at the time of detection, if necessary. The position and size are obtained from the start point coordinates and end point coordinates.

  The detection information 41 is accessed by the object determination unit 33 through the control unit 31. The operation is writing or deleting detection information. If it is determined that the object is detected even once, the detection information is written or held, and if it is determined that the object is not detected even once, the detection information is deleted. The information is also read from the second detection area setting unit 37 as information for setting the detection area.

  Reference numeral 42 denotes information of a plurality of detection methods, and index information is stored including the detection methods of executed and remaining. 44 stores in advance main body data of a plurality of detection methods. FIG. 4B shows an example of the contents of the detection method data 44 stored in the memory 4. A serial number 441, a detection method of an object candidate for the first image, that is, a first detection method 442, a detection method of an object candidate for the second image, that is, a second detection method 443, and the like. These are stored in advance prior to the object detection process, and are not rewritten or deleted during the process.

  Similarly, the detection method information 42 is accessed for detection method setting by the first detection method setting unit 35 and the second detection method setting unit 38 through the control unit 31. Further, in the repetition of the detection method, an access is made to confirm a detection method that has been executed or the remaining method and to switch the processing flow.

  The procedure in which each component of the image processing unit 3 operates will be described in detail using the following processing example.

(Processing example 1)
A processing example 1 of the object detection processing will be described with reference to FIG. FIG. 5 is a flowchart showing the flow of object detection processing according to Processing Example 1.

  When the object detection process is started, first, the first image and the second image stored in the memory 4 are read in step S11. Or you may receive and acquire from the stereo camera 1 or another memory | storage device.

  Next, in step S12, which is a first detection area setting step, the first detection area setting unit 34 sets a first detection area for the first image. Specifically, an appropriate subwindow is set, and the entire area of the first image is scanned in the subwindow.

  The sub-window scan executed in step S12 will be described with reference to FIG. FIG. 6A shows the setting of the sub window 61 for the entire area 51 of the first image.

  The area of the sub window 61 is set as the first detection area. An object, that is, a face detection process is executed on the set first detection area, that is, the subwindow 61. However, the detection process is executed not only on one subwindow but also on the entire subimage of the first image. By scanning over the area 51, the detection process is performed on the entire first image area 51.

  FIG. 6A shows a state in which the sub window 61 is shifted in the arrow direction in units of pixels and scanned. When scanning in the horizontal direction is completed, the pixel is shifted in the vertical direction again in units of pixels, and the scanning is continued. In this way, the object detection process is executed on the subwindow 61 located at any position in the first image 51, that is, the detection area.

  FIG. 6B shows a case where a sub window 62 having a size different from that in FIG. 6A is set for the entire area 51 of the first image. Even if there is an object to be detected in the first image 51, its size is unknown. In order to apply the same detection method to objects of different sizes, the size of the subwindow is changed little by little, and the entire area 51 of the first image is also scanned. Even if the object size is different, it can be detected.

  Therefore, the size of the sub-window is changed variously, and scanning is performed and detection processing is performed each time.

  As another method for changing the size, the resolution of the first image is converted, the size of the first image itself is changed little by little, the window is the same size, and the plurality of resolution-converted images are scanned. There is also a way to go. According to this method, since the size of the detection area is not changed, the detection process can be executed efficiently without considering the size.

  As described above, the scanning method may have various forms. In short, the first image is detected so that the object can be detected at any position and in any position in the entire area of the first image. On the other hand, the first detection area is set comprehensively.

  In this processing example, by repeating the processing, the first detection area setting is sequentially changed, and the entire area of the first image is scanned comprehensively.

  Returning to FIG. 5, in step S13, which is the first detection method setting step, the first detection method setting unit 35 sets the object detection method for the first detection region set in step S12.

  There are various known object detection methods, and any method may be set. For example, a method such as template matching may be used. If various templates are prepared and determined from the degree of matching with each template, and the determination of a large number of prepared templates is comprehensively determined, the accuracy as a whole can be improved.

  There is also a method in which various feature amounts are extracted from the image of the detection area, and determination is made from the values of the feature amounts. It is possible to increase the detection accuracy by experimentally setting or learning what value of what feature value is important for object determination.

  In addition, there are many detection methods for setting the discrimination space by positioning the image of the detection area in the vector space, and there are many variations on the space setting and discrimination indicators.

  In this processing example, a specific detection method is not limited. However, in order to have both the detection accuracy and the detection speed, a simple detection method, that is, a number of weak identification means are configured in a cascade manner, and are sequentially tested. , Based on the premise of comprehensive judgment.

  In this way, it is possible to perform object detection processing in which step S13 is repeated many times, and a different detection method is set each time. Further, the detection method set in step S13 may be a detection method including a plurality of detection methods.

  In this processing example, a plurality of detection methods are prepared in advance, and repeated processing is performed so that all of them are sequentially executed. Details of these detection processing methods will be described later.

  Next, in step S14 which is the first object detection step, the first object detection unit 36 applies the first detection method set in step S13 to the first detection region set in step S12. The object candidate detection process is executed.

  The detection step in step S14 outputs a result indicating whether there is an object candidate. At this stage, it is a candidate, and object detection is determined by completing all object detection steps.

  In step S15, it is determined whether or not an object candidate has been detected for the result of the first object detection step in step S14. If an object candidate is detected (step S15: YES), step S16 is executed, the object candidate detection information is stored in the memory, and the process proceeds to step S18. In step S16, the position information of the detection area, the size information of the detection area, and the like are stored. Depending on the detection method, the evaluation value of the object candidate is also included.

  Note that the object candidate detection information 41 stored in the memory 4 increases when an object candidate is detected in a different detection area, and decreases when an object candidate is detected using a different detection method. Eventually, the object candidates remaining after trying all the detection methods are determined as objects.

  If no object candidate is detected in step S15 (step S15: NO), step S17 is executed, and it is determined that no object candidate is detected in the detection area at this time. If the position information of the corresponding detection area and the size information of the detection area are already stored, the detection information is deleted. Once the detection information is deleted, the detection information is not referred to, that is, no new detection method is subsequently tried for the same detection area. Therefore, the next step S18 is omitted, and the process proceeds to step S19.

  In short, only when object candidates are detected for all the prepared detection methods, the object detection is judged for the detection area, and the object is detected even once among all the prepared detection methods. If a candidate is not detected, the detection area makes a determination that the object is not detected, and the subsequent detection processing for the detection area is terminated.

  The determination in step S15 for the first detection area and the storage management of detection information in steps S16 and S17, and the determination in step S24 for the second detection area described later and the storage management of detection information in steps S25 and S26 The determination unit 33 repeats the object detection process many times before the end of the object detection process, but all these processes are combined to form an object determination process.

  In step S18, it is determined whether or not a plurality of detection methods prepared in advance have been tried. If all the detection methods have been completed for the set first detection region (step S18: YES), the process proceeds to step S19. If an unexecuted detection method remains (step S18: NO), the process returns to the first detection method setting step of step S13, and after setting a new detection method, the subsequent processing is repeated.

  In step S19, it is determined whether or not scanning of the first detection area for the first image has been completed. When the scan is completed and all the detection area settings are completed (step S19: YES), the process proceeds to step S20. When the scan of the detection area remains (step S19: NO), the process returns to the first detection area setting step of step S12, and the scan is continued to set a new detection area and repeat the subsequent processing. Become.

  When step S19 is completed and the process proceeds to step S20, all the prepared first detection methods are applied to all the first detection areas scanned in all areas of the first image. That is, the object detection process 1 is completed.

  Of course, the object candidate is not detected in the process so far, and there should be many detection areas that are not stored in the memory 4 as the detection information 41 or deleted. The detection area stored in the memory 4 as the detection information 41 at the time of proceeding to Step S20 means that the object candidate is still being detected, and the detection information 41 is also detected in the subsequent second image detection process. If the memory 4 holds the memory, it is finally determined that the object is detected in the detection area.

  Accordingly, the following processing for the second image is sequentially executed only for the detection area stored as detection information so far.

  In step S20, the detection information 41 is sequentially read from the memory 4. This is because the first detection area is comprehensively set by the scanning operation for the first image, whereas there are object candidates for the second image based on the result for the first image. This is an operation of setting only the estimated detection area as the second detection area.

  In step S21, which is the next second detection area setting step, the second detection area setting unit 37 is based on the detection information 41 read in step S20, in the second image corresponding to the detection information 41. A second detection area is set.

  The detection information (position information and size information) stored in step S16 and read out in step S20 represents the position and size of the candidate object detected in the first image, and the second image Used to estimate the position and size of the corresponding object candidate.

  That is, in the second image, unlike the case of the first image, the setting of the detection area does not scan the entire area of the second image, but the detection information of the object candidates detected in the first image. From this, the position and size of the corresponding object candidate in the second image are estimated, and the area is set as the second detection area.

  Details of the second detection area setting method executed in step S21 will be described later.

  Next, in step S22 which is a second detection method setting step, the second detection method setting unit 38 sets an object detection method for the second detection region set in step S21.

  In the present processing example, as in the case of the first image, a plurality of detection methods are prepared in advance for the second image, and all of them are sequentially executed by repeated processing. These detection methods use the same detection method as the plurality of detection methods for the first image in the present processing example, but of course, a detection method different from that for the first image may be used. One detection method may be composed of a plurality of detection methods.

  Next, in step S23 which is the second object detection step, the second object detection unit 39 applies the second detection method set in step S22 to the second detection region set in step S21. The object candidate detection process is executed.

  As in the case of the first image, the detection step using the set detection method outputs a result indicating whether there is an object candidate.

  In step S24, it is determined whether or not an object candidate has been detected for the result of the second object detection step in step S23. If an object candidate is detected (step S24: YES), step S25 is executed, the object candidate detection information is stored in the memory, and the process proceeds to step S27.

  Although it is the detection information 41 of the object candidate that has been continuously stored in the memory 4 in the detection process for the first image, it can be further reduced by trying more detection methods for the second image. Object candidates remaining after trying all the detection methods for the second image are determined as objects.

  If no object candidate is detected in step S24 (step S24: NO), step S26 is executed, and it is determined that no object candidate is detected in the detection area at this time. Therefore, the detection information is deleted from the position information and size information of the corresponding detection area. Once the detection information is deleted, the detection information is not referred to, that is, a new detection method is not tried for the same detection area, as in the case of the first image. is there. Accordingly, the next step S27 is omitted, and the process proceeds to step S28.

  In step S27, it is determined whether or not all the plurality of detection methods prepared in advance have been tried. If all the detection methods have been completed for the set second detection area (step S27: YES), the process proceeds to step S28. If an unexecuted detection method remains (step S27: NO), the process returns to the second detection method setting step of step S22, and a new detection method is set and subsequent processing is repeated.

  In step S28, all detection information stored in the memory is read out, and it is determined whether or not all processes corresponding to the detection information have been completed for the second image. If the processing for all the detection information has been completed (step S28: YES), all the object detection processing is ended. The detection information remaining in the memory at this time is finally determined to be object detection.

  When reading of detection information and processing remain (step S28: NO), it returns to reading of detection information of step S20, and after reading the remaining new detection information, the subsequent processing is repeated.

  As an example of detection method setting and object candidate detection processing executed in steps S13 to S18 and steps S22 to S27 in FIG. 5, a cascade detection process will be described.

  In this detection processing method, a plurality of simple detection methods are prepared in advance, and an object is accurately and efficiently processed by a simple detection method by repeatedly performing all of them in order. Can be detected. Details thereof will be described with reference to FIG. FIG. 7A shows a procedure in which a plurality of detection methods are executed in cascade, that is, sequentially and subjected to discrimination processing.

  In FIG. 7A, 101 is a step of executing a certain detection method, and 102 is a step of executing another certain detection method. A plurality of processes for executing such different detection methods are arranged, and the final M detection method execution process. Execution processing is performed in the order of the arrows. That is, the process 101 and the process 102 are executed in this order, and the process ends.

  First, when the detection area is set, step 101 is executed. If the object is detected as a candidate at this time point according to the result of step 101, the next step 102 is executed. If no object candidate is detected, it is finally determined that no object is detected at this point, and the subsequent processing is not performed.

  Similarly, when the process proceeds to step 102, if an object candidate is detected based on the result of step 102, the next step 103 (not shown) is executed. If no object candidate is detected, it is finally determined that no object is detected at this point, and the subsequent processing is not performed.

  The detection process proceeds in this way, and only when the object candidate is detected in all detection method execution steps until the result of the last detection method execution step M is reached, it is finally determined that the object is detected. If an object candidate is not detected even once in the middle, it is finally determined that no object is detected at that time, and the detection process ends.

  As can be seen from the above, the characteristics of this detection process are to prepare multiple simple detection methods to ensure the overall detection accuracy, and each detection method is a weak detection, that is, an object-like one is a candidate. By eliminating those that are not objects reliably, it is possible to eliminate detection regions that probably do not contain the majority of objects at an early stage of processing, thereby shortening the overall processing time.

  These multiple detection methods are performed sequentially but need not be performed sequentially. For example, when the result of the process 101 is obtained, the process 101 may be performed for another detection region without moving to the process 102. However, the result of step 101 is retained, and step 102 will be executed. Thus, the order is preserved but need not be executed continuously.

  A plurality of detection methods may be configured and combined as one detection method. An example is shown in FIG.

  In FIG. 7B, two sub detection methods are combined into one detection method. In comparison with FIG. 7A, the steps 101 and 102 are each composed of two sub-detection methods. Therefore, these two sub detection methods are executed in succession. However, it is the same as FIG. 7A that an object candidate is detected only when an object candidate is detected in both of them.

  As described above, there are many possible combinations of detection methods. However, it is only necessary to select an appropriate combination based on the idea that non-objects are excluded at an early stage.

  A specific example of the method of setting the second detection area from the detection information 41 in the first detection area executed in step S21 of FIG. 5 will be described with reference to FIG. FIG. 8 is a diagram illustrating a positional relationship between a stereo camera that captures an object and an object to be captured.

  In FIG. 8, reference numeral 1 denotes a stereo camera used for photographing, and X denotes the position of the object P to be photographed. X is assumed to be equidistant from the first optical system A and the second optical system B of the stereo camera 1, and D is the distance from the stereo camera 1 in the optical axis direction of the optical systems A and B. d is the distance between the optical axes of the optical systems A and B. θ represents the angle of view of each optical system, that is, the range of angles that appear in the captured image.

  FIG. 9 shows an image obtained by photographing the object P (here, a face) at the position X with the stereo camera 1. The image AX1 is a first image acquired by the first optical system A, and the image BX1 is a second image acquired by the second optical system B.

  In both the images AX1 and BX1, the object P at the same X position exists as an image. However, since the optical axes of the first optical system and the second optical system are separated from each other by the distance d, the position of the object in the image AX1 and the position of the object in the image BX1 are different from each other.

  In the present embodiment, object detection is first performed on the first image, and then object detection is performed on the second image. When an object candidate is not detected in at least one of the first image and the second image, the detection process for the detection area is terminated there. That is, the object detection process is performed on the second image only for the detection region where the object candidate is detected for the first image.

  When an object candidate is detected in the first image, the response in the second image is detected from detection information of the detected object candidate, that is, the position and size of the first detection area in the first image. The position and size of the candidate object to be present are estimated, and a second detection area corresponding to the first detection area is set.

  As an actual example of the above estimation, the image sizes of the first image AX1 and the second image BX1 are both 640 × 480 pixels, and the distance d between the first optical system A and the second optical system B is 0.1 m. Each angle of view θ is 50 degrees. The distance D from the stereo camera 1 at the position X of the object P is 1 m.

  The difference between the horizontal coordinate position of the object P in the first image AX1 and the horizontal coordinate position of the object in the second image BX1 is set as x, and this is obtained.

  Since the distance d of the first optical system A and the second optical system B is 0.1 m, the horizontal direction 0.1 m at the point of the distance D = 1 m from the stereo camera 1 is in the image AX1 and the image BX1. This corresponds to the difference x (pixel) in the coordinate position in the horizontal direction.

Considering the angle of view θ = 50 degrees, the entire horizontal image range at a distance D = 1 m corresponds to 640 pixels.
x / 640 = d / (2Dtan (θ / 2))
Therefore, when this is calculated, x = 68 (pixels)
It becomes.

  Table 1 shows the detection area in the second image BX1 based on the detection information (the position and size of the object, that is, the position and size of the detection area) in the first image AX1 when the distance D is different. An example of calculating the position and size of is shown. Both position and size are based on horizontal coordinate values (pixels).

  For example, in the above example, when the detection information in the first image AX1 (the position and size of the first detection region where the object candidate is detected) is 250 and 137, respectively, x = 68 (pixels) The position and the size of the second detection region in the second image BX1 are 250-68 = 182 and the size is 137. In this case, since it is a stereo camera and the distance from the first and second optical systems to the object in the optical axis direction of each optical system is the same, the size does not change just by shifting the position in the horizontal direction. .

  FIG. 9 shows the coordinate position in the horizontal direction with the above numerical example. The horizontal position of the first detection region AX2 in the first image AX1 is 250 and the size is 137, whereas the horizontal position of the second detection region BX2 in the second image BX1 is 182 and the size is 137. It is.

  In this way, from the object candidate detection information in the first image, that is, the position and size of the first detection region in the first image, the position where the corresponding object candidate exists in the second image, and The size can be estimated and a second detection area corresponding to the first detection area can be set.

  Actually, it is considered that the estimation of the position and size of the object candidate varies. For example, suppose that it is detected when the window size is 137. If the window size is discrete and the next size is 114, it is assumed that there is an appropriate window size between 137 and 114.

  In the case of the window size 114, since the positional shift is calculated as x = 57 (pixels), the position of the second detection area 64 is actually set to the position of the first detection area 63 as shown in FIG. On the other hand, it is preferable to scan by changing from −68 to −57 to set an optimal detection area.

  As described above, executing the procedure of Processing Example 1 has the following effects of improving detection accuracy and reducing processing time.

  (1) In the object detection process, both the first image and the second image are subjected to the detection process, and if it is detected on the one hand, it must be detected on the other hand. By omitting the other detection step, the increase in processing time can be minimized while increasing the number of images and contributing to improvement in accuracy.

  (2) Further, when an object candidate is detected in the first image, the second information is detected from the detection information in the first image, that is, information on the position and size of the object candidate detection in the first image. By calculating the information on the position and size of the object candidate in the image, the detection area can be specified for the second image, and the process of repeatedly detecting by changing the detection area can be reduced. The processing time can be greatly shortened.

  (3) In order to improve accuracy, it is effective to apply a plurality of detection steps in which a plurality of detection methods are set, but on the other hand, the processing becomes complicated and the processing time is increased. However, non-objects that occupy the majority in the image can be obtained by executing a screening method determination that immediately terminates subsequent processing for areas where object candidates cannot be detected while being combined with the multiple images and multiple detection areas. The area can be eliminated at an early stage, and the processing time can be greatly shortened.

  The object detection process in the present embodiment is configured to sequentially repeat a plurality of detection steps in order to improve accuracy. The elements that make a plurality of detection steps are a plurality of images (first and second images as stereo images), a plurality of detection regions, and a plurality of detection methods. Each detection step is a combination thereof, but there are various multi-stage configurations when viewed in order by a combination of these repetitions.

  For example, in the processing example 1 described above, the settings of the first image and the second image are first changed, then the detection area is changed for each image, and the detection method is set for each detection area. Change and repeat.

  The order in which the iterations are combined is arbitrary from the viewpoint of eliminating non-objects at an early stage and skipping unnecessary detection steps, that is, improving the overall processing efficiency and reducing the processing time. Just choose.

  In the following processing example 2 and processing example 3, another combination of iterations will be exemplified and described.

(Processing example 2)
Another example 2 of the object detection process will be described with reference to FIG. FIG. 11 is a flowchart showing the flow of object detection processing according to Processing Example 2.

  Process example 2 differs from process example 1 only in the order of the iterative processing, and each step is almost the same as process example 1. The same steps as those of the processing example 1 are denoted by the same reference numerals, and description of the steps is omitted.

  When the object detection process is started, first, the first image and the second image stored in the memory are read in step S11. This is the same as Processing Example 1 including the previous shooting.

  Next, in step S12, which is a first detection area setting step, a first detection area is set for the first image. An appropriate sub-window is set, and the entire area of the first image is scanned in the sub-window as in the processing example 1.

  Next, in step S13, which is a first detection method setting step, an object detection method is set for the first detection area set in step S12. A plurality of detection methods are prepared in advance, and repeated processing is performed in order to sequentially execute all of them, as in the first processing example.

  Next, in step S14, which is the first object detection step, the first detection method set in step S13 is applied to the first detection region set in step S12, and object candidate detection processing is executed. To do. It is an object candidate at this stage, and object detection is determined after all object detection steps are completed, as in the first processing example.

  In step S15, it is determined whether or not an object candidate has been detected for the result of the first object detection step in step S14. If an object candidate is detected (step S15: YES), the process proceeds to step S21, that is, the process of the second image is performed when the object candidate is detected.

  If no object candidate is detected in step S15 (step S15: NO), step S17 is executed, and at this time, it is determined that no object candidate is detected in the detection area, and if the detection information has already been detected. If it is remembered, delete it. Therefore, the detection process is not performed on the second image, the process after the next step S21 is omitted, and the process proceeds to step S19.

  In step S21, which is the second detection area setting step, the second detection area in the second image corresponding to the detection area is set based on the first detection area in which the object candidate is detected in step S14. To do. The method for estimating the position and size of the second detection area from the position information and size information of the first detection area is the same as in the first processing example.

  Next, in step S22, which is a second detection method setting step, an object detection method is set for the second detection region set in step S21. The same detection method as a plurality of detection methods for the first image may be used, or a different detection method may be used as in Processing Example 1.

  Next, in step S23 which is the second object detection step, the object detection processing is executed by applying the second detection method set in step S22 to the second detection region set in step S21. To do.

  In step S24, it is determined whether or not an object candidate has been detected for the result of the second object detection step in step S23. If an object candidate is detected (step S24: YES), step S16 is executed, the object candidate detection information is stored in the memory, and the process proceeds to step S18.

  If no object candidate is detected in step S24 (step S24: NO), step S17 is executed. At this time, it is determined that no object candidate is detected in the detection area, and the detection information is deleted. . Accordingly, a new detection method is not tried after that for the detection region, as in the first processing example. Therefore, the next step S18 is omitted, and the process proceeds to step S19.

  Only when object candidates are detected for all detection methods prepared, the detection area is determined to detect an object, and if no object candidate is detected, the detection area is not detected. Thus, the subsequent detection processing for the detection area including the second image is aborted.

  Therefore, in the case of the process example 2, the determination in step S15 and the storage and deletion processes in steps S16 and S17 that are repeated many times for the first detection area and the second detection area form the object determination process. Will do.

  In step S18, it is determined whether or not a plurality of detection methods prepared in advance have been tried. When all the detection methods have been completed for the set detection area (step S18: YES), the process proceeds to step S19. If an unexecuted detection method remains (step S18: NO), the process returns to the first detection method setting step of step S13, and after setting a new detection method, the subsequent processing is repeated.

  In step S19, it is determined whether or not scanning of the first detection area for the first image has been completed. If the scan is finished and all the detection area settings are finished (step S19: YES), all the object detection processes are finished here. The detection information remaining in the memory at this time is finally determined to be object detection.

  When the scan of the detection area remains (step S19: NO), the process returns to the first detection area setting step of step S12, and the scan is continued to set a new detection area and repeat the subsequent processing. Become.

(Processing example 3)
Another process example 3 of the object detection process will be described with reference to FIGS. 12 to 15 are flowcharts showing the flow of object detection processing according to Processing Example 3.

  The processing example 3 is different from the processing examples 1 and 2 in that the order of the iterative processing is different, and further, when the number of detected areas where the object candidates are detected is equal to or less than a predetermined threshold, Is the point that has changed. Most of the steps are the same as those in Processing Examples 1 and 2. The same steps as those of the processing examples 1 and 2 are referred to by the reference numerals of the processing examples 1 and 2, and the description of the steps is omitted.

  When the object detection process is started, first, the first image and the second image stored in the memory are read in step S301 in FIG. The process is the same as Step S11 in Processing Example 1 including the previous shooting.

  Next, in step S302, which is a first detection method setting step, an object detection method is set. A plurality of detection methods are prepared in advance, and repeated processing is performed in order to sequentially execute all of them, as in the first processing example step S13.

  In step S303, the detection information is sequentially read from the memory. Processing example 1 is the same as step S20. However, the first image is scanned only in the initial stage. That is, this step S303 is omitted, and the first detection area is comprehensively set by a scanning operation in the next step S304.

  Next, in step S304, which is a first detection area setting step, a first detection area is set for the first image. As described above, an appropriate subwindow is initially set, and the entire area of the first image is scanned in the subwindow. When the detection information is read in the preceding step S303, the detection information is set based on the detection information.

  Next, in step S305, which is the first object detection step, the first detection method set in step S302 is applied to the first detection region set in step S304, and object candidate detection processing is executed. To do. Processing example 1 is the same as step S14.

  In step S306, it is determined whether an object candidate has been detected for the result of the first object detection step in step S305. Processing example 1 is the same as step S15. If an object candidate is detected (step S306: YES), step S307 is executed to store or hold the object candidate detection information in the memory.

  If no object candidate is detected in step S306 (step S306: NO), step S308 is executed, and it is determined that no object candidate is detected in the detection area at this point. That is, the detection information is not stored in the memory and is deleted if it is already stored.

  In step S309, all the detection information stored in the memory is read out, and it is determined whether or not all the processes corresponding to the detection information have been completed for the first image. Processing example 1 is the same as step S28. If the processing for all the detection information has been completed (step S309: YES), all the object detection processing is ended. The detection information remaining in the memory at this time is finally determined to be object detection.

  When reading of detection information and processing remain (step S309: NO), the process returns to reading of detection information in step S303, and after reading the remaining new detection information, the subsequent processing is repeated.

  Next, in step S310, it is determined whether or not the number of detection areas in which object candidates are detected, that is, the number of detection information stored in the memory is equal to or less than a predetermined threshold value. However, since all the detection areas are initially being scanned, that is, the detection information is being stored, this step S310 is omitted during the scan.

  If the number of pieces of detection information is equal to or less than the predetermined threshold value in step S310 (step S310: YES), the process proceeds to step S312 in order to shift to the processing of the second image without waiting for the end of the previous detection method. If the number of pieces of detection information is not less than or equal to the predetermined threshold (step S310: NO), the process proceeds to step S311.

  In step S311, it is determined whether or not a plurality of detection methods prepared in advance have been tried. Processing example 1 is the same as step S18. If all the detection methods have been completed (step S311: YES), the process proceeds to step S312. If an unexecuted detection method remains (step S311: NO), the process returns to the first detection method setting step of step S302, and after setting a new detection method, the subsequent processing is repeated.

  In step S312, information on detection methods already executed so far is stored in the memory. This is because the same detection method that has already been executed is executed in the processing for the second image in FIG.

  In step S321 of FIG. 13, information on the detection method already executed for the first image stored in the memory is read from the memory.

  Next, in step S322, which is the second detection method setting step, the second object detection method is set to be sequentially executed based on the detection methods that have been read in step S321. Processing example 1 is the same as step S22.

  In step S323, the detection information stored in the memory is sequentially read. Processing example 1 is the same as step S20.

  In step S324, which is a second detection area setting step, the second detection area in the second image corresponding to the detection information is set based on the detection information read in step S323. The setting method of the second detection area is the same as that in step S21 of processing examples 1 and 2.

  Next, in step S325, which is the second object detection step, the second detection method set in step S322 is applied to the second detection region set in step S324, and object candidate detection processing is executed. To do. This is the same as Step S23 in Processing Examples 1 and 2.

  In step S326, it is determined whether or not an object candidate has been detected for the result of the second object detection step in step S325. This is the same as step S306. If an object candidate is detected (step S326: YES), step S327 is executed, and the object candidate detection information is stored or held in the memory.

  If no object candidate is detected in step S326 (step S326: NO), step S328 is executed, and it is determined that no object candidate is detected in the detection area at this point. That is, the detection information stored in the memory is deleted.

  In step S329, all the detection information stored in the memory is read out, and it is determined whether or not the processing for the second image is completed. This is the same as Step S28 in Processing Examples 1 and 2. If the processing for all detection information has been completed (step S329: YES), the process proceeds to step S330.

  When reading of detection information and processing remain (step S329: NO), the process returns to reading of detection information in step S323, and after reading the remaining new detection information, the subsequent processing is repeated.

  In step S330, all executed detection methods stored as executed detection method information in the memory are read out, and it is determined whether or not the second image has been executed. When the execution process for all the executed detection methods has been completed (step S330: YES), the process proceeds to step S331.

  When the readout of the executed detection method and the process remain (step S330: NO), the process returns to the readout of the executed detection method information in step S321, and after reading the remaining executed detection method, the subsequent process Will be repeated.

  In step S331, it is determined whether or not all of the plurality of detection methods prepared in advance are included in the executed detection method, that is, whether or not all the detection methods have been tried on the first and second images. If all the detection methods have been completed (step S331: YES), all the object detection processes are completed. The detection information remaining in the memory at this time is finally determined to be object detection.

  If there is still a detection method that has not been executed (step S331: NO), the process proceeds to step S341 in FIG. 14 in order to execute an unexecuted detection method on the first and second images.

  In step S341 in FIG. 14, information on detection methods that are not included in the executed detection methods among the plurality of detection methods prepared in advance, that is, information on all remaining unexecuted detection methods is stored in the memory as residual detection method information. Remember.

  In step S342, the residual detection method information stored in the memory is sequentially read from the memory.

  Next, in step S343, which is the first detection method setting step, the first object detection method is set so as to be sequentially executed based on the residual detection method information read in step S342. This is the same as Step S13 in Processing Examples 1 and 2.

  In step S344, the detection information stored in the memory is sequentially read. This is the same as Step S20 in Processing Examples 1 and 2.

  In step S345, which is the next first detection area setting step, the first detection area in the first image corresponding to the detection information is set based on the detection information read in step S344. This is the same as Step S12 in Processing Examples 1 and 2.

  Next, in step S346, which is the first object detection step, the first detection method set in step S343 is applied to the first detection region set in step S345, and object candidate detection processing is executed. To do. This is the same as Step S14 in Processing Examples 1 and 2.

  In step S347, it is determined whether or not an object candidate has been detected with respect to the result of the first object detection step in step S346. This is the same as step S306. If an object candidate is detected (step S347: YES), step S348 is executed, and the object candidate detection information is held in the memory.

  If no object candidate is detected in step S347 (step S347: NO), step S349 is executed, and it is determined that no object candidate is detected in the detection area at this time. That is, the detection information stored in the memory is deleted.

  In step S350, all the detection information stored in the memory is read out, and it is determined whether or not the processing for the first image is completed. This is the same as step S329. If the processing for all the detection information has been completed (step S350: YES), the process proceeds to step S351.

  When reading of detection information and processing remain (step S350: NO), the process returns to reading of detection information in step S344, and the subsequent processing is repeated after reading the remaining new detection information.

  In step S351, all the residual detection methods stored as the residual detection method information in the memory are read out, and it is determined whether or not the first image has been executed. If the execution process for all the residual detection methods has been completed (step S351: YES), the process proceeds to step S361 in FIG. 15 to execute the residual detection method for the second image.

  When the remaining detection method reading and processing remain (step S351: NO), the process returns to reading the residual detection method information in step S342, and after reading the remaining new residual detection method, the subsequent processing is repeated. become.

  In step S361 in FIG. 15, the residual detection method information stored in the memory is sequentially read from the memory.

  Next, in step S362, which is the second detection method setting step, the second object detection method is set to sequentially execute them based on the residual detection method information read in step S361. This is the same as Step S22 in Processing Examples 1 and 2.

  In step S363, the detection information stored in the memory is sequentially read out. This is the same as Step S20 in Processing Examples 1 and 2.

  In step S364, which is the second detection area setting step, the second detection area in the second image corresponding to the detection information is set based on the detection information read in step S363. This is the same as Step S21 in Processing Examples 1 and 2.

  Next, in step S365, which is the second object detection step, the object detection processing is executed by applying the second detection method set in step S362 to the second detection region set in step S364. To do. This is the same as Step S23 in Processing Examples 1 and 2.

  In step S366, it is determined whether or not an object candidate is detected with respect to the result of the second object detection step in step S365. This is the same as step S306. If an object candidate is detected (step S366: YES), step S367 is executed, and the detection information of the object candidate is held in the memory.

  If no object candidate is detected in step S366 (step S366: NO), step S368 is executed, and it is determined that no object candidate is detected in the detection area at this time. That is, the detection information stored in the memory is deleted.

  In step S369, all detection information stored in the memory is read out, and it is determined whether or not the processing for the second image is completed. This is the same as Step S28 in Processing Examples 1 and 2. If the processing for all the detection information has been completed (step S369: YES), the process proceeds to step S370.

  When reading of detection information and processing remain (step S369: NO), the process returns to reading of detection information in step S363, and after reading the remaining new detection information, the subsequent processing is repeated.

  In step S370, all the residual detection methods stored as residual detection method information in the memory are read out, and it is determined whether or not the second image has been executed. When the execution process for all the residual detection methods has been completed (step S370: YES), all the object detection processes are completed. The detection information remaining in the memory at this time is finally determined to be object detection.

  If the remaining detection method reading and processing remain (step S370: NO), the process returns to reading the remaining detection method information in step S361, and after reading the remaining new residual detection method, the subsequent processing is repeated. become.

  According to the embodiments as shown in the above processing examples 1 to 3, in the detection of the object in the image, a plurality of images obtained by photographing the same subject are used as a plurality of detection steps, and the interval between the images is determined. By effectively utilizing the relationship, it is possible to provide an object detection method and an object detection device that are simpler, more efficient, and have fewer false detections.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modified forms are allowed within the scope as long as the gist of the present invention is met.

1 is an external view of a stereo camera that is an embodiment of a photographing apparatus according to the present invention. It is a block diagram which shows the structure of the object detection system which consists of the stereo camera of FIG. 1 and the image processing apparatus which is one Embodiment of the object detection apparatus based on this invention. FIG. 3 is a block diagram illustrating a configuration of an image processing unit and a memory in the image processing apparatus of FIG. 2. It is a figure which shows the example of the content of the detection information memorize | stored in the memory of FIG. 3, and the detection method. It is a flowchart which shows the flow of the object detection process which concerns on the process example 1. FIG. It is a figure which shows the state which sets the 1st detection area | region with respect to a 1st image, and scans. It is a figure which shows the actual example of the detection process which detects and discriminate | determines sequentially using several different detection methods. It is a figure which shows the actual example of the positional relationship of the stereo camera which image | photographed the object, and the object of imaging | photography object. It is a figure which simplifies and shows the example of the 1st image and 2nd image which were image | photographed with the stereo camera. It is a figure which shows the example of the state of the 2nd detection area setting with respect to a 1st detection area. 10 is a flowchart showing a flow of object detection processing according to Processing Example 2; It is a flowchart (1/4) which shows the flow of the object detection process which concerns on the process example 3. FIG. It is a flowchart (2/4) which shows the flow of the object detection process which concerns on the process example 3. FIG. It is a flowchart (3/4) which shows the flow of the object detection process which concerns on the process example 3. It is a flowchart (4/4) which shows the flow of the object detection process which concerns on the process example 3.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Stereo camera 2 Image processing apparatus 3 Image processing part 4 Memory 5 Image input part 6 Output part 9 1st image pick-up element 10 2nd image pick-up element 11 Output image generation part 12 Image output part 13 Communication path 31 Control part 32 Image switching Unit 33 object determination unit 34 first detection region setting unit 35 first detection method setting unit 36 first object detection unit 37 second detection region setting unit 38 second detection method setting unit 39 second object detection unit 41 detection information 42 detection Method information 51 Image area 61, 62 Subwindow 63, 64 Detection area

Claims (14)

A specific object in the first image and the second image is detected using the first image and the second image acquired by a photographing device that photographs a plurality of images of the same subject from different positions. In the object detection method,
A first object detection step of detecting object candidates in the first image;
A second detection region setting step of setting a detection region in the second image based on the object candidates detected in the first object detection step;
A second object detection step of detecting an object candidate in the detection region set in the second detection region setting step;
An object determination step of determining object detection based on a detection result of the object candidate in the second object detection step. The first object detection step detects information regarding the position and size of the object candidate in the first image,
The second detection area setting step includes:
The position and size of the object candidate in the second image corresponding to the object candidate are estimated from information on the position and size of the object candidate detected in the first object detection step in the first image. Then, an area where an object candidate is to be detected in the second image is set.
The object detection method according to claim 1. The first object detection step includes a first detection region setting step for setting a plurality of different detection regions in the first image, and each set in the first detection region setting step. In this detection area, object candidates are detected.
The second detection region setting step sets a detection region in the second image associated with each object candidate when the object candidate is detected in the first object detection step,
The second object detection step detects object candidates in each detection region set by the second detection region setting step,
The object determination step is based on a detection result of each object candidate detected in the first object detection step and a detection result of each object candidate detected in association with the second object detection step. To determine object detection,
The object detection method according to claim 1, wherein the object detection method is an object detection method. In the first object detection step, object candidates are detected by a plurality of different object detection methods in the first image,
In the object determination step, in the first object detection step, when no object candidate is detected by at least one detection method among the respective object detection methods with respect to the set detection region, Determine that the object is not detected,
The object detection method according to claim 1, wherein the object detection method is an object detection method. The second object detection step detects object candidates by a plurality of different object detection methods in the second image,
In the object determination step, in the second object detection step, when no object candidate is detected by at least one detection method among the respective object detection methods for the set detection region, Determine that the object is not detected,
The object detection method according to claim 1, wherein the object detection method is an object detection method. In the first object detection step, object candidates are detected by different object detection methods in each detection region set in the first detection region setting step,
When the number of object candidates detected by one detection method is equal to or less than a predetermined threshold value, the process proceeds to the second object detection step.
The object detection method according to claim 1, wherein the object detection method is an object detection method. The object detection method according to claim 1, wherein the object to be detected is a face. A specific object in the first image and the second image is detected using the first image and the second image acquired by a photographing device that photographs a plurality of images of the same subject from different positions. In the object detection device,
First object detection means for detecting object candidates in the first image;
Second detection area setting means for setting a detection area in the second image based on the object candidates detected by the first object detection means;
Second object detection means for detecting an object candidate in the detection area set by the second detection area setting means;
An object detection apparatus comprising: object determination means for determining object detection based on a detection result of object candidates in the second object detection means. The first object detection means detects information related to the position and size of an object candidate in the first image,
The second detection area setting means includes:
The position and size of the object candidate in the second image corresponding to the object candidate are estimated from information on the position and size of the object candidate detected by the first object detection means in the first image. Then, an area where an object candidate is to be detected in the second image is set.
The object detection apparatus according to claim 8. The first object detection means includes first detection area setting means for setting a plurality of different detection areas in the first image, and each set by the first detection area setting means. In this detection area, object candidates are detected.
The second detection area setting means sets a detection area in the second image associated with each object candidate when the object candidate is detected by the first object detection means,
The second object detection means detects an object candidate in each detection area set by the second detection area setting means,
The object determination means is based on a detection result of each object candidate detected by the first object detection means and a detection result of each object candidate detected in association with the second object detection means. To determine object detection,
The object detection apparatus according to claim 8, wherein the object detection apparatus is an object detection apparatus. The first object detection means detects object candidates by a plurality of different object detection methods in the first image,
The object determination unit is configured to detect, in the detection region, when the first object detection unit detects no object candidate by at least one detection method among the respective object detection methods for the set detection region. Determine that the object is not detected,
The object detection device according to claim 8, wherein the object detection device is an object detection device. The second object detection means detects object candidates by a plurality of different object detection methods in the second image,
The object determination means, when the object detection is not detected by at least one detection method among the respective object detection methods for the set detection area in the second object detection means, Determine that the object is not detected,
The object detection device according to claim 8, wherein the object detection device is an object detection device. The first object detection means detects object candidates by a plurality of different object detection methods in each detection area set by the first detection area setting means,
Operating the second object detection means when the number of object candidates detected by one detection method is less than or equal to a predetermined threshold;
The object detection apparatus according to claim 8, wherein the object detection apparatus is an object detection apparatus. The object detection apparatus according to claim 8, wherein the object to be detected is a face.

Images