JP2015002401A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology with which detection and tracking of a plurality of types of objects having different sizes and travel speeds from each other can be performed with high accuracy.SOLUTION: An image input unit 3 receives the input of frame images of a target area photographed with video cameras 10 and 11. An image processing processor 4 creates, on the frame images input to the image input unit 3, for each of a plurality of processing areas in which the type of an object, which is a detection object, is determined, a resizing image in which the image of the processing area is converted into an image with resolution according to the type of the object, which is the detection object. The image processing processor 4 further processes the created resizing image to detect the object of the type of the detection object.

Description

  The present invention relates to a technique for processing a frame image of a target area captured by a camera and detecting a plurality of types of objects having different sizes.

  2. Description of the Related Art Conventionally, there is an image processing apparatus that processes a frame image captured by a camera, detects an object such as a person or a vehicle captured in the frame image, and tracks (tracks) the detected object. For example, Patent Literature 1 describes a traffic flow measuring device that processes a frame image obtained by imaging an intersection and counts the number of vehicles for each branch direction (straight, right, left).

  Also, control the signal lights installed at this intersection using the number of vehicles by branching direction (straight, right turn, left turn) counted at the intersection and the presence / absence of pedestrians crossing the pedestrian crossing. Has also been done.

JP 2005-190142 A

  However, when a frame image captured by a camera is processed to detect or track an imaged object, the accuracy can be ensured if the resolution of the image is not appropriate for the detected object. Can not. On the other hand, depending on the size of the object to be detected or tracked, the moving speed, etc., the appropriate range of the resolution of the image for that object (a range where sufficient accuracy can be ensured) varies.

  Therefore, a system for detecting and tracking a plurality of types of objects having different sizes and moving speeds (for example, detecting and tracking a plurality of types of objects such as an automobile (four-wheeled vehicle), a motorcycle (two-wheeled vehicle), and a pedestrian at an intersection). In the system), if the image resolution is set to an appropriate range for any type of object, the image resolution for other types of objects may be out of the appropriate range. The accuracy of detection and tracking cannot be ensured for objects of a type whose image resolution is out of the proper range.

  As described above, in the conventional apparatus, it is difficult to accurately detect and track a plurality of types of objects having different sizes and moving speeds.

  For each type of object, the range of image resolution appropriate for the object has a certain range (the appropriate image resolution is not a specific value).

  An object of the present invention is to provide a technique capable of accurately detecting and tracking a plurality of types of objects having different sizes and moving speeds.

  In order to achieve the above object, the image processing apparatus of the present invention is configured as follows.

  A frame image of the target area captured by the camera is input to the image input unit. On the frame image input to the image input unit, a plurality of processing regions that define the types of objects (vehicles, pedestrians, etc.) that are detection targets are set. Each processing region may or may not overlap with another processing region.

  The resizing processing unit generates, for each processing area, a resizing image obtained by converting the image of the processing area into an image having a resolution corresponding to the type of object to be detected.

  Then, the object detection unit processes the resizing image generated by the resizing processing unit, and detects the type of object that is the detection target.

  In this way, instead of directly processing the frame image captured by the camera to detect the object, for each processing region that defines the type of object to be detected, the processing region is changed to the type of object to be detected. A resizing image with a corresponding resolution is generated, and the resizing image is processed to detect an object. Therefore, even when a plurality of types of objects having different sizes, moving speeds, and the like are detected and tracked, detection and tracking can be performed with accuracy for each type of object.

  In addition, since it is not necessary to install a camera for each type of object, an increase in the size and cost of the system can be suppressed.

  The resizing processing unit may generate a resizing image having a preset number of pixels for each processing region. In this case, if the size of the processing area set on the frame image is determined so as to have a resolution corresponding to the type of object to be detected set in the processing area in the generated resizing image. Good.

  The resizing processing unit may be configured to generate an image having a resolution determined in advance for each processing region. In this case, the resolution can be set regardless of the size of the processing area set on the frame image. However, the number of pixels of the resizing image generated for each processing region is not uniform.

  Moreover, you may provide the object map production | generation part which produces | generates the object map which shows the change of the position on the target area with progress of time for every object which the object detection part detected. As a result, a change in position (movement trajectory) with time can be obtained for each detected object.

  In addition, frame images captured by a plurality of cameras having different angles with respect to the target area may be input to the image input unit, and a resizing processing unit and an object detection unit may be provided for each camera. As a result, in a frame image of a certain camera, it is possible to prevent an object that has not been captured due to a shadow of an object positioned in front of the camera from being overlooked.

  Also, it is preferable to provide an object map generation unit for each camera. In this case, an integration unit that integrates the object map generated by the object map generation unit provided for each camera in units of detected objects may be provided. Thereby, the object detected from the frame image of a some camera can be processed as one object.

  In addition, when the target area is an intersection, the type of object to be detected may be a vehicle (automobile or motorcycle), a pedestrian, or the like. Moreover, what is necessary is just to set it as the structure which outputs the output signal which shows the passage of the vehicle in the intersection which is a target area, or a person based on the detection result of an object detection part. Thereby, control of the signal lamp according to traffic volume can be performed.

  According to the present invention, it is possible to accurately detect and track a plurality of types of objects having different sizes and moving speeds.

It is a block diagram which shows the structure of the principal part of an image processing apparatus. It is a figure which shows the example of installation of the video camera in an intersection. It is a functional block diagram which shows the function structure of an image processor. It is a figure which shows the frame free image of the intersection which the video camera imaged. It is a figure which shows a resizing image. It is a figure which shows an object map. It is a flowchart which shows operation | movement of an image processing apparatus. It is a flowchart which shows an image process.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing the configuration of the main part of the image processing apparatus according to this example. The image processing apparatus 1 according to this example includes a control unit 2, an image input unit 3, an image processing processor 4, and an output unit 5. The image processing apparatus 1 according to this example detects and tracks vehicles and pedestrians at intersections. In this example, in order to simplify the description, the types of objects to be detected are two types: vehicles (automobiles (four-wheeled vehicles) and motorcycles (two-wheeled vehicles)) and pedestrians.

  The control unit 2 controls the operation of each part of the main body of the image processing apparatus 1. Further, an integration process (a configuration corresponding to the integration unit in the present invention) and an output signal generation process described later are performed.

  The image input unit 3 receives a frame image of an intersection captured by the two connected video cameras 10 and 11. The frame rate of the video cameras 10 and 11 is several tens of frames / sec (for example, 30 frames / sec).

  The image processor 4 processes a frame image input from one video camera 10 to detect and track a vehicle to be detected and a pedestrian, and the other video camera. 11, a second image processing unit 42 that processes the frame image input from 11 to detect and track a vehicle to be detected and a pedestrian. Details of the image processor 4 will be described later. The image processor 4 corresponds to a computer that executes the image processing method according to the present invention. The image processor 4 corresponds to a computer that executes the image processing program according to the present invention.

  The output unit 5 outputs a signal (an output signal generated by the control unit 2) based on a detection result of a vehicle and a pedestrian to be detected and a processing result related to tracking. For example, the output unit 5 outputs a passing signal (passing pulse) of a vehicle or a pedestrian to a signal control device (not shown) that controls a signal lamp installed at an intersection. Further, the output unit 5 may be configured to count and output the number of vehicles by the branch direction (straight forward, right turn, left turn) at the intersection with respect to a device that monitors traffic at the intersection. That is, a signal output from the output unit 5 (an output signal generated by the control unit 2) may be determined according to a device that inputs the signal and an application.

  As shown in FIG. 2, the two video cameras 10 and 11 are installed in the direction in which an intersection is imaged facing each other. In addition, each video camera 10 and 11 uses a wide-angle lens as an imaging lens so that the entire intersection can be accommodated in the imaging area. In addition, the video cameras 10 and 11 are provided with an image sensor of megapixels (for example, vertical × horizontal is about 1300 × 1000). As is apparent from FIG. 2, the angles of the video cameras 10 and 11 with respect to the intersection are different.

  The two video cameras 10 are vehicles (ordinary cars), pedestrians, etc. that are located behind a large vehicle (such as a bus or a truck) captured by one video camera 10 (or 11). Even when the camera is hidden and is not imaged, the other video camera 11 (or 10) is installed so that the vehicle, a pedestrian, and the like can image. In other words, if there is an environment in which all objects located in the imaging area can be imaged without being hidden by other objects, one camera may be installed at the intersection. Further, three or more cameras may be installed at the intersection. Here, as described above, a case where two video cameras 10 and 11 are installed will be described as an example.

  The image input unit 3 inputs a first image input unit (not shown) that inputs an intersection frame image captured by one video camera 10 and a second image input unit inputs an intersection frame image captured by the other video camera 11. And an image input unit (not shown). The image input unit 3 includes an image memory (not shown) that temporarily stores the input frame image.

  FIG. 3 is a functional block diagram showing the functional configuration of the image processor. The image processor 4 includes a first image processing unit 41 and a second image processing unit 42. The first image processing unit 41 and the second image processing unit 42 have the same functional configuration. The first image processing unit 41 processes the frame image of the intersection imaged by one video camera 10, and the second image processing unit 42 processes the frame image of the intersection imaged by the other video camera 11.

  When there is one video camera 10 (or 11) installed at the intersection, the second image processing unit 42 (or the first image processing unit 41) can be omitted. If there are three or more video cameras installed at the intersection, the image processing processor 4 includes image processing units as many as the number of video cameras.

  The first image processing unit 41 includes a resizing processing function unit 411, a vehicle detection processing function unit 412, a vehicle tracking processing function unit 413, a vehicle determination processing function unit 414, a pedestrian detection processing function unit 415, A pedestrian tracking processing function unit 416 and a pedestrian determination processing function unit 417 are provided.

  The resizing processing function unit 411 generates resizing images with converted resolutions for each processing region set for the frame image input to the first image input unit of the image input unit 3. I do. FIG. 4 is a diagram showing a frame image of an intersection captured by one video camera. As shown in FIG. 4, a plurality of processing areas are set for the frame image captured by the video camera 10. In the example shown in FIG. 4, the two processing areas 100 and 101 are areas where the types of objects to be detected are determined as vehicles (hereinafter referred to as vehicle detection processing areas 100 and 101). The four processing areas 110 to 113 are areas in which the types of objects to be detected are defined as pedestrians (hereinafter referred to as pedestrian detection processing areas 110 to 113).

  As shown in FIG. 4, the vehicle detection processing area 100 is an area covering the inside of the intersection, and the vehicle detection processing area 101 covers the vicinity of the stop line on the opposite side to the installation side of the video camera 10 that captured the frame image. It is an area. In FIG. 4, the vehicle detection processing area 100 and the vehicle detection processing area 101 are an example in which a part thereof overlaps. The four pedestrian detection processing areas 110 to 113 are areas that cover pedestrian crossings provided at intersections. The pedestrian detection processing areas 110 to 113 do not overlap with the other pedestrian detection processing areas 110 to 113. However, the pedestrian detection areas 110 to 113 partially overlap the vehicle detection processing areas 100 and 101.

  The resizing processing function unit 411 has an image with a preset number of pixels for each of the vehicle detection processing areas 100 and 101 and the pedestrian detection processing areas 110 to 113 (in this example, a VGA image of 640 × 480 pixels). Resizing process to convert to. Here, the image generated by the resizing process is referred to as a resizing image. The resolution of the resizing images in the vehicle detection processing areas 100 and 101 is within an appropriate range for vehicle detection. Moreover, the resolution of the resizing image of the pedestrian detection process area | regions 110-113 is in the appropriate range in a pedestrian detection. In other words, the vehicle detection processing areas 100 and 101 are set so that the resolution of the generated resizing image falls within an appropriate range in vehicle detection, and the pedestrian detection processing areas 110 to 113 are generated. The resolution of the resizing image is set so as to be within an appropriate range in detecting a pedestrian. This resizing process may be performed using a method such as a known nearest neighbor interpolation (nearest neighbor), bilinear interpolation (bilinear interpolation), or bicubic interpolation (bicubic interpolation). FIG. 5A is a diagram showing a resizing image related to the vehicle detection processing area 100, and FIG. 5B is a diagram showing a resizing image related to the pedestrian detection processing area 110.

  Note that the resizing image resolution for the vehicle detection processing areas 100 and 101 only falls within the resolution range suitable for vehicle detection, and the resizing image resolutions of these two areas are the same. Not that. Similarly, the resolution of the resizing image relating to the pedestrian detection processing areas 110 to 113 only falls within the resolution range suitable for detection of the pedestrian, and the resizing images of these four areas have the same resolution. It doesn't mean that.

  The vehicle detection processing function unit 412 detects a vehicle that has been picked up by processing resizing images for the vehicle detection processing regions 100 and 101 whose detection targets are vehicles. The vehicle detection processing function unit 412 detects a vehicle being imaged using known pattern matching or the like.

  For each vehicle detected by the vehicle detection processing function unit 412, the vehicle tracking processing function unit 413 corresponds to the vehicle ID, detection time (frame image capturing time), position (coordinates indicating the position on the intersection), and state. Generate the attached object map. In this object map, since the position of the detected vehicle is set to the position on the intersection, it is possible to identify a vehicle located in a portion where the two vehicle detection processing areas 100 and 101 overlap. Therefore, the same vehicle detected in the resizing image related to the vehicle detection processing area 100 and the resizing image related to the vehicle detection processing area 101 can be detected as one vehicle.

  The vehicle determination processing function unit 414 determines the state of each vehicle for which an object map has been generated by the vehicle tracking processing function unit 413 based on the movement trajectory of the vehicle obtained from the object map already generated for the vehicle. The vehicle state to be determined here is before the intersection inflow, in the intersection, and in the intersection outflow.

  The pedestrian detection processing function unit 415 detects pedestrians that are picked up by processing resizing images for the pedestrian detection processing areas 110 to 113 whose detection targets are pedestrians. The pedestrian detection processing function unit 415 detects a pedestrian being imaged using a known pattern matching or the like.

  The pedestrian tracking processing function unit 416, for each pedestrian detected by the pedestrian detection processing function unit 415, the pedestrian ID, detection time (frame image capturing time), and position (coordinates indicating the position on the intersection). An object map in which is associated is generated. Also in this object map, since the position of the detected pedestrian is set to the position on the intersection, it is possible to identify a pedestrian located in a portion where a plurality of pedestrian detection processing areas 110 to 113 overlap.

  In addition, the example shown in FIG. 4 is an example in which the pedestrian detection processing areas 110 to 113 do not overlap.

  For each pedestrian whose object map has been generated by the pedestrian tracking processing function unit 416, the pedestrian determination processing function unit 417 is based on the movement trajectory of the pedestrian obtained from the object map already generated for the pedestrian. Determine the state. The pedestrian status to be judged here is pedestrians outside the pedestrian crossing (pedestrians who have not crossed the pedestrian crossing due to traffic lights etc.), pedestrians in the middle (pedestrians crossing the center of the pedestrian crossing), It is a pedestrian who has not reached the center (a pedestrian who has not reached the center of the pedestrian crossing)

  As shown in FIG. 3, the second image processing unit 42 also has a resizing processing function unit 421, a vehicle detection processing function unit 422, and a vehicle tracking process in the same manner as the first image processing unit 41 described above. A functional unit 423, a vehicle determination processing function unit 424, a pedestrian detection processing function unit 425, a pedestrian tracking processing function unit 426, and a pedestrian determination processing function unit 427 are provided. Resizing processing function unit 421, vehicle detection processing function unit 422, vehicle tracking processing function unit 423, vehicle determination processing function unit 424, pedestrian detection processing function unit 425, pedestrian tracking processing function unit 426, and pedestrian determination processing function The units 427 are respectively the resizing processing function unit 411, the vehicle detection processing function unit 412, the vehicle tracking processing function unit 413, the vehicle determination processing function unit 414, and the pedestrian detection process included in the first image processing unit 41 described above. Since it is the same functional block as the function unit 415, the pedestrian tracking processing function unit 416, and the pedestrian determination processing function unit 417, description thereof is omitted here.

  Note that the settings of the vehicle detection processing areas 100 and 101 and the pedestrian detection processing areas 110 to 113 for the frame image captured by one video camera 10 and the setting of the processing area for the frame image captured by the other video camera 11 Are done individually. Even if the number of processing areas set for the frame image captured by one video camera 10 and the number of processing areas set for the frame image captured by the other video camera 11 are different. There is no particular problem.

  Here, the object map generated by the image processor 4 will be briefly described with reference to FIG.

  FIG. 6A is a diagram showing an object map for the vehicle. FIG. 6B is a diagram illustrating an object map for a pedestrian. This object map is generated for each detected vehicle or pedestrian. FIG. 6A is an object map of a vehicle assigned ID “p0001”. As shown in FIG. 6A, the object map includes, for each resizing image in which the vehicle is detected, the imaging time (the imaging time of the frame image used for generating the resizing image), and the position of the vehicle on the intersection. , And records in which the vehicle states determined by the vehicle determination processing function unit 414 are associated with each other are accumulated in a time series. Therefore, from this object map, the change (movement trajectory) of the position on the intersection with the passage of time is obtained for the corresponding vehicle. For this reason, it can be obtained from which direction the corresponding vehicle enters the intersection, and which direction (right turn, left turn, straight travel) the passing direction is. That is, the branching rate of the vehicle at the intersection can be obtained.

  The object map for the pedestrian shown in FIG. 6B is the same as the object map for the vehicle described above. FIG. 6B is an object map of a pedestrian assigned ID “q0001”. As described above, from this object map, a change in the position on the intersection with the passage of time can be obtained for the corresponding direction person. For this reason, it can be obtained in which direction the corresponding pedestrian crosses the pedestrian crossing.

  Hereinafter, the operation of the image processing apparatus 1 according to this example will be described. FIG. 7 is a flowchart showing the operation of the image processing apparatus.

  As described above, a frame image in which one video camera 10 images an intersection is input to the image input unit 3 (first image input unit), and a frame image in which the other video camera 11 images an intersection is imaged. The data is input to the input unit 3 (second image input unit). The image input unit 3 includes an image memory (not shown) that stores the input frame image.

  The image processing apparatus 1 performs image processing on the frame image stored in the image memory (s1). In s1, the first image processing unit 41 processes a frame image captured by one video camera 10, and the second image processing unit 42 processes a frame image captured by the other video camera 11. The processes in the first image processing unit 41 and the second image processing unit 42 are performed in parallel.

  In addition, the image processing executed by the first image processing unit 41 and the image processing executed by the second image processing unit 42 are substantially the same processing except that the image to be processed is different. Here, the image processing relating to s1 will be described using the first image processing unit 41 as an example. FIG. 8 is a flowchart showing the image processing according to s1.

  The first image processing unit 41 is an unprocessed frame image stored in the image memory, and the frame image at the earliest time in time (the oldest frame image in time) is used as the processing target frame image. Select and capture (s11). The first image processing unit 41 selects a frame image to be processed from frame images captured by one video camera 10.

  In the first image processing unit 41, the resizing processing function unit 411 performs resizing processing on the processing target frame image captured in s11 (s12). The resizing processing function unit 411 has an image with a preset number of pixels (640 × 480 in this example) for each of the vehicle detection processing areas 100 and 101 and the pedestrian detection processing areas 110 to 113 shown in FIG. Resizing processing for conversion into a VGA image of pixels) is performed.

  The first image processing unit 41 performs processing for detecting the vehicle imaged by the vehicle detection processing function unit 412 for each resizing image in the vehicle detection processing regions 100 and 101 (s13). In s13, the imaged vehicle is detected by a known method such as pattern matching. In s13, coordinate data indicating the position on the intersection is acquired for the detected vehicle.

  For the vehicle detected by the vehicle tracking processing function unit 413 in s13, the first image processing unit 41 uses both the resizing image for the vehicle detection processing region 100 and the resizing image for the vehicle detection processing region 101. An identification process for extracting the detected vehicle is performed (s14). s14 is a process of associating one-to-one with a vehicle located in an overlapping area where the vehicle detection processing area 100 and the vehicle detection processing area 101 overlap on the intersection. That is, this is a process for detecting a vehicle located in an overlapping area where the vehicle detection processing area 100 and the vehicle detection processing area 101 overlap on the intersection as one vehicle. This identification process is performed using the position of the vehicle on the intersection acquired in s13.

  The vehicle tracking processing function unit 413 generates an object map for each vehicle detected this time (s15). In s15, for each vehicle detected this time, a record in which the imaging time of the frame image to be processed selected in s11 is associated with the position on the intersection acquired in s13 is generated. The record generated here is not associated with the state of the vehicle shown in FIG. Further, the vehicle tracking processing function unit 413 determines whether or not an ID has already been assigned to the vehicle for each record generated this time. Vehicles to which no ID is assigned at this time are detected in the vehicle detection processing regions 100 and 101 between the time when the frame image that was the previous processing target is captured and the time when the frame image that is the current processing target is captured. The vehicle has entered the vehicle detection processing areas 100 and 101 from the outside, and the object map shown in FIG. 6A is not generated at this time.

  The vehicle tracking processing function unit 413 assigns an ID to a vehicle to which no ID is assigned at this time, and generates an object map (an object map in which only the record generated this time is registered) shown in FIG. To do. On the other hand, for a vehicle that has already been given an ID at this time, an object map is generated by adding the record generated this time to the object map of the corresponding vehicle (the current map is generated for the object map of the corresponding vehicle). Add a record.) Whether the ID is assigned can be determined based on the vehicle position on the intersection acquired in s13 and the temporal change in the vehicle position on the intersection obtained from the object map of the vehicle assigned the ID at this time. .

  Further, the first image processing unit 41 determines the state of the vehicle (before entering the intersection, inside the intersection, and flowing out of the intersection) for each vehicle detected by the vehicle determination processing function unit 414 this time, and the state is determined in s15. It is registered in the record additionally registered in the generated object map of the corresponding vehicle (s16).

  Moreover, the 1st image process part 41 performs the process which detects the pedestrian currently imaged by the pedestrian detection process function part 415 for every resizing image concerning the pedestrian detection process area | regions 110-113 (s17). . In s17, the imaged pedestrian is detected by a known method such as pattern matching. In s17, coordinate data indicating the position on the intersection is acquired for the detected pedestrian.

  The first image processing unit 41 identifies the pedestrian detected by the pedestrian tracking processing function unit 416 detected from the resizing images of the plurality of pedestrian detection processing areas 110 to 113 for the pedestrian detected in s17. Processing is performed (s18). s18 is a process of associating pedestrians in a one-to-one correspondence on overlapping areas where a plurality of pedestrian detection processing areas 110 to 113 overlap on an intersection. That is, it is a process for detecting a pedestrian located in an overlapping area where a plurality of pedestrian detection processing areas 110 to 113 overlap on the intersection as one pedestrian. This identification process is performed using the position of the pedestrian on the intersection acquired in s17.

  In the example illustrated in FIG. 4, the pedestrian detection processing areas 110 to 113 do not overlap with the other pedestrian detection processing areas 110 to 113, and therefore the processing according to s 18 may be unnecessary.

  The pedestrian tracking processing function unit 416 generates an object map for each pedestrian detected this time (s19). In s19, for each pedestrian detected this time, a record in which the imaging time of the processing target frame image selected in s11 is associated with the position on the intersection acquired in s17 is generated. The record generated here is not associated with the pedestrian state shown in FIG. In addition, the pedestrian tracking processing function unit 416 determines whether or not an ID has already been assigned to the pedestrian for each record generated this time. At this time, the pedestrian to whom no ID is assigned is the pedestrian detection processing area 110 between the imaging time of the frame image that was the previous processing target and the imaging time of the frame image that was the current processing target. ~ 113 is a pedestrian who has entered the pedestrian detection processing areas 110 to 113 from outside, and the object map shown in FIG. 6B is not generated at this point.

  The pedestrian tracking processing function unit 416 assigns an ID to a pedestrian not assigned an ID at this time, and the object map shown in FIG. 6B (an object map in which only the record generated this time is registered). Is generated. On the other hand, for a pedestrian whose ID has already been assigned at this time, an object map is generated by adding the record generated this time to the corresponding pedestrian object map. Whether or not the ID is given is a temporal change in the position of the pedestrian on the intersection acquired in s17 and the position of the pedestrian on the intersection obtained from the object map of the pedestrian assigned the ID at this time. It can be done based on.

  Further, the first image processing unit 41, for each pedestrian detected by the pedestrian determination processing function unit 417 this time, the state of the pedestrian (a pedestrian outside the pedestrian crossing, a central passing pedestrian, a central unachieved pedestrian). Is registered in the record additionally registered in the object map of the corresponding pedestrian generated in s19 (s20).

  The process according to s13 to s16 and the process according to s17 to s20 are the difference between whether the target is a vehicle or a pedestrian, and the processes are substantially the same. Moreover, the order which performs the process concerning s13-s16 and the process concerning s17-s20 is reverse order to the above (after execution of the process concerning s17-s20, the process concerning s13-s16 is performed). It may be. Further, the first image processing unit 41 may be configured to perform parallel processing on the processing on s13 to s16 and the processing on s17 to s20.

  Furthermore, since the resizing images of the processing areas 100, 101, 110 to 113 generated in s12 are images having the same number of pixels (here, VGA images), an object (vehicle or The processing program related to the detection of pedestrians) can be made substantially the same. More specifically, this can be dealt with by changing parameters relating to pattern matching or the like due to the difference in the type of object (vehicle or pedestrian) to be detected.

  Returning to FIG. 7, when the image processing apparatus 1 completes the image processing related to s1 described above, the control unit 2 performs integration processing (s2). The integration process related to s2 is a process of integrating the object map generated by the first image processing unit 41 and the object map generated by the second image processing unit 42 in the image processing related to s1. Specifically, an object (vehicle and pedestrian) detected by both the first image processing unit 41 and the second image processing unit is detected as one object, and the first image processing unit 41, Or it is the process which detects the object (a vehicle and a pedestrian) detected only by one side of the 2nd image process part as one object. In other words, the object map generated by the first image processing unit 41 and the object map generated by the second image processing unit 42 are integrated to obtain one object map for each detected object. . As a result, when one of the video cameras 10 (or 11) captures an image of a large vehicle (such as a bus or truck), the video camera 10 (or 11) takes a vehicle (ordinary car) or a pedestrian that was not imaged because it is located behind it. You can also get an object map. Thereby, oversight of an object (a vehicle or a pedestrian) can be prevented.

  In the image processing apparatus 1, the control unit 2 generates an output signal based on the processing result of the integration processing related to s 2, performs output processing for outputting this output signal from the output unit 5 (s 3), and returns to s 1. This output signal is, for example, a signal that is output to a signal control device that controls a signal lamp installed at an intersection, and is a pulse signal that indicates the passage of a vehicle or a pedestrian. Further, it may be a signal output to a device for monitoring traffic at an intersection, and may be a digital signal obtained by measuring the number of vehicles at the intersection according to the branch direction (straight, right turn, left turn).

  In the above description, the image processing device according to the present invention is exemplified by a vehicle at an intersection and a device that detects a pedestrian, but the vehicle may be distinguished by an automobile (four-wheeled vehicle) and a motorcycle (two-wheeled vehicle). . In this case, instead of the vehicle detection processing areas 100 and 101 shown in FIG. 4, the vehicle detection processing area and the motorcycle detection processing area may be set to frame images captured by the video cameras 10 and 11. In addition, a bicycle may be added as the type of object to be detected. In this case, the bicycle detection processing area may be set to a frame image captured by the video cameras 10 and 11. The image processor 4 may include a detection processing function unit, a tracking processing function unit, and a determination processing function unit for each type of object to be detected and tracked.

  The present invention is not limited to the above example, and can also be used as an apparatus for detecting a plurality of types of objects having different sizes. In addition, the imaging areas of the video cameras 10 and 11 are not limited to intersections, but may be other places such as station platforms.

  In the above example, the resizing image generated for each of the vehicle detection processing areas 100 and 101 and the pedestrian detection processing areas 110 to 113 is a VGA image. The number of pixels of the sizing image may be different. However, the resolution of the resizing image generated for each processing region is within the appropriate range for the detection target object set for the processing region.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Control part 3 ... Image input part 4 ... Image processing processor 5 ... Output part 10, 11 ... Video camera 100, 101 ... Vehicle detection processing area 110-113 ... Pedestrian detection processing area 41 ... 1st Image processing unit 42 ... Second image processing unit 411, 421 ... Resizing processing function unit 412, 422 ... Vehicle detection processing function unit 413, 423 ... Vehicle tracking processing function unit 414, 424 ... Vehicle determination processing function unit 415, 425 ... Pedestrian detection processing function unit 416, 426 ... Pedestrian tracking processing function unit 417, 427 ... Pedestrian determination processing function unit

Claims (10)

An image input unit for inputting a frame image of the target area captured by the camera;
On the frame image input to the image input unit, for each of a plurality of processing areas in which the type of the object to be detected is determined, the image of the processing area is an image having a resolution corresponding to the type of the object to be detected. A resizing processing unit that generates a resizing image converted into
An image processing apparatus comprising: an object detection unit that processes a resizing image generated by the resizing processing unit and detects an object of a type that is a detection target.   The image processing apparatus according to claim 1, wherein the resizing processing unit generates a resizing image having a preset number of pixels for each processing region.   The image processing apparatus according to claim 1, further comprising: an object map generation unit configured to generate an object map indicating a change in position on the target area with time for each object detected by the object detection unit. . The image input unit receives frame images captured by a plurality of cameras having different angles with respect to the target area,
The image processing apparatus according to claim 1, wherein the resizing processing unit and the object detection unit are provided for each camera. The image input unit receives frame images captured by a plurality of cameras having different angles with respect to the target area,
The resizing processing unit, the object detection unit, and the object map generation unit are provided for each camera,
The image processing apparatus according to claim 1, further comprising an integration unit that integrates an object map generated by the object map generation unit provided for each camera in units of detected objects.   The image processing apparatus according to claim 1, wherein the target area is an intersection.   The image processing apparatus according to claim 6, wherein the processing area set on the frame image input to the image input unit is a vehicle or a person as a type of object to be detected.   The image processing apparatus according to claim 6, further comprising an output unit configured to output an output signal indicating the passage of a vehicle or a person at an intersection that is a target area based on a detection result of the object detection unit. An image processing method in which a computer processes a frame image of a target area captured by a camera, input to an image input unit,
On the frame image input to the image input unit, for each of a plurality of processing areas in which the type of the object to be detected is determined, the image of the processing area is an image having a resolution corresponding to the type of the object to be detected. A resizing process step for generating a resizing image converted into
An image processing method in which a computer executes an object detection step of processing a resizing image generated in the resizing processing step and detecting an object of a type to be detected. An image processing program for causing a computer to process a frame image of a target area captured by a camera, input to an image input unit,
On the frame image input to the image input unit, for each of a plurality of processing areas in which the type of the object to be detected is determined, the image of the processing area is an image having a resolution corresponding to the type of the object to be detected. A resizing process step for generating a resizing image converted into
An image processing program that causes a computer to execute an object detection step of processing a resizing image generated in the resizing processing step and detecting an object of a type that is a detection target.

Images