JP2009301495A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and an image processing method capable of effectively removing reflection components projected integrally with a detection object in a picked up image and correctly detecting the object from the picked up image. <P>SOLUTION: The control part of the image processor determines a difference by a background difference or the like on the basis of image data acquired from an imaging apparatus (S2), and extracts it as the candidate of a detection object region where the detection object is projected. On the basis of the position in the image data of the extracted candidate, a distance to the detection object is calculated (S3). The control part decides whether or not the reflection component is included by deciding whether or not the size in the image data of the extracted candidate is proper as the size of a region to be occupied by the detection object region on the basis of the calculated distance (S4). When the reflection component is included, the control part specifies a boundary to removes the reflection component (S5). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for detecting an object appearing in a captured image captured by an imaging apparatus. In particular, the present invention relates to an image processing apparatus and an image processing method that can effectively remove only a reflection component reflected in a captured image integrally with a target and correctly detect only the target from the captured image.

  Conventionally, various traffic control systems for preventing traffic accidents at intersections such as T-shaped roads and crossroads with poor visibility have been proposed. In particular, a system has been proposed in which an imaging device that captures road conditions in the visible light wavelength band from above is installed on the roadside in the vicinity of the intersection, and the driver can be alerted and alerted by transmitting the captured image to the automobile. (For example, refer to Patent Document 1).

Further, preprocessing for detecting a car from a captured image may be performed (see, for example, Patent Document 2). In this case, by using techniques such as “background difference”, “edge extraction”, “time difference (frame difference)”, and “optical flow” as preprocessing, an area corresponding to a car in the captured image is determined. Yes.
JP 2003-109199 A JP 2008-46761 A

  In order to reduce various traffic accidents such as a collision between a car and a bicycle, a collision between a car and a person as well as a collision between cars, it is expected to realize a more advanced safe driving support system. Specifically, information on the presence and location of other automobiles, bicycles, people, or fallen objects at the intersection is transmitted to the automobile about to enter the intersection. And the control for avoiding an accident is automatically performed based on the information received by the vehicle side. As a result, accidents can be prevented and damage is expected to be minimized. In this case, it is necessary to identify and detect not only the presence / absence of the target object but also the type of detection target such as another automobile, bicycle, person, or fallen object from the captured image.

  In order to identify and detect the type of detection target from the captured image, it is effective to determine whether or not a different feature amount can be detected for each type of detection target from the detection target region in the captured image. For example, the shape and size of the detection target area in the captured image is set as a feature amount for each type, and the detection target is determined by whether or not the shape and size of the detection target area match the set shape and size. Can be identified.

  However, when the detection target is reflected in the captured image, the reflected image of the detection target may be reflected in the captured image. For example, when a detection target such as a vehicle or a person is imaged when the road surface is wet, such as in rainy weather, the reflection of the detection target generated on the road surface is also captured. When trying to extract a detection target region by image processing such as background difference or time difference, there is a high possibility that a reflection component is detected as a part of the detection target. This is because when the boundary between the detection target region and the reflection component is not clearly separated, it is extracted as a series of regions. In this case, since the size of the detection target area is detected to be larger by the size of the reflection component, there is a possibility that the type of detection target is erroneously detected.

  On the other hand, it is determined whether there is reflection on the road surface based on the average luminance value in the captured image, and when it can be determined that there is reflection, the following processing is performed to determine the boundary between the detection target and the reflection component A way to identify is being done. In the preceding example, first, the range extracted as the detection target region is edge-processed, and the luminance value is added in each horizontal line. As the edge is detected by the edge processing, the luminance value added in each horizontal line increases. Then, the distribution of the total luminance value of each horizontal line with respect to the vertical direction of the image corresponds to the minimum value of the luminance value between the peaks on the assumption that both the detection target itself and its reflection component have peaks. The line is specified as a boundary.

However, with the above-described method, it is difficult to determine the presence or absence of reflection and to specify the boundary between the detection target region and the reflection component region with high accuracy.
First, in the preceding example, the average luminance value of the captured image is compared with a reference value, and the presence or absence of reflection on the road surface is determined. However, it is difficult to accurately determine the presence or absence of reflection by this method. The captured image in rainy weather is generally darker than in clear weather, but even if the captured image is generally bright, reflection may occur due to locally remaining puddles. In this case, since it is determined that there is no reflection and the boundary is not specified, the detection target region cannot be specified correctly.
In general, the degree of wetness of the road surface is different in each part of the road, and the strength of reflection is also different. On the road surface portion that is wet but not puddle, there may be a case where there is no clear peak of the edge luminance value due to the reflection component. In the method of the previous example, it is assumed that both the detection target itself and the reflection component have peaks. Therefore, when there is no peak even if there is a reflection component, the boundary cannot be specified.

  The present invention has been made in view of such circumstances, and an image processing apparatus and an image processing method capable of accurately detecting a target based on a captured image even when the captured image includes a reflection component of the detection target. The purpose is to provide.

  An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that includes an imaging unit, and performs image processing for detecting an object within an imaging range based on a captured image obtained from the imaging unit. Extraction means for extracting a candidate for a detection target area in which the detection target is shown, a position in the captured image of the candidate extracted by the extraction means, the size of the candidate in the vertical direction and the horizontal direction, and pre-stored Based on dimensional information indicating the size of a certain detection target, a determination unit that determines whether or not the candidate includes a reflection component of the detection target; and the determination unit that determines that the determination unit includes a reflection component, the candidate A specifying unit for specifying a boundary between the region of the detection target itself and the region of the reflection component, a unit for removing the reflection component region from the candidate based on the boundary specified by the specifying unit, and the reflection component Based on the detection target region obtained by removing, characterized in that it comprises a means for detecting a detection target.

  In the image processing device according to the second invention, the specifying means specifies an edge extraction processing means for extracting an edge portion in the captured image, and a vertical coordinate in the captured image of the upper end of the candidate extracted by the extraction means. Based on the dimension information, means for determining the vertical size that the detection target area should occupy in the captured image, and the coordinates in the vertical direction determined from the coordinates specified by the upper end specifying means. Lower end specifying means for specifying coordinates in a captured image that is lower by the amount, and each edge included in a horizontal band region passing through the coordinates for each of a plurality of lower coordinates from the coordinates specified by the lower end specifying means Means for obtaining an index value indicating the edge strength of the band region based on the edge strength of the part, and means for obtaining a distribution curve for each coordinate in the vertical direction of the index value obtained by the means And means for identifying a characteristic point in the cloth curve, a band region in the captured image corresponding to the identified feature points, characterized in that are to be identified as a boundary.

  In the image processing apparatus according to a third aspect, the lower end specifying means further comprises means for calculating a distance in real space from the imaging means to the detection target based on the coordinates specified by the upper end specifying means. Based on the calculated distance and the dimensional information, the coordinates of the lower end of the area that should be occupied by the detection target area in the captured image when the detection target exists at the distance from the imaging means in real space are specified. It is characterized by.

  The image processing apparatus according to a fourth aspect of the invention is characterized in that the determination unit calculates a distance in real space from the imaging unit to the detection target based on a position in the candidate captured image extracted by the extraction unit, Based on the distance calculated by the means and the vertical size of the candidate extracted by the extracting means, the size information indicates the means for calculating the size of the detection target in real space, and the size calculated by the means And a means for determining whether or not the size corresponds to the size in the real space. If it is determined that the size does not correspond, the candidate is determined to include a reflection component.

  An image processing apparatus according to a fifth aspect of the invention detects an imaging range of the imaging unit from a second imaging unit that images simultaneously from a different angle from the imaging unit, and a captured image captured by the second imaging unit. From a means for extracting a candidate for a detection target area in which the object is shown, a candidate for a detection target area extracted from a captured image obtained by the extraction means from the imaging means, and a captured image captured by the second imaging means Based on the difference between the means for associating the extracted candidates for the detection target region, the means for identifying the positions of the two candidates associated with the means in the captured image, and the positions where the two candidates are identified, Means for calculating a distance in real space from the imaging means or the second imaging means to the detection target; a distance calculated by the means; and a size of a candidate in the vertical direction extracted by the extraction means And a means for calculating the size in the real space to be detected, and a means for determining whether the size calculated by the means corresponds to the size in the real space indicated by the dimension information. When it is determined that the reflection component is not included, the determination unit determines that the reflection component is included.

  The image processing apparatus according to a sixth aspect of the present invention further includes means for determining whether or not the detection target is a specific detection target.

  The image processing apparatus according to a seventh aspect is characterized in that the specific detection target is a person.

  An image processing method according to an eighth aspect of the present invention is a method of performing image processing for detecting an object within an imaging range based on a captured image, and extracting a detection target area candidate in which the detection target is reflected from the captured image. Based on the extraction step, the position in the captured image of the candidate extracted in the extraction step, the size of the candidate in the vertical and horizontal directions, and the dimension information indicating the size of the detection target stored in advance, A determination step for determining whether or not the candidate includes a reflection component of a detection target, and when it is determined that the candidate includes a reflection component in the determination step, between the region of the detection target itself in the candidate and the region of the reflection component A step of identifying a boundary of the target, a step of removing a reflection component from the candidate based on the boundary identified by the identification step, and a detection target region obtained by removing the reflection component Based on, characterized in that it comprises the steps of detecting a detection target.

  In the image processing method according to a ninth aspect of the invention, the specifying step specifies an edge extraction processing step for extracting an edge portion in the captured image, and specifies vertical coordinates in the captured image of the upper end of the candidate extracted in the extraction step. A step of obtaining a vertical size that the detection target region should occupy in the captured image based on the dimension information, and a detection target itself that is below the specified coordinate by the determined vertical size Identifying the coordinates in the captured image corresponding to the lower end of the image, and for each of a plurality of coordinates further below the identified coordinates, based on the edge strength of each edge portion included in the horizontal band region passing through the coordinates A step of obtaining an index value indicating the edge strength of the band area, and a step of obtaining a distribution curve for each coordinate in the vertical direction of the index value obtained by the step. And-up, identifying a characteristic point in the distribution curve obtained, a band region in the captured image corresponding to the feature points identified, characterized in that it comprises the steps of: identifying as a boundary.

  In the first invention and the eighth invention, a candidate for a region corresponding to the detection target is extracted from the captured image. Then, based on the position of the extracted area candidate in the captured image and the vertical and horizontal sizes of the extracted area candidate, whether or not each candidate area includes a reflection component is determined by the determination unit. The determination by the determination means is based on the following grounds. The size of the detection target has a predetermined range corresponding to the type. Therefore, when the detection target is actually captured at a position in the captured image of the region extracted as the detection target region candidate, the vertical and horizontal sizes that the detection target region should occupy in the image can be inevitably determined. In the present invention, this is utilized, and based on the position in the captured image of the detection target region, the dimension information indicating the size that should be in accordance with the position, and the vertical and horizontal sizes of the actually extracted region, the reflection component Whether or not is included is determined. When it is determined that the reflection component is included, the boundary between the detection target region and the reflection component region is specified, and the reflection component region delimited by the boundary is removed.

In the second and ninth inventions, the boundary between the detection target area and the reflection component area in the first and seventh inventions is specified as follows. Edge processing is performed on the captured image to extract an edge portion.
And the coordinate of the upper end of the candidate of the detection object area | region in a captured image is specified. When there is reflection, in the image range that shows the vicinity of the grounding point to be detected, there is a high possibility that the reflection component area and the detection target area are in a series, and the lower end is the lower end of the reflection component area and is detected It may not be the lower end of the area of the subject itself. Therefore, the coordinates of the upper end are specified.
When the head of the detection target is located at the position of the specified upper end coordinate, the coordinate that should correspond to the lower end of the detection target area is specified from the size that the detection target area should occupy. The Since it is presumed that there is a boundary between the area of the detection target itself and the reflection component in the vicinity of the coordinate corresponding to the lower end, the coordinate corresponding to the lower end is directed downward in the vertical direction with respect to each of a plurality of different coordinates. Then, an index value obtained by integrating the edge strengths of the edge portions included in the band region passing through each coordinate is obtained. Then, based on the index value, a band region where the edge strength is relatively weak is specified. The band region in which the edge strength is relatively weak is specified by obtaining a distribution curve of the index value of the edge strength in the horizontal direction and specifying the characteristic points of the distribution curve.
By starting from the coordinates that should correspond to the lower end, i.e., the portion where the boundary is likely to exist, the processing is lightened, and it is avoided that the boundary is erroneously specified inside the detection target region, and the reflection component is accurately detected. It is possible to identify and remove the boundary of the.

  In the third invention, the lower end of the detection target area is specified as follows. First, when the edge of the head of the detection target is captured at the upper end of the region candidate extracted by the extraction means, the distance from the imaging means is calculated according to the position of the edge in the captured image and the type of the detection target. It is possible. The size that the detection target area should occupy is determined according to the calculated distance, and the lower end coordinates can be accurately identified based on the upper end coordinates and the determined size. Accordingly, it is possible to avoid erroneously specifying the boundary, to accurately specify the boundary, and to effectively remove the reflection component.

  In the fourth aspect of the invention, the determination as to whether or not the reflection target region is included in the detection target region candidates extracted by the extraction means is performed as follows. Based on the size of the extracted candidate area in the captured image, the position of the extracted candidate image in the captured image, and the distance in real space from the imaging means to the detection target obtained from the position, The size in space is calculated. The calculated size in the real space is compared with the size indicated by the dimension information determined by the type of the detection target, and the size in the real space obtained from the region extracted as a detection target region candidate from the captured image is When the size is not possible for the detection target, it is highly likely that the size is increased by the amount of the reflection component, and thus it is determined that the reflection component is included. Thereby, it can be accurately determined whether or not a reflection component is included.

  In the fifth invention, there is further provided second imaging means for simultaneously imaging the same range at a different angle by a predetermined angle or more, and the distance from the imaging means to the detection target based on the captured images captured at different angles Is calculated. Then, based on the position and distance of each extracted region candidate in the captured image, the size of the detection target in real space is calculated. When the size in the real space obtained from the region extracted as the detection target region candidate from the captured image is a size that is impossible for the detection target, it is likely that the size is increased by the amount of the reflection component. Therefore, it is determined that the reflection component is included. Thereby, it can be accurately determined whether or not a reflection component is included.

  In the sixth invention, it is determined whether or not the detection target is a specific detection target. Thereby, it becomes possible to identify and detect a detection target.

  In the seventh invention, when the detection target is a person, it is statistically utilized that the characteristic points of the distribution curve of the edge strength with respect to the vertical direction are easily clarified. As a result, it is possible to detect a person with high accuracy even when the reflected component of the person is reflected in the captured image.

  According to the present invention, since the reflection component from the background of the detection target can be removed, it is possible to accurately detect the detection target region. This makes it possible to correctly identify the detection target with high accuracy.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

  In the embodiment shown below, it is possible to accurately detect a person at an intersection by the image processing apparatus according to the present invention, and to perform a traveling control for preventing an accident on a car entering the intersection. A case where the present invention is applied to a driving support system will be described as an example.

(Embodiment 1)
FIG. 1 is a perspective view from above showing an example of an intersection where the safe driving support system according to Embodiment 1 is installed. Reference numeral 1 in FIG. 1 denotes an image processing apparatus configured integrally with an imaging apparatus that images an intersection. The image processing apparatus 1 is attached to a column installed on the road side and extending to the road side at a predetermined height. The lens of the image pickup apparatus configured integrally with the image processing apparatus 1 is installed so as to overlook the intersection and the range in the vicinity thereof, and the image processing apparatus 1 and the image pickup apparatus have a shooting wavelength band on the lens side of the image pickup apparatus. It is housed in a casing made of transparent material. Reference numeral 2 in FIG. 1 denotes a roadside machine, which is attached to a roadside column like the image processing apparatus 1. The image processing apparatus 1 and the roadside machine 2 are connected by a communication line.

  The image processing apparatus 1 can transmit and receive data via wireless communication via the roadside device 2. For example, two cars are shown in FIG. The vehicle-mounted device of each automobile is configured to be able to perform wireless communication with the roadside device 2 by, for example, DSRC (Dedicated Short Range Communication) technology.

  In the safe driving support system according to the first embodiment, the image processing apparatus 1 performs the operation described below, thereby realizing traveling control in each automobile. The image processing apparatus 1 detects an automobile, a bicycle, a pedestrian, a fallen object, or the like at an intersection from image data obtained by an imaging apparatus that captures an image of the intersection. Then, the image processing apparatus 1 transmits information to be detected to each automobile via the roadside device 2. Thereby, in each automobile, it is possible to realize traveling control such as automatically operating a brake when the presence of a pedestrian is detected in the traveling direction based on information received via the roadside device 2. In addition, even when a collision accident occurs, such as operating a shock absorber, it is possible to realize safety control for reducing the impact caused by the collision.

  In order to realize such a safe driving support system, image processing performed by the image processing apparatus 1 based on image data and various information obtained by the image processing will be described below.

  FIG. 2 is a block diagram illustrating an internal configuration of the image processing device 1, the roadside device 2, and the imaging device 3 that constitute the safe driving support system in the first embodiment. The image processing apparatus 1 includes a control unit 10 that controls each component using a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc., an EEPROM (Electrically Erasable and Programmable Read Only Memory), a flash memory, and the like. A storage unit 11 using a non-volatile memory, an image memory 12 using a memory such as SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory), and an image acquisition unit that receives an image signal from the imaging device 3 13 and a communication unit 14 that realizes communication with the roadside machine 2.

  The control unit 10 reads out and executes a control program stored in a built-in ROM (Read Only Memory) or the storage unit 11 to control each component unit and perform image processing described later. is there. The storage unit 11 may be built in the control unit 10. Various data generated by the processing of the control unit 10 is temporarily stored in a non-volatile memory provided separately or in a memory built in the control unit 10.

  The image memory 12 stores the image data acquired by the image acquisition unit 13. The control unit 10 instructs writing to the image memory 12 when the image acquisition unit 13 acquires image data.

  The image memory 12 may be a partial storage area of a RAM (Random Access Memory) built in the control unit 10. The storage unit 11 or the image memory 12 may be installed outside the image processing apparatus 1 and connected to the image processing apparatus 1 so that the control unit 10 of the image processing apparatus 1 can read and write. Each process described below is performed on the image data stored in the image memory 12 or a copy thereof.

  The image acquisition unit 13 receives an image signal output from the imaging device. The image acquisition unit 13 extracts image data from the received image signal and stores it in the image memory 12. Specifically, according to an instruction from the control unit 10, the image acquisition unit 13 specifies a luminance value or a color difference for each pixel constituting the image from the received image signal, and writes it to the image memory 12 as image data.

  The communication unit 14 realizes transmission / reception of information via a communication line connecting the roadside device 2. The control unit 10 transmits information obtained by the image processing to the roadside device 2 through the communication unit 14 and receives information from the roadside device 2 through the communication unit 14.

  The roadside machine 2 includes a control unit 20 that controls each component using a CPU or MPU, a communication unit 21 that realizes communication with the image processing apparatus 1, and communication with an on-vehicle device mounted on each vehicle. And a wireless communication unit 22 that realizes the above wirelessly.

  The communication unit 21 can transmit and receive information in correspondence with the communication unit 14 of the image processing apparatus 1. When the communication unit 21 receives information transmitted from the image processing apparatus 1, the control unit 20 receives a notification from the communication unit 21 and receives it.

  The wireless communication unit 22 can perform wireless communication using a DSRC technique. The control unit 20 transmits information received from the image processing apparatus 1 by the communication unit 22 to the vehicle-mounted device of each automobile by the wireless communication unit 22. The wireless communication unit 22 may be configured not only to communicate with the vehicle-mounted device by the DSRC method but also to transmit information to a communication device in a traffic control center at a remote location.

  The imaging device 3 is a far-infrared camera that includes an infrared lens and a far-infrared imaging device and performs imaging in the far-infrared wavelength band. The infrared lens is manufactured using zinc sulfide, chalcogen glass, germanium, zinc selenium, or the like as a raw material. As the far-infrared imaging device, an uncooled type using vanadium oxide (VOx), amorphous silicon, SOI (Silicon on Insulator) diode, thermopile or the like is used. The wavelength band that can be detected by the far-infrared imaging device is, for example, 8 μm to 12 μm. A filter may be provided to block light of other wavelengths.

  The far-infrared imaging device of the imaging device 3 converts the temperature difference into a luminance difference and outputs it. The number of pixels of the far-infrared imaging element of the imaging device 3 is, for example, 320 × 240, and an image signal of an image generated at a rate of about 30 frames per second is output.

  Next, details of the image processing performed on the image data acquired from the image signal output from the imaging device 3 by the control unit 10 of the image processing device 1 configured as described above will be described.

  The control unit 10 detects a person from the image data acquired by the image acquisition unit 13. Specifically, the control unit 10 extracts a person region from the image data. There are various techniques for detecting a person based on image data. In the first embodiment, the background difference method is used. However, a frame difference method that obtains a difference from one or more previous frames of image data may be used, or a method called an optical flow may be used. Good.

  FIG. 3 is an explanatory diagram illustrating an example of the content of background image data stored in advance in the storage unit 11 of the image processing apparatus 1 according to the first embodiment. In addition, background image data for far infrared rays as shown in FIG.

  As shown in FIG. 3, the road image which is a fixed object is reflected in the background image data. By calculating the difference between the image data acquired from the imaging device 3 and the background image data, a person region candidate is extracted.

  The imaging range is determined according to the installation position and installation height of the image processing apparatus 1 and the depression angle and field angle of the imaging apparatus 3. In the case of Embodiment 1, the imaging device 3 is installed so as to look down at the intersection. Therefore, there is a correspondence relationship between the position in the image data, particularly the position in the vertical direction and the position at the intersection of the real space. Each horizontal line represented by a broken line in FIG. 3 indicates the distance from the image processing apparatus 1 in the real space corresponding to the position in the vertical direction in the image data. For example, a part of the road surface shown on the broken line at the bottom in FIG. 3 is located at a distance of 20 m from the position where the image processing apparatus 1 is installed. A part of the road surface shown on the broken line at the center is located at a distance of 50 m from the position where the image processing apparatus 1 is installed. A part of the road surface shown on the uppermost broken line is located at a distance of 100 m from the position where the image processing apparatus 1 is installed.

  The size of the person is determined to some extent, and the height x width x height should be approximately within a volume of 80 cm x 80 cm x 2 m (height). When a person is imaged by the imaging device 3, the size of the person area in the image data is smaller as it is farther from the image processing device 1 and larger as it is closer. The size of the person area in the image data is determined according to the distance in real space from the imaging device 3 to the person and the angle of view of the imaging device.

  Therefore, the vertical and horizontal sizes to be occupied in the image data of the person area located at a predetermined distance from the image processing apparatus 1 are stored in the storage unit 11. As described above, the person falls within a volume of 80 cm × 80 cm × 2 m (height). When a person located 50 m from the image processing apparatus 1 is imaged by the imaging apparatus 3, the size of the area that the person's area should occupy is 15 pixels in the horizontal direction (number of pixels) in consideration of errors. The vertical size is within 30 pixels.

  When the position in the image data of the area extracted as the person area is a position different from a predetermined position (for example, a position at a distance of 50 m) corresponding to the size information stored in the storage unit 11, the control is performed. The unit 10 calculates the size in the horizontal direction and the vertical direction that the person area should occupy according to the difference. For example, when the person area is located at a position corresponding to a distance of 100 m from the image processing apparatus 1, the distance to be occupied is approximately half because it is twice the distance from the predetermined distance of 50 m.

  In this way, the vertical and horizontal sizes that should be occupied by the person area in the image data are stored in the storage unit 11 in advance. Then, the control unit 10 determines whether or not the extracted area is appropriate as the person area based on whether or not the vertical and horizontal sizes of the area extracted as the person area meet the size condition of the area that the person area should occupy. Determine whether. For example, when the region extracted as the human region is at a position in the image data corresponding to a distance of 50 m from the image processing apparatus 1, the control unit 10 has a horizontal size (number of pixels) of 45 pixels. If the size in the vertical direction is 10 pixels, it does not match the size of the area that the person's area should occupy, so it is determined that it is not appropriate.

  Then, the control unit 10 determines that the extracted area is not valid as the person area, that the area extracted as the person area is larger in the vertical direction including the reflection component.

  The control unit 10 of the image processing apparatus 1 performs the following process using the information stored in the storage unit 11 as described above. FIG. 4 is a flowchart showing an example of an image processing procedure performed by the control unit 10 of the image processing apparatus 1 according to the first embodiment.

  The control unit 10 acquires the image data based on the image signal output from the imaging device 3 by outputting an instruction signal to the image acquisition unit 13 (step S1). In addition, the acquisition timing of the image data in step S1 is arbitrary. For example, it may be configured to be performed at regular intervals such as 1 second, or may be configured to be performed when the vehicle detects entry into an intersection, such as when the roadside device 2 receives information from the vehicle-mounted device.

  The control unit 10 obtains difference data between the image data acquired by the image acquisition unit 13 in step S <b> 1 from the image signal output from the imaging device 3 and the far-infrared background image data stored in the storage unit 11. Calculate (step S2). The calculated difference data is temporarily stored in the image memory 12 or the control unit 10 built-in memory. Thereby, from the image data obtained from the imaging device 3 that captures an image in the far-infrared wavelength band, a detection target that is not reflected in the background image data, that is, a candidate for a region where a person exists is extracted.

  The control unit 10 calculates the difference data as follows, for example. For each pixel, the control unit 10 calculates a difference in luminance value between the background image data and the acquired image data, and distinguishes between a pixel whose difference is greater than or equal to a predetermined value and a pixel whose difference is less than the predetermined value. The control unit 10 calculates, as difference data, a two-dimensional array of 1-bit information indicating whether the difference is greater than or equal to a predetermined value for each pixel constituting the image data. It should be noted that the 1-bit information of the pixel may be set to false unless the true pixel is true, that is, if the true pixel is not continuous. This is to remove pixels whose difference is equal to or greater than a predetermined value due to noise.

  More specifically, the control unit 10 performs an operation based on Expression 1 shown below for each pixel of the acquired image data.

However, f (x, y) in Expression 1 is 1-bit information of each pixel specified by the position x in the horizontal direction and the position y in the vertical direction constituting the acquired image data. The position x in the horizontal direction is the number of pixels from the left end of the image data, and y in the vertical direction is the number of pixels from the upper end of the image data. I _{x, y} is a luminance value for each pixel specified by _{x, y} in the acquired image data. I ′ _{x, y} is a luminance value for each pixel specified by _{x and y} in the background image data. Threshold is an arbitrary threshold value.

  In addition, when calculating the difference data as shown in Expression 1, the difference data may be calculated based on the luminance value after performing the edge extraction process on each of the acquired image data and background image data.

  Then, the control unit 10 calculates the distance from the image processing apparatus 1 to the person to be detected based on the difference position of the far-infrared image data calculated in step S2, that is, the position of the person area candidate (step S3). Specifically, the control unit 10 specifies a rectangle circumscribing the continuous area of the difference data calculated in step S2, and sets the specified rectangle as the extracted person area candidate. And the control part 10 specifies the coordinate of the perpendicular direction in the image data of the lower end of a person area candidate. Since the lower end of the person area is likely to correspond to a contact point with the road surface of the person, the distance in the real space from the image processing apparatus 1 to the person can be calculated by specifying the coordinates corresponding to the contact point. Because it can. When there are a plurality of continuous regions of pixels whose difference values in the difference data are equal to or larger than a predetermined value, each continuous region is set as a human region candidate, and a distance is calculated for each human region candidate. The correspondence between the coordinates of the specified lower end and the distance in the real space is obtained based on what is stored in advance as shown in FIG. Thereby, the arithmetic processing becomes easy.

  Next, based on the distance from the image processing apparatus 1 to the person in the real space calculated in step S3 and the size of the area occupied by the person area candidate in the image data, the control unit 10 detects the detection target area candidate, that is, the person area. It is determined whether or not the size of the area occupied by the candidate in the image data is appropriate (step S4). Thereby, it is determined whether the person area candidate extracted by the calculation of the difference data in step S2 includes a reflection component.

  Specifically, the control unit 10 determines whether the number of pixels (pixels) in the horizontal direction and the vertical direction of the person area candidate corresponds to the size that the person area should occupy. For example, when the distance calculated in step S3 is 50 m, the size that the person area should occupy in the image data is 15 pixels in the horizontal direction and 30 pixels in the vertical direction. If the error is plus or minus 2 pixels, when the horizontal size (number of pixels) of the human region candidate is 13 pixels and the vertical size is 25 pixels, the control unit 10 determines the size of the human region. Is determined to be appropriate. When the size of the circumscribed rectangle in the horizontal direction is 14 pixels and the size in the vertical direction is 36 pixels, the control unit 10 determines that the size of the person area is not appropriate.

  In step S4, the control unit 10 determines whether the size of the person area candidate is appropriate as the size of the area that the person area should occupy as described above. Therefore, even when the area extracted by calculating the difference data is a passenger car other than a person, a vehicle such as a bicycle, or a fallen object area, it is determined that it is not appropriate in step S4. In the first embodiment, a person is a specific detection target. Accordingly, in the determination in step S4, the control unit 10 determines whether or not there is a high possibility of being a person area based on the horizontal size or aspect ratio of the person area candidate extracted by calculating the background difference. Then, the control unit 10 performs processing when the size of the person area is not appropriate only when the possibility of the person area is high. For example, when the distance calculated in step S3 is 50 m and the horizontal size of the extracted person area candidate is 20 pixels or more, the control unit 10 determines that the possibility of being a person area is low. . The control unit 10 determines that the possibility of being a person region is low when the aspect ratio is 1: 1 even when the horizontal size is 15 pixels or less.

  If the control unit 10 determines that it is appropriate in step S4 (S4: YES), the communication unit 14 transmits information about the person to be detected, including the distance obtained in step S3 (step S6). End the process.

  Note that if the control unit 10 determines in step S4 that the person area can be determined to be invalid, the following process may be performed in step S6. The control unit 10 may identify the type of detection target, for example, a passenger car, a bicycle, or the like from the size or aspect ratio of the extracted region, and transmit the type of information included in the detection target information.

  When it is determined that the control unit 10 is not valid in step S4 (S4: NO), the extracted person region candidate is likely to contain a reflection component, and thus the process of removing the reflection component is performed (step S5). . And the control part 10 transmits the information of the detected person including the distance recalculated after reflection component removal by the communication part 14 (S6), and complete | finishes a process. Details of the reflection component removal process and the process of recalculating the distance in step S5 will be described later.

  When there are a plurality of extracted person area candidates in the image data, the control unit 10 performs the processing from step S3 to step S6 on each candidate.

  Next, details of the reflection component removal processing in step S5 will be described. FIG. 5 is a flowchart illustrating an example of the procedure of the reflection component removal process performed by the control unit 10 of the image processing apparatus 1 according to the first embodiment.

  The control unit 10 cuts out the same range as the human region candidate from the image data, and performs edge extraction processing in the range (step S501). The controller 10 performs edge extraction processing using, for example, a Prewitt filter, but of course, other methods may be used. By the edge extraction process, the luminance of the pixel in the edge portion (the portion corresponding to the contour) is increased, and the luminance of the pixel that is not the edge portion is decreased.

  The control unit 10 calculates again the distance from the image processing apparatus 1 to the person to be detected based on the upper end position of the person area candidate in the image data (step S502). In the flowchart of FIG. 4 described above, the control unit 10 calculates the distance based on the lower end position of the person area candidate extracted from the image data. However, in this processing procedure, the extracted person area includes a reflection component, and the lower end of the extracted person area is not likely to correspond to the contact point between the person and the road surface. Based on this, the distance is calculated.

  However, when the image data is acquired by capturing images of a person standing at different positions at the intersection, the upper end of each person area in the image data is located at the same vertical coordinate due to the difference in the actual height of the person There is. Therefore, the depression angle of the image pickup device 3 is appropriately adjusted and installed. For example, it is desirable that the upper end of a person who is actually 50 meters or more away in the direction corresponding to the vertical direction of the image data does not match in the image data. Specifically, the depression angle is adjusted to be in a range from 15 degrees to 25 degrees.

  Next, based on the distance calculated in step S502, the control unit 10 specifies the vertical coordinate for starting the search for the boundary in the image data (step S503). When starting the search downward from the identified coordinates, the control unit 10 resets a counter for counting the number of coordinates that satisfy a condition described later to zero.

  Here, details of the vertical coordinate specifying process in step S503 will be described with reference to FIG.

  FIG. 6 is an explanatory diagram showing a process in which the control unit 10 specifies the boundary search start coordinates in the person area candidate. The rectangle shown in FIG. 6 schematically illustrates an example in which the control unit 10 cuts out the same range as the human region candidate from the image data and performs edge extraction processing on the cut-out range. The darker the hatching density is, the lower the luminance is, and the white part indicates the high luminance part. As a result, it can be seen that the outline of the person is indicated by a portion with high luminance. Further, it can be seen that the lower part of the cutout range includes a portion with high luminance that is considered to be a reflection component of a person. When viewed with human eyes, it is possible to easily determine that the lower part is a reflection component, but it is difficult for the control unit 10 to determine the reflection component.

  The control unit 10 recalculates the distance from the image processing apparatus 1 to the person in the real space in the above-described step S501 based on the position in the image data at the upper end of the cutout range. Based on the recalculated distance, the control unit 10 obtains the size that the person area should occupy in the imaging data when there is a person at the distance. Specifically, when the distance is 50 m, the size that the person area should occupy is 15 pixels in the horizontal direction and 30 pixels in the vertical direction. When the recalculated distance is 100 m, it can be calculated that, for example, it should be 7.5 pixels in the horizontal direction and 15 pixels in the vertical direction.

  The control unit 10 recognizes the portion below the upper limit of the cutout range by the vertical size that the person region should occupy as a portion that is highly likely to have a boundary between the person's own region and the reflection component region. . Here, in the case where a part of the upper end of the human region is missing due to image noise, or in consideration of a difference in human height, the control unit 10 coordinates Y (image data of the image data) offset by a predetermined amount vertically above the portion. The number of pixels from the lower end) is specified and used as the coordinates for starting the boundary search (search start coordinates in the figure). The offset amount may be a fixed value, but may be determined at a rate corresponding to the size of the extracted person area in the vertical direction, for example.

  Returning to the flowchart of FIG. Next, the control unit 10 calculates an index value based on the edge intensity of each pixel at the same horizontal position for each coordinate, further downward from the coordinates specified in step S503 within the cut-out range (step S504). Specifically, the control unit 10 extracts the luminance after edge extraction processing of each pixel (each pixel on the same line) having the same horizontal position within the cut-out range. Then, the control unit 10 selects the top several (for example, three) luminances from the extracted luminances, calculates an average value of the selected luminances, and sets it as an index value. For each line, the index value becomes higher as the edge becomes clearer and the number of edges becomes higher.

  Next, the control unit 10 determines whether or not the edge strength index value calculated in step S504 satisfies a condition for determining that it is a boundary (step S505). Specifically, the control unit 10 determines that the index value of the edge strength is lower than the index value calculated by one pixel in the vertical direction, that is, the line one level higher, and the decrease width is smaller than the predetermined value. If it is smaller, it is determined that the condition is satisfied.

  More specifically, the control unit 10 performs an operation based on Expression 2 shown below for the index value of each line.

However, Ave (I _{y = n−1} ) in Equation 2 indicates an index value calculated in the line above, and Ave (I _{y = n} ) is an index value calculated in the current line. y is a coordinate, and n is an arbitrary number. Threshold is an arbitrary threshold value.

  If the control unit 10 determines that the condition shown in Expression 2 is not satisfied (S505: NO), the control unit 10 moves the coordinates downward in the vertical direction, resets the counter to zero (step S506), and determines the next line. In order to perform the process, the process returns to step S504.

  If the control unit 10 determines that the condition shown in Formula 2 is satisfied (S505: YES), the counter moves the coordinates downward in the vertical direction and counts the number of vertical coordinates (lines) that satisfy the condition. 1 is added (step S507). And the control part 10 advances a process to the next.

  Next, the control unit 10 determines whether or not the counted counter is N (for example, 2) or more (step S508). If the control unit 10 determines that the counter is less than N (S508: NO), the control unit 10 returns the process to step S504 and determines the next line. However, the value N is an arbitrary threshold value.

  If the control unit 10 determines that the counter is greater than or equal to N (S508: YES), the control unit 10 demarcates the coordinate in the vertical direction where the counter starts to be incremented, that is, the coordinate on the N lines from the current vertical coordinate. (Step S509). That is, the control unit 10 determines that the edge (contour) is the most ambiguous line when the index value indicating the edge strength for each line is decreased, but the decrease width is continuously N lines or more and continuously becomes a predetermined value or less. It is judged that. This is because the edge is ambiguous at the boundary between the person's own area and the reflection component area, and the edge is the most ambiguous line.

  Then, the control unit 10 removes the region below the line identified as the boundary in step S509 in the cut-out range because it is a reflection component region (step S510), and returns the process to step S6 shown in the flowchart of FIG.

  At the end of the processing procedure shown in the flowchart of FIG. 5, assuming that the vertical coordinate in the image data of the line as the boundary is the position corresponding to the contact point between the person and the road surface, the control unit 10 revisits the image in the real space. You may make it calculate the distance from the processing apparatus 1 to a person.

  In the processing procedure shown in the flowchart of FIG. 5, processing for specifying a line where the edge strength is weakened as a boundary is performed. This is due to the observation that the edge intensity index value has the following tendency at the boundary between the person's own area and the reflection component area. Specifically, in the case of daytime, when the reflection surface is disturbed, or when the reflection intensity is weak or there is no reflection, the decrease in edge intensity for each line tends to stop near the boundary. On the other hand, when the reflection intensity is strong, such as at night or when the reflection surface is smooth, the edge intensity for each line tends to increase near the boundary. Therefore, a line corresponding to a feature point such as an inflection point or a point corresponding to a minimum value in the distribution curve of the distribution of the edge intensity for each line in the vertical direction is used as a boundary.

  Further, the boundary search start coordinates are specified based on the distance to the person in the real space by the processing in step S503, and the edge strength is weakened even in the area of the person itself, and there is a possibility that the line satisfying the condition is specified. This is to avoid this effectively.

  As described above, the index value indicating the magnitude of the edge strength for each line is calculated, and the above feature points can be obtained only by comparing the index values in successive lines. It becomes possible to specify the boundary stably without depending on it. Even when there is no reflection, the boundary between the person's own area and the road surface area can be specified by the same processing.

  Next, processing by the control unit 10 of the image processing apparatus 1 will be described with a specific example. FIG. 7 is an explanatory diagram illustrating a specific example of an image processing result performed by the control unit 10 of the image processing apparatus 1 according to the first embodiment. FIG. 7A schematically shows an example of the content of image data acquired by the image acquisition unit 13 from the imaging device 3, that is, far-infrared image data.

  In the far-infrared image data of FIG. 7A, the road surface is shown as shown in the hatched area. In the vicinity of the lower right center of the image data, there is a white area having a brightness value higher than that of other road surfaces. It is assumed that the area with the high luminance value is a person area. However, the range surrounded by the broken line is a reflection component from the road surface when the road surface is wet.

  In this case, the control unit 10 performs the processing shown in the flowchart of FIG. 4 and calculates difference data, so that a region indicated by white in the drawing is extracted as a human region candidate. The control unit 10 calculates the distance to the person based on the position in the vertical direction in the image data at the lower end of the extracted person area candidate. At this time, the lower end is not an accurate value because it is a lower end including a reflection component. Based on the calculated distance, the size to be occupied by the person area is obtained, and it is determined whether or not the size of the area occupied by the difference data is appropriate. In the case of the example shown in FIG. 7A, the region is vertically long by the amount of the reflection component. Therefore, it is determined by the control unit 10 that it is not appropriate.

  Therefore, the control unit 10 performs processing for cutting out the same range as the human region candidate from the image data and removing the reflection component. FIG. 7B is an enlarged view of the image after performing the edge processing on the image data in the same range as the human region candidate extracted from FIG. The black and white gradation indicates the level of brightness, the white part indicates high brightness, and the black part indicates low brightness. When viewed with human eyes, it is possible to recognize the outline of a person walking in the left direction in the upper half.

  The control unit 10 recalculates the distance from the position in the image data at the upper end of the clipped range. Then, the control unit 10 specifies the search start coordinates that are highly likely to be a boundary from the size in the vertical direction of the area that the person area should occupy when the person is at the recalculated distance. Then, the control unit 10 calculates an average value of the top three luminance values for each line as an index value. The control unit 10 specifies a line where the index value is decreased and the decrease is less than a predetermined value as a boundary.

  FIG. 7C is a graph showing the vertical distribution of edge strength obtained by performing the processing procedure shown in the flowchart of FIG. 5 on the person region candidate shown in FIG. 7B. The horizontal axis indicates the edge strength (index value), and the vertical axis indicates the vertical coordinate in the range of the human area candidate. The vertical axis corresponds to the person area candidate shown in FIG. In FIG. 7C, for clarity of explanation, the distribution of the edge strength with respect to the entire vertical direction of the range of the human region candidate is shown. The index value is calculated only until it is determined that the condition is satisfied.

  As shown in FIG. 7C, the edge strength is relatively high in the area of the person itself, and the edge strength decreases toward the boundary. And the way of decreasing the edge strength near the boundary line is dull. A black circle in FIG. 7C is a feature point corresponding to the boundary specified by the control unit 10. In FIG. 7 (c), the peak is unclear in the part vertically below the feature point of the black circle. However, as shown in the flowchart of FIG. 5, the control unit 10 uses a feature point that satisfies the index value as a boundary, and therefore can effectively specify the boundary even if the peak is unknown.

  In this way, it is determined whether the person area is appropriate based on the distance information in the real space obtained from the position in the image data of the area extracted as the person area candidate in which the person to be detected is shown. Whether or not a reflection component is included is determined, and even when the reflection component is included, the boundary can be specified with high accuracy from the distance information and the tendency of the distribution of edge strength.

  In addition, when specifying the boundary, the control unit 10 specifies the boundary that should be the lower end of the person itself, and then specifies the boundary based on the distribution of the edge strength, thereby specifying the boundary incorrectly. By avoiding this, the boundary can be specified with high accuracy. Thereby, the reflection component reflected in the image data can be effectively removed, and an excellent effect is achieved.

(Modification)
The image processing apparatus 1 according to the first embodiment described above determines the size of the area that the human area should occupy at the position of the human area candidate in the image data when determining whether or not the human area candidate includes a reflection component. The obtained size was compared with the size of the human region candidate actually extracted. In the present invention, the size of the person in the real space is calculated from the size of the extracted person area candidate, and it is determined whether or not the reflection component is included depending on whether or not the calculated size is appropriate as the size of the person. May be.

  FIG. 8 is a flowchart illustrating an example of an image processing procedure performed by the control unit 10 of the image processing apparatus 1 according to the modification of the first embodiment. Among the processing procedures shown below, procedures common to the processing procedures shown in the flowchart of FIG. 4 are denoted by the same step numbers, and detailed description thereof is omitted.

  The control unit 10 calculates the distance in the real space from the image processing apparatus 1 to the person to be detected (S3), and calculates the actual size of the person to be detected based on the calculated distance (step S7). Then, the control unit 10 determines whether or not the actual size calculated by Expression 3 is appropriate as the size of the person (step S8). For example, the control unit 10 determines that the size of the person is appropriate when the calculated actual size is 1.7 m, but determines that it is not appropriate when the calculated actual size is 2.5 m.

  When it is determined that the actual size is appropriate (S8: YES), the control unit 10 transmits information on the person to be detected (S6), and ends the process. On the other hand, when it is determined that the actual size is not appropriate (S8: NO), the control unit 10 removes the reflection component (S5), transmits information of the person to be detected (S6), and ends the process. .

Details of the calculation of the actual size of the person in step S7 will be described. In the image data, the control unit 10 obtains the height H _o in the vertical direction of the person area candidate extracted in the detailed process of step S3 in units of the number of pixels. The control unit 10 includes a distance D calculated at step S3, the height H _o determined, Equation 3 shown size in the real space of a person on the basis of the other fixed value (actual size) T below Calculate based on

In Equation 3, H _c is the number of pixels in the vertical direction of the image data. θ _v is the angle of view of the imaging device 3.

  FIG. 9 is an explanatory diagram schematically illustrating the principle by which the control unit 10 of the image processing apparatus 1 according to the modification of the first embodiment calculates the size of a person in real space. FIG. 9A shows the relationship between the position of the image processing apparatus 1 where the imaging apparatus 3 is installed and the position of a person at an intersection that is a detection target by the image processing apparatus 1. FIG. 9B schematically illustrates an example of the content of image data captured by the imaging device when the image processing apparatus 1 and a person are in the positional relationship illustrated in FIG.

  It should be noted that the distance D of the person from the image processing apparatus 1 shown in FIG. 9A is obtained from information stored in the storage unit 11 in advance as described in the first embodiment. Specifically, the control unit 10 obtains the distance from the image processing apparatus 1 in the real space stored in association with the vertical position Y in the image data at the lower end of the region extracted as the region corresponding to the person. .

However, the relationship between the position Y in the vertical direction in the image data and the distance D from the image processing apparatus 1 including the imaging device 3 is the angle of view θ _v and the image shown in FIGS. 9A and 9B. In addition to the number of pixels H _c in the vertical direction of the data, the relational expression shown below is established, including the height h at which the image processing apparatus 1 including the imaging apparatus 3 is installed and the depression angle φ _c of the imaging apparatus 3.

  By storing Expression 4 in the storage unit 11, when the control unit 10 specifies the detection target region (circumscribed rectangle), the control unit 10 specifies the vertical coordinate Y in the image data at the lower end of the region. Based on this, the distance D from the image processing apparatus 1 to the detection target in the real space may be calculated.

  In the first embodiment, the correspondence between the coordinate Y calculated in advance based on Expression 4 and the distance D is stored in the storage unit 11, and the distance D corresponding to the specified coordinate Y is stored in the storage unit. The distance from a predetermined position in the real space was obtained by reading from 11. Thereby, the position (distance from the imaging device 3) of the detection target in the real space can be easily specified without depending on the calculation. Therefore, position information in the real space within the intersection can be transmitted to the automobile as information on the person to be detected, and high-precision traveling control can be realized for safe driving support.

(Embodiment 2)
FIG. 10 is a schematic diagram schematically illustrating an example of a vehicle in which the safe driving support system according to the second embodiment is installed. Reference numerals 4 and 5 in FIG. 10 denote a first imaging device and a second imaging device that perform imaging with far infrared rays. Reference numeral 6 in FIG. 10 denotes an image processing apparatus, which is connected to a travel control apparatus 8 that performs a travel control process of the automobile via the in-vehicle LAN 7. The first and third imaging devices 4 and 5 are connected to the image processing device 6 through the signal line 9.

  The first imaging device 4 is arranged side by side on the left side of the front portion of the automobile. The second imaging device 5 is installed forward on the right side of the front portion of the automobile. The first and second imaging devices 4 and 5 have substantially the same range as the imaging range, and the installation direction is adjusted so that imaging with the same angle of view is possible. The first and second imaging devices 4 and 5 are both configured to output an image signal to the image processing device 6 via the communication line 9.

  When the image processing device 6 acquires image data from the image signals output from the first and second imaging devices 4 and 5 and detects a person based on the acquired image data, the detected fact and the vehicle to the person Is transmitted to the travel control device 8 via the in-vehicle LAN 7. When the fact that the person is detected is received, the travel control device 8 performs control such as automatically instructing braking based on the distance to the person or starting the shock absorber inside and outside the vehicle. Is possible.

  FIG. 11 is a block diagram showing an internal configuration of the first and second imaging devices 4 and 5 and the image processing device 6 constituting the safe driving support system in the second embodiment. The details of the configuration of the first and second imaging devices 4 and 5 are the same as those of the imaging device 3 in the first embodiment, and a description thereof will be omitted.

  The image processing device 6 transmits information to the travel control device via the control unit 60, the storage unit 61, the image memory 62, the first image acquisition unit 63 and the second image acquisition unit 64, and the in-vehicle LAN 7. And a communication unit 65 to be realized. Details of the control unit 60, the storage unit 61, and the image memory 62 are the same as those of the control unit 10, the storage unit 11, and the image memory 12 of the image processing apparatus 1 in the first embodiment, and thus detailed description thereof is omitted.

  Each of the first image acquisition unit 63 and the second image acquisition unit 64 extracts image data from the received image signal and stores it in the image memory 62, similarly to the image acquisition unit 13 of the image processing apparatus 1 in the first embodiment. . Specifically, according to an instruction from the control unit 60, the luminance value or color difference for each pixel constituting the image is specified from the received image signal, and is written as image data in the image memory 62.

  The communication unit 65 transmits and receives information based on standards for in-vehicle LANs such as CAN (Controller Area Network), LIN (Local Interconnect Network), and FlexRay (registered trademark). The control unit 60 transmits the person information detected by the communication unit 65 to the travel control device.

  The image processing apparatus 6 according to the second embodiment configured as described above processes the image data acquired from the first and second imaging apparatuses 4 and 5 in the same manner as the image processing procedure described in the first embodiment. To do. That is, the first and second image acquisition units 63 and 64 acquire image data simultaneously, calculate difference data, and extract person area candidates. However, in the first and second imaging devices 4 and 5 mounted on the automobile, the background changes at any time during traveling, so it is desirable to calculate the frame difference instead of the background difference.

  Then, the control unit 60 of the image processing device 6 according to the second embodiment uses the image data of the person area extracted from each of the image data acquired from the second imaging device 5 installed away from the first imaging device 4. Based on the difference in position, the distance from the head of the car body to the person to be detected is calculated by the process described below.

  Since the second imaging device 5 is installed so as to have the same angle of view as the first imaging device 4, the second imaging device 5 can also image a person imaged by the first imaging device 4. Since the installation directions are different, the position of the person region in the image data obtained from the second image pickup device 5 is the position of the person region in the image data obtained from the first image pickup device 4 even if the images are the same person. And different. This is because the incident angle of light from a person is different.

  The control unit 60 acquires image data based on the image signals output from the first and second imaging devices 4 and 5 at the same time, and based on the shape or position of the person region candidate extracted by taking the frame difference, respectively. The person area candidates extracted by the similarity are associated with each other. That is, the control unit 60 associates person area candidates extracted corresponding to the same person. The control unit 10 obtains a difference in position within the same angle of view. Since the difference in position is a difference in incident angle, the distance in real space to the person can be obtained based on the difference in incident angle and the distance between the first and second imaging devices 4 and 5. Then, similarly to the image processing procedure shown in the first embodiment, the control unit 10 determines whether or not the human area candidate includes a reflection component based on the distance information, and removes the reflection component if it includes the reflection component. To do.

  As described above, the configuration in which the first and second imaging devices 4 and 5 and the image processing device 6 are mounted on the automobile accurately calculates the distance to the detection target person necessary for the presence / absence of the reflection component by stereo vision. It becomes possible to do.

  In the first and second embodiments described above, the imaging device 3 and the first and second imaging devices are configured to use an infrared camera that captures an image in the far infrared wavelength band. However, the present invention is not limited to this, and any of the configurations may use a camera that captures an image in the visible light wavelength band or the near infrared wavelength band. In any case, since the imaging device also captures the reflection component due to the road surface, the reflection component can be effectively removed by performing the processing by the control unit of the image processing apparatus according to the present invention.

  Also, in the first embodiment, similarly to the configuration in the second embodiment, a third imaging device that performs imaging from different angles may be separately provided, and the distance in the real space may be calculated by stereo vision.

  In the first and second embodiments described above, the control units 10 and 60 perform image acquisition and image processing by the image acquisition unit 13 or the first and second image acquisition units 63 and 64. However, the present invention is not limited to this, and image acquisition processing and image processing may be realized by an integrated circuit in which the image processing portion is separated. In particular, in the first and second embodiments described above, the processing procedure shown in the flowcharts of FIGS. 4, 5, and 8 is configured so that the control unit 10 controls each component. However, the image processing apparatus 1 according to the present invention may be configured to have an application specific integrated circuit (ASIC) further integrated with an integrated circuit designed to perform each processing procedure shown in the flowchart. Good. For example, a circuit that calculates difference data corresponding to step S2 shown in the flowchart of FIG. 4, a circuit that realizes the reflection component removal process corresponding to step S5 shown in the flowcharts of FIGS. 5 and 8, and more specifically, By designing and using an ASIC that combines the edge extraction processing circuit corresponding to step S501 shown in the flowchart of FIG. 5 and the boundary identification circuit corresponding to the processing from step S503 to step S509, each processing is executed at high speed. can do.

  Furthermore, Embodiments 1 and 2 have shown the case where they are applied to a safe driving support system. However, the detection of the moving object by the image processing apparatus according to the present invention is not limited to the safe driving support system, and can be applied to other fields. For example, it is possible to detect a target accurately based on a captured image such as a security system or a monitoring system.

  It should be understood that the embodiment disclosed above is illustrative in all respects and is not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a perspective view from the upper part which shows the example of the intersection in which the safe driving assistance system in Embodiment 1 is installed. 1 is a block diagram illustrating an internal configuration of an image processing device, a roadside device, and an imaging device that constitute a safe driving support system in Embodiment 1. FIG. 4 is an explanatory diagram illustrating an example of the content of background image data stored in advance in a storage unit of the image processing apparatus according to Embodiment 1. FIG. 3 is a flowchart illustrating an example of an image processing procedure performed by a control unit of the image processing apparatus according to the first embodiment. 5 is a flowchart illustrating an example of a procedure of reflection component removal processing performed by a control unit of the image processing apparatus according to Embodiment 1. It is explanatory drawing shown about the process in which a control part specifies a boundary search start coordinate within a person area | region candidate. 6 is an explanatory diagram illustrating a specific example of an image processing result performed by a control unit of the image processing apparatus according to Embodiment 1. FIG. 10 is a flowchart illustrating an example of an image processing procedure performed by a control unit of the image processing apparatus according to a modification of the first embodiment. FIG. 10 is an explanatory diagram schematically illustrating a principle by which a control unit of an image processing apparatus according to a modification of the first embodiment calculates the size of a person in real space. It is a schematic diagram which shows typically the example of the vehicle in which the safe driving assistance system in Embodiment 2 is installed. FIG. 6 is a block diagram showing an internal configuration of first and second imaging devices and an image processing device that constitute a safe driving support system in a second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,6 Image processing apparatus 10,60 Control part 11,61 Memory | storage part 3 Imaging device 4 1st imaging device 5 2nd imaging device

Claims (9)

In an image processing apparatus that includes an imaging unit and performs image processing for detecting an object within an imaging range based on a captured image obtained from the imaging unit.
Extraction means for extracting a candidate for a detection target area in which the detection target is reflected from the captured image;
Based on the position in the captured image of the candidate extracted by the extraction means, the vertical and horizontal sizes of the candidate, and the dimension information indicating the size of the detection target stored in advance, the candidate is A determination means for determining whether or not a reflection component to be detected is included;
If it is determined that the determination means includes a reflection component, a specifying means for specifying a region between the detection target itself in the candidate and a boundary between the reflection component regions;
Means for removing a region of the reflection component from the candidate based on the boundary specified by the specifying means;
An image processing apparatus comprising: means for detecting a detection target based on a detection target region obtained by removing the reflection component. The specifying means is:
Edge extraction processing means for extracting an edge portion in a captured image;
Upper end specifying means for specifying the vertical coordinates of the upper end of the candidate extracted by the extracting means in the captured image;
Based on the dimension information, means for determining the vertical size that the detection target area should occupy in the captured image;
From the coordinates specified by the upper end specifying means, the lower end specifying means for specifying the coordinates in the captured image that is lower by the determined vertical size,
An index value indicating the edge strength of the band region based on the edge strength of each edge portion included in the horizontal band region passing through the coordinates for each of a plurality of coordinates further below the coordinate specified by the lower end specifying means A means of seeking
Means for obtaining a distribution curve for each coordinate in the vertical direction of the index value obtained by the means;
Means for specifying the characteristic points in the obtained distribution curve,
The image processing apparatus according to claim 1, wherein a band region in the captured image corresponding to the identified feature point is identified as a boundary. The lower end specifying means includes
Based on the coordinates specified by the upper end specifying means, further comprising means for calculating a distance in real space from the imaging means to the detection target,
Based on the distance calculated by the means and the dimension information, when the detection target is located at the distance from the imaging means in real space, the coordinates of the lower end of the area that the detection target area should occupy in the captured image are specified. The image processing apparatus according to claim 2, wherein: The determination means includes
Means for calculating a distance in real space from the imaging means to the detection target based on the position in the captured image of the candidate extracted by the extraction means;
Means for calculating the size of the detection target in real space based on the distance calculated by the means and the vertical size of the candidate extracted by the extraction means;
Means for determining whether the size calculated by the means corresponds to the size in the real space indicated by the dimension information;
The image processing apparatus according to claim 1, wherein if it is determined that the candidate does not correspond, the candidate is determined to include a reflection component. A second imaging means for simultaneously imaging the imaging range of the imaging means from an angle different from that of the imaging means;
Means for extracting a detection target area candidate in which a detection target is shown from a captured image captured by the second imaging means;
Means for associating a candidate for a detection target area extracted from a captured image obtained by the extraction means from the imaging means and a candidate for a detection target area extracted from the captured image captured by the second imaging means;
Means for specifying a position in each captured image of two candidates associated by the means;
Means for calculating a distance in real space from the imaging means or the second imaging means to the detection object based on a difference between positions where two candidates are specified;
Means for calculating the size in the real space of the detection target based on the distance calculated by the means and the vertical size of the candidate extracted by the extraction means;
Means for determining whether the size calculated by the means corresponds to the size in the real space indicated by the dimension information;
The image processing apparatus according to any one of claims 1 to 3, wherein when it is determined that the information does not correspond, the determination unit determines that a reflection component is included. The image processing apparatus according to claim 1, further comprising means for determining whether or not the detection target is a specific detection target.   The image processing apparatus according to claim 6, wherein the specific detection target is a person. In a method of performing image processing for detecting an object within an imaging range based on a captured image,
An extraction step of extracting a detection target area candidate in which the detection target is shown from the captured image;
The candidate is detected based on the position in the captured image of the candidate extracted in the extraction step, the vertical and horizontal sizes of the candidate, and the dimension information indicating the size of the detection target stored in advance. A determination step of determining whether or not the target reflection component is included;
When it is determined that the reflection component is included in the determination step, a specifying step of specifying a region between the detection target itself in the candidate and a boundary between the reflection component regions;
Removing a reflection component from the candidate based on the boundary identified by the identifying step;
An image processing method comprising: detecting a detection target based on a detection target region obtained by removing the reflection component. The specific step includes
An edge extraction processing step for extracting an edge portion in the captured image;
Identifying the vertical coordinates in the captured image of the top of the candidate extracted in the extraction step;
Based on the dimension information, obtaining a vertical size that the detection target region should occupy in the captured image;
Identifying the coordinates in the captured image corresponding to the lower end of the detection target itself that is lower by the determined vertical size from the identified coordinates;
Obtaining an index value indicating the edge strength of the band region based on the edge strength of each edge portion included in a horizontal band region passing through the coordinates for each of a plurality of coordinates further below the identified coordinates; ,
Obtaining a distribution curve for each vertical coordinate of the index value obtained in the step;
Identifying feature points in the obtained distribution curve;
The image processing method according to claim 8, further comprising: specifying a band region in the captured image corresponding to the specified feature point as a boundary.