JP2003296710A - Identification method, identification device and traffic control system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an identifying method and an identifying device capable of recognizing the existence/absence or types of objects to be measured with high by reducing the influence of an approximation between a background color and the color of an object to be measured. <P>SOLUTION: White line parts of a crosswalk are set to a first identification area, and parts other than the white line parts are set to a second identification area. First and second-temporal images to be reference are generated in advance in the first and second identification areas with no existence of objects to be measured. A first spatio-temporal image generated on the basis of image data obtained from the first identification area, a second spatio-temporal image generated on the basis of image data obtained from the second identification area, and the first and second reference spatio-temporal images are used to generate a difference spatio-temporal image. The existence/absence or types of objects to be measured are identified on the basis of the generated difference spatio-temporal image. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an identification method, an identification device, and a traffic control system for identifying a measurement object from imaged image data, and more particularly, to a pedestrian or an automobile from image data obtained by imaging the zebra zone of a pedestrian crossing. The present invention relates to an identification method, an identification device, and a traffic control system for identifying the presence or absence or the type.

[0002]

2. Description of the Related Art A traffic control system that detects a vehicle or a pedestrian using a camera or the like installed at an intersection and controls a traffic light according to the detection result for the purpose of smoothing traffic flow and safety of pedestrians. Is being developed. JP-A-10
Japanese Patent No. 63863 discloses a moving object counting processing method in which the movement of a moving object on a screen is acquired through a slit and the acquired slit-shaped moving images are arranged to perform time change analysis and shape analysis. This is to detect a moving object by arranging image data obtained from a counting line in time series to generate a spatiotemporal image and analyzing the shape of the generated spatiotemporal image.

Further, in Japanese Unexamined Patent Application Publication No. 11-275562, a camera is installed on a signal pole installed at an intersection of roads, and an image of a surveillance area in a predetermined range including a pedestrian crossing is taken by the camera. , It is input as images that are continuous in time, and pedestrians waiting for crossing near the pedestrian traffic light are extracted from this input image, and various information obtained at that time (number of people, etc.) ) Discloses a moving person monitoring device for controlling a vehicular traffic light and a pedestrian traffic light.

[0004]

However, the moving person monitoring device disclosed in Japanese Patent Laid-Open No. 11-275562 performs pedestrian detection based on a simple background difference image and time difference image. It has been difficult to detect a person who is dressed without a pattern or a person whose clothes are similar to the background color. Further, in the moving object counting processing method disclosed in Japanese Patent Laid-Open No. 10-63863, the brightness or color of the measurement object output on the counting line approximates the brightness or color on the road where the measurement object does not exist. In that case, there is a problem that the difference does not occur and the detection accuracy of the moving object is lowered. If detection of a pedestrian walking on a pedestrian crossing fails and the traffic signal is controlled by mistake, it may lead to a serious accident.Therefore, it is possible to identify with high accuracy the presence or type of a measurement target moving on the road. It was desired to develop a possible identification device.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a first identification area in which characteristic values specified based on image data obtained from the identification area differ by a predetermined value or more. And a second identification region are provided to generate a spatiotemporal image, and a differential spatiotemporal image is further generated from a reference spatiotemporal image, and the differential spatiotemporal image is analyzed to obtain a background color and measurement. An object of the present invention is to provide an identification method and an identification device capable of recognizing the presence or absence or type of a measurement object with higher accuracy by reducing the influence of the color approximation of the object.

Another object of the present invention is to provide a plurality of sets of a set of identification regions consisting of a first identification region and a second identification region, and to identify and measure a measuring object moving between the sets. An object of the present invention is to provide an identification device capable of smoothing a traffic flow by measuring the speed of an object and controlling the signal switching time.

Further, another object of the present invention is to provide a pedestrian and a driver with safety by outputting information regarding the presence or type of the measured object to be identified to a signal control device, an electronic display board, a traffic control center or the like. An object of the present invention is to provide an identification device and a traffic control system capable of walking or driving.

[0008]

According to a first aspect of the present invention, there is provided an identification method for identifying an object to be measured from image data of a pedestrian crossing on which a white line is drawn. The first identification area and the second identification area are set as the reference in the state where the measurement target is not present and the portion other than the white line is set as the second identification area. The first reference spatiotemporal image and the second reference spatiotemporal image are generated in advance, and the first spatiotemporal image and the second spatiotemporal image generated based on the image data obtained from the first identification region are generated.
Second spatiotemporal image generated based on image data obtained from the identification area of the first reference spatiotemporal image and the second spatiotemporal image
It is characterized in that a difference spatiotemporal image is generated using the reference spatiotemporal image, and the presence or type of the measurement target is identified based on the generated difference spatiotemporal image.

The identification method according to the second aspect of the present invention is the identification method for identifying an object to be measured from image data of an image of a pedestrian crossing on which a white line is drawn.
Of the first reference image data and the second identification region obtained from the first identification region in the state where the measurement object is not present, with the portion other than the white line being set as the second identification region. Second pre-stored image data obtained from the second reference image data is stored in advance, a first spatiotemporal image is generated based on the image data obtained from the first identification region and the stored first reference image data, A second spatiotemporal image based on the image data obtained from the identification area and the stored second reference image data, and the generated first spatiotemporal image and the second spatiotemporal image.
A difference spatiotemporal image is generated using the spatiotemporal image, and the presence or type of the measurement target is identified based on the generated difference spatiotemporal image.

The identification method according to the third aspect of the present invention is an identification method for identifying a measurement object from image data taken in from an image pickup section, which is used for identifying a measurement object in an image data fetching area of the image pickup section. Setting a first identification region; setting a second identification region in which a characteristic value specified based on image data obtained from the identification region differs from the first identification region by a predetermined value or more; Generating a first reference spatio-temporal image and a second reference spatio-temporal image in advance based on the image data obtained from the first identification region and the second identification region, respectively; A first spatiotemporal image generated based on image data obtained from an identification region, a second spatiotemporal image generated based on image data obtained from the second identification region, and the first reference space-time And a step of generating a differential spatiotemporal image using the image and the second reference spatiotemporal image, and an identification step of identifying the presence or type of the measurement target based on the generated differential spatiotemporal image. To do.

The identification method according to a fourth aspect of the present invention is an identification method for identifying a measurement object from image data taken from an image pickup section, which is used to identify a measurement object in an image data fetching area of the image pickup section. A step of setting a first identification area, a step of setting a second identification area in which a characteristic value specified based on image data obtained from the identification area is different from the first identification area by a predetermined value or more; Storing the first reference image data obtained from the first identification area and the second reference image data obtained from the second identification area, and the image data obtained from the first identification area and the storage Generating a first spatiotemporal image based on the first reference image data, the image data obtained from the second identification region, and the stored second reference image data Generating a second spatiotemporal image based on the generated spatiotemporal image and a second spatiotemporal image using the generated spatiotemporal image, and a measurement target based on the generated differential spatiotemporal image And the step of identifying the presence or type of an object.

According to a fifth aspect of the present invention, there is provided an identification device for identifying an object to be measured from image data taken from an image pickup section, which is used for identifying an object to be measured in an image data fetching area of the image pickup section. A unit for setting a first identification region, a unit for setting a second identification region in which a characteristic value specified based on image data obtained from the identification region differs from the first identification region by a predetermined value or more; Means for respectively generating a first reference spatiotemporal image and a second reference spatiotemporal image, which serve as a reference, based on image data obtained from the first identification region and the second identification region, and from the first identification region A first spatiotemporal image generated based on the obtained image data and a second spatiotemporal image generated based on the image data obtained from the second identification region, the first reference spatiotemporal image and the second group Characterized in that it comprises means for generating a differential space-time image, the identifying means for identifying the presence, absence or type of the measurement object based on the time-space image generated difference using a space-time image.

According to a sixth aspect of the present invention, there is provided a discrimination device for discriminating a measuring object from image data captured from an image pickup section, wherein the discriminating apparatus is used for identifying a measuring object in an image data capturing area of the image pickup section. Means for setting a first identification area, means for setting a second identification area whose characteristic value specified based on image data obtained from the identification area differs from the first identification area by a predetermined value or more, Means for storing first reference image data obtained from the first identification area and second reference image data obtained from the second identification area, and image data obtained from the first identification area and the storage Based on the first reference image data
Means for generating a spatiotemporal image, means for generating a second spatiotemporal image based on the image data obtained from the second identification region and the stored second reference image data, and the generated first spatiotemporal image And a second spatiotemporal image to generate a differential spatiotemporal image, and an identification unit that discriminates the presence or type of the measurement object based on the generated differential spatiotemporal image.

According to a seventh aspect of the invention, in the fifth or sixth aspect of the invention, the characteristic value is a lightness average value in a discrimination area, and the first lightness average value in the discrimination area and the second lightness average value. It is characterized in that the difference from the brightness average value of the identification region is different by a predetermined value or more.

According to an eighth aspect of the present invention, in the fifth or sixth aspect of the invention, the first identification area includes a pixel group corresponding to a white line portion of a pedestrian crossing arranged in a direction in which a pedestrian crosses. It is characterized in that the area is defined as the area, and the second identification area is structured such that the pixel group corresponding to a portion other than the white line is defined as the area.

An identification device according to a ninth invention is the identification device according to any one of the fifth invention to the eighth invention, wherein the first identification area and the second identification area are provided.
When the measurement target is identified by the means for setting a plurality of sets of the first identification region and the second identification region, the identification time of each measurement target in each set And means for calculating the moving speed of the measurement object based on the distance between the identification regions of each set.

According to a tenth aspect of the present invention, in the identification device according to any one of the fifth to ninth aspects of the present invention, the identification device further comprises means for outputting information regarding the presence or absence or the type of the measurement object identified by the identification means to the outside. Characterize.

A traffic control system according to an eleventh invention is a traffic control system comprising the identification device according to any one of the fifth to tenth inventions and a traffic signal control device connected to the identification device, wherein the identification device is Further comprising means for outputting information on the presence or type of the measurement object identified by the identifying means to the traffic signal control device, wherein the traffic signal control device switches the traffic signal based on the information output from the identification device. It is characterized by comprising means for controlling the time.

In the first to eighth inventions, the first identification region and the second identification region used for identifying the measurement object are set in the image data capturing region of the image pickup section. The characteristic value specified based on the image data of the second identification area is set to be different from the characteristic value of the first identification area by a predetermined value or more. For example, when the characteristic value is the average brightness value of the identification area, the difference between the average brightness value of the first identification area and the average brightness value of the second identification area is set to be different by a predetermined value or more. Alternatively, a pixel group corresponding to a white line part of a pedestrian crossing, that is, a white line part of a pedestrian crossing arranged in a direction crossing a pedestrian is set as a region, and a second identification region is defined as the white line. The pixel group corresponding to the other portion is set as the area.

At the timing when there is no measurement target, the first
The first reference spatiotemporal image and the second reference spatiotemporal image, which serve as the reference, are generated in advance based on the image data obtained from the identification region and the second identification region. Then, a first spatiotemporal image generated based on image data obtained from the first identification region, a second spatiotemporal image generated based on image data obtained from the second identification region, and a first reference spatiotemporal image And a differential spatiotemporal image is generated using the second reference spatiotemporal image. Alternatively, the first reference image data obtained from the first identification area and the second reference image data obtained from the second identification area are stored in advance in the state where the measurement target does not exist, and the first identification area is stored. The first spatiotemporal image is generated by taking the difference from the first reference image data from the respective image data obtained in time series. Similarly, for the second identification region, the second spatiotemporal image is generated by taking the difference from the stored second reference image data from the image data obtained in time series from the second identification region. Finally, these first spatiotemporal image and second
A difference spatiotemporal image is generated using the spatiotemporal image. Then, the presence or absence or the type of the measurement target is identified based on the generated difference-time space image. For the identification, the presence or absence of the measurement object and the type of the vehicle or the pedestrian are identified by performing histogram analysis, edge detection, Fourier analysis, etc. of the differential spatiotemporal image.

That is, on the white line, it is difficult to recognize a pedestrian wearing white clothes or a vehicle of a white system,
It is easy to recognize pedestrians wearing black or neutral clothes or vehicles of black or neutral colors. On the other hand, on parts other than the white line, that is, on the road, it is difficult to recognize pedestrians wearing black (particularly gray) or neutral clothes or vehicles with black or neutral colors, but wearing white clothes. It is easy to recognize pedestrians who are walking or white vehicles. in this way,
By using the zebra zone of the pedestrian crossing to generate a differential spatiotemporal image that combines these to compensate for the defects of either one of the identification areas, and to perform the identification, the influence of the color of the measurement target It is difficult and highly accurate to be able to recognize the presence or type of the measurement target.

In the ninth aspect of the invention, a plurality of sets of the first identification area and the second identification area are set with the first identification area and the second identification area as one set. When the measurement target is identified, the moving speed of the measurement target is calculated based on the identification time of the measurement target in each set and the distance between the identification regions of each set. Can be detected and the elderly person is determined to control the switching time of the signal for a long time, and when the traveling speed is fast, the switching time of the signal is controlled to be short and the traveling of the vehicle is prioritized. As a result, the traffic flow can be made smooth and pedestrians can be safe.

In the tenth invention and the eleventh invention, the information regarding the presence or absence or the type of the identified measurement object is output to the outside. For example, by outputting these pieces of information to the traffic light control device, the traffic light control device controls the switching time of the traffic light. Also, when a pedestrian is identified, information such as "there is a pedestrian" is output to the electronic bulletin board and the vehicle-mounted device,
Alternatively, when it is identified as a vehicle, information such as “vehicle approaching” is output to the mobile terminal, so that a pedestrian and a driver can safely walk or drive.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1 Hereinafter, the present invention will be described in detail with reference to the drawings showing an embodiment. FIG. 1 is a schematic diagram showing an outline of a traffic control system according to the present invention. In the figure, S is a pedestrian traffic light provided at the end of a pedestrian crossing, and under the control of the traffic light control device 3, a light emitting element (not shown) provided inside the traffic light S is caused to emit light. An image pickup unit 2 such as a CCD (Charge Couple Device) camera for picking up an image of a pedestrian crossing near the traffic light S
Are adjacent to each other, and the image data captured from the image pickup unit 2 is input to the identification device 1.

The identification device 1 determines the presence or absence or type (pedestrian or vehicle) of an object to be measured based on the input image data.
The control time of the traffic light S is adjusted if necessary. The object to be measured is identified by the image data (pixel group) relating to the first identification area indicated by the alternate long and short dash line and the second by the alternate long and two short dashes line
Processing is performed using the image data (pixel group) related to the identification area. The first identification area is a white line portion of the pedestrian crossing, that is, a pixel group corresponding to the white line portion of the pedestrian crossing lined up in the direction in which the pedestrian crosses, and the second identification area is a portion other than the white line. The pixel group corresponding to is set as an area.

FIG. 2 is a block diagram showing the hardware configuration of the identification device 1. As shown in the figure, CPU (Central
A RAM (Ra) is connected to the processing unit) 11 via a bus 17.
ndomAccess Memory) 12, storage unit 1 such as a hard disk
5, a communication unit 16 for transmitting and receiving information to and from the traffic light control device 3, a display unit 14 such as a liquid crystal display, and an input unit 13 such as a keyboard, a mouse, and an operation panel are connected. The RAM 12 stores a control program 12P for controlling the storage unit 15, the display unit 14, the input unit 13, and the like.

The video signal output from the image pickup unit 2 is A / D.
The conversion unit 19 digitizes the YC separation unit 10YC.
Then, the luminance signal is extracted as image data. As the image data, the image data for one field is stored in the RAM 12. The clock unit 18 outputs the date and time data to the CPU 11. Although the luminance signal is used in the image data in the present embodiment, it may be processed by a color signal instead.

The processing procedure of the identifying apparatus according to the present invention having the above-mentioned configuration will be described with reference to a flowchart and the like.
FIG. 3 is a flowchart showing the procedure of initial setting. First, the first identification area is set (step S31).
FIG. 4 is an explanatory diagram showing an image when the image data obtained from the image pickup unit 2 is output to the display unit 14. The horizontal direction of the screen is the X axis, the vertical direction is the Y axis, and X is displayed on the display unit 14.
Image data composed of 1024 pixels in the axial direction and 768 pixels in the Y-axis direction is output.

In FIG. 4, 10 is one pixel in the Y-axis direction, and X is
This is a first identification area consisting of 500 pixels in the axial direction, and the first identification area 10 is in a white line portion, that is, in a white line portion of a pedestrian crosswalk arranged in a direction in which a pedestrian crosses (direction of an arrow in the figure). The corresponding pixel group is set as an area. The first identification area 10 may partially include a portion other than the white line. This first identification area 10 is displayed on the display unit 14 by the operator.
It may be set by designating coordinate values from the input unit 13 while watching. Note that this setting may be automatically set as described later.

Then, the second identification area 20 is set (step S32). As shown in FIG. 4, the second identification region 20 is a pixel group corresponding to a portion other than the white line, that is, a portion other than the white line of a pedestrian crosswalk arranged in a direction crossed by a pedestrian (arrow direction in the figure). Is the area. This second
The operator may also set the identification area 20 by specifying the coordinate value from the input section 13 while looking at the display section 14. In the present embodiment, the rectangular area has one pixel on the short side, but the short side does not necessarily have to be one pixel, and the shape may be a shape other than the rectangular shape.

After setting the identification area, the CPU 11 stores the image data of the first identification area 10 for several frames (for example, 100 frames) in the RAM 12 to generate a first reference spatiotemporal image as a reference for recognition ( Step S33). Specifically, the brightness data for one frame in the first identification area 10 is made to correspond to the date and time data output from the clock unit 18 by RA.
Store in M12. Then, this is repeated for 100 frames. Similarly, the image data of the second identification area 20 for several frames (for example, 100 frames) is stored in the RAM 12 and a second reference spatiotemporal image serving as a reference for recognition is generated (step S34). Through the above processing, the first reference spatiotemporal image and the second reference spatiotemporal image that serve as a reference for recognition are completed. The generation of the reference spatiotemporal image is executed in the state where no vehicle or pedestrian exists.

FIG. 5 is an explanatory view showing images of the first reference spatiotemporal image and the second reference spatiotemporal image generated. Figure 5
5A is a first reference spatiotemporal image, and FIG. 5B is a second reference spatiotemporal image. The horizontal axis represents the pixel position in the X-axis direction, and the vertical axis represents the time axis in units of frames. It is a thing.
Since the first reference spatiotemporal image (FIG. 5A) captures the image corresponding to the white line, the spatiotemporal image having high brightness is generated as a whole. On the other hand, the second spatiotemporal image (see FIG.
Since the image corresponding to the asphalt portion is captured in (b), a spatiotemporal image having low brightness is generated as a whole.

The CPU 11 refers to the date and time data output from the clock unit 18 and determines whether or not a fixed time (for example, one hour) has passed (step S35). That is, since the brightness value obtained from the imaging unit 2 changes with the passage of time (for example, in the evening), the reference spatiotemporal image is recreated at an appropriate timing. If the fixed time has not elapsed (NO in step S35), this process is repeated. On the other hand, when the fixed time has passed (YES in step S35), it is determined whether or not the measurement target object on the pedestrian crossing is recognized (step S36). The recognition process will be described later.

When the object to be measured is recognized (step S3)
(YES in 6), and waits until the measurement object is no longer recognized. That is, this processing is performed in order to avoid the situation in which the measurement target object exists when the reference spatiotemporal image is generated. On the other hand, when the measurement target is not recognized (N in step S36)
O), and it is determined whether or not a signal indicating interrupt processing is input from the input unit 13 (step S37). When the interrupt process is not performed (NO in step S37), the process proceeds to step S33, and the first reference spatiotemporal image and the second reference spatiotemporal image are generated again. If interrupt processing has been performed (YES in step S37), all processing ends.

FIG. 6 is a flowchart showing the procedure for setting the identification area. The setting of the identification area is input by the operator from the input unit 13 as described above, and in addition to the following,
It may be executed using image processing. First, the coordinate values (x, y) of two adjacent identification areas are changed as appropriate (step S51). Specifically, one identification area is (x, y), (x + 500, y), (x, y +).
1) and (x + 500, y + 1), and other adjacent identification areas are (x, y + 5), (x + 500, y + 5),
Let (x, y + 5 + 1) and (x + 500, y + 5 + 1). Then, the CPU 11 appropriately changes x and y within a predetermined range to calculate the characteristic value specified based on the image data obtained from each area. In the present embodiment, the interval between one identification area and another identification area is set to 5 pixels, but other values may be input from the input unit 13 and set.

An appropriate value may be input from the input unit 13 or stored in the storage unit 15 in advance so that the range of changing x and y includes the white line portion of the pedestrian crossing. The characteristic values include, for example, the average brightness of the identification area, the variance value, the total brightness, and the like. In this embodiment, the average brightness of the identification area will be described as a characteristic value.

The CPU 11 changes the coordinate values (x, y) and divides the total sum of the brightness data obtained from the identification area by the number of images in the area to calculate the brightness average value, and the calculated brightness of the two identification areas. The average value and the coordinate value (x, y) are stored in the RAM 12 (step S52). The CPU 11 calculates the difference between the average brightness values of the two identification areas and determines whether or not the difference is equal to or greater than a predetermined value (step S53). If it is less than the predetermined value (NO in step S53), the RAM 12
The brightness average value and the coordinate value (x, y) stored from are deleted (step S54). The predetermined value serving as the reference may be stored in the storage unit 15 in advance or an appropriate value may be set from the input unit 13. For example, when the brightness data is output in 8-bit 256 gradations, the predetermined value may be set to about 150.

When the difference between the average brightness values is equal to or larger than the predetermined value (YES in step S53), the stored coordinate values (x, y) are not erased but saved. In this way, one coordinate value (x, y) is extracted from the plurality of coordinate values (x, y) extracted by changing the coordinate value (x, y) (step S55). Then, the first identification area and the second identification area are set by specifying the extracted one coordinate value (x, y) (step S56). Note that one coordinate value (x, y) may be extracted by extracting the one having the largest difference in average brightness.

FIG. 7 is a flow chart showing the procedure for generating the differential spatiotemporal image when the measurement is started. First, in the same procedure as when the reference spatiotemporal image is generated, a first spatiotemporal image is generated for the first identification area 10 and a second spatiotemporal image is generated for the second identification area 20 (step S71). . Then, the first difference spatiotemporal image is generated from the difference between the generated first spatiotemporal image and the stored first reference spatiotemporal image. Similarly, a second difference spatiotemporal image is generated from the difference between the second spatiotemporal image generated and the stored second reference spatiotemporal image (step S72).

Subsequently, the difference spatiotemporal image is generated by averaging the first difference spatiotemporal image and the second difference spatiotemporal image (step S73). In this way, the zebra zone of the pedestrian crossing is used to generate a differential spatiotemporal image that combines these so as to compensate for the defects of either one of the identification areas, and the identification is performed. It is possible to recognize the presence or absence of the measurement target or the type of the measurement target with high accuracy without being easily affected by.

FIG. 8 is an explanatory diagram showing an image of the generated differential spatiotemporal image. FIG. 8A is a differential spatiotemporal image of a pedestrian walking on a pedestrian crossing, and FIG. 8B is a differential spatiotemporal image of a vehicle traveling across the pedestrian crossing. The CPU 11 analyzes the generated difference spatiotemporal image and determines whether or not a measurement target object exists (step S74). When it is determined that the measurement target does not exist (NO in step S74), the process proceeds to step S71 and the above processing is repeated. When it is determined that the measurement target is present (YES in step S74), it is identified whether the measurement target is a vehicle or a pedestrian (step S7).
5). The details of these processes will be described later.
The CPU 11 outputs information regarding the presence or absence or the type of the measurement target to the outside such as the traffic light control device 3 via the communication unit 16 (step S76).

In the above-described embodiment, the first discrimination area 10 and the second discrimination area 20 are set to one pixel in the Y-axis direction. However, in order to reduce the discrimination error, two pixels are arranged in the Y-axis direction. The area may be set. FIG. 9 is a flowchart showing a processing procedure when an area for two pixels is set in the Y-axis direction, and FIG. 10 is an explanatory diagram showing an image when an area for two pixels is set in the Y-axis direction. As shown in FIG. 10, the first identification area 10A and the first identification area 10
The first identification area 10B adjacent to A is set. Similarly, the second identification area 20A and the second identification area 20B adjacent to the second identification area 20A are set.

In this case, the reference spatiotemporal image and the differential spatiotemporal image are generated according to the following procedure. First,
As described with reference to FIG. 3, the first reference space-time image A, the first reference space-time image B, and the second reference space-time image A are obtained from the image data obtained from the first identification regions 10A, 10B and the second identification regions 20A, 20B. The spatiotemporal image A and the second reference spatiotemporal image B are generated (step S91). When the measurement is started, the image data is captured to generate the first spatiotemporal image A and the first spatiotemporal image B, and the second spatiotemporal image A and the second spatiotemporal image B (step S92).

Then, a differential spatiotemporal image between the first spatiotemporal image A and the first reference spatiotemporal image A and a differential spatiotemporal image between the second spatiotemporal image A and the second reference spatiotemporal image A are obtained, A first difference spatiotemporal image is generated from the sum of these two calculated difference spatiotemporal images. Similarly, a differential spatiotemporal image between the first spatiotemporal image B and the first reference spatiotemporal image B and a differential spatiotemporal image between the second spatiotemporal image B and the second reference spatiotemporal image B are obtained,
A second differential spatiotemporal image is generated from the calculated sum of the two differential spatiotemporal images (step S93). The subsequent processing is the same as that in step S73 and will not be described.

FIG. 11 is a flow chart showing the procedure for identifying the measuring object in the generated differential spatiotemporal image. First, the generated differential spatiotemporal image is read (step S11).
1). Further, the threshold value stored in advance in the storage unit 15 is read (step S112). The CPU 11 determines whether or not the brightness of each pixel is equal to or greater than the stored threshold value (step S113). If it is greater than or equal to the threshold (step S1
13), the count number N stored in the RAM 12
Increment (initial value N = 0) (step S1)
14). On the other hand, if it is not greater than or equal to the threshold (step S11
No in step 3), the process of step S114 is skipped.

The CPU 11 determines whether or not the process of step S113 has been performed for all pixels (step S).
115). If the determination is not made for all pixels (NO in step S115), the process proceeds to step S113 and the above processing is repeated. When the determination is completed for all the pixels (YES in step S115), it is determined whether or not the count number N is a predetermined value or more (step S).
116). The predetermined value is stored in the storage unit 15 in advance. If it is equal to or larger than the predetermined value (YES in step S116), it is determined that the measurement target object exists, and the flag of the measurement target existence is set in the RAM 12 (step S1)
17). On the other hand, if it is not equal to or more than the predetermined value (step S11
(NO in 6), it is determined that the measurement object does not exist, and the flag of no measurement object is set in the RAM 12 (step S118).

When it is determined that the measurement object is present, it is determined whether the count number N is equal to or more than the set value (step S119). Note that this set value is also stored in the storage unit 15 in advance, and a value larger than the predetermined value used in step S116 is set. CPU 11 counts N
Is determined to be equal to or greater than the set value (step S119
If YES, the type of the measurement object is determined to be a vehicle, and the vehicle flag is set (step S1111). On the other hand, when the count number N is not equal to or greater than the set value (NO in step S119), the type of the measurement target object is determined to be a pedestrian, and the pedestrian flag is set (step S1110).

In order to identify the type of the measuring object, the following processing may be performed. For example, from the generated differential spatiotemporal image, a histogram is created in which the horizontal axis is the pixel position on the X axis and the vertical axis is the sum of lightness of the pixel group in the Y axis direction at each pixel position on the X axis, and this shape is analyzed. It is also possible to identify the type by doing so. For example, in the case of the pedestrian shown in FIG. 8A, the shape has an abrupt peak (particularly, when a plurality of pedestrians cross, the shape has an abrupt peak). In the case of the vehicle shown in b), the histogram has a shape showing a high brightness value over almost the entire area of the X axis. From this, the degree of variation from the lightness sum average of the histogram is defined as a variance value, the variance value is calculated, and if the variance value is large, it is determined to be a pedestrian, and if the variance value is small, it is determined to be a vehicle. do it.

Further, the difference between adjacent pixels in the X-axis direction of the histogram is calculated. Then, the difference is calculated for all the adjacent pixels, and the sum of the differences is divided by the number of pixels on the X axis to calculate the average value. If this average value is large, it is determined to be a pedestrian, and if the average value is small, it is determined to be a vehicle. In addition to these, the histogram is FF
Frequency analysis is performed using T (Fast Fourier Transform). When the frequency is high, it is determined to be a pedestrian, and when the frequency is low, it is determined to be a vehicle. Further, instead of analyzing the histogram, edge detection may be performed on the differential spatiotemporal image to recognize the type. First, the edges are detected and the direction of each detected edge is detected. It should be noted that the edge detection is performed not only on the edge of the contour portion of the object area but also on the edge inside the object area. Next, 360 degrees is divided into eight directions, and the ratio of the existence of each angle is calculated from the detected edge direction. With this, the “texture orderliness” of the spatiotemporal image can be quantified,
It is possible to determine whether the measurement target is a pedestrian or a vehicle. The determination of the type is not limited to the method described above, and analysis may be performed using another processing method.

Next, a processing procedure for calculating the walking speed of a pedestrian using the above-described identification processing will be described. FIG. 12 is an explanatory diagram showing an image of an identification area according to another embodiment. In addition to one identification region R including a set of the first identification region 10 and the second identification region 20, another identification including a different set of the first identification region 10 and the second identification region 20. Area Q is set. The CPU 11 calculates the walking speed of the pedestrian based on the time when the pedestrian flag is set in each identification area and the distance between the identification areas.

FIG. 13 is a flowchart showing the processing procedure for calculating the walking speed of a pedestrian. After the above-described setting of the one identification area R and the other identification area Q is completed, the distance between these areas QR is measured and stored in the storage unit 15 in advance and set (step S131). CPU11
Determines whether the pedestrian flag is set in one identification region R (step S132). If the pedestrian flag is not set (N in step S132)
O), this process is repeated.

On the other hand, when the pedestrian flag is set (YES in step S132), the date / time data output from the clock unit 18 is stored in the RAM 12 (step S1).
33). Subsequently, the CPU 11 determines whether or not the pedestrian flag is set in another identification area Q (step S).
134). If the pedestrian flag is not set (NO in step S134), this process is repeated.

On the other hand, when the pedestrian flag is set (YES in step S134), the date and time data output from the clock section 18 is stored in the RAM 12 (step S1).
35). The CPU 11 calculates the time required to pass through these areas from the stored two date / time data (step S136). Then, the pedestrian speed is calculated by dividing the stored distance by the required reading time (step S137).

Finally, the processing procedure when the information regarding the presence or absence or the type of the measuring object is output to the traffic light control device 3 will be described. FIG. 14 is a flowchart showing a processing procedure of the traffic light control device 3 that has received the information regarding the presence or absence or the type of the measurement target. The traffic light control device 3 calculates the switching time from the green signal to the red signal based on the output from the internal clock section (not shown), and determines whether it is 10 seconds before the switching to the red signal ( Step S14
1). If it is not 10 seconds before (N in step S141
O), this process is repeated.

On the other hand, if it is 10 seconds before (step S14)
If YES in step 1), then it is determined whether a pedestrian flag is output from the identification device 2 (step S142). When the pedestrian flag is output (YES in step S142)
S), processing for extending the switching time by several seconds is performed (step S143). After the process of extending the switching time, the process proceeds to step S142 again and it is determined whether or not a hand pedestrian exists. If the pedestrian flag is continuously set for a certain period of time due to a failure or the like, a warning signal may be output to an external monitoring center computer or the like. On the other hand, when the pedestrian flag is not output (NO in step S142), without extending the switching time,
The switching signal of the traffic signal S is output to the traffic signal S, and the traffic signal S
Is switched to a red traffic light (step S144). This ensures the safety of pedestrians and facilitates smooth traffic flow.

Note that the traffic signal S when information regarding the presence or absence of the measurement object or the type of the measurement object is output to the traffic signal control device 3.
The control for is not limited to this, and other control may be performed. For example, the switching time of the traffic light S is optimized from the walking speed of the pedestrian, and when the pedestrian is not continuously detected (for example, at night), the switching time is set shorter than the default value. Is also good. Furthermore, although the information regarding the presence or absence or the type of the measurement object is output to the traffic light control device 3, the information is not limited to this, and may be output to another device. For example, when the pedestrian flag is output at the time of a red light, such information is output to the traffic flow comprehensive management center.
A warning signal may be output to the mobile phone of a pedestrian having the h function, a signal of "there is a pedestrian" to the electronic bulletin board, or information on pedestrian crossing may be output to an in-vehicle device such as a car navigation system. .

Embodiment 2 The difference spatiotemporal image generation processing may be executed by the following processing different from the processing described in Embodiment 1. FIG. 15 is a flowchart showing the generation process of the differential spatiotemporal image. First, the identification device 1 sets the first identification area 10 in the same manner as the procedure described in the first embodiment (step S
151) and the second identification area 20 is set (step S152).

Then, the image data for one frame is fetched from the first identification area 10 and stored in the RAM 12 as the first reference image data. Further, one frame of image data is fetched from the second identification area 20 and stored in the RAM 12 as second reference image data (step S153).
After the measurement is started (step S154), the difference between the image data obtained from the first identification area 10 and the first reference image data stored in the RAM 12 is calculated for each pixel, and the difference data for each pixel is calculated. The data is stored in the RAM 12 in association with the date and time data output from the clock unit 18 (step S
155). Similarly, the difference between the image data obtained from the second identification area 20 and the second reference image data stored in the RAM 12 is calculated for each pixel, and the date and time when the difference data for each pixel is output from the clock unit 18. RA in association with data
It is stored in M12 (step S156).

Subsequently, it is determined whether or not the difference data stored in the RAM 12 has been stored for a predetermined time (for example, one second) (step S157). If not stored for the predetermined time (NO in step S157), the processes of steps S155 and S156 are repeated to store the difference data of the next frame. On the other hand, when the data is stored for the predetermined time (YES in step S157), the difference data stored in association with the date and time data in step S155 is read in time series to generate the first spatiotemporal image, and further in step S156. In step S158, the second spatiotemporal image is generated by reading out the difference data stored in association with the date / time data in time series. Finally, the brightness of each pixel of the first spatiotemporal image and the second spatiotemporal image is added to generate a differential spatiotemporal image (step S159). The differential spatiotemporal image may be generated by adding the first spatiotemporal image and the second spatiotemporal image, or may be generated by taking an average or the like.

Next, another processing procedure when the width of the identification area is set to a plurality of pixels as shown in FIG. 10 will be described. 16 and 17 are flowcharts showing another processing procedure when the width of the identification area is set to a plurality of pixels. First, the identification device 1 sets the first identification areas 10A and 10B and the second identification areas 20A and 20B in the same manner as the procedure described in the first embodiment (step S16).
1).

Then, one frame of image data is fetched from the first identification area 10A and stored in the RAM 12 as the first reference image data A. Similarly, the first identification area 1
Image data for one frame is taken in from 0B and stored in the RAM 12 as the first reference image data B. Also, the second
Image data for one frame is fetched from the identification area 20A, and is stored in the RAM 12 as the second reference image data A.
Image data for one frame is fetched from the second identification area 20B and stored in the RAM 12 as the second reference image data B (step S162). Measurement starts (step S1
63), the difference between the image data obtained from the first identification area 10A other than the white line and the first reference image data A stored in the RAM 12 is calculated for each pixel, and the difference for each pixel is calculated. The data 10A is stored in the RAM 12 in association with the date and time data output from the clock unit 18 (step S164). Further, the difference between the image data obtained from the second identification area 20A, which is the white line portion, and the second reference image data A stored in the RAM 12 is calculated for each pixel, and the difference data 20A for each pixel is calculated as the clock unit 18. The data is stored in the RAM 12 in association with the date and time data output from (step S165).

Similar to the processing procedure described above, similar processing is performed for the first identification area 10B and the second identification area 20B. First, the difference between the image data obtained from the first identification area 10B other than the white line and the first reference image data B stored in the RAM 12 is calculated for each pixel, and the difference data 10B for each pixel is calculated. The data is stored in the RAM 12 in association with the date and time data output from the unit 18 (step S166). In addition, the difference between the image data obtained from the second identification area 20B, which is the white line portion, and the second reference image data B stored in the RAM 12 is calculated for each pixel, and the difference data 20B for each pixel is calculated as the clock unit 18. It is stored in the RAM 12 in association with the date and time data output from (step S167).

Subsequently, the difference data 1 stored in the RAM 12
0A, 20A, 10B, 20B for a predetermined time (for example,
It is determined whether or not it has been stored (for one second) (step S16).
8). If not stored for a predetermined time (step S1
No in step 68), the processes of steps S164 to S167 are repeated, and the difference data 10A, 20A, 1 of the next frame,
Store 0B and 20B. On the other hand, if the predetermined time has been stored (YES in step S168), step S1
The difference data 10A stored corresponding to the date and time data in 64 is read in time series to obtain the first spatiotemporal image 10A.
And further, the difference data 20A stored in association with the date and time data in step S165 is read in time series to generate the second spatiotemporal image 20A (step S1).
71).

Similarly, the difference data 10B stored in association with the date and time data in step S166 is read in time series to generate the first spatiotemporal image 10B, and the difference stored in association with the date and time data in step S167. The data 20B is read in time series to generate the second spatiotemporal image 20B (step S172). The brightness of each pixel of the first spatiotemporal image 10A and the second spatiotemporal image 20A generated in this way is added, and each pixel of the generated first spatiotemporal image 10B and the second spatiotemporal image 20B is added. , And the sum of these values is divided by 2 which is the width of the identification area to generate a differential spatiotemporal image (step S17).
3). If a plurality of pixels (for example, three pixels (six in total)) are provided as the width of the identification region, each spatiotemporal image generated in step S173 may be divided by the number of widths (three in this example). Good.

The second embodiment is configured as described above, and other configurations and operations are similar to those of the first embodiment. Therefore, corresponding parts are designated by the same reference numerals and detailed description thereof will be given. Omit it.

[0066]

As described above in detail, in the first to eighth inventions, the first identification area and the second identification area, which are used to identify the measurement object, in the image data acquisition area of the image pickup unit. Set the identification area. The characteristic value specified based on the image data of the second identification area is set to be different from the characteristic value of the first identification area by a predetermined value or more. A first reference spatiotemporal image and a second reference spatiotemporal image, which are references in advance, are generated based on the image data obtained from the first identification region and the second identification region at the timing when the measurement target does not exist. deep. The first spatiotemporal image generated based on the image data obtained from the first identification area and the second spatiotemporal image.
The differential spatiotemporal image is generated using the second spatiotemporal image generated based on the image data obtained from the identification area, the first reference spatiotemporal image, and the second reference spatiotemporal image. Alternatively, the first reference image data obtained from the first identification region and the second reference image data obtained from the second identification region are stored in advance in the state where the measurement object does not exist,
The first spatio-temporal image is generated by taking the difference from the first reference image data from the respective image data obtained in time series from the identification area. Similarly for the second identification area,
A second spatiotemporal image is generated by taking the difference from the stored second reference image data from the respective image data obtained in time series from the second identification region. Finally, a differential spatiotemporal image is generated using the generated first spatiotemporal image and second spatiotemporal image. Then, the presence or absence or the type of the measurement target is identified based on the generated difference-time space image.

As described above, the zebra zone of the pedestrian crossing is used to generate the differential spatiotemporal image by combining these so as to compensate for the defect in either one of the identification areas, and the identification is performed. The presence or type of the measurement object can be recognized with high accuracy, being less affected by the color of the object.

In the ninth invention, a plurality of sets of the first identification area and the second identification area are set with the first identification area and the second identification area as one set. When the measurement target is identified, the moving speed of the measurement target is calculated based on the identification time of the measurement target in each set and the distance between the identification regions of each set. Can be detected and the elderly person is determined to control the switching time of the signal for a long time, and when the traveling speed is fast, the switching time of the signal is controlled to be short and the traveling of the vehicle is prioritized. As a result, the traffic flow can be made smooth and pedestrians can be safe.

In the tenth invention and the eleventh invention, the information regarding the presence or absence or the type of the identified measurement object is output to the outside. For example, by outputting these pieces of information to the traffic light control device, the traffic light control device controls the switching time of the traffic light. Also, when a pedestrian is identified, information such as "there is a pedestrian" is output to the electronic bulletin board and the vehicle-mounted device,
Alternatively, when the vehicle is identified as a vehicle, information such as “vehicle approaching” is output to the mobile terminal, so that the pedestrian and the driver can walk or drive safely, and the present invention is excellent. It can be effective.

[Brief description of drawings]

FIG. 1 is a schematic diagram showing an outline of a traffic control system according to the present invention.

FIG. 2 is a block diagram showing a hardware configuration of an identification device.

FIG. 3 is a flowchart showing a procedure of initial setting.

FIG. 4 is an explanatory diagram showing an image when image data obtained from an imaging unit is output to a display unit.

FIG. 5 is an explanatory diagram showing images of a first reference spatiotemporal image and a second reference spatiotemporal image generated.

FIG. 6 is a flowchart showing a procedure for setting an identification area.

FIG. 7 is a flowchart showing a procedure for generating a differential spatiotemporal image when measurement is started.

FIG. 8 is an explanatory diagram showing an image of a generated differential spatiotemporal image.

FIG. 9 is a flowchart showing a processing procedure when an area for two pixels is set in the Y-axis direction.

FIG. 10 is an explanatory diagram showing an image when a region for two pixels is set in the Y-axis direction.

FIG. 11 is a flowchart showing a procedure for identifying an object to be measured in a generated differential spatiotemporal image.

FIG. 12 is an explanatory diagram showing an image of an identification area according to another embodiment.

FIG. 13 is a flowchart showing a processing procedure for calculating a walking speed of a pedestrian.

FIG. 14 is a flowchart showing a processing procedure of a traffic light control device that has received information regarding the presence or absence or type of a measurement object.

FIG. 15 is a flowchart showing a process of generating a differential spatiotemporal image.

FIG. 16 is a flowchart showing another processing procedure when the width of the identification area is set to a plurality of pixels.

FIG. 17 is a flowchart showing another processing procedure when the width of the identification area is set to a plurality of pixels.

[Explanation of symbols] 1 identification device 18 clock 13 Input section 14 Display 16 Communication unit 19 A / D converter 10YC YC separation unit 2 Imaging unit 3 Traffic light control device S traffic light 10 First identification area 20 Second identification area

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Masakatsu Higashikubo             1-3-3 Shimaya, Konohana-ku, Osaka City, Osaka Prefecture             Sumitomo Electric Industries, Ltd. Osaka Works (72) Inventor Kayo Tanaka             1-3-3 Shimaya, Konohana-ku, Osaka City, Osaka Prefecture             Sumitomo Electric Industries, Ltd. Osaka Works (72) Masanori Aoki, inventor             1-3-3 Shimaya, Konohana-ku, Osaka City, Osaka Prefecture             Sumitomo Electric Industries, Ltd. Osaka Works (72) Inventor Kenji Terada             3-6-10 Ataka, Tokushima City, Tokushima Prefecture-211 F term (reference) 5B057 AA16 AA19 BA02 DA06 DA12                       DA15 DB02 DB05 DB09 DC22                       DC33 DC36 DC39                 5H180 AA01 AA21 CC04 JJ02 JJ05                       JJ15                 5L096 AA06 BA02 CA04 CA07 GA08                       HA03

Claims (11)

[Claims] 1. An identification method for identifying an object to be measured from image data of a pedestrian crossing with a white line drawn therein, wherein a white line portion of the pedestrian crossing is set as a first identification region, and a portion other than the white line is set. Is set as the second identification region, and the first reference space-time image and the second reference space-time image serving as the reference are set in the first identification region and the second identification region in the state where the measurement object does not exist. A first spatiotemporal image generated based on image data obtained from the first identification region and a second spatiotemporal image generated based on image data obtained from the second identification region; Discrimination characterized by generating a differential spatiotemporal image using the first reference spatiotemporal image and the second reference spatiotemporal image, and discriminating the presence or absence or the type of the measurement object based on the generated differential spatiotemporal image Method. 2. An identification method for identifying an object to be measured from image data of a pedestrian crossing with a white line drawn therein, wherein a white line portion of the pedestrian crossing is set as a first identification area, and a portion other than the white line is set. Is set in the second identification area, and the first reference image data obtained from the first identification area and the second reference image data obtained from the second identification area are stored in advance in the state where the measurement object does not exist. Aside from the first
A first spatiotemporal image based on the image data obtained from the identification region and the stored first reference image data, and the image data obtained from the second identification region and the stored second reference image data are generated. A second spatiotemporal image is generated based on the generated spatiotemporal image and a second spatiotemporal image, and a differential spatiotemporal image is generated based on the generated differential spatiotemporal image. An identification method characterized by identifying a type. 3. An identification method for identifying a measurement target from image data captured from an image capturing unit, wherein a first identification region used for identifying a measurement target is set in an image data capturing region of the image capturing unit. A step of setting a second identification area in which a characteristic value specified based on image data obtained from the identification area is different from the first identification area by a predetermined value or more; Generating a first reference spatiotemporal image and a second reference spatiotemporal image in advance based on image data obtained from the second identification region; and image data obtained from the first identification region. And a second spatiotemporal image generated based on image data obtained from the second identification region, the first reference spatiotemporal image, and the second reference Identification method characterized by comprising the steps of: generating a differential space-time image, an identification step of identifying the presence, absence or type of basis measurement object when space image generated difference with a spatial image. 4. An identification method for identifying a measurement target from image data captured from an image capturing unit, wherein a first identification region used for identifying a measurement target is set in an image data capturing region of the image capturing unit. A step of setting a second identification area in which a characteristic value specified based on image data obtained from the identification area differs from the first identification area by a predetermined value or more; and a first identification serving as a reference in advance. Storing the first reference image data obtained from the area and the second reference image data obtained from the second identification area; the image data obtained from the first identification area and the stored first reference image data; Generating a first spatiotemporal image based on the image data obtained from the second identification region and the stored second reference image data. And a step of generating a differential spatiotemporal image using the generated first spatiotemporal image and second spatiotemporal image, and identifying the presence or type of the measurement target based on the differential spatiotemporal image generated And a step of performing. 5. An identification device for identifying a measurement target from image data captured from an image capturing unit, wherein a first identification region used for identifying a measurement target is set in an image data capturing region of the image capturing unit. Means for setting a second identification area in which the characteristic value specified based on the image data obtained from the identification area is different from the first identification area by a predetermined value or more, the first identification area and the first identification area Means for generating a first reference spatiotemporal image and a second reference spatiotemporal image, respectively, based on image data obtained from the second identification region; and generation based on image data obtained from the first identification region Using the first spatiotemporal image and the second spatiotemporal image generated based on the image data obtained from the second identification region, the first reference spatiotemporal image, and the second reference spatiotemporal image Minute means for generating a spatial image, the identification device characterized by comprising an identifying means for identifying the presence, absence or type of basis measurement object in the generated differential space-time images. 6. An identification device for identifying a measurement target from image data captured from an image capturing unit, wherein a first identification region used for identifying a measurement target is set in an image data capturing region of the image capturing unit. Means for setting a second identification area in which the characteristic value specified based on the image data obtained from the identification area differs from the first identification area by a predetermined value or more; and the first identification serving as a reference in advance. Means for storing first reference image data obtained from the area and second reference image data obtained from the second identification area; and image data obtained from the first identification area and the stored first reference image data. Means for generating a first spatiotemporal image based on the image data obtained from the second identification region and the stored second reference image data, and means for generating a second spatiotemporal image A means for generating a differential spatiotemporal image using the generated first spatiotemporal image and the second spatiotemporal image, and identification means for identifying the presence or type of the measurement object based on the differential spatiotemporal image. An identification device comprising: 7. The characteristic value is a lightness average value in an identification area, and a difference between a lightness average value of the first identification area and a lightness average value of the second identification area is different by a predetermined value or more. The identification device according to claim 5, wherein the identification device is configured as described above. 8. The first identification area includes a pixel group corresponding to a white line portion of a pedestrian crosswalk arranged in a direction crossing a pedestrian, and the second identification area includes areas other than the white line. The identification device according to claim 5 or 6, wherein a pixel group corresponding to the portion is configured as an area. 9. A means for setting a plurality of sets of the first identification area and the second identification area, with the first identification area and the second identification area as one set, and an object to be measured by the identification means. 9. When it is identified, it further comprises means for calculating the moving speed of the measurement object based on the identification time of the measurement object in each set and the distance between the identification regions of each set. The identification device according to any one. 10. The identification device according to claim 5, further comprising a unit that outputs information regarding the presence or absence or the type of the measurement target identified by the identification unit to the outside. 11. A traffic control system comprising the identification device according to any one of claims 5 to 10 and a traffic light control device connected to the identification device, wherein the identification device is an object to be measured identified by the identification means. Further comprising means for outputting information regarding the presence or absence of the type to the traffic light control device, wherein the traffic light control device comprises means for controlling the switching time of the traffic light based on the information output from the identification device. Traffic control system.