JP2013114652A - Crosswalk detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect a crosswalk from a photographic image by an in-vehicle camera which moves with a vehicle while reducing a load on the arithmetic processing of a control system.SOLUTION: The edge processing of the portion of the image recognition area of a photographic image from a camera 60 loaded on a vehicle is performed by a microcomputer 30, and the high speed Fourier inverse transformation of the edge-processed image signal is performed by image recognition LSI 51, and a zebra pattern having strong periodicity is extracted from the image recognition area. When the periodicity of the extracted zebra pattern is within the range of the periodicity of the crosswalk, it is recognized that the zebra pattern is the crosswalk by the microcomputer 30.

Description

  The present invention relates to an apparatus for detecting a pedestrian crossing from an image captured by an in-vehicle camera.

  When a pedestrian crossing is detected from an image taken by an in-vehicle camera, a white line portion in the image is usually recognized as a pattern, and this is pattern-matched with a road marking pattern of a pedestrian crossing registered in advance.

  This pattern matching places a heavy load on the arithmetic processing of the control system. In view of this, a proposal has been made to reduce the load on the control system by stopping the pattern matching of the lane or the like while the pedestrian crossing is detected by pattern matching (for example, Patent Document 1).

JP 2003-252148 A

  However, since the content of the image taken by the on-vehicle camera always changes as the vehicle travels, even if the pedestrian crossing is detected by pattern matching each time the content of the image changes, the burden on the control system becomes considerable.

  In addition, in response to changes in the position and shape of the pedestrian crossing in the image as the vehicle travels, either a large number of road marking patterns for pattern matching should be registered or new pedestrian crossing patterns Every time it is recognized, it must be learned and additionally registered, so the detection of pedestrian crossings by pattern matching inevitably increases the scale of the device.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to accurately detect a pedestrian crossing from an image captured by an in-vehicle camera that moves with the vehicle while reducing the load on arithmetic processing of the control system. An object of the present invention is to provide a pedestrian crossing detection device.

In order to achieve the above object, a pedestrian crossing detection apparatus according to the present invention described in claim 1 is provided.
Each time a captured image of the camera is updated by updating a signal input from an in-vehicle camera, a zebra with a predetermined cycle corresponding to a pedestrian crossing is obtained based on a spectrum pattern obtained by Fourier transform of a predetermined area of the captured image. Extracting means for extracting a pattern from the predetermined area;
The extracted zebra pattern is recognized as a pedestrian crossing when the front end close to the vehicle of the extracted zebra pattern and the rear end far from the vehicle are separated from each other by a predetermined interval or more according to a pedestrian crossing on the predetermined region. Recognizing means for recognizing that the zebra pattern is not a pedestrian crossing when the front and rear ends are not separated by more than the predetermined interval on the predetermined area;
A pedestrian crossing detection device comprising:

  According to the pedestrian crossing detection apparatus of the present invention described in claim 1, as the vehicle travels, the position, size, and shape of the pedestrian crossing that appears in the captured image of the camera change. Therefore, in order to detect a pedestrian crossing from a captured image by pattern matching, it is necessary to register a number of road marking patterns for pattern matching in accordance with the size and shape of the pedestrian crossing that appears in the captured image.

  On the other hand, according to the present invention, a zebra pattern having a predetermined period corresponding to a pedestrian crossing can be extracted from a captured image depending on whether a spectrum pattern obtained by Fourier transform of the captured image is strong in a predetermined period portion. In addition, by limiting the portion where the zebra pattern is extracted to a predetermined area in the captured image, the range in which the shape and size change according to the position of the zebra pattern is limited, and the interval between the front and rear ends of the zebra pattern is used as an index. It is possible to easily determine whether it is a crosswalk.

  As a result, it is possible to detect a pedestrian crossing with high accuracy from an image captured by an in-vehicle camera while reducing the burden on arithmetic processing of the control system.

  Moreover, the pedestrian crossing detection apparatus of the present invention described in claim 2 is the pedestrian crossing detection apparatus of the present invention described in claim 1, wherein the extraction means is configured such that the front and rear ends are not separated by the predetermined interval or more. The zebra recognized by the recognition means as a pedestrian crossing without extracting the zebra pattern from the predetermined area and extracting the zebra pattern whose front end does not exist on the predetermined area from the predetermined area Update based on the amount of displacement of the zebra pattern in which the front end does not exist on the predetermined region with respect to the pattern in the predetermined region at the rear end and the amount of movement of the vehicle before and after the update of the captured image A discriminating unit for discriminating whether or not the zebra patterns extracted from the predetermined areas of the respective front and rear captured images are the same pedestrian crossing; For example, characterized in that is.

  According to the pedestrian crossing detection apparatus of the present invention described in claim 2, in the pedestrian crossing detection apparatus of the present invention described in claim 1, recognition based on the premise that the front and rear ends of the zebra pattern exist on a predetermined region. Even if the pedestrian crossing cannot be recognized by the means, if the rear end of the zebra pattern exists on the predetermined area, the discriminating means causes the rear end displacement amount and the vehicle travel amount in the predetermined area with respect to the pre-update of the photographed image. Based on the above, the identity of the trailing edge of the zebra pattern can be determined.

  Therefore, if it is recognized that the zebra pattern before update is a pedestrian crossing, the back end of the zebra pattern having the same identity is reflected in the predetermined area of the updated image, so that the front end is reflected. Even if it is not a zebra pattern, it can be substantially recognized that it is a pedestrian crossing.

  Furthermore, the pedestrian crossing detection apparatus of the present invention described in claim 3 is the pedestrian crossing detection apparatus of the present invention described in claim 2, wherein the extraction means does not extract the zebra pattern from the predetermined area. In addition, the determining means is based on the rear end of the zebra pattern extracted by the extracting means from the predetermined area of the previous photographed image and the amount the vehicle has traveled before and after updating the photographed image. The position of the rear end of the zebra pattern is estimated, and based on the relative position with respect to the vehicle, it is determined whether or not the zebra pattern of the estimated rear end position is a pedestrian crossing.

  According to the pedestrian crossing detection apparatus of the present invention described in claim 3, in the pedestrian crossing detection apparatus of the present invention described in claim 2, when not only the front end of the zebra pattern but also the rear end does not exist in the predetermined area. Whether the zebra pattern extracted by the extraction means from the predetermined area of the photographed image before the update and the zebra pattern estimated from the amount of travel of the vehicle is a pedestrian crossing is determined based on the relative position of the estimated zebra pattern to the rear end of the vehicle. Can be determined.

  Therefore, even if the zebra pattern of the pedestrian crossing does not exist in a predetermined area of the photographed image as the vehicle progresses, for a while (for example, until the vehicle is expected to pass the pedestrian crossing) Recognizes that the pedestrian crossing exists in the vicinity of the vehicle, and can continue the processing on the assumption that the vehicle exists in the pedestrian crossing or in the vicinity thereof.

  According to the present invention, it is possible to detect a pedestrian crossing with high accuracy from an image taken by an in-vehicle camera that moves with the vehicle while reducing the load on the arithmetic processing of the control system.

It is a block diagram which shows schematic structure of the drive recorder incorporating the pedestrian crossing detection apparatus which concerns on one Embodiment of this invention. It is a schematic diagram of the vehicle by which the drive recorder and camera of FIG. 1 are mounted. It is explanatory drawing which shows the standard of the road marking (zebra pattern) of the pedestrian crossing which the image recognition LSI of FIG. 1 detects. It is explanatory drawing which shows the edge period of the zebra pattern of the pedestrian crossing of FIG. FIG. 2 is an explanatory diagram illustrating an image recognition area in which fast Fourier inverse transform processing is performed on an image signal of a captured image edge-processed by the image recognition LSI in order to detect a zebra pattern of a pedestrian crossing from the captured image of the camera of FIG. 1. It is explanatory drawing which shows schematic structure of the image recognition LSI of FIG. It is a schematic diagram which shows the process of obtaining the average value and intensity threshold value of an amplitude spectrum from the effective data part obtained by carrying out the inverse fast Fourier transform of the picked-up image of a camera with each processor element (PE) of FIG. It is a graph which shows an example of distribution of the amplitude spectrum for every period with respect to the intensity | strength threshold value obtained by the process of FIG. FIG. 7 is an explanatory diagram illustrating a state in which the image recognition area in FIG. 5 is rotated by 90 ° in order to perform fast Fourier inverse transform processing in each processor element (PE) in FIG. 6. It is a flowchart which shows the outline | summary of the process regarding the detection of the zebra pattern applicable to the pedestrian crossing which the microcomputer shown in FIG. 1 performs according to the program of ROM. It is explanatory drawing which shows the case where the front end of a zebra pattern does not exist in the image recognition area | region of FIG. (A)-(c) is explanatory drawing which shows the content of the label used when the microcomputer shown in FIG. 1 performs the tracking process of the zebra pattern applicable to a pedestrian crossing. It is a flowchart which shows the outline | summary of the procedure of the tracking process of the pedestrian crossing which the microcomputer shown in FIG. 1 performs according to the program of ROM. FIG. 2 is an explanatory diagram illustrating a current recognition rear end (current recognition position) and a predicted rear end position predicted from a previous recognition rear end on a captured image of the camera of FIG. 1. It is explanatory drawing which shows the case where a preceding vehicle exists on the pedestrian crossing reflected in the picked-up image of the camera of FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a drive recorder incorporating a pedestrian crossing detection apparatus according to an embodiment of the present invention. A drive recorder 10 shown in FIG. 1 is used by being mounted on a vehicle 1 together with a camera 60, as shown in the schematic diagram of FIG.

  As shown in FIG. 1, the drive recorder 10 decodes an input signal of a photographed image from a camera 60 photographed in front of the vehicle 1 by a main decoder 20 and compresses it in a microcomputer 30 constituted by an image processing LSI or the like. After the processing, a moving image for a certain period of time and a still image with a certain period are stored in a removable storage medium such as the SD card 40. The moving image for a certain time is overwritten and recorded so that the latest past certain time is always stored.

  The drive recorder 10 has an image recognition unit 50 that constitutes a pedestrian crossing detection apparatus together with the main decoder 20 and the microcomputer 30. The image recognition unit 50 includes an image recognition processing LSI (hereinafter abbreviated as “image recognition LSI”) 51, a ROM 52, and a RAM 53. The ROM 52 stores a program that is executed by the image recognition LSI 51 to perform fast Fourier inverse transform (IxFFT) processing on an image captured by the camera 60 based on the decoded image signal from the main decoder 20. The RAM 53 provides a work area or the like necessary for the image recognition LSI 51 to execute the program of the ROM 52, and can be constituted by, for example, an SDRAM (Synchronous Dynamic Random Access Memory).

  Next, the principle at the time of detecting a pedestrian crossing from the image captured by the camera 60 will be described. As shown in the explanatory diagram of FIG. 3, according to the standard of the road marking (zebra pattern) of the pedestrian crossing 70 determined by the public safety committee of each prefecture, the width L1 of the white line portion 71 and the non-white line portion (road surface) The width L2 of the exposed portion 72 is 0.45 to 0.50 m. Moreover, the space | interval of the front end of the white line part 71 and the non-white line part 72 seen from the vehicle 1 and a rear end is about 2.5-3.0 m. The white lines 73 and 74 may be provided at the front and rear ends of the white line portion 71 and the non-white line portion 72. In this case, the thickness (front-rear length) of the white lines 73 and 74 is 0.15 to 0. It is determined to be 30m.

  Therefore, when the edge of the zebra pattern (the edge of the white line portion 71 or the non-white line portion 72) of the pedestrian crossing 70 that appears in the photographed image of the camera 60 is extracted by contrast, white → black (road surface) as shown in the explanatory diagram of FIG. Color) edge exists at a period of 0.9 to 1.0 m, and black → white edge also exists at a period of 0.9 to 1.0 m.

  By the way, when the front of the vehicle 1 is photographed with the camera 60, the minimum value 0.45m of the width L1 of the white line portion 71 and the width L2 of the non-white line portion 72 is converted into the number of pixels on the photographed image. In the case of the embodiment, it is about 7 dots for 20 m ahead of the vehicle 1, about 5 dots for 27 m ahead, and about 3 dots for 35 m ahead. Since the minimum value (0.9 m) of the edge period of white → black (or black → white) is doubled, it is about 14 dots for 20 m ahead of vehicle 1, about 10 dots for 27 m ahead, and about 6 dots for 35 m ahead. It becomes.

  On the other hand, in this embodiment, the detection resolution of the edge period of the zebra pattern performed by the image recognition LSI 51 performing inverse Fourier transform (IxFFT) processing on the image signal obtained by performing edge processing on the captured image has a lower limit of 8 pixels of the video signal. (8 dots), the upper limit is 85.3 pixels (85.3 dots) of the video signal. The lower limit depends on the recognition resolution of the image signal decoded by the image recognition LSI 51. The upper limit depends on the horizontal width (= 512 dots) of an image recognition area (corresponding to a predetermined area in the claims, see FIG. 5) to recognize an image signal decoded by the image recognition LSI 51.

  From the detection resolution of the edge period (8 to 85.3 dots) described above, it is expected that the edge period of white → black (or black → white) of the zebra pattern can be detected in front of 20 m or 27 m in front of the vehicle 1. , It is expected to be difficult to detect at a resolution of 35 m ahead.

  Further, 0.15 m, which is the minimum thickness of the white lines 73 and 74 at the front end and the rear end of the pedestrian crossing 70, is converted into the number of pixels on the captured image. Is about 2.5 dots, 27 meters ahead, about 1.4 dots, and 35 ahead ahead is about 0.8 dots.

  Since the thickness of the white lines 73 and 74 at the front end and the rear end (edges before and after the white lines 73 and 74) cannot be distinguished if it is less than 2 dots, the detection of the white lines 73 and 74 at the front end and the rear end of the pedestrian crossing 70 Although it is expected to be possible at 20 meters ahead, it is expected to be difficult in terms of resolution at 27 meters and 35 meters ahead.

  Therefore, in the present embodiment, it is roughly possible to detect both the white → black (or black → white) edge period of the zebra pattern of the pedestrian crossing 70 and the white lines 73 and 74 at the front and rear ends of the pedestrian crossing 70. This is considered to be an area 20 m ahead of the vehicle 1. Therefore, in the present embodiment, the area around 20 m ahead of the vehicle 1 in the image captured by the camera 60 is set as the upper limit and the lower limit of the above-described image recognition area. Then, the range of 0.9 m ± 25% (46 dots ± 11.5 dots) in the photographed image 20 m ahead of the vehicle 1 is used as the white → black (or black → white) edge period of the zebra pattern detected as the pedestrian crossing 70. Yes.

  FIG. 5 is an explanatory diagram showing an image recognition area in which fast Fourier inverse transform processing is performed on an image signal of a photographed image edge-processed by the image recognition LSI 51 in order to detect a zebra pattern of a pedestrian crossing from a photographed image of the camera 60. . In the image recognition area indicated by reference numeral 62 in FIG. 5, the Vd pixel (dot) is the upper end from the upper end of the captured image 61 of the camera 60, and the lower part is the lower portion by Vh pixels. The horizontal width of the image recognition area 62 is 256 dots from the horizontal center of the captured image 61 to the left and right (a total of 512 dots).

  The reason why the horizontal width of the image recognition area 62 is 512 dots depends on the configuration of the image recognition LSI 51 of this embodiment shown in the explanatory diagram of FIG. As shown in FIG. 6, the image recognition LSI 51 controls 64 processor elements (PE0 to PE63) by a control processor (CP) and a floating point number arithmetic unit (FPU) (not shown) to perform floating point arithmetic. High speed processing. Each PE is connected to a RAM (not shown) that provides a work area for arithmetic processing.

  In this embodiment, the image recognition LSI 51 having the above-described configuration designates a real part and an imaginary part by 2 bytes in each of PE0 to PE63, and for one line of an image signal obtained by performing edge processing on a captured image of the camera 60, Fixed-point arithmetic processing by fast Fourier inverse transform (IxFFT) is executed in parallel. Data after inverse fast Fourier transform (IxFFT) by each PE0 to PE63 is stored in each corresponding RAM (not shown), and 256-byte data stored at addresses 0 to 255 is an effective data portion.

  In the address portion of 0 to 255 of each RAM corresponding to each of PE0 to PE63, data of a real part and an imaginary part of 2 bytes are stored in order of 1 to 256 frequencies. That is, as shown in the schematic diagram of FIG. 7, the imaginary part and the real part are arranged in the column direction by 64 PEs. Therefore, the imaginary part and the real part of each of PE0 to PE63 are combined in one column to calculate the average value Save and the period t of the amplitude spectrum S in each column.

  The amplitude spectrum S for each frequency in each column can be obtained by S = (real part / 256) 2 + (imaginary part / 256) 2. Further, the period t of each column can be obtained by the number of samples / frequency. Therefore, an appropriate amplitude spectrum intensity threshold Y is calculated and set from the combination of the average value Save of the amplitude spectrum S and the period t.

  FIG. 8 is a graph showing an example of the distribution of the amplitude spectrum for each period with respect to the intensity threshold obtained by the process of FIG. As shown in the graph of FIG. 8, the intensity threshold value Y of the amplitude spectrum S has a curved shape that decreases exponentially as the period t increases. If the period t in which the average value Save of the amplitude spectrum S greatly exceeds the intensity threshold Y is in the range of 0.9 m ± 25% (46 dots ± 11.5 dots) in the captured image 61 20 m ahead of the vehicle 1, A zebra pattern of the pedestrian crossing 70 exists in the image recognition area 62.

  In order to improve the efficiency of inverse fast Fourier transform (IxFFT) processing by each of PE0 to PE63 in FIG. 6, the image recognition LSI 51 of the present embodiment, as shown in the explanatory diagram of FIG. The image of the image recognition area 62 is edge-processed from the left side of FIG. 9 (from the front position side to the rear position side), with each line in the vertical direction rotated 90 ° clockwise (clockwise). Perform an inverse fast Fourier transform (IxFFT) process on the signal.

  The microcomputer 30 shown in FIG. 1 receives a result of inverse fast Fourier transform (IxFFT) processing on the image signal subjected to edge processing of the image recognition area 62 by the image recognition LSI 51, and a zebra pattern corresponding to the pedestrian crossing 70 in the image recognition area 62. Is detected. For this purpose, the microcomputer 30 includes a ROM 31 in which a program for detecting a zebra pattern corresponding to the pedestrian crossing 70 is stored, and a RAM 32 that provides a work area necessary for the microcomputer 30 to execute the program in the ROM 52. And are connected. The RAM 32 can be configured by, for example, an SDRAM (Synchronous Dynamic Random Access Memory).

  Next, an outline of processing related to detection of a zebra pattern corresponding to a pedestrian crossing 70 executed by the microcomputer 30 according to a program stored in the ROM 31 will be described with reference to a flowchart of FIG.

  First, the microcomputer 30 receives a signal input of a captured image 61 of the camera 60 and collects a video (captured image 61) (step S1), and performs edge processing on the collected video (captured image 61) (step S3). The edge-processed video (the image signal of the image recognition area 62 in the photographed image 61) is subjected to inverse Fourier transform processing by the image recognition LSI 51 as described above, so that the amplitude spectrum of the zebra pattern existing in the image recognition area 62 is obtained. An average value Save and a period t of S are obtained (step S5).

  Next, based on the comparison between the average value Save of the amplitude spectrum S and the intensity threshold Y, the microcomputer 30 selects a zebra pattern candidate corresponding to the pedestrian crossing 70 for the image signal subjected to the edge processing of the image recognition area 62 in FIG. Whether or not exists is determined (step S7).

  In this determination, the microcomputer 30 determines the y coordinate value and the rear end (the upper end of the image recognition area 62) of the front end (the lower end side of the image recognition area 62) of the zebra pattern obtained from the edge-processed image signal of the image recognition area 62 in FIG. Side), the y coordinate value of the front end (front position in FIG. 9) of the zebra pattern (y coordinate value in the image recognition area 62 in FIG. 5) and the rear end of the zebra pattern (rear position in FIG. 9). ) Y coordinate value (y coordinate value in the image recognition area 62 in FIG. 5).

  Then, when the pixel position difference in the y coordinate of the front and rear ends of the registered zebra pattern is an amount corresponding to 2.2 m or more in the captured image 61 20 m ahead of the vehicle 1, the microcomputer 30 selects the zebra pattern. The zebra pattern corresponding to the candidate for the pedestrian crossing 70 is determined to exist.

  If there is no zebra pattern corresponding to the candidate for the pedestrian crossing 70 (NO in step S7), the microcomputer 30 returns to step S1. If it exists (YES in step S7), it is determined that the zebra pattern existing in the image recognition area 62 is a zebra pattern corresponding to the candidate for the pedestrian crossing 70 (step S9). It is confirmed whether or not (step S11).

  If the determination that the zebra pattern corresponds to the candidate for the pedestrian crossing 70 is not continued twice (NO in step S11), the process returns to step S1, and if the determination is continued twice (YES in step S11). The zebra pattern corresponding to the pedestrian crossing 70 is determined (step S13).

  Then, as part of the processing of step S13, the microcomputer 30 sets the lines corresponding to the front and rear ends of the first zebra pattern determined as candidates for the pedestrian crossing 70 to the front end of the pedestrian crossing 70 (front side of FIG. 9). Position) y coordinate value (y coordinate value in the image recognition area 62 in FIG. 5) and y coordinate value (y position in the image recognition area 62 in FIG. 5) of the rear end of the pedestrian crossing 70 (rear position in FIG. 9). (Coordinate value) is registered in the RAM 32.

  The front end and the rear end of the zebra pattern determined as a candidate for the pedestrian crossing 70 are specified as follows. That is, for each line in the horizontal direction of the image signal subjected to the edge processing in the image recognition area 62, the zebra pattern having the periodicity of the pedestrian crossing 70 does not exist from the lower end side to the upper end side of the image recognition area 62. If the existing line continues twice, the y-coordinate value of the first line where the zebra pattern having the periodicity of the pedestrian crossing 70 exists is identified as the front end of the zebra pattern determined to be a candidate for the pedestrian crossing 70 Is done.

  Further, when a line that does not exist continues two times continuously from the lower end side to the upper end side of the image recognition area 62 and the zebra pattern having the periodicity of the pedestrian crossing 70 exists, the periodicity of the pedestrian crossing 70 is changed. The y coordinate value of the first line having no zebra pattern is specified as the rear end of the zebra pattern determined as a candidate for the pedestrian crossing 70.

  The y-coordinate values at the front and rear ends of the identified zebra pattern are registered in the RAM 32 as the front and rear ends of the pedestrian crossing 70. Further, a tracking label (see FIG. 12C) used in the tracking process of the pedestrian crossing 70 described later (see FIG. 13), the pedestrian crossing valid flag is “valid (◯)”, and the valid flag is “valid (○ ) ”, The current value of the takeover processing flag“ unconfirmed (identity unconfirmed) ”, the number of lost times“ 0 ”, and the final confirmation flag“ × (unconfirmed) ”) Coordinate values are registered.

  And the microcomputer 30 repeats the process of step S1 thru | or step S13 until the ignition switch (IGN) not shown turns off (it is YES at step S15).

  Next, the tracking process of the pedestrian crossing 70 executed by the microcomputer 30 according to the program stored in the ROM 31 will be described. As described above, an amount corresponding to 2.2 m or more in the captured image 61 20 m ahead of the vehicle 1 in the image recognition area 62 is determined to be a zebra pattern corresponding to a candidate for the pedestrian crossing 70. Therefore, there must be a pixel position difference in the y-coordinate at the front and rear ends of the zebra pattern.

  Therefore, as shown in the explanatory diagram of FIG. 11, when the front end of the zebra pattern does not exist in the image recognition area 62, the zebra pattern is determined to be a zebra pattern corresponding to a candidate for the pedestrian crossing 70 according to the above-described conditions. It becomes impossible. However, in the case of an image captured by the camera 60 of the traveling vehicle 1, there is a possibility that the front end of the zebra pattern does not exist temporarily in the image recognition area 62 due to the vibration of the vehicle 1 or the like.

  Therefore, when the front end of the zebra pattern no longer exists in the image recognition area 62, the zebra pattern and the image recognition area 62 before update, which has been determined to be a zebra pattern previously corresponding to the candidate for the pedestrian crossing 70, are used. It is the tracking process of the pedestrian crossing 70 that determines whether or not the identity of the zebra pattern existing in the above is recognized using a new condition.

  The identity of the zebra pattern is determined with reference to the rear end of the zebra pattern existing in each image recognition area 62 before and after the update. If the zebra pattern existing in each image recognition area 62 before the update is recognized as being identical, the zebra pattern corresponding to the pedestrian crossing 70 is obtained even if the front end of the zebra pattern does not exist in the image recognition area 62 after the update. If it does not exist in the image recognition area 62, it will not be immediately determined.

  In order to execute the tracking process of the pedestrian crossing 70 described above, the microcomputer 30 performs the processes shown in FIGS. 12A to 12C for the rear end of the zebra pattern determined as a candidate for the pedestrian crossing 70 (the rear position in FIG. 9). The flag and data having the contents shown in the explanatory diagram are managed as labels. Specifically, a pedestrian crossing valid flag (◯ = valid, x = invalid) indicating that it is the rear end of a zebra pattern determined to fall under the pedestrian crossing 70, and a pedestrian crossing 70 candidate are determined. A valid flag (flag ○ = candidate, flag × = not candidate) indicating the rear end of the zebra pattern is managed.

  In addition, the microcomputer 30 confirms the identity with the trailing edge of the zebra pattern that has existed in the past image recognition area 62 and indicates whether or not the inheritance processing flag has been taken over (unidentified unidentified, finished = (Identity confirmed), the number of lost times (count number) in which the trailing edge of the zebra pattern did not exist in the image recognition area 62 without the takeover process, and the lost confirmation flag (◯ = confirmed, x = unconfirmed).

  Further, the microcomputer 30 performs the y-coordinate value (recognition position Z) of the rear end of the zebra pattern before and after the update of the captured image 61 before and after the update of the captured image 61 by the update of the signal input from the camera 60 (previous and current). To manage the amount of travel of the vehicle 1 (vehicle travel distance). Each label is assigned an independent label number (label No.).

  In the tracking process of the pedestrian crossing 70, as shown in FIG. 13, the microcomputer 30 first sets the transfer process flag of all labels to “unchecked (identity not confirmed)” (step S21), and the pedestrian crossing valid flag is set. It is confirmed whether there is no “valid (◯)” label (step S23). If there is no label (YES in step S23), the current recognition rear end that can be recognized as the rear end of the zebra pattern is extracted from the updated image recognition region 62, and the y coordinate values are increased in ascending order (in the image recognition region 62). The extracted y-coordinate value of each rear end of current recognition is substituted into the current value of the label for each recognition position Z created (in order of increasing distance Zi from the lower end side) (step S25). Thereafter, the process proceeds to step S41 described later.

  Here, as shown in FIG. 12B, the contents of each label created by the microcomputer 30 in step S25 are “invalid (×)” for the pedestrian crossing valid flag, “invalid (×)” for the valid flag, and takeover. The processing flag is “unconfirmed (identity unconfirmed)”, the number of lost times is “0”, and the lost confirmation flag is “× (unconfirmed)”. Each label is numbered sequentially from the smallest available number. In step S25, when the y-coordinate value at the rear end of the current recognition is substituted for the current value of the label, the valid flag is changed from “invalid (×)” to “valid (◯)”, and the takeover processing flag is set to “not (identical). From “unconfirmed”) to “done (identity confirmed)”.

  Also, in step S23 of FIG. 13, when the pedestrian crossing valid flag has a label of “valid (◯)” (NO), the distance Zi from the lower end side of the image recognition area 62 is small in ascending order of the y-coordinate value. In order), the labels (label Nos. 1 and 2 shown in FIG. 12A) whose valid flag is “valid (◯)” and whose takeover process flag is “unidentified (identity unconfirmed)” are read ( Step S27).

  Subsequently, the microcomputer 30 checks whether or not all the y-coordinate values of the rear end recognized this time are assigned to the current value of any label (step S29), and if all are assigned (YES in step S29), The process proceeds to step S41. On the other hand, if all of the values have not been assigned (NO in step S29), the average values of the previous and current vehicle speeds ((( The amount of travel of the vehicle 1 (the vehicle travel distance) is calculated by multiplying the previous vehicle speed + current vehicle speed) / 2) by the time elapsed between the previous and current tracking processes (step S31).

  Then, the microcomputer 30 calculates the current recognized rear end from the y coordinate value of the rear end recognized last time, which is substituted as the previous value in the label read in step S27, and the own vehicle travel distance calculated in step S31. Y coordinate value (y coordinate value of the predicted rear end position) is determined (step S33). Furthermore, the minimum y-coordinate value of the current recognition rear end that is read out in step S25 and not substituted for the current value of any label is a predetermined allowable error range with respect to the y-coordinate value of the predicted rear end position determined in step S33. (Step S35).

  Note that, as shown in the explanatory diagram of FIG. 14, a range of equal intervals in the image recognition area 62 centered on the predicted rear end position 75 is an allowable error range 76. In step S <b> 35, it is confirmed whether or not the current recognition rear end 77 exists within the allowable error range 76.

  If present (YES in step S35), the microcomputer 30 performs in-range processing (step S37). If not present (NO in step S35), the microcomputer 30 performs out-of-range processing (step S39). In either case, the process returns to step S29.

  In the in-range processing in step S37, the y-coordinate value of the rear end of the current recognition is substituted for the current value of the label read in step S27, the number of lost times is reset to “0”, and the takeover processing flag is set to “completed ( Set to "Identity confirmed."

  In the out-of-range processing in step S39, the pedestrian crossing valid flag is set to “invalid (×) as shown in FIG. 12 (b) on the condition that it is not substituted for the current value of any label read in step S27. ”, Valid flag is“ invalid (×) ”, takeover processing flag is“ unidentified (identity unconfirmed) ”, the number of times is“ 0 ”, and the final confirmation flag is“ × (unconfirmed) ”. Is assigned as a label number. Then, as the current value of the created label, the y coordinate value at the rear end of the current recognition is substituted, the valid flag is changed from “invalid (×)” to “valid (O)”, and the takeover processing flag is set to “unidentified (identity not confirmed). ) ”To“ Done (identity confirmed) ”respectively.

  In step S41, the travel amount (vehicle travel distance) of the vehicle 1 is calculated in the same manner as in step S31. Subsequently, the previous value is assigned to a label in which the y-coordinate value of the current recognition rear end is not assigned to the current value. The y-coordinate value of the rear end of the current recognition in calculation (y-coordinate value of the predicted rear end position) is determined from the y-coordinate value of the rear end of the previous recognition and the own vehicle travel distance calculated in step S31. (Step S43).

  Further, it is confirmed whether or not the tracking label exists (step S45). If it does not exist (NO in step S45), the process proceeds to step S49 described later, and if it exists (YES in step S45). The microcomputer 30 has the y-coordinate value of the rear end of current recognition assigned to the current value of the tracking label within a predetermined allowable error range with respect to the y-coordinate value of the predicted rear end position determined in step S43. Whether or not (step S47).

  If it does not exist (NO in step S47), the microcomputer 30 proceeds to step S49, and if present (YES in step S47), the microcomputer 30 proceeds to step S51 described later.

  In step S49, the microcomputer 30 assigns the y-coordinate value of the predicted rear end position determined in step S43 as the y-coordinate value (current value) of the rear end recognized this time to the label to which the current value is not assigned. Then, the process proceeds to step S53. On the other hand, in step S51, the microcomputer 30 sets the y-coordinate value of the rear end of current recognition assigned to the current value of the tracking label to the y-coordinate value of the rear end of current recognition (ie, the label not assigned the current value). This value is substituted, and the process proceeds to step S53.

  In step S53, the microcomputer 30 confirms whether or not the y-coordinate value of the rear end of current recognition that is substituted for the current value of each label exists in the MASK area. Here, the MASK region is a region including a portion where the camera 60 becomes a blind spot due to the body of the vehicle 1 and a portion of the captured image 61 other than the image recognition region 62.

  If it exists in the MASK area (YES in step S53), it is determined whether or not the y coordinate position of the rear end recognized this time is a position that is more than 2 m ahead of the camera 60 in the traveling direction of the vehicle 1. Confirm (step S55). If the position is more than 2 m ahead of the camera 60 (YES in step S55), the process proceeds to step S61 described later. If not (NO in step S55), the process proceeds to step S63 described later. Transition.

  On the other hand, if it is not present in the MASK area in step S53 (NO), the number of lost times is incremented by “1” (step S57), and it is confirmed whether the number of lost times exceeds the threshold value N (step S59). If exceeded (YES in step S59), the microcomputer 30 proceeds to step S61. If not (NO in step S59), the microcomputer 30 proceeds to step S63.

  In step S61, for the label whose number of lost times exceeds the threshold value N, the number of lost times is reset to “0” and the valid flag is set to “invalid (×)”. In step S63, end processing is performed. In this end processing, the y-coordinate value (recognition position) of the rear end of the current recognition assigned to the current value for all labels with the valid flag “valid (◯)” and the takeover processing flag “completed (identity confirmed)”. Z) Compare them. When there are a plurality of labels in which the same value as the y-coordinate value at the rear end of the current recognition is assigned to each current value, the label with the smallest label number is used, and other labels have the valid flag “ Switch from ○ (valid) to “x (invalid)”.

  As is apparent from the above description, in the present embodiment, steps S1 to S5 in the flowchart of FIG. 10 are processing corresponding to the extraction means in the claims. Further, in the present embodiment, steps S7 to S13 in FIG. 10 are processes corresponding to the recognition means in the claims.

  Further, in the present embodiment, steps S23 to S39 and steps S53 to S55 in the flowchart of FIG. 13 are processing corresponding to the determination means in the claims.

  In the drive recorder 10 of the present embodiment configured as described above, the following operation is performed by the pedestrian crossing detection apparatus configured by the main decoder 20, the microcomputer 30, and the image recognition unit 50.

  That is, the image signal obtained by performing edge processing on the image recognition area 62 of the photographed image 61 from the camera 60 is inversely fast Fourier transformed by the image recognition LSI 51 to extract a zebra pattern having a period t corresponding to the pedestrian crossing 70 and a front and rear end interval. The zebra pattern corresponding to the pedestrian crossing 70 is specified. At this time, even if the vehicle 1 travels and the shape and size of the zebra pattern in the captured image 61 change, the amount of change can be limited to the change in the image recognition area 62. Therefore, when the period t of the zebra pattern and the interval between the front and rear ends are specified by the range of the number of pixels in the captured image 61, the range is limited to a certain range, and the pedestrian crossing 70 by image recognition without image processing is performed. Can be easily determined without imposing a burden on the microcomputer 30.

  In addition, even if the front end of the zebra pattern does not exist in the image recognition area 62 as the vehicle 1 travels, the rear ends of the zebra pattern determined to be the pedestrian crossing 70 in the image recognition area 62 before the update Based on the displacement amount and the traveling amount of the vehicle 1, it is determined whether or not the zebra pattern extracted in the updated image recognition region 62 and having no front end in the image recognition region 62 is the pedestrian crossing 70. it can.

  Furthermore, even if the trailing edge of the zebra pattern extracted from the image recognition area 62 before update does not exist in the updated image recognition area 62, the trailing edge of the zebra pattern estimated from the amount of progress of the vehicle 1 is obtained from the camera 60. Recognize that the zebra pattern estimated for the position of the rear end is identical to the zebra pattern corresponding to the pedestrian crossing 70 if it is 2m before (in front of the vehicle 1) or in front of it. Can do. Therefore, the existence of the pedestrian crossing 70 can be recognized until the vehicle 1 passes through the pedestrian crossing 70, and the information on the pedestrian crossing 70 can be used for travel control of the vehicle 1 and the like.

  In addition, since the strong periodicity of the zebra pattern is determined based on the average value Save of the amplitude spectrum S by the inverse fast Fourier transform performed by the image recognition LSI 51, it is determined whether the zebra pattern is a pedestrian crossing 70 or not. Thus, even if the preceding vehicles 2 and 3 exist on the zebra pattern of the pedestrian crossing 70, the pedestrian crossing 70 can be detected from the zebra pattern of the image processing area 62 with high accuracy.

  The present invention is suitable for use in an apparatus that detects a pedestrian crossing from an image captured by an in-vehicle camera.

DESCRIPTION OF SYMBOLS 1 Vehicle 2, 3 Advancing vehicle 10 Drive recorder 20 Main decoder 30 Microcomputer 31 ROM
32 RAM
40 SD card 50 Image recognition unit 51 Image recognition LSI
52 ROM
53 RAM
60 Camera 61 Photographed image 62 Image recognition area 70 Crosswalk 71 White line part 72 Non-white line part 73, 74 White line 75 Prediction rear end position 76 Allowable error range 77 Current recognition rear end S Amplitude spectrum Save Amplitude spectrum average value Y Intensity threshold Z recognition Position Zi Distance t Period

Claims (3)

Each time a captured image of the camera is updated by updating a signal input from an in-vehicle camera, a zebra with a predetermined cycle corresponding to a pedestrian crossing is obtained based on a spectrum pattern obtained by Fourier transform of a predetermined area of the captured image. Extracting means for extracting a pattern from the predetermined area;
The extracted zebra pattern is recognized as a pedestrian crossing when the front end close to the vehicle of the extracted zebra pattern and the rear end far from the vehicle are separated from each other by a predetermined interval or more according to a pedestrian crossing on the predetermined region. Recognizing means for recognizing that the zebra pattern is not a pedestrian crossing when the front and rear ends are not separated by more than the predetermined interval on the predetermined area;
A pedestrian crossing detection device comprising:   The extraction means does not extract the zebra pattern in which the front and rear ends are not separated by the predetermined interval or more from the predetermined area, and the zebra pattern in which the front end does not exist on the predetermined area is extracted from the predetermined area. The amount of displacement of the zebra pattern in which the front end does not exist on the predetermined region with respect to the zebra pattern recognized by the recognition means as a pedestrian crossing in the predetermined region at the rear end, Discriminating means for discriminating whether or not the zebra patterns extracted from the predetermined areas of the respective photographed images before and after the update are the same pedestrian crossing based on the amount of travel of the vehicle before and after the photographed image is updated. The pedestrian crossing detection apparatus according to claim 1, further comprising:   When the extraction unit does not extract the zebra pattern from the predetermined region, the determination unit is configured to extract a trailing end of the zebra pattern extracted by the extraction unit from the predetermined region of the previous captured image. The position of the rear end of the zebra pattern is estimated based on the amount of travel of the vehicle before and after the update of the captured image, and the zebra pattern of the estimated rear end position is a crosswalk based on the relative position with respect to the vehicle. The pedestrian crossing detection apparatus according to claim 2, wherein it is determined whether or not there is a pedestrian crossing.

Images