JP5513238B2 - Image processing system and image processing program - Google Patents

Description

  The present invention relates to an image processing system and an image processing program that can appropriately recognize a pedestrian from a photographed image.

  Detect the presence of a pedestrian located in the traveling direction of the vehicle from an image taken from a camera mounted at the head position of the vehicle, and display the position of the pedestrian on a display device in the vehicle, or the front An apparatus for projecting the position of a pedestrian on a window has been proposed.

  In addition to pedestrians, the captured images include other vehicles, traffic lights, signs, etc., so in order to recognize pedestrians, images are taken using an infrared camera, and the pixels in accordance with the amount of infrared radiation are captured. There has been proposed a technique for obtaining an image having a data value and specifying a pedestrian area therefrom. More specifically, the shape of a region showing a value larger than a certain threshold is specified in the captured image, and a pedestrian is detected based on the shape.

  However, simple shape recognition may emit infrared rays in the same way as a pedestrian and may recognize a relatively elongated object as a pedestrian. On the other hand, in the judgment based on the shape, when the pedestrians are lined up in the horizontal direction or when a part of the pedestrian is hidden by a shield such as a tree or a guardrail, the pedestrian may not be recognized. is there.

JP 2005-159392 A JP 2008-65463 A

  Patent Document 1 extracts a region having a near-infrared radiation amount greater than or equal to a certain value from an image photographed by a near-infrared camera, and further obtains a variance of data values of pixels included in the region. A technique for determining that the area corresponds to a pedestrian when the value is equal to or greater than a predetermined value is disclosed.

  In Patent Document 2, a pedestrian candidate area is extracted from an image taken by a near-infrared camera, and a value larger than a predetermined value is referred to by referring to a data value of a pixel included in the area. A technique is disclosed in which a pixel is counted, and when the ratio of the area of the pixel to the candidate area is smaller than a threshold, the area is determined to correspond to a pedestrian.

  As described above, even when pedestrians are lined up sideways or when a part of the pedestrian is hidden by a shield such as a tree or guardrail, the presence of the pedestrian is surely and quickly. A technology that can detect this is desired.

  An object of the present invention is to provide an image processing system and an image processing program that can reliably detect the presence of a pedestrian in a short processing time.

An object of the present invention is to accept image data including a data value of each pixel based on the temperature of an object to be photographed, which is captured by an infrared camera, and from the image data, each pixel is a multi-value having three or more values based on temperature. Multi-value image data acquisition means for acquiring multi-value image data having the data values of
In the multi-value image data, based on the data value of each pixel, a pedestrian candidate area extracting unit that extracts a pixel area indicating a predetermined temperature or more as a pedestrian candidate area;
By referring to the data value of the pixel included in the pedestrian candidate area, a histogram indicating the frequency ratio for each data value is generated, and based on the frequency ratio for each value, the pedestrian candidate area is First determination means for determining whether the pedestrian is applicable;
For the pedestrian candidate area determined to be a pedestrian by the first determining means, an outer frame area surrounding the outer edge is generated, and the outer frame area is divided into a plurality of blocks in the horizontal direction and the vertical direction. Then, based on the data value of the pixel of the pedestrian candidate area included in the block for each block, the index value indicating the change in the data value in the horizontal block and the change in the data value in the vertical block This is achieved by an image processing system comprising: second determination means for further determining whether the pedestrian candidate area corresponds to a pedestrian based on an index value to be indicated.

  In a preferred embodiment, the first determination means makes a determination based on whether the frequency ratio for each value is within a predetermined range.

  In a preferred embodiment, the second determination means calculates an average value of data values of pixels of a pedestrian candidate area included in the block for each block, and changes in the average value in the horizontal direction. Is calculated, and a second index value indicating a change in average value in the vertical direction is calculated.

In a more preferred embodiment, the data value of the multi-value image data increases as the temperature increases,
The second determination means is
When the difference between the maximum value and the minimum value of the average value is calculated as the first index value in the horizontal direction, and the first index value is within a predetermined range To determine that the first condition is met,
As the second index value, for the blocks adjacent in the vertical direction, the average value of the blocks positioned on the upper side−the average value of the blocks positioned on the lower side is calculated, and when the second index value is positive It is determined that the second condition is satisfied, and it is determined that the pedestrian candidate area corresponds to a pedestrian when the first condition and the second condition are satisfied.

In another preferred embodiment, the data value of the multi-valued image data increases as the temperature increases,
The second determination means is
As the first index value, a difference between average values of blocks adjacent in the horizontal direction is calculated, and when the first index value is within a predetermined range, the first condition is met. Judgment
As the second index value, for the blocks adjacent in the vertical direction, the average value of the blocks positioned on the upper side−the average value of the blocks positioned on the lower side is calculated, and when the second index value is positive Judge that the second condition is met,
When the first condition and the second condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian.

In a preferred embodiment, the second determination means is
For each block, calculate a frequency ratio for each pixel data value of the pedestrian candidate area included in the block, calculate a third index value indicating a change in the frequency ratio in the lateral direction, and A fourth index value indicating a change in the frequency ratio in the vertical direction is calculated.

In a more preferred embodiment, the second determination means is
As the third index value, a difference between the maximum value and the minimum value of the frequency change ratio for each value in the block adjacent in the horizontal direction is calculated, and the third index value is within a predetermined range. Is determined to meet the third condition,
As the fourth index value, the blocks adjacent to the longitudinal direction, and calculates the difference between the rate of change of frequency of each value in accordance with the value, when the index value of the fourth exhibits a predetermined same trends To the fourth condition,
When the third condition and the fourth condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian.

In another preferred embodiment, the second determination means is
As the third index value, in the block adjacent in the horizontal direction, the difference in the frequency change rate for each value is calculated, and when the third index value is within a predetermined range, Judging that the third condition is met,
As the fourth index value, the blocks adjacent to the longitudinal direction, and calculates the difference between the rate of change of frequency of each value in accordance with the value, when the index value of the fourth exhibits a predetermined same trends To the fourth condition,
When the third condition and the fourth condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian.

  Further, in a preferred embodiment, referring to the coordinates of the pixels constituting the pedestrian candidate area and the data value of the pixels, a center of gravity position is calculated for each data value, and based on the coordinates of the center of gravity position, Third determining means for further determining whether the pedestrian candidate area corresponds to a pedestrian is provided.

  In a more preferred embodiment, the third determination means determines that the pedestrian candidate area corresponds to a pedestrian when the x-coordinate of the center of gravity for each data value is adjacent within a predetermined range. To do.

  In another preferred embodiment, when the third determining means increases the y-coordinate of the center of gravity for each data value as the data value indicating that the temperature is high, the pedestrian candidate area Is considered a pedestrian.

Further, an object of the present invention is to accept image data including data values of each pixel based on the temperature of an object to be photographed, which is captured by an infrared camera, and from the image data, each pixel has three or more values based on temperature. A multi-value image data acquisition step for acquiring multi-value image data that is multi-value data values and increases in value as the temperature increases;
In the multi-value image data, based on the data value of each pixel, a pedestrian candidate area extraction step for extracting a pixel area indicating a predetermined temperature or more as a pedestrian candidate area;
By referring to the data value of the pixel included in the pedestrian candidate area, a histogram indicating the frequency ratio for each data value is generated, and based on the frequency ratio for each value, the pedestrian candidate area is A first determination step for determining whether the pedestrian is applicable;
For the pedestrian candidate area determined to be a pedestrian in the first determination step, an outer frame area surrounding the outer edge is generated, and the outer frame area is divided into a plurality of blocks in the horizontal direction and the vertical direction. Then, based on the data value of the pixel of the pedestrian candidate area included in the block for each block, the index value indicating the change in the data value in the horizontal block and the change in the data value in the vertical block And a second determination step of further determining whether or not the pedestrian candidate area corresponds to a pedestrian based on the indicated index value. This is achieved by an image processing program that causes a computer to execute the second determination step. .

  According to the present invention, it is possible to provide an image processing system and an image processing program capable of reliably detecting the presence of a pedestrian in a short processing time.

FIG. 1 is a block diagram showing an outline of an image processing system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of a pedestrian recognition processing unit according to the present embodiment. FIGS. 3A and 3B are diagrams illustrating examples of image data in a state where the processing by the pedestrian candidate area specifying unit 22 is executed. FIG. 4 is a flowchart illustrating an example of processing executed in the heat source body region extraction unit according to the present embodiment. FIG. 5 is a flowchart illustrating an example of a process (pedestrian candidate area specifying process) executed in the pedestrian candidate area specifying unit. FIG. 6 is a flowchart illustrating an example of the first determination process in the first determination processing unit according to the present embodiment. FIG. 7 is an example of a histogram for an actual pedestrian area and a histogram for an area other than a pedestrian (artifact) in the image data. FIG. 8 is a flowchart illustrating an example of the second determination process in the second determination processing unit according to the present embodiment. FIGS. 9A and 9B are diagrams illustrating examples of pedestrian candidate regions, outer frame regions, and blocks extracted in the image data. FIG. 10 is a flowchart illustrating an example of the third determination process by the third determination processing unit according to the present embodiment. FIG. 11 is a diagram illustrating an example of a pedestrian candidate region and a barycentric position for each value. 12A and 12B are diagrams illustrating examples of image data. FIG. 13 is a flowchart illustrating an example of the second determination process according to the second embodiment of the present invention. FIG. 14A is a diagram illustrating an example of an outer frame region and divided blocks, and FIG. 14B is a diagram illustrating an example of a histogram related to each block.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing an outline of an image processing system according to a first embodiment of the present invention. The image processing system 10 according to the present embodiment determines whether a pedestrian is captured in an image based on image data acquired by a far-infrared camera. As shown in FIG. 1, the image processing system 10 includes a far-infrared camera 11, an interface (I / F) 12, a CPU 13, an input unit 14, a display unit 15, a RAM 16, a ROM 17, and a pedestrian recognition processing unit 18. The I / F 12 to the pedestrian recognition processing unit 18 are connected to a bus. An image processing system 10 according to the present embodiment is mounted on a vehicle, takes an image of the traveling direction of the vehicle, and detects the presence or absence of a pedestrian based on the image data.

  The far-infrared camera 11 can detect a far-infrared region and obtain an image signal according to the amount of far-infrared radiation of the object to be photographed. For example, the image signal is composed of multi-value data values indicating the amount of far-infrared radiation received in each pixel for each pixel. Based on the image signal received from the far-infrared camera 11, the I / F 12 associates pixel position information (coordinates) with a data value, and stores them in a predetermined area of the RAM 16 as image data. The far-infrared camera 11 is, for example, disposed at the front end portion of the vehicle and can take an image of the front (the normal traveling direction of the vehicle).

  The CPU 13 controls the information processing system 10 as a whole, processes based on the switch operation of the input unit 14 by the operator, controls image display on the display unit 15, activation of the far-infrared camera 11, shooting instructions, and pedestrian recognition processing. An operation instruction to the unit 18 is executed. The input unit 14 includes switches (not shown) and gives an operation signal based on the switch operation to the CPU 13. The display unit 15 includes a display device (for example, a liquid crystal display device), and can display an image including an image taken by the far-infrared camera 11 on the screen.

  The RAM 16 stores image data based on the image signal photographed by the far-infrared camera 11, data generated by the processing, parameters, and calculation values. The ROM 17 controls the entire information processing system 10, processing based on the switch operation of the input unit 14 by the operator, control of image display on the display unit 15, activation of the far-infrared camera 11 and shooting instruction, pedestrian recognition processing A processing program such as an operation instruction for the unit 18 is stored.

  FIG. 2 is a block diagram showing an example of a pedestrian recognition processing unit according to the present embodiment. As shown in FIG. 2, the pedestrian recognition processing unit 18 includes a heat source body region extraction unit 21, a pedestrian candidate region specification unit 22, a pedestrian determination unit 23, and a display image generation unit 24.

  The heat source body region extraction unit 21 receives image data, and extracts a region having a predetermined temperature or more as a heat source body region based on the image data. The pedestrian candidate area specifying unit 22 further narrows down the pedestrian candidate area by making a predetermined determination from the heat source body area. The pedestrian determination unit 23 finally finds a pedestrian area from the pedestrian candidate areas by performing three types of determinations as described later. The display image generation unit 24 generates image data in which a frame indicating the finally recognized pedestrian area is superimposed on the image data, and outputs the generated image data to the display unit 15.

  In the present embodiment, the heat source region processing unit 21 includes a quaternization processing unit 31, a noise removal processing unit 43, and a labeling processing unit 33. The pedestrian determination unit 23 includes a first determination processing unit 34, a second determination processing unit 35, and a third determination processing unit 36. Processing executed in the pedestrian recognition processing unit 18 according to the present embodiment will be described in detail later.

  FIGS. 3A and 3B are diagrams illustrating examples of image data in a state where the processing by the pedestrian candidate area specifying unit 22 is executed. For example, in FIG. 3A, the image data 300 includes three pedestrian candidate areas 301 to 303. In the actual image, the pedestrian candidate areas 301 and 302 correspond to a single pedestrian, while the pedestrian candidate area 302 corresponds to a state in which a plurality of pedestrians are arranged side by side (in parallel). . The pedestrian candidate area 302 is simply determined by the aspect ratio of the frame (outer frame) that forms the outline of the pedestrian candidate area 302. For example, the size (length) in the vertical direction is the size in the horizontal direction like a normal pedestrian. Since it is not established that it is sufficiently larger than (width), there is a high possibility that it is not determined to be a pedestrian.

  In FIG. 3B, the image data 310 includes two pedestrian candidate areas 311 and 312. Actually, both of them indicate pedestrians, but the lower half is blocked by a shield such as a wall or guardrail and is not included in the candidate pedestrian area. In the present embodiment, a pedestrian candidate area where a plurality of pedestrians are arranged (or overlapped) as shown in FIG. 3A, or a pedestrian as shown in FIG. The pedestrian candidate area in which a part of the outline of the pedestrian is missing can be appropriately recognized as a pedestrian.

  FIG. 4 is a flowchart illustrating an example of processing executed in the heat source body region extraction unit according to the present embodiment. As shown in FIG. 4, the quaternarization processing unit 31 of the heat source body region extraction unit 21 quaternizes the data values of each pixel constituting the image data, and is quaternized with the data values of new pixels. The generated image data is generated (step 401). The data value of the new pixel is also stored in a predetermined area of the RAM 16.

The data value of the pixels constituting the image data is, for example, multi-value data indicating any value from 0 to 127 with 8 bits. In this embodiment, the value indicating the lowest temperature is “0”, and the data value increases as the temperature increases. The quaternarization processing unit 31 uses threshold values Pth1, Pth2, and Pth3 (Pth1 <Pth2 <Pth3),
0 ≦ pixel data value <Pth1: first value (lowest value) v1
Pth1 ≦ pixel data value <Pth2: second value v2
Pth2 ≦ pixel data value <Pth3: third value v3
Pth3 ≦ pixel data value: fourth value (maximum value) v4
(However, v1 <v2 <v3 <v4)
A new data value for each pixel may be determined so that Hereinafter, for convenience, the first value v1, the second value v2, the third value v3, and the fourth value v4 are set to “0”, “1”, “2”, and “3”, respectively. The second value v2 is also referred to as “low temperature value”, the third value v3 is also referred to as “medium temperature value”, and the fourth value v4 is also referred to as “high temperature value”.

  Of course, the data value is not limited to that described above, and the value indicating the highest temperature is “0”, and the data value may increase as the temperature decreases. Needless to say.

  The noise removal unit 43 sets the second value (low temperature value) to the fourth value (high temperature value) except for the pixel indicating the lowest value (that is, the lowest temperature) in the four-valued image data. A region composed of pixels having the same is extracted as an initial heat source region candidate (step 402). Further, the noise removing unit 43 excludes an area having an extremely small number of pixels (for example, an area where the number of pixels is smaller than a predetermined lower limit Pmin) from the heat source area from the initial heat source area candidates (step 403). At the same time, a region having an extremely large number of pixels (for example, a region where the number of pixels exceeds a predetermined upper limit value Pmax) is excluded from the heat source region candidates (step 404).

  Next, the labeling processing unit 33 gives the same label to the pixels belonging to the same heat source region by labeling (step 405). As a result of labeling, the RAM 16 stores coordinate values (x-coordinate, y-coordinate) indicating pixel positions, data values indicating either low-temperature values to high-temperature values, and pixel data including labels.

  FIG. 5 is a flowchart illustrating an example of a process (pedestrian candidate area specifying process) executed in the pedestrian candidate area specifying unit. As shown in FIG. 5, the pedestrian candidate area specifying unit 22 refers to the x coordinate and y coordinate of the pixels constituting the heat source body area, and the x coordinate minimum value xmin, maximum value xmax, and y coordinate minimum value. ymin and the maximum value ymax are acquired, and the outer frame region of the heat source region defined by the four points (xmin, ymin), (xmin, ymax), (xmax, ymin) and (xmax, ymax) is specified (Step 501). In step 501, for each heat source body region, a rectangular outer frame region in contact with the heat source body region is obtained.

  Next, the pedestrian candidate area specifying unit 22 considers the number of pixels constituting the image data in the vertical direction and the horizontal direction, and the focal length of the lens of the far-infrared camera 11, and the width of the outer frame area is a predetermined value. The larger heat source body region is excluded from the pedestrian candidate region (step 502). For example, the actual size of the outer frame region can be grasped from the number of pixels and the focal length of the lens. Therefore, an outer frame region corresponding to a width that is not normally considered even if pedestrians are arranged side by side (for example, several meters or more) is excluded from the pedestrian candidate regions. Of course, the same judgment may be made for the vertical direction.

  The pedestrian candidate area specifying unit 22 excludes, from the pedestrian candidate area, heat source body areas in which the aspect ratio (vertical length / width) of the outer diameter area is smaller than a predetermined value (step 503). This is based on the judgment that the possibility that the horizontal width is several times the vertical length is small even if the pedestrians are arranged side by side or even if a part of the pedestrians is hidden.

Moreover, the pedestrian candidate area | region specific | specification part 22 calculates the coordinate of the gravity center position of each heat source body area | region, and the coordinate of a center position (step 504). The x-coordinate and y-coordinate of the gravity center position of the heat source body region can be calculated as follows.
X-coordinate of the center of gravity position = (1 / N) · Σ (x-coordinate of pixels included in the heat source region)
Y coordinate of the center of gravity position = (1 / N) · Σ (y coordinate of the pixel included in the heat source region)
N is the number of pixels constituting the heat source region.

Further, the x coordinate and the y coordinate of the center position of the heat source body region can be calculated as follows.
X coordinate of center position = (1/2) · (xmin + xmax)
Center position y coordinate = (1/2) · (ymin + ymax)
The pedestrian candidate region specifying unit 22 determines that the difference between the x-coordinate of the center of gravity position and the x-coordinate of the center position is outside a predetermined range, that is, | (x coordinate of the center of gravity position) − (x coordinate of the center position) | The heat source body region having a value is excluded from the pedestrian candidate regions (step 505). Further, the pedestrian candidate region specifying unit 22 selects a heat source region where the y coordinate of the center of gravity is below the y coordinate of the center, that is, (y coordinate of the center of gravity) <(y coordinate of the center). The pedestrian candidate area is excluded (step 506). As a result, the heat source region that has not been excluded becomes a pedestrian candidate region, and information on the pedestrian candidate region stored in the RAM 16 is referred to in the next processing by the pedestrian determination unit 23.

  In the pedestrian candidate area specifying process according to the present embodiment, based on the width of the outer diameter area, the aspect ratio of the outer diameter area, the x coordinate of the centroid position and the center position, and the y coordinate of the centroid position and the center position. Thus, the pedestrian candidate area is narrowed down from the heat source body area. However, the predetermined value and the predetermined range for narrowing down may not be very sensitive. That is, in the pedestrian candidate area specifying process, it is not necessary to exclude many heat source body areas. Moreover, it is not necessary to perform all the above-mentioned narrowing down. In the present embodiment, the process performed in the pedestrian determination unit 23 to be executed next is the main process for pedestrian recognition, and the pedestrian candidate area specifying process is a pre-process. Therefore, in the pedestrian candidate area specifying process, it is not necessary to narrow down the pedestrian candidate areas very strictly.

  Next, processing in the pedestrian determination unit 23 will be described. FIG. 6 is a flowchart illustrating an example of the first determination process in the first determination processing unit according to the present embodiment. As illustrated in FIG. 6, the first determination processing unit 34 acquires information on a pedestrian candidate area to be processed from the RAM 16 (step 601). The first determination processing unit 34 generates a histogram for each value based on the pixel data value in the pedestrian candidate area (step 602). As described above, the pixel data value is one of a low temperature value, a medium temperature value, and a high temperature value. For example, in the histogram, the ratio of the number of pixels having a low temperature value, a medium temperature value, and a high temperature value is shown.

  The first determination processing unit 34 determines whether the ratio of the number of pixels of each value is within a predetermined range (step 603). FIG. 7 is an example of a histogram for an actual pedestrian area and a histogram for an area other than a pedestrian (artifact) in the image data. In FIG. 7, reference numerals 701 to 703 indicate histograms for the pedestrian area, and reference numerals 711 to 713 indicate histograms for the artifact area.

  As shown in FIG. 7, the artificial region tends to be biased to a specific value as compared to the pedestrian region. In the present embodiment, a pedestrian region is actually cut out from a large number of image data in advance, and a ratio between a low temperature value and a high temperature value is obtained, and a range of each value in the pedestrian region is acquired. The acquired value range is stored in the RAM 16. In FIG. 7, ranges indicated by reference numerals 721, 722, and 723 are ranges of high temperature values, medium temperature values, and low temperature values that are considered to be pedestrian regions, respectively.

  The first determination processing unit 34 determines whether the ratio of the number of pixels of each value is within a predetermined range, and if all are within the range (Yes in Step 604), walking in the RAM 16 Information on the candidate area is left (step 605). This may be performed without deleting the information on the candidate pedestrian area in the RAM 16. On the other hand, when it is determined No in step 604, the first determination processing unit 34 deletes the information on the pedestrian candidate area in the RAM 16 (step 606). Thereafter, the first determination processing unit 34 refers to the information on the pedestrian candidate areas in the RAM 16 and determines whether the process has been completed for all the pedestrian candidate areas (step 607). If NO is determined in step 607, the process returns to step 601. On the other hand, if it is determined Yes in step 607, the first determination process is terminated.

  Next, the second determination processing unit 35 executes a second determination process. FIG. 8 is a flowchart illustrating an example of the second determination process in the second determination processing unit according to the present embodiment. As illustrated in FIG. 8, the second determination processing unit 35 acquires information on a pedestrian candidate area to be processed from the RAM 16 (step 601). The second determination processing unit 35 refers to the x-coordinate and y-coordinate of each pixel in the pedestrian candidate area information, and the x-coordinate minimum value xmin, the maximum value xmax, and the y-coordinate minimum value in the pedestrian candidate area. ymin, the maximum value ymax is acquired, and the outer frame area of the pedestrian candidate area defined by the four points (xmin, ymin), (xmin, ymax), (xmax, ymin) and (xmax, ymax) is specified (Step 802). This is substantially the same processing as step 501 in FIG. By step 802, a rectangular outer frame area in contact with the pedestrian candidate area is obtained.

  Next, the second determination processing unit 35 divides the outer frame area into a plurality of blocks, and specifies pixels belonging to each block (step 803). The number of blocks may be 3 × 3 = 9, for example. Or according to the size and aspect ratio of an outer frame area | region, it is good also as changeable in the range of a predetermined range (for example, 2-4) both in length and width. Hereinafter, a case where the number of blocks is 9 will be described. FIGS. 9A and 9B are diagrams illustrating examples of pedestrian candidate regions, outer frame regions, and blocks extracted in the image data. In FIG. 9A, the original image data 900 is overlaid with a curve 911 indicating the outer edge of the pedestrian candidate area, and a straight line 910 defining the outer frame and the block.

  The second determination processing unit 35 calculates the average value of the data values of the pixels constituting the pedestrian candidate area included in the block for each block (step 804). In FIG. 9B, for example, the average value of the pixel data values of the pedestrian candidate area 921 included in the upper left block and the data value of the pixels of the pedestrian candidate area 922 included in the upper central block. An average value is calculated.

  Next, the second determination processing unit 35 acquires an index value indicating a change in the horizontal direction of the average value of each block (step 805). 9B, for example, each of the pedestrian candidate area 921 included in the upper left block, the pedestrian candidate area 922 included in the upper central block, and the pedestrian candidate area 923 included in the upper right block. Then, an index value (first index value) indicating a change in average value is acquired. Similarly, an index value indicating a change in average value is acquired for the middle and lower stages. In the horizontal direction, for example, an absolute value (| maximum value−minimum value |) of the difference between the maximum value and the minimum value of the average value can be used as the index value indicating the change of the average value.

  Further, the second determination processing unit 35 acquires an index value indicating a change in the average value of each block in the vertical direction (step 806). In FIG. 9B, for example, each of the pedestrian candidate area 921 included in the upper left block, the pedestrian candidate area 931 included in the middle left block, and the pedestrian candidate area 932 included in the lower left block. Then, an index value (second index value) indicating a change in average value is acquired. Similarly, an index value indicating a change in average value is acquired for the center column and the right column. For the vertical direction, as an index value indicating a change in average value, (average value for the block located on the upper side) − (average value for the block located on the lower side) can be used.

  In the second determination processing unit 35, the first index value indicating the change in the average value in the horizontal direction in each of the upper, middle, and lower stages matches the predetermined first condition, and the left column, the center It is determined whether the second index value indicating the change in the average value in the vertical direction of each of the columns and the right column meets a predetermined second condition (step 807).

  In the present embodiment, for example, the first condition is that the first index value is smaller than a predetermined threshold value. That is, in the horizontal direction, the condition is that the change in average value is smaller than a predetermined value, that is, the average value is substantially constant. The second condition is that the second index value is a positive value. That is, in the vertical direction, the condition is that the average value decreases from top to bottom (temperature tends to decrease from top to bottom).

  When it is determined Yes in step 807, the second determination processing unit 35 leaves information on the pedestrian candidate area in the RAM 16 (step 808). On the other hand, when it is determined No in step 807, the second determination processing unit 35 deletes the information on the pedestrian candidate area in the RAM 16 (step 809). Thereafter, the second determination processing unit 35 refers to the information on the pedestrian candidate areas in the RAM 16 and determines whether the process has been completed for all the pedestrian candidate areas (step 810). If NO is determined in step 810, the process returns to step 801. On the other hand, if it is determined Yes in step 810, the second determination process is terminated.

Next, the third determination processing unit 36 executes a third determination process. FIG. 10 is a flowchart illustrating an example of the third determination process by the third determination processing unit according to the present embodiment. As illustrated in FIG. 10, the third determination processing unit 36 acquires information on a pedestrian candidate area to be processed from the RAM 16 (step 1001). The third determination processing unit 36 refers to the information on the pedestrian candidate area, and acquires the coordinates of the center of gravity position for each value, that is, for each of the high temperature value, the medium temperature value, and the low temperature value (step 1002). The x coordinate and y coordinate of M pixels having the same value can be obtained as follows.
X coordinate of centroid position = (1 / M) · Σ (x coordinate of pixels having the same value)
Y coordinate of the center of gravity position = (1 / M) · Σ (y coordinate of pixels having the same value)
The third determination processing unit 36 compares the x-coordinates of the centroid positions for each value (step 1003),
It is determined whether the x-coordinates of the centroid positions of the respective values are adjacent within a predetermined range (step 1004). In step 1004, for example, the absolute value of the difference between the maximum value of the x coordinate and the minimum value of the x coordinate (| the maximum value of the x coordinate−the minimum value of the x coordinate |) is calculated, and the absolute value of the difference is predetermined. It is judged whether it is smaller than the threshold value. When it is determined Yes in step 1004, the third determination processing unit compares the y-coordinates of the centroid positions for the respective values (step 1005), and the y-coordinate becomes more as the value becomes higher. It is determined whether it is larger (step 1006). In step 1006, it is determined whether (y coordinate for high temperature value)> (y coordinate for medium temperature value)> (y coordinate for low temperature value) is satisfied.

  If it is determined Yes in step 1006, the third determination processing unit 36 leaves information on the pedestrian candidate area in the RAM 16 (step 1007). On the other hand, if it is determined No in step 1004 or if it is determined No in step 1006, the third determination processing unit 36 deletes the information on the pedestrian candidate area in the RAM 16 (step 1008). ). Thereafter, the third determination processing unit 36 refers to the information on the pedestrian candidate areas in the RAM 16 and determines whether the process has been completed for all the pedestrian candidate areas (step 1009). If NO in step 1009, the process returns to step 1001.

  When it is determined Yes in step 1009, the third determination processing unit 36 uses information on the pedestrian candidate area stored in the RAM 16 as final pedestrian area information (step 1010). In step 1010, information on the pedestrian candidate area may be newly stored in the RAM 16 as information on the pedestrian area, or a pedestrian area confirmation flag is provided in the information on the pedestrian candidate area to confirm the pedestrian area. The flag may be set to “1”.

  FIG. 11 is a diagram illustrating an example of a pedestrian candidate region and a barycentric position for each value. In FIG. 11, the centroid position for the high temperature value, the centroid position for the medium temperature value, and the centroid position for the low temperature value of the pedestrian candidate area 1100 are indicated by reference numerals 1101, 1102, and 1103, respectively. The coordinates of the centroid position for the high temperature value are (X4, Y4), the coordinates of the centroid position for the medium temperature value are (X3, Y3), and the coordinates of the centroid position for the low temperature value are (X2, Y2).

  In step 1004, the maximum value X3 of the x coordinate and the minimum value X2 are compared, and it is determined that X3-X2 is smaller than a predetermined threshold value Xth. In step 1006, the y-coordinates are compared, and it is determined that Y4> Y3> Y2 holds. Therefore, the pedestrian candidate area 1100 shown in FIG. 11 is determined as a pedestrian area because the condition for the center of gravity position is satisfied.

  When the processing in the pedestrian determination unit 23 is completed, the display image generation unit 24 acquires information on the pedestrian area from the RAM 16, adds a frame surrounding the pedestrian area to the image data, and adds the frame. The image data is output to the display unit 15. Thereby, on the screen of the display part 15, the image which surrounded and highlighted the pedestrian area | region is displayed.

  12A and 12B are diagrams illustrating examples of image data. In FIG. 12A, in an image 1200, there are a plurality of pedestrian candidate areas (see, for example, reference numerals 1202, 1202, and 1203). As a result of the processing, it is determined that the pedestrian candidate area 1201 is a pedestrian area, and a pedestrian area frame 1211 is added to the image 1210 as shown in FIG.

  Next, a second embodiment of the present invention will be described. In the first embodiment, in the second determination process, the outer frame area is divided into a plurality of blocks, and the average value of the data values of the pixels constituting the pedestrian candidate area included in each block is calculated. The pedestrian candidate area is narrowed down by looking at the value trends. In the second embodiment, the temperature distribution of each block is acquired, and the pedestrian candidate area is narrowed down from the tendency of the temperature distribution. The processing by the heat source body region extraction unit 21 and the processing by the pedestrian candidate region specification unit 22 are the same as those in the first embodiment. The processing in the first determination processing unit 34 and the third determination processing unit 26 in the pedestrian determination unit 23 is the same as that in the first embodiment.

  FIG. 13 is a flowchart illustrating an example of the second determination process according to the second embodiment of the present invention. In FIG. 13, steps 1301 to 1303 are the same as those in the first embodiment. After step 1303, in each block, the second determination processing unit 35 generates a histogram for each value based on the data value of the pixel that calibrates the pedestrian candidate area included in the block (step 1304). Since the data value of the pixel is one of a low temperature value, a medium temperature value, and a high temperature value, for example, the histogram shows the ratio of the number of pixels having a low temperature value, a medium temperature value, and a high temperature value. In the present embodiment, since the outer frame area is divided into nine blocks, nine histograms are generated.

  Next, the second determination processing unit 35 acquires an index value (third index value) indicating a change in the frequency ratio of the horizontal blocks (step 1305). Further, the second determination processing unit 35 acquires an index value (fourth index value) indicating a change in the frequency ratio of the vertical blocks (step 1306). The second determination unit 35 determines whether all of the third index values are in accordance with a predetermined third condition and whether all of the fourth index values are in accordance with a predetermined fourth condition ( Step 1307).

  FIG. 14A is a diagram illustrating an example of an outer frame region and divided blocks, and FIG. 14B is a diagram illustrating an example of a histogram related to each block. 14B, among the blocks A11 to A13 obtained by dividing the outer frame region 1400 into nine as shown in FIG. 14A, A11 to A13 (reference numerals 1411 to 1413), A21 and A31 (reference numeral 1421). , 1431).

For example, as the third index value, the absolute value of the difference between the maximum value and the minimum value of the frequency ratio can be used for each value in the horizontal direction. In the example of FIG. 14B, the following values are calculated as absolute values of the difference between the maximum value and the minimum value of the frequency ratio for the upper block.
| V4 (A11) -V4 (A12) |
| V3 (A13) -V3 (A11) |
| V2 (A13) -V3 (A12) |
Regarding the horizontal direction, when the absolute value of the difference (third index value) is smaller than a predetermined threshold, it is determined that the third condition is being followed. Similarly, the third index value is calculated for the middle stage and the lower stage, and it is determined whether it is smaller than a predetermined threshold value.

As the fourth index value, (the ratio of the frequency of the block positioned on the upper side) − (the ratio of the frequency positioned on the lower side) can be used for the high temperature value and the low temperature value of the block adjacent in the vertical direction. . In the example of FIG. 14B, the following values are calculated as (the ratio of the frequency of the block positioned on the upper side) − (the ratio of the frequency positioned on the lower side) for the block in the left column.
V4 (A11) -V4 (A21)
V2 (A11) -V2 (A21)
V4 (A21) -V4 (A31)
V2 (A21) -V2 (A31)
Regarding the vertical direction, the high temperature value V4 decreases from the top to the bottom, and the low sound value V2 is determined to comply with the fourth condition when rising from the top to the bottom. In other words, in the left column block,
V4 (A11) -V4 (A21)> 0, V4 (A21) -V4 (A31)> 0, and
The fourth condition is that V2 (A11) −V2 (A21) <0 and V2 (A21) −V2 (A31) <0.

  If it is determined Yes in step 1307, the second determination processing unit 35 leaves information on the pedestrian candidate area in the RAM 16 (step 1308). On the other hand, if it is determined No in step 1307, the second determination processing unit 35 deletes the information on the pedestrian candidate area in the RAM 16 (step 1309). Thereafter, the second determination processing unit 35 refers to the information on the pedestrian candidate areas in the RAM 16 and determines whether the process has been completed for all the pedestrian candidate areas (step 1310). If it is determined No in step 1310, the process returns to step 1301. On the other hand, if it is determined Yes in step 1310, the second determination process is terminated.

  The present invention is not limited to the above embodiments, and various modifications can be made within the scope of the invention described in the claims, and these are also included in the scope of the present invention. Needless to say.

  For example, in the second determination process of the first embodiment, the absolute value of the difference between the maximum value and the minimum value of the average value is used as the first index value indicating the change in the horizontal direction. The absolute value of the difference between the average values of adjacent blocks may be used. In this case, the first condition is that all the absolute values of the difference between the average values are smaller than a predetermined threshold value.

In the second determination process according to the second embodiment, the absolute value of the difference between the maximum value and the minimum value of the frequency ratio is used as the third index value indicating the change in the horizontal direction. However, the present invention is not limited to this. For example, as the third index value, the absolute value of the frequency ratio difference between adjacent values can be used for each value of the adjacent blocks in the horizontal direction. In the example of FIG. 14B, the following values are calculated as the absolute value of the frequency ratio difference between adjacent values for the upper block.
| V4 (A11) -V4 (A12) |
| V3 (A11) -V3 (A12) |
| V2 (A11) -V3 (A12) |
| V4 (A12) -V4 (A13) |
| V3 (A12) -V3 (A13) |
| V2 (A12) -V3 (A13) |
In this example, in the horizontal direction, when the absolute value of the difference is smaller than a predetermined threshold, it is determined that the third condition is being followed. Similarly, the third index value is calculated for the middle stage and the lower stage, and it is determined whether it is smaller than a predetermined threshold value.

  In the present invention, the image data captured by the far-infrared camera is converted into four values to obtain new four-valued image data. However, the present invention is not limited to this and is converted into three values. New image data or new image data having a value of five or more values may be generated, and the new image data may be used for processing.

DESCRIPTION OF SYMBOLS 10 Image processing system 11 Far-infrared camera 12 Interface 13 CPU
14 Input unit 15 Display unit 16 RAM
17 ROM
DESCRIPTION OF SYMBOLS 18 Pedestrian recognition part 21 Heat source body area | region extraction part 22 Pedestrian candidate area | region specific | specification part 23 Pedestrian determination part 31 Four-valued process part 32 Noise removal process part 33 Labeling process part 34 1st determination process part 35 2nd determination process part 36 Third determination processing unit

Claims (12)

Accepting image data captured by an infrared camera and including data values of each pixel based on the temperature of the object to be imaged, and from the image data, each pixel has a multivalued data value of three or more values based on the temperature; Multi-value image data acquisition means for acquiring multi-value image data;
In the multi-value image data, based on the data value of each pixel, a pedestrian candidate area extracting unit that extracts a pixel area indicating a predetermined temperature or more as a pedestrian candidate area;
By referring to the data value of the pixel included in the pedestrian candidate area, a histogram indicating the frequency ratio for each data value is generated, and based on the frequency ratio for each value, the pedestrian candidate area is First determination means for determining whether the pedestrian is applicable;
For the pedestrian candidate area determined to be a pedestrian by the first determining means, an outer frame area surrounding the outer edge is generated, and the outer frame area is divided into a plurality of blocks in the horizontal direction and the vertical direction. Then, based on the data value of the pixel of the pedestrian candidate area included in the block for each block, the index value indicating the change in the data value in the horizontal block and the change in the data value in the vertical block An image processing system comprising: a second determination unit that further determines whether the pedestrian candidate area corresponds to a pedestrian based on an index value indicated.   The image processing system according to claim 1, wherein the first determination unit makes a determination based on whether the ratio of the frequency for each value is within a predetermined range.   The second determination means calculates, for each block, an average value of data values of pixels of a pedestrian scavenging area included in the block, and a first index value indicating a change in the average value in the horizontal direction The image processing system according to claim 1, wherein a second index value indicating a change in average value in the vertical direction is calculated. The data value of the multi-value image data increases as the temperature increases,
The second determination means is
When the difference between the maximum value and the minimum value of the average value is calculated as the first index value in the horizontal direction, and the first index value is within a predetermined range To determine that the first condition is met,
As the second index value, for the blocks adjacent in the vertical direction, the average value of the blocks located on the upper side−the average value of the blocks located on the lower side is calculated, and when the second index value is positive 4. The method according to claim 3, wherein the pedestrian candidate area is determined to correspond to a pedestrian when the second condition is determined and the first condition and the second condition are satisfied. 5. Image processing system. The data value of the multi-value image data increases as the temperature increases,
The second determination means is
As the first index value, a difference between average values of blocks adjacent in the horizontal direction is calculated, and when the first index value is within a predetermined range, the first condition is met. Judgment
As the second index value, for the blocks adjacent in the vertical direction, the average value of the blocks located on the upper side−the average value of the blocks located on the lower side is calculated, and when the second index value is positive Judge that the second condition is met,
4. The image processing system according to claim 3, wherein when the first condition and the second condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian. The second determination means is
For each block, calculate a frequency ratio for each pixel data value of the pedestrian candidate area included in the block, calculate a third index value indicating a change in the frequency ratio in the lateral direction, and The image processing system according to claim 1, wherein a fourth index value indicating a change in the frequency ratio in the vertical direction is calculated. The second determination means is
As the third index value, a difference between the maximum value and the minimum value of the frequency change ratio for each value in the block adjacent in the horizontal direction is calculated, and the third index value is within a predetermined range. Is determined to meet the third condition,
As the fourth index value, the blocks adjacent to the longitudinal direction, and calculates the difference between the rate of change of frequency of each value in accordance with the value, when the index value of the fourth exhibits a predetermined same trends To the fourth condition,
The image processing system according to claim 6, wherein when the third condition and the fourth condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian. The second determination means is
As the third index value, in the block adjacent in the horizontal direction, the difference in the frequency change rate for each value is calculated, and when the third index value is within a predetermined range, Judging that the third condition is met,
As the fourth index value, the blocks adjacent to the longitudinal direction, and calculates the difference between the rate of change of frequency of each value in accordance with the value, when the index value of the fourth exhibits a predetermined same trends To the fourth condition,
The image processing system according to claim 6, wherein when the third condition and the fourth condition are met, it is determined that the pedestrian candidate area corresponds to a pedestrian.   Referring to the coordinates of the pixels constituting the pedestrian candidate area and the data value of the pixel, a centroid position for each data value is calculated, and the pedestrian candidate area is determined to be a pedestrian based on the coordinates of the centroid position. The image processing system according to any one of claims 1 to 8, further comprising a third determination unit that further determines whether or not it is applicable.   The third determining means determines that the pedestrian candidate area corresponds to a pedestrian when the x-coordinate of the center of gravity for each data value is adjacent within a predetermined range. Item 10. The image processing system according to Item 9.   The third determining means determines that the pedestrian candidate area corresponds to a pedestrian when the y-coordinate of the center of gravity for each data value increases as the data value indicates that the temperature is high. The image processing system according to claim 9 or 10, wherein Accepting image data captured by an infrared camera and including data values of each pixel based on the temperature of the object to be imaged, each pixel having a multi-value data value of three or more values based on the temperature. A multi-value image data acquisition step for acquiring multi-value image data whose value increases as the temperature increases;
In the multi-value image data, based on the data value of each pixel, a pedestrian candidate area extraction step for extracting a pixel area indicating a predetermined temperature or more as a pedestrian candidate area;
By referring to the data value of the pixel included in the pedestrian candidate area, a histogram indicating the frequency ratio for each data value is generated, and based on the frequency ratio for each value, the pedestrian candidate area is A first determination step for determining whether the pedestrian is applicable;
For the pedestrian candidate area determined to be a pedestrian in the first determination step, an outer frame area surrounding the outer edge is generated, and the outer frame area is divided into a plurality of blocks in the horizontal direction and the vertical direction. Then, based on the data value of the pixel of the pedestrian candidate area included in the block for each block, the index value indicating the change in the data value in the horizontal block and the change in the data value in the vertical block An image processing program for causing a computer to execute a second determination step of further determining whether or not the pedestrian candidate area corresponds to a pedestrian based on an index value to be indicated.

Images