JP5512697B2 - TRAVEL IMAGE DATA PROCESSING DEVICE AND METHOD, COMPUTER PROGRAM FOR PROCESSING TRAVEL IMAGE DATA, AND RECORDING MEDIUM CONTAINING THE COMPUTER PROGRAM - Google Patents

Description

  The present invention relates to a traveling image data processing apparatus and method.

In recent years, in-vehicle cameras mounted on vehicles are becoming widespread for purposes such as back monitors and drive recorders. As a result, video information captured by an in-vehicle camera has been used for various purposes.
One of the above uses is to use travel images taken with an in-vehicle camera for production of a map database.
However, since the in-vehicle camera is usually in a shooting state even when the vehicle is stopped, such as when there is a traffic jam or waiting for a signal, the traveling image used for producing the map database also includes an image that is stopped. For this reason, there is a problem that the work for checking the moving image takes time and the moving image data capacity increases.
Here, Patent Document 1 proposes an apparatus for performing a vehicle stop determination using a captured image. In this device, when the vehicle speed is equal to or lower than a predetermined speed, the amount of change per predetermined time of the captured image is equal to or less than the predetermined amount using the image of a part of the image captured by the camera mounted on the vehicle. If it is determined that the vehicle is stopped.

JP 2009-2754 A

The inventors of the present invention have made extensive studies in order to more accurately identify an image that is included in a running image while the vehicle is stopped. As a result, the following issues were found.
In the image processing for producing the map database, the stop image is specified only from the travel image without depending on the vehicle speed information, and the stop image determination using a partial area of the travel image is not sufficient.
In particular, there is a concern that the accuracy of stop image identification may be reduced in a running image including an image during high-speed running.
Moreover, if the entire travel image is used, stop determination can be made with high accuracy, but this is not desirable because the burden on the image processing apparatus increases.

The present invention has been made in view of the above-described problems, and an object thereof is to provide a technique capable of specifying a stop image included in travel image data with higher accuracy than the conventional technique.
That is, the first aspect of the present invention is defined as follows.
First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
A coordinate information storage unit for storing coordinate information A and coordinate information B for selecting the comparison area a and the comparison area b from the first frame image and the second frame image;
A first comparison area selecting means for selecting a comparison area 1a based on the coordinate information A and a comparison area 1b based on the coordinate information B from the first frame image;
Second comparison area selection means for selecting a comparison area 2a based on the coordinate information A and a comparison area 2b based on the coordinate information B from the second frame image;
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
Stop frame image specifying means for specifying the first frame image and the second frame image as stop frame images based on the first change amount and the second change amount;
A travel image data processing device comprising:

According to the traveling image data processing apparatus defined in the first aspect defined as described above, in order to detect the amount of change between the frame images, a plurality of comparison areas selected from within the frame images are separated from another frame. A comparison is made with a plurality of comparison areas selected from within the image, and whether or not the frame image is a stop frame image is specified based on the change amount. For this reason, it is possible to specify a stop frame image with high accuracy and a reduced burden on the image processing apparatus.
Here, the comparison area refers to an area selected from within the frame images in order to detect a change between the frame images, and is selected based on the coordinate information stored in the coordinate information storage unit.

The second aspect of the present invention is defined as follows. That is,
In the traveling image data processing device defined in the first aspect, the coordinate information storage unit has predetermined coordinates such that the distance between the comparison area a and the comparison area b in the frame image is a maximum distance. Information A and coordinate information B are stored.
Generally, when the first frame image and the second frame image are compared with each other when the vehicle is stopped, there is no change in the image, and there is a change in both images when the vehicle is running. However, even when the vehicle is stopped, if a disturbance factor such as an oncoming vehicle or a pedestrian is captured, a change occurs between the first frame image and the second frame image.
In this case, if the comparison area a and the comparison area b are close to each other in the frame image, there is a high possibility that a disturbance factor such as an oncoming vehicle or a pedestrian is reflected in the comparison area a and the comparison area b. However, if the comparison area a and the comparison area b are separated from each other, the possibility that a disturbance factor affects both areas is reduced.
On the other hand, when the change in scenery is poor even during traveling, there is a possibility that no large image change will appear between the first frame image and the second frame image. The separation is easier to identify the change.
Therefore, the stop / run determination accuracy is improved by arranging the comparison area a and the comparison area b farthest from each other as in the travel image data processing apparatus defined in the second aspect.

The third aspect of the present invention is defined as follows. That is,
In the travel image data processing apparatus defined in the first aspect, the coordinate information storage unit selects coordinates for selecting a comparison area c and a comparison area d from the first frame image and the second frame image. Further storing information C and coordinate information D;
The first comparison area selecting means further selects a comparison area 1c based on the coordinate information C and a comparison area 1d based on the coordinate information D,
The second comparison area selection means further selects a comparison area 2c based on the coordinate information C and a comparison area 2d based on the coordinate information D,
A third change amount detecting means for detecting a third change amount by comparing the comparison region 1c and the comparison region 2c;
A fourth change amount detecting means for detecting a fourth change amount by comparing the comparison region 1d and the comparison region 2d;
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.

According to the traveling image data processing apparatus defined in the third aspect defined in this way, four comparison areas are selected for each extracted frame image and they are separated from each other. In the image, the four areas are located at the four corners of the frame image.
Usually, in a running image, the amount of change between frame images during vehicle running increases from the center of the frame image toward the periphery. For this reason, if the coordinate information is determined in advance so that the comparison area can be selected from the four corners of the frame image, the amount of change in each comparison area can be detected with higher accuracy.
In this case, when there are three or more comparison regions whose change amount values corresponding to the four comparison regions are equal to or less than a predetermined change amount threshold value, the stop frame image specifying unit extracts each of the extracted The frame image is identified as a stop frame image (fourth aspect).

The fifth aspect of the present invention is defined as follows. That is,
In the traveling image data processing apparatus defined in the first aspect, the apparatus further comprises coordinate information setting means for setting the coordinate information A and the coordinate information B stored in the coordinate information storage unit,
When the first change amount and / or the second change amount is less than a predetermined value, the coordinate information setting means includes at least one of the position, area, and shape of the coordinate information A and / or the coordinate information B. Change one.
There is a possibility that the image change amount between the comparison areas (first change amount and / or second change amount) is small even at the time of traveling in a location where the scenery changes poorly, and the state of the frame image is specified as “stop”. is there.
At this time, by changing at least one of the position, area, and shape of the comparison region, it is possible to detect the amount of change caused by vehicle travel more accurately.
Even when the comparison area is dynamic as defined in the fifth aspect at the time of stopping, there is no problem because the image does not change between the comparison areas.

The sixth aspect of the present invention is defined as follows. That is,
A first frame image extraction step for extracting a first frame image from travel image data that is captured by an imaging device attached to the vehicle and is composed of a plurality of temporally continuous frame images;
A second frame image extraction step for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. A first comparison area selection step of selecting
A second comparison region selection step of selecting a comparison region 2a based on the coordinate information A and a comparison region 2b based on the coordinate information B from the second frame image;
A first change amount detecting step of comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detection step of comparing the comparison region 1b and the comparison region 2b to detect a second change amount;
A stop frame image specifying step of specifying the first frame image and the second frame image as stop frame images based on the first change amount and the second change amount;
A running image data processing method comprising:
According to the invention defined in the sixth aspect defined in this manner, the same effects as in the first aspect are achieved.

The seventh aspect of the present invention is defined as follows. That is,
In the traveling image data processing method defined in the sixth aspect, the coordinate information storage unit has predetermined coordinates such that the distance between the comparison area a and the comparison area b in the frame image is a maximum distance. Information A and coordinate information B are stored.
According to the invention defined in the seventh aspect defined in this manner, the same effects as in the second aspect are achieved.

The eighth aspect of the present invention is defined as follows. That is,
In the traveling image data processing method defined in the sixth aspect, the coordinate information storage unit selects coordinates for selecting a comparison region c and a comparison region d from the first frame image and the second frame image. Further storing information C and coordinate information D;
The first comparison area selection step further selects a comparison area 1c based on the coordinate information C and a comparison area 1d based on the coordinate information D,
The second comparison area selection step further selects a comparison area 2c based on the coordinate information C and a comparison area 2d based on the coordinate information D,
A third change amount detection step of comparing the comparison region 1c and the comparison region 2c to detect a third change amount;
A fourth change amount detecting step of comparing the comparison region 1d and the comparison region 2d to detect a fourth change amount;
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.
According to the invention defined in the eighth aspect defined in this manner, the same effects as in the third aspect are achieved.

The ninth aspect of the present invention is defined as follows. That is,
In the traveling image data processing method according to the eighth aspect, in the stop frame image specifying step, the first to fourth change values corresponding to the comparison regions a to d are equal to or less than a predetermined change threshold value. When there are three or more comparison areas, the first frame image and the second frame image are identified as the stop frame image.
According to the invention defined in the ninth aspect defined in this manner, the same effects as in the fourth aspect are achieved.

The tenth aspect of the present invention is defined as follows. That is,
In the traveling image data processing method of the sixth aspect, further comprising a coordinate information setting step for setting the coordinate information A and the coordinate information B stored in the coordinate information storage unit,
The coordinate information setting step includes at least one of a position, an area, and a shape of the coordinate information A and / or the coordinate information B when the first change amount and / or the second change amount is less than a predetermined value. Change one.
According to the invention defined in the tenth aspect defined in this manner, the same effects as in the fifth aspect are achieved.

Furthermore, the eleventh aspect of the present invention is defined as follows. That is,
A computer program for processing travel image data, comprising:
First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. First comparison region selection means for selecting
Second comparison area selection means for selecting a comparison area 2a based on the coordinate information A and a comparison area 2b based on the coordinate information B from the second frame image;
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
Stop frame image specifying means for specifying the first frame image and the second frame image as a stop frame image based on the first change amount and the second change amount;
A computer program characterized by functioning as:
According to the computer program defined in the eleventh aspect defined in this manner, the same effects as in the first aspect are achieved.

The twelfth aspect of the present invention is defined as follows. That is,
In the computer program defined in the eleventh aspect, the coordinate information storage unit includes predetermined coordinate information A and a predetermined distance so that a distance between the comparison area a and the comparison area b in the frame image is a maximum distance. Coordinate information B is saved.
According to the computer program defined in the twelfth aspect defined in this manner, the same effects as those in the second aspect are achieved.

The thirteenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the eleventh aspect, the coordinate information storage unit includes coordinate information C for selecting a comparison region c and a comparison region d from the first frame image and the second frame image. Further save the coordinate information D,
The first comparison area selecting means further selects a comparison area 1c based on the coordinate information C and a comparison area 1d based on the coordinate information D,
The second comparison area selection means further selects a comparison area 2c based on the coordinate information C and a comparison area 2d based on the coordinate information D,
A third change amount detecting means for detecting a third change amount by comparing the comparison region 1c and the comparison region 2c;
A fourth change amount detecting means for detecting a fourth change amount by comparing the comparison region 1d and the comparison region 2d;
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.
According to the computer program defined in the thirteenth aspect defined in this way, the same effects as in the third aspect are achieved.

The fourteenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the thirteenth aspect, the stop frame image specifying means has a value of the first to fourth change amounts corresponding to the comparison areas a to d that is equal to or less than a predetermined change amount threshold value. When there are three or more comparison areas, the first frame image and the second frame image are specified as the stop frame image.
According to the computer program defined in the fourteenth aspect defined in this manner, the same effects as in the fourth aspect are achieved.

The fifteenth aspect of the present invention is defined as follows. That is,
In the computer program defined in the eleventh aspect, the computer program further comprises coordinate information setting means for setting the coordinate information A and the coordinate information B stored in the coordinate information storage unit,
When the first change amount and / or the second change amount is less than a predetermined value, the coordinate information setting means includes at least one of the position, area, and shape of the coordinate information A and / or the coordinate information B. Change one.
According to the computer program defined in the fifteenth aspect defined in this way, the same effects as in the fifth aspect are achieved.

  The recording medium which records the computer program as described in any one of Claims 11-15.

It is a block diagram which shows the structure of the driving | running | working image data processing apparatus of embodiment of this invention. It is a block diagram which shows the detailed structure of the variation | change_quantity detection part and stop frame image specific | specification part which are shown in FIG. It is a flowchart which shows operation | movement of the driving | running | working image data processing apparatus of embodiment of this invention. It is a flowchart which shows the detailed operation | movement of step 5 and step 7 which are shown in FIG. It is a block diagram which shows the structure of the driving | running | working image data processing apparatus of other embodiment of this invention. It is a flowchart which shows operation | movement of the driving | running | working image data processing apparatus of other embodiment of this invention. It is a block diagram which shows the structure of the driving | running | working image data processing apparatus of other embodiment of this invention. It is a block diagram which shows the detailed structure of the variation | change_quantity detection part and stop frame image specific | specification part which are shown in FIG. It is a flowchart which shows operation | movement of the driving | running | working image data processing apparatus of other embodiment of this invention. It is a flowchart which shows the detailed operation | movement of step 8 shown in FIG. The computer system which comprises the driving | running | working image data processing apparatus of other embodiment of this invention is shown.

A traveling image data processing apparatus according to an embodiment of the present invention will be described.
FIG. 1 shows a schematic configuration of a traveling image data processing apparatus 1 according to an embodiment of the present invention.
As shown in FIG. 1, the travel image data processing apparatus 1 includes a travel image data storage unit 2, a frame image extraction unit 3, a coordinate information storage unit 4, a comparison area selection unit 5, a change amount detection unit 6, and a stop frame image specification. The unit 7 is provided.
The traveling image data storage unit 2 stores traveling image data captured by an imaging device attached to the vehicle. The travel image data is composed of a plurality of temporally continuous frame images.
Here, the installation direction and angle of the imaging device attached to the vehicle are not particularly limited as long as a traveling image that can identify the stop frame image in the traveling image data can be captured. From the viewpoint that it is easy to compare the frame image while the vehicle is running and the frame image when the vehicle is stopped, it is preferable to install the camera so as to photograph the traveling direction or to photograph the direction opposite to the traveling direction. For the same reason, the installation angle of the imaging device is preferably set to an angle at which the ratio of the road surface on which the vehicle is traveling in the frame image is about 1/3 to half. This is because, if it is set to be extremely upward or downward, most of the frame image captures the sky or the road surface, so that it is not easy to specify the stop frame image.

The frame image extraction unit 3 extracts a frame image from the travel image data stored in the travel image data storage unit 2. In the travel image data processing apparatus 1 shown in FIG. 1, the frame image extraction unit 3 includes a first frame image extraction unit 31 that extracts a first frame image and a second frame image extraction unit 32 that extracts a second frame image. And. The first frame image and the second frame image may be temporally continuous, or there may be a time interval between both frame images. Further, the number of frame images to be extracted may be a plurality of two or more, and the number is not limited.
When the time interval between frame images is extremely small, one frame image is arbitrarily extracted from a plurality of frame images included in a short time interval (for example, 0.1 second), and a comparison area a is set in the same. It is also possible to set the comparison region b in another extracted frame image. In this case, both the former frame image and the latter frame image are handled as the first frame image (or the second frame image).

  The coordinate information storage unit 4 stores coordinate information for selecting a comparison area to be described later from the frame image extracted by the frame image extraction unit 3. In the traveling image data processing device 1, coordinate information A for selecting the comparison area a from the frame image and coordinate information B for selecting the comparison area b are stored. The coordinate information storage unit 4 may store predetermined coordinate information, or may store coordinate information set by a coordinate information setting unit described later. As the coordinate information, a plurality of pieces of coordinate information are stored according to the number of comparison areas to be selected.

The comparison area selection unit 5 selects a comparison area from the frame image extracted by the frame image extraction unit 3 based on the coordinate information stored in the coordinate information storage unit 4. The comparison region selection unit 5 includes a first comparison region selection unit 51 that selects a comparison region from the first frame image, and a second comparison region selection unit 52 that selects a comparison region from the second frame image. The first comparison region selection unit 51 includes a comparison region 1a selection unit 511 that selects the comparison region 1a based on the coordinate information A, and a comparison region 1b selection unit 512 that selects the comparison region 1b based on the coordinate information B. Provided. Similarly, the second comparison region selection unit 52 includes a comparison region 2a selection unit 521 and a comparison region 2b selection unit 522. Since the comparison area 1a selected from the first frame image and the comparison area 2a selected from the second frame image are both selected based on the coordinate information A, they indicate the same area on each frame image. . Similarly, the comparison region 1b and the comparison region 2b selected based on the coordinate information B indicate the same region on each frame image.
The number of comparison areas to be selected is not particularly limited as long as it is two or more. However, since the frame image is rectangular, it is preferable to select four comparison areas.

The change amount detection unit 6 detects the change amount between the first frame image and the second frame image. In the traveling image data processing device 1, in the same manner as the first change amount detection unit 61 that detects the change amount between the comparison region 1a selected from the first frame image and the comparison region 2a selected from the second frame image. A second change amount detection unit 62 that detects a change amount between the selected comparison region 1b and comparison region 2b is provided.
In order to detect the change amount between the extracted frame images, for example, a luminance value or the like can be used. With reference to FIG. 2, the configuration of the change amount detection unit when the average value of luminance is used will be described in detail.

The first change amount detection unit 61 includes a region 1a average luminance calculation unit 611, a region 2a average luminance calculation unit 612, and a region a change amount detection unit 613. The area 1a average luminance calculation unit 611 calculates the average luminance per pixel from the number of pixels constituting the comparison area 1a and the luminance value of each pixel. Similarly, the area 2a average luminance calculation unit 612 calculates the average luminance in the comparison area 2a. The region a change amount detection unit 613 detects the change amount in the comparison region a. That is, the calculated average brightness of the comparison area 1a is compared with the average brightness of the comparison area 2a, and the difference is detected as the change amount of the comparison area a. The second change amount detection unit 62 also detects the change amount in the comparison region b in the same manner as described above.
The detection of the amount of change is not particularly limited as long as an element that detects the amount of change between frame images is used. In addition to using the average value of luminance, a standard deviation of luminance can also be used.

Returning to FIG. 1, the change amount of each area detected by the change amount detection unit 6 is sent to the stop frame image specifying unit 7.
The stop frame image specifying unit 7 specifies a stop region based on the change amount of each region detected by the change amount detecting unit 6, and whether or not the frame image having the stop region corresponds to the stop frame image is determined in advance. Judge according to the rules. The configuration of the stop frame image specifying unit 7 will be described in detail with reference to FIG.
The stop frame image specifying unit 7 includes a stop region specifying unit 71 and a stop frame image determining unit 72.
The stop region specifying unit 71 compares the change amount of the region a sent from the region a change amount detection unit 613 with a predetermined change amount threshold value, and when the change amount of the region a is equal to or less than the change amount threshold value. The area a is identified as a stop area. For the region b, the stop region is specified in the same manner.

The stop frame image determination unit 72 determines a stop frame image based on the specification result of the stop region specification unit 71. The determination is not particularly limited as long as it is a method capable of accurately determining whether or not the frame is a stop frame image. For example, a plurality of comparison areas selected from the frame image are all set as stop areas in the stop area specifying unit 71. When specified, the frame image can be determined as a stop frame image. In the case of the traveling image processing apparatus 1, when both the region a and the region b are specified as the stop region, the first frame image and the second frame image to be processed are determined as the stop frame images. .
As another method, when the number of stop comparison regions with respect to the number of selected comparison regions is a predetermined number or more, or the total area of stop comparison regions with respect to the total area of the selected comparison regions is greater than or equal to a predetermined value. In this case, it may be determined as a stop frame image.
Further, it is possible to make a judgment as to whether or not the operation is stopped based on the amount of change in one comparison region, and to refer to the amount of change in the other comparison region. For example, when the change amount of the comparison region b is larger than the change amount of the comparison region a, for example, by 50% or more, the specific result based on the change amount of the comparison region a is canceled. Or let a specific result be a driving state. Thus, it is preferable to specify the state in which the amount of change in the comparison area a and the comparison area b varies as the running state. Even if the stopped image in the running image referred to when creating the map database is identified as being “running” and this is displayed, the burden on the operator will increase somewhat, but there will be no effect on the map database production accuracy. Because.
A frame image that has not been identified as a stop frame image may be regarded as a stop frame image if the frame image before and after the frame image is identified as a stop frame image.

Next, the operation of the traveling image data processing apparatus 1 according to the embodiment of the present invention will be described with reference to the flowcharts of FIGS.
First, in step 1, a frame image pair is extracted from the running image data stored in the running image data storage unit 2. A plurality of frame images may be extracted, but the traveling image data processing apparatus 1 extracts the first frame image and the second frame image as a pair of frame images.
Next, a plurality of comparison areas are selected from the extracted first frame image (step 3). The comparison area is selected with reference to the coordinate information storage unit 4 in which coordinate information A for selecting the comparison area a and coordinate information B for selecting the comparison area b are stored in advance. In the traveling image data processing device 1, the comparison area 1a based on the coordinate information A and the comparison area 1b based on the coordinate information B are selected from the first frame image. Similarly, the comparison area 2a and the comparison area 2b are selected from the second frame image.

Subsequently, among the comparison areas selected in step 3, the amount of change between the comparison area 1a and the comparison area 2a selected based on the same coordinate information, and the amount of change between the comparison area 1b and the comparison area 2b are detected. Then, based on the detection result, the first frame image and the second frame image are identified as stop frame images (step 7).
The detailed operation of step 5 and step 7 is shown in FIG.
In step 51, the average luminance of the comparison region k is calculated for each frame image. That is, the area 1a average luminance calculation unit 611 in the first change amount detection unit 61 calculates 1 from the number of pixels included in the area 1a and the luminance value of each pixel for the comparison area 1a selected by the comparison area selection unit 5. The average luminance per pixel is calculated. Similarly, the area 2a average brightness calculation unit 612 calculates the average brightness in the comparison area 2a.
The calculated average luminance of the region 1a and the region 2a is sent to the region a change amount detection unit 613, and a difference in average luminance between the two regions is detected (step 53). The difference is the amount of change in the comparison area a between the first frame image and the second frame image.

The detected change amount of the region a is compared with a predetermined change amount threshold value by the stop region specifying unit 71 in the stop frame image specifying unit 7, and when the change amount of the region a is equal to or less than the change amount threshold value. The area a is identified as a stop area (step 71).
Similarly, the amount of change in the comparison area b is detected (steps 51 and 53), and the stop area is specified for the area b (step 71).
Based on the stop region specifying result in step 71, the stop frame image determination unit 72, for example, determines the first frame image and the second frame image to be determined when both the region a and the region b are specified as stop regions. It is determined as a stop frame image (step 73).

FIG. 5 shows a running image data processing apparatus 20 according to another embodiment. In FIG. 5, the same elements as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is partially omitted.
In locations where the change of scenery is poor, the amount of image change between the comparison regions (first change amount and / or second change amount) is small even during travel, and therefore “stop” may be output as processing. The traveling image data processing device 20 changes at least one of the position, area, and shape of the comparison area on the frame image in order to avoid the stop determination as much as possible even when the vehicle is traveling. Let According to this traveling image data processing device 20, it is possible to detect the amount of change caused by vehicle traveling more accurately.
Therefore, the traveling image data processing device 20 shown in FIG. 5 includes a coordinate information setting unit 8 for setting the coordinate information stored in the coordinate information storage unit 4 in the traveling image data processing device 1 shown in FIG. .
The coordinate information setting unit 8 is a coordinate information A setting unit 81 for setting coordinate information A for selecting the comparison area a on the frame image, and a coordinate for setting coordinate information B for selecting the comparison area b. And an information B setting unit 82. The coordinate information setting unit 8 includes a setting unit for setting each coordinate information corresponding to the number of selected comparison areas.

The coordinate information A setting unit 81 resets the predetermined coordinate information A to the coordinate information A ′ based on a predetermined rule.
The predetermined rule is, for example, the amount of change between the comparison region 1a selected from the first frame image by the first change amount detection unit 61 based on the coordinate information A and the comparison region 2a selected from the second frame image. When the change amount less than the predetermined value is detected, the coordinate information A ′ is set. As the predetermined value, a value different from the change amount threshold value which is a criterion for determining whether or not the stop region is the stop region is set.
As an example other than the above, when the comparison area 1a and the comparison area 2a are specified as stop areas by the stop area specifying section 71, the coordinate information A setting section 81 may set the coordinate information A ′. .
The set coordinate information A ′ is sent to the coordinate information storage unit 4.
The coordinate information can be set by setting coordinate information that can select a comparison region in which the change amount between the comparison region 1a and the comparison region 2a can be easily detected. For example, the position of the coordinate information can be changed so that the comparison area a slides toward the center of the horizontal axis on the frame image. In addition to the position of the coordinate information, the area and shape of the coordinate information may be changed, or the coordinate information may be changed by combining them.
Similarly to the coordinate information A setting unit 81, the coordinate information B setting unit 82 sets coordinate information B ′ for selecting the comparison area b and sends it to the coordinate information storage unit 4.
The set coordinate information A ′ and coordinate information B ′ are compared from the third frame image and the fourth frame image extracted by the frame image extraction unit 3 from the travel image data storage unit 2 as a new stop determination target. Used when selecting a ′ and comparison region b ′. That is, the comparison area selection unit 5 selects the comparison area 3a ′ from the third frame image and the comparison area 4a ′ from the fourth frame image based on the coordinate information A ′. Similarly, the comparison region 3b ′ and the comparison region 4b ′ are selected.
In the present embodiment, the newly extracted frame image pair is the third frame image and the fourth frame image. However, the second frame image and the third frame image may be extracted to determine the stop frame image. Good.

The operation of the travel image data processing device 20 shown in FIG. 5 will be described with reference to FIG. In FIG. 6, steps that perform the same processing as in FIG. 3 are assigned the same reference numerals, and descriptions thereof are partially omitted.
In FIG. 6, when there is a comparison region in which the amount of change detected in step 5 is less than a predetermined value (step 9: Yes), step 9 shows the coordinate information of the comparison region in which the amount of change less than the predetermined value is detected. Set (step 11). The coordinate information set in step 11 is referred to when the comparison area of the next frame image pair is selected. That is, if the next frame image pair specified in step 13 is not the final image (step 15: No), the comparison area is selected from the frame image pair (third frame image and fourth frame image) extracted in step 1 Is performed based on the reset coordinate information for the comparison area in which the coordinate information is reset in step 11, and based on the original coordinate information for the comparison area in which the coordinate information is not reset.

In FIG. 7, the functional block diagram of the driving | running | working image data processing apparatus 30 of other embodiment is shown. In FIG. 7, the same elements as those in FIGS. 1 and 5 are denoted by the same reference numerals, and the description thereof is partially omitted.
In the traveling image data processing device 30 shown in FIG. 7, in the traveling image data processing device 1 shown in FIG. 1, the comparison region selection unit includes four comparison regions a to d for each of the first frame image and the second frame image. And the first to fourth change amount detection units 601 to 604 that detect four change amounts corresponding to the four comparison regions. The comparison areas a to d are selected based on the coordinate information A to D stored in the coordinate information storage unit 40. The coordinate information A to D is preferably coordinate information that maximizes the distance between the comparison areas a to d in the frame image.
The four comparison regions may be selected from one frame image as the first frame image, or a plurality of frame images that are temporally approximated as the first frame image are defined as the first frame image group, and the first Four comparison regions may be selected from the frame image group. Similarly, for the second frame image, four comparison regions may be selected from the second frame image group.

FIG. 8 is a detailed block diagram of the change amount detection unit 60 and the stop frame image specifying unit 70. The same elements as those in FIG. 2 are denoted by the same reference numerals, and the description thereof is partially omitted.
The change amount detection unit 60 includes first to fourth change amount detection units that detect change amounts of the comparison regions a to d. The change amount detected by the change amount detection unit 60 is sent to the stop frame image specifying unit 70.
The stop frame image specifying unit 70 includes a stop region specifying unit 710, a stop frame image determining unit 720, and a counter 730. The stop region specifying unit 710 specifies a stop region for each region by comparing the change amounts of the regions a to d detected by the change amount detection unit 60 with a predetermined change amount threshold value. The counter 730 counts the number of comparison areas identified as stop areas. Based on the counted number of stop areas, the stop frame image determination unit 720 determines whether or not the frame image is a stop frame image. As a determination method, for example, the stop frame image determination unit 720 uses the first frame image and the second frame image as stop frames when there are three or more regions identified as stop regions among the comparison regions a to d. It can be determined as an image.

The operation of the travel image data processing device 30 will be described with reference to FIGS. 9 and 10. 9 and 10, steps that perform the same processing as in FIGS. 3, 4, and 6 are assigned the same reference numerals, and descriptions thereof are partially omitted.
When a change amount is detected for each of the comparison areas a to d selected in step 3, the stop frame image specifying unit 70 specifies whether or not the frame image is a stop frame image based on the detection result. (Step 8).
The detailed operation of step 8 will be described with reference to FIG.
When the amount of change in the comparison area a is detected in step 53 via step 51, the amount of change in the area a is compared with a predetermined change amount threshold by the stop area specifying unit 710 (step 81). When the change amount of the comparison area a is equal to or smaller than the change amount threshold (step 81: Yes), the comparison area a is specified as a stop area and stored (steps 83 and 85). Subsequently, the comparison area b as the next area is designated (step 86). The same step is executed for the comparison area b, and the specific result is stored. Similar steps are performed for the comparison areas c and d. When the stop areas are specified for all the comparison areas (step 87: Yes), the ratio of the stop areas in all the comparison areas is determined. In step 88, when the number of areas identified as stop areas among the comparison areas a to d is three or more (step 88: Yes), the first frame image and the second frame image that are the targets of the stop determination are stopped. The frame image is determined (step 89).

FIG. 11 is a block diagram illustrating a hardware configuration of the traveling image data processing device 30.
The hardware configuration of the device 30 is such that various elements are coupled to the central control device 221 via the system bus 222 as in a general computer system.
The central controller 221 includes a general-purpose CPU, a memory controller, a bus controller, an interrupt controller, and a DMA (direct memory access) device. The system bus 222 also includes a data line, an address line, and a control line. A memory circuit including a RAM (Random Access Memory) 223 and a nonvolatile memory (ROM 224, CMOS-RAM 225, etc.) is connected to the system bus 222. Data stored in the RAM 223 is read or rewritten by the central controller 221 or other hardware elements. The data in the non-volatile memory is read-only, and the data is not lost when the device is turned off. The system program for controlling the hardware is stored in the hard disk device 227 and also stored in the RAM 223, and is read and used by the central control device 221 as appropriate via the disk drive control device 226. The hard disk device 227 has an area for storing a computer program for operating a general-purpose computer system as the running image data processing device 30.
A predetermined area of the hard disk device 227 is allocated for the traveling image data storage unit 2.
The selected link and the extracted node to another area of the hard disk device 227 and the buffer memory for temporarily storing the regulation time can be allocated.

Connected to the system bus 222 are a flexible drive control device 231 that reads and writes data from and to the flexible disk 232, and a CD / DVD control device 233 that reads data from the compact disk 234. In this apparatus 3, a printer 238 is connected to the printer interface 237.
A keyboard / mouse control device 241 is connected to the system bus 222 to enable data input from the keyboard 242 and the mouse 243. A monitor 245 is connected to the system bus 222 via the monitor control device 244. The monitor 245 can be a CRT type, a liquid crystal type, a plasma display type, or the like.
An empty slot 251 is prepared in order to allow the addition of various elements (such as a modem).

  Programs (OS program and application program (including those of the present invention)) necessary for operating the running image data processing apparatus 30 comprising this computer system are installed in the system via various recording media. . For example, it is possible to install in the form of a non-write recording medium (CD-ROM, ROM card, etc.), a writable recording medium (FD, DVD, etc.), or a communication medium using the network N. Of course, these programs can be written in advance in the nonvolatile memories 224 and 225 and the hard disk device 227.

  As mentioned above, although embodiment of this invention has been described, you may implement combining 2 or more embodiment among these. Alternatively, one of these embodiments may be partially implemented. Furthermore, among these, two or more embodiments may be partially combined.

  According to the traveling image data processing device of the present invention, for example, it is possible to delete the stop frame image in the traveling image data or set a specific flag. By deleting unnecessary stop frame images from the travel image data, it is possible to efficiently perform the travel image data confirmation work when producing the map database.

  The present invention is not limited to the description of the embodiment of the invention. Various modifications may be included in the present invention as long as those skilled in the art can easily conceive without departing from the description of the scope of claims.

1 20 30 Traveling image data processing device 2 Traveling image data storage unit 3 Frame image extraction unit 4 40 Coordinate information storage unit 5 50 Comparison area selection unit 6 60 Change amount detection unit 7 70 Stop frame image specification unit 8 Coordinate information setting unit 71 710 Stop region specifying unit 72 720 Stop frame image determining unit 611 612 621 622 Region average luminance calculation unit 613 623 Region change amount detecting unit 730 Counter

Claims (13)

First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
Coordinate information A, coordinate information B, coordinate information C, and coordinate information for selecting comparison area a, comparison area b, comparison area c, and comparison area d from the first frame image and the second frame image. A coordinate information storage unit for storing D ,
A comparison area 1a based on the coordinate information A, a comparison area 1b based on the coordinate information B , a comparison area 1c based on the coordinate information C, and a comparison area 1d based on the coordinate information D are selected from the first frame image. First comparison area selection means for
A comparison area 2a based on the coordinate information A, a comparison area 2b based on the coordinate information B , a comparison area 2c based on the coordinate information C, and a comparison area 2d based on the coordinate information D are selected from the second frame image. Second comparison region selection means for
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
A third change amount detecting means for detecting a third change amount by comparing the comparison region 1c and the comparison region 2c;
A fourth change amount detecting means for detecting the fourth change amount by comparing the comparison region 1d and the comparison region 2d;
Based on the first change amount, the second change amount, the third change amount, and the fourth change amount , the first frame image and the second frame image are identified as stop frame images. Stop frame image specifying means ,
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.
A traveling image data processing apparatus characterized by the above. The stop frame image specifying means has three or more comparison regions in which the values of the first to fourth change amounts corresponding to the comparison regions a to d are equal to or less than a predetermined change amount threshold value. Identifying the first frame image and the second frame image as the stop frame image;
The traveling image data processing apparatus according to claim 1 . First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
A coordinate information storage unit for storing coordinate information A and coordinate information B for selecting the comparison area a and the comparison area b from the first frame image and the second frame image;
A first comparison area selecting means for selecting a comparison area 1a based on the coordinate information A and a comparison area 1b based on the coordinate information B from the first frame image;
Second comparison area selection means for selecting a comparison area 2a based on the coordinate information A and a comparison area 2b based on the coordinate information B from the second frame image;
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
Stop frame image specifying means for specifying the first frame image and the second frame image as stop frame images based on the first change amount and the second change amount;
Coordinate information setting means for setting coordinate information A and coordinate information B stored in the coordinate information storage unit,
When the first change amount and / or the second change amount is less than a predetermined value, the coordinate information setting means includes at least one of the position, area, and shape of the coordinate information A and / or the coordinate information B. Change one,
A traveling image data processing apparatus characterized by the above. The coordinate information storage unit stores predetermined coordinate information A and coordinate information B so that the distance between the comparison area a and the comparison area b in the frame image is a maximum distance.
The traveling image data processing apparatus according to claim 3 . A first frame image extraction step for extracting a first frame image from travel image data that is captured by an imaging device attached to the vehicle and is composed of a plurality of temporally continuous frame images;
A second frame image extraction step for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. A first comparison region selection step for selecting a comparison region 1c based on the coordinate information C for selecting the comparison region c and a comparison region 1d based on the coordinate information D for selecting the comparison region d ;
A comparison area 2a based on the coordinate information A, a comparison area 2b based on the coordinate information B , a comparison area 2c based on the coordinate information C, and a comparison area 2d based on the coordinate information D are selected from the second frame image. A second comparison area selection step,
A first change amount detecting step of comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detection step of comparing the comparison region 1b and the comparison region 2b to detect a second change amount;
A third change amount detection step of comparing the comparison region 1c and the comparison region 2c to detect a third change amount;
A fourth change amount detection step of detecting a fourth change amount by comparing the comparison region 1d and the comparison region 2d;
Based on the first change amount, the second change amount, the third change amount, and the fourth change amount , the first frame image and the second frame image are identified as stop frame images. A stop frame image specifying step ,
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.
A running image data processing method characterized by the above. In the stop frame image specifying step, when there are three or more comparison areas in which the values of the first to fourth change amounts corresponding to the comparison areas a to d are equal to or less than a predetermined change amount threshold value. Identifying the first frame image and the second frame image as the stop frame image;
The traveling image data processing method according to claim 5 . A first frame image extraction step for extracting a first frame image from travel image data that is captured by an imaging device attached to the vehicle and is composed of a plurality of temporally continuous frame images;
A second frame image extraction step for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. A first comparison area selection step of selecting
A second comparison region selection step of selecting a comparison region 2a based on the coordinate information A and a comparison region 2b based on the coordinate information B from the second frame image;
A first change amount detecting step of comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detection step of comparing the comparison region 1b and the comparison region 2b to detect a second change amount;
A stop frame image specifying step of specifying the first frame image and the second frame image as stop frame images based on the first change amount and the second change amount;
A coordinate information setting step for setting coordinate information A and coordinate information B stored in the coordinate information storage unit,
The coordinate information setting step includes at least one of a position, an area, and a shape of the coordinate information A and / or the coordinate information B when the first change amount and / or the second change amount is less than a predetermined value. Change one,
A running image data processing method characterized by the above. The coordinate information storage unit stores predetermined coordinate information A and coordinate information B so that the distance between the comparison area a and the comparison area b in the frame image is a maximum distance.
The traveling image data processing method according to claim 7 . A computer program for processing travel image data, comprising:
First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. A first comparison region selection means for selecting a comparison region 1c based on the coordinate information C for selecting the comparison region c and a comparison region 1d based on the coordinate information D for selecting the comparison region d ;
A comparison area 2a based on the coordinate information A, a comparison area 2b based on the coordinate information B , a comparison area 2c based on the coordinate information C, and a comparison area 2d based on the coordinate information D are selected from the second frame image. Second comparison region selection means for
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
A third change amount detecting means for detecting a third change amount by comparing the comparison region 1c and the comparison region 2c;
A fourth change amount detecting means for detecting a fourth change amount by comparing the comparison region 1d and the comparison region 2d;
Based on the first change amount, the second change amount, the third change amount, and the fourth change amount , the first frame image and the second frame image are identified as stop frame images. Function as stop frame image specifying means ,
The coordinate information storage unit defines the coordinate information A, B, C, and D so that the distance between the comparison areas a, b, c, and d in the frame image is maximized.
A computer program characterized by the above. The stop frame image specifying means has three or more comparison regions in which the values of the first to fourth change amounts corresponding to the comparison regions a to d are equal to or less than a predetermined change amount threshold value. Identifying the first frame image and the second frame image as the stop frame image;
The computer program according to claim 9 . A computer program for processing travel image data, comprising:
First frame image extraction means for extracting a first frame image from travel image data that is captured by an imaging device attached to a vehicle and is composed of a plurality of temporally continuous frame images;
Second frame image extraction means for extracting a second frame image from the running image data;
From the first frame image, the comparison area 1a based on the coordinate information A for selecting the comparison area a stored in the coordinate information storage unit, and the comparison area 1b based on the coordinate information B for selecting the comparison area b. First comparison region selection means for selecting
Second comparison area selection means for selecting a comparison area 2a based on the coordinate information A and a comparison area 2b based on the coordinate information B from the second frame image;
First change amount detecting means for comparing the comparison region 1a and the comparison region 2a to detect a first change amount;
A second change amount detecting means for detecting the second change amount by comparing the comparison region 1b and the comparison region 2b;
Stop frame image specifying means for specifying the first frame image and the second frame image as stop frame images based on the first change amount and the second change amount ;
Function as coordinate information setting means for setting coordinate information A and coordinate information B stored in the coordinate information storage unit,
When the first change amount and / or the second change amount is less than a predetermined value, the coordinate information setting means includes at least one of the position, area, and shape of the coordinate information A and / or the coordinate information B. Change one,
A computer program characterized by the above. The coordinate information storage unit stores predetermined coordinate information A and coordinate information B so that the distance between the comparison area a and the comparison area b in the frame image is a maximum distance.
The computer program according to claim 11 . The recording medium which records the computer program as described in any one of Claims 9-12 .

Images