JP4966788B2 - Shooting direction determination device and shooting direction determination program - Google Patents

Description

  The present invention relates to a technique for determining a shooting direction of an image.

  2. Description of the Related Art Conventionally, there is a traveling investigation vehicle that is equipped with a front camera and a rear camera for road data maintenance and travels while photographing with the camera. In the travel investigation vehicle, data that associates image data captured by the camera with the latitude and longitude of the travel position is also created. Thereby, it is possible to associate the image data with the position on the map.

  It should be noted that the camera of the vehicle for traveling investigation as described above has a problem that when the power supply is restarted, the identification value fluctuates, so it is not known whether or not each image data was photographed by the front photographing camera. there were.

  As a method for solving the above problem, there is an installation state determination device that can determine an installation state of an in-vehicle camera by obtaining an optical flow of image data (see, for example, Patent Document 1).

JP 2007-8325 A

  When the optical flow is used as in the installation state determination device of Patent Document 1, there is a problem that the processing efficiency is not good because the calculation amount increases.

  Examples of the problems to be solved by the present invention include those described above. An object of the present invention is to provide a photographing direction determination device that can easily determine the direction of a camera that has photographed image data.

The invention according to claim 1 is a photographing direction determination device, and defines image data holding means for holding image data taken by a photographing camera mounted on a moving body, and a relationship between a photographing time and the image data. A shooting-related data holding unit that holds the shooting-related data , and a comparison that determines a comparison image data group that is a set of image data that is composed of a plurality of image data and has the latest shooting time based on the shooting time Image data group determining means, template image extracting means for extracting a template image from image data of the comparative image data group, shape pattern matching means for performing shape pattern matching using the template image, and shape pattern matching , Template images detected in each of the image data included in the comparison image data group Based on the magnitude relationship between the determination result of the comparison image data group shooting direction of the image data of the comparison image data group to determine whether the direction opposite to the traveling direction of the traveling direction is one mobile mobile Based on the shooting direction determined by the shooting direction determination means and the comparison image data group shooting direction determination means, whether the shooting direction of the image data is the moving direction of the moving body or the direction opposite to the moving direction of the moving body And a shooting direction output means for outputting a result determined by the shooting direction determination means.

The invention according to claim 6 is a method for determining a shooting direction, wherein an image data holding step for holding image data shot by a shooting camera mounted on a moving body, and a relationship between a shooting time and the image data are defined. A shooting-related data holding step for holding the shooting-related data , and a comparison for determining a comparison image data group that is a set of image data composed of a plurality of image data and having the latest shooting time based on the shooting time An image data group determining step, a template image extracting step of extracting a template image from image data of the comparison image data group, a shape pattern matching step of performing shape pattern matching using the template image, and shape pattern matching , Template images detected in each of the image data included in the comparison image data group Based on the magnitude relationship between the determination result of the comparison image data group shooting direction of the image data of the comparison image data group to determine whether the direction opposite to the traveling direction of the traveling direction is one mobile mobile Whether the shooting direction of the image data is the moving direction of the moving body or the reverse direction of the moving body based on the shooting direction determined by the shooting direction determining step and the comparison image data group shooting direction determining step A shooting direction determination step for determining the shooting direction, and a shooting direction output step for outputting a result determined by the shooting direction determination step;
It is characterized by providing.

The invention according to claim 7 is a shooting direction determination program executed by an apparatus including a computer, an image data holding means for holding image data shot by a shooting camera mounted on a moving body, a shooting time, A shooting-related data holding unit that holds shooting-related data that defines a relationship with the image data, and a comparison that is a set of image data that is composed of a plurality of image data based on the shooting time and that has the latest shooting time A comparison image data group determining unit for determining an image data group, a template image extracting unit for extracting a template image from the image data of the comparison image data group, and a shape pattern for performing shape pattern matching using the template image and matching means, the shape pattern matching image included in the comparison image data group Based on the determination result of the magnitude relationship between the detected template image in each chromatography data, the shooting direction of the image data of the comparison image data group in the direction opposite to the traveling direction if the moving body is a moving direction of the moving body Based on the comparison image data group shooting direction determination means for determining whether or not the comparison image data group shooting direction determination means determines the shooting direction of the image data is the traveling direction of the moving body The computer is caused to function as an imaging direction determination unit that determines whether the direction of movement is opposite to a body traveling direction and an imaging direction output unit that outputs a result determined by the imaging direction determination unit.

  In a preferred embodiment of the present invention, the photographing direction determining device includes an image data holding unit that holds image data taken by a photographing camera mounted on a moving body, and a photographing that defines a relationship between the photographing time and the image data. Shooting related data holding means for holding related data, comparison image data group determining means for determining a comparison image data group composed of a plurality of image data based on the shooting time, and the comparison image data group A template image extraction unit that extracts a template image from image data of the image, a shape pattern matching unit that performs shape pattern matching using the template image, and a scale of the shape pattern matching result in the comparison image data group are compared. A ratio for determining the photographing direction of the image data of the comparative image data group based on the result An image data group shooting direction determination unit; a shooting direction determination unit that determines a shooting direction of image data based on the shooting direction determined by the comparison image data group shooting direction determination unit; and the shooting direction determination unit determines Photographing direction output means for outputting the result.

  Specifically, the above-described shooting direction determination device can be applied to a car navigation device, a personal computer, or the like. The photographing direction determination device holds in advance image data taken by a photographing camera mounted on a moving body. The photographing direction determination apparatus holds photographing related data that defines a relation between the photographing time of the image data and the image data.

  Then, the comparison image data group determination means of the imaging direction determination device determines a comparison image data group composed of a plurality of image data based on the imaging time. Then, the template image extraction means extracts a template image from the image data of the comparison image data group.

  The shape pattern matching means performs shape pattern matching on image data other than the image data from which the template image extraction means has extracted the template image out of the image data of the comparison image data group, using the template image.

  The comparison image data group shooting direction determination means determines the shooting direction of the image data of the comparison image data group based on the result of comparing the scales of the shape pattern matching results.

  The shooting direction determining means finally determines the shooting direction of the image data in the comparison image data group based on the content determined by the comparative image data group shooting direction determining means, and the shooting direction output means The shooting direction is output.

  As described above, the imaging direction determination device uses the template image extracted from one image data in the comparison image data group, performs shape pattern matching on the other image data, and compares the size of the matched portion with the size of the template image. By doing so, it can be determined whether or not the image data is image data obtained by photographing the traveling direction of the moving body.

  In this case, the photographing direction determination device can easily determine the photographing direction as compared with the case of obtaining the optical flow of the image data and determining the photographing direction of the image data.

  In one aspect of the above-described photographing direction determination apparatus, there are a plurality of comparison image data groups, and the number of photographing direction determination means is the largest among the photographing directions determined by the comparison image data group photographing direction determination means. The shooting direction is determined as the shooting direction of the image data.

  According to this, since the shooting direction determination device determines the shooting direction of the image data among the shooting directions determined for the image data of the plurality of comparative image data groups, the shooting direction of some image data is determined. Even if an erroneous determination is made depending on the shooting state, an appropriate determination can be made as a whole.

  In one aspect of the photographing direction determination device, the template image extraction unit extracts a plurality of template images, and the comparison image data group photographing direction determination unit determines the shape pattern matching result using the plurality of template images. The shooting direction is determined based on the largest number of scale comparison results. According to this, the imaging direction determination device may not be able to accurately determine the imaging direction of the image data of the comparison image data group because, for example, the size of a part corresponding to the template image does not change for some template images. However, as a whole, the photographing direction of the image data of the comparative image data group can be appropriately determined.

  In one aspect of the above-described shooting direction determination apparatus, the template image extraction unit re-scans another template image when the comparison image data group shooting direction determination unit cannot determine the shooting direction using the extracted template image. Extract. According to this, the imaging direction determination device determines “unknown” as a result of performing shape pattern matching on the other image data using the template image extracted from one image data of the comparison image data group. Even so, since another template image is re-extracted, the possibility of appropriate determination can be improved.

  In one aspect of the photographing direction determining device, the image capturing apparatus further includes a curve determining unit that determines image data captured during curve traveling, and the comparison image data group extracting unit includes image data captured during the curve traveling. The order of determination of the shooting direction of the comparative image data group including is postponed. According to this, there is a possibility that the shooting direction determination device matches the image data during the curve traveling by deferring the shooting direction determination order of the comparison image data group including the image data during the curve traveling. It is possible to preferentially determine a comparative image data group having a high.

  In another aspect of the present invention, an imaging direction determination method includes an image data holding step for holding image data shot by a shooting camera mounted on a moving body, and a shooting-related function that defines a relationship between a shooting time and the image data. A shooting-related data holding step for holding data, a comparison image data group determination step for determining a comparison image data group composed of a plurality of image data based on the shooting time, and the comparison image data group A template image extraction step for extracting a template image from image data, a shape pattern matching step for performing shape pattern matching using the template image, and a scale of the result of the shape pattern matching in the comparison image data group were compared. Based on the result, the comparison image for determining the photographing direction of the image data of the comparison image data group Data group shooting direction determination step, shooting direction determination step for determining the shooting direction of the image data based on the shooting direction determined by the comparative image data group shooting direction determination step, and results of the determination of the shooting direction determination step And a shooting direction output step for outputting.

  Also by the above method, by using the template image extracted from one image data in the comparison image data group, shape pattern matching is performed on the other image data, and by comparing the size of the matched portion and the template image, It can be determined whether or not the image data is image data obtained by photographing the traveling direction of the moving body.

  In another aspect of the present invention, a shooting direction determination program executed by an apparatus including a computer includes an image data holding unit that holds image data shot by a shooting camera mounted on a moving body, a shooting time, and the image data. Shooting-related data holding means for holding shooting-related data defining the relationship between the image data, and comparison image data group determining means for determining a comparison image data group composed of a plurality of image data based on the shooting time Template image extracting means for extracting a template image from image data of the comparison image data group, shape pattern matching means for performing shape pattern matching using the template image, and the shape pattern in the comparison image data group Based on the result of comparing the scales of the matching results, the comparison image A comparison image data group shooting direction determination unit that determines a shooting direction of the image data of the data group, and a shooting direction of the image data based on the shooting direction determined by the comparison image data group shooting direction determination unit. The computer is caused to function as a shooting direction determination unit and a shooting direction output unit that outputs a result determined by the shooting direction determination unit. By executing this program on a computer, the above-described photographing direction determination device can be realized. The shooting direction determination program can be suitably handled in a state where it is recorded on a recording medium.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[Shooting direction determination device]
FIG. 1 shows a configuration of an imaging direction determination apparatus 1 according to an embodiment of the present invention. The imaging direction determination device 1 includes image data captured by a vehicle (for example, a vehicle travel type field survey vehicle that is a map survey vehicle) equipped with a front shooting camera and a rear shooting camera, and information that the vehicle has traveled. Is a device that determines whether each image data is captured by a camera in front of the vehicle using the image data and the travel log data. The photographing direction determination device 1 is substantially constituted by a personal computer or the like. The imaging direction determination apparatus 1 includes a control unit 20, a storage unit 36, and an input device 60 as illustrated. The storage unit 36 is a hard disk or the like, and holds a travel log DB 120, a plurality of image data 140, and a detection result list 150. The control unit 20 is a CPU (Central Processing Unit) or the like, and includes a comparison image data group determination unit 101, a template image extraction unit 102, a curve determination unit 103, a shape pattern matching unit 104, and a comparison image data group. A shooting direction determination unit 105, a shooting direction determination unit 106, and a shooting direction output unit 107 are provided. That is, when the control unit 20 executes a predetermined program, the comparison image data group determination unit 101, the template image extraction unit 102, the curve determination unit 103, the shape pattern matching unit 104, and the comparison image data group. An imaging direction determination unit 105, an imaging direction determination unit 106, and an imaging direction output unit 107 are realized.

  The travel log DB 120 is a database that holds the travel log data 121. FIG. 2 shows the data structure of the travel log data 121. The travel log data 121 includes position information 122, an imaging time 123, a travel direction 124, image file information (image data storage destination and travel survey identifier) 125 that is information related to image data, camera information 126, and the like. This information includes back travel presence / absence information 127, speed information 128, and gyro data 129. Here, the position information 122 is specifically longitude / latitude information and indicates the position of the investigation vehicle at the time of shooting. The shooting time 123 indicates the time when an image is shot by the camera. The traveling direction 124 is an angle with respect to the north. The camera information 126 is information relating to the camera that captured the image data (such as an identification number). The back travel presence / absence information 127 is information indicating whether or not back travel was performed when the travel log data 121 was generated. The speed information 128 indicates the traveling speed at the time of shooting. The gyro data 129 is angular velocity information or the like, and is data for determining whether or not the vehicle is traveling on a curve.

  The detection result list 150 is a list that holds the determination result of the shooting direction and the like. FIG. 3 shows an example of the detection result list 150. As illustrated, the detection result list 150 includes an image ID 151, a cycle ID 152, a shooting time 153, and a shooting direction 154 that can uniquely identify the image data 140. Here, the cycle ID 152 is an identifier indicating a travel survey unit (for example, “001” for the first survey, “002” for the second survey, etc.), and the cycle ID 152 indicates the image file information 125 in the travel log data 121. Corresponds to the travel survey identifier. The shooting time 153 indicates the time when the camera of the traveling research vehicle has captured the image data, and corresponds to the shooting time 123 of the travel log data 121. The shooting direction 154 indicates the shooting direction information of the image data 140 shot by the camera, and is set by the shooting direction output unit 107 described later.

  The comparison image data group determination unit 101 extracts a comparison image data group that is a set for comparing image data based on the shooting time 123 of the travel log data 121. Specifically, a set of image data 140 with the latest photographing time 123 is extracted.

  Note that, when the extracted image data 140 is image data that is running on a curve, the comparison image data group determination unit 101 delays the shooting direction determination order of the comparison image data group including the image data 140.

  The template image extraction unit 102 sets a predetermined area (hereinafter also referred to as a template area) of one image data 140 in the comparison image data group as a template, and extracts the image data of the template. Note that the template image extraction unit 102 in this embodiment sets a plurality of regions of the image data 140 as templates.

  When the curve determination unit 103 refers to the speed information 128 of the travel log data 121 corresponding to the image data 140 of each comparison image data group and is below a predetermined threshold, the image data 140 is captured during the curve travel. It is determined that

  The shape pattern matching unit 104 performs shape pattern matching on the image data 140 other than the image data 140 from which the template image is extracted from the comparison image data group, using the template image extracted by the template image extraction unit 102. .

  The comparison image data group shooting direction determination unit 105 determines the shooting direction of the image data of the comparison image data group based on the result of the shape pattern matching. Specifically, when the template image extraction unit 102 has extracted the template image from the image data with the shooting time in the image data of the comparison image data group, the shape pattern matching unit 104 has the shooting time. As a result of performing shape pattern matching on the subsequent image data using the template, the comparison image data group shooting direction determination unit 105 determines that the template portion in the image data after the shooting time is enlarged. If the template portion in the image data after the shooting time is reduced, it is determined that the direction opposite to the traveling direction of the vehicle is being shot.

  Here, a method in which the comparison image data group photographing direction determination unit 105 compares the two image data 140 will be described with reference to FIG. It is assumed that there are two image data 140A and image data 140B, and the shooting order is the order of image data 140A and image data 140B.

  The comparison image data group photographing direction determination unit 105 selects two features A (Xa, Ya) and feature B (Xb, Yb) that are reflected in both the image data 140A and the image data 140B, and the image data The representative points of the feature A and feature B of 140A are PA1 and PB1, and the representative points of the feature A and feature B of the image data 140B are PA2 and PB2.

  The comparison image data group photographing direction determination unit 105 calculates a representative point distance L1 of the image data 140A and a representative point distance L2 of the image data 140B.

  The comparative image data group photographing direction determination unit 105 calculates a representative point distance ratio obtained by dividing the representative point distance L1 of the image data 140A by the representative point distance L2 of the image data 140B.

  As a result of calculating the representative point distance ratio, if the representative point distance ratio is smaller than 1, the representative point distance L2 is longer than the representative point distance L1, and the features A and B Indicates that it is expanding. Therefore, the comparative image data group shooting direction determination unit 105 determines that the image data 140A and the image data 140B are image data obtained by shooting the traveling direction of the vehicle. Further, as a result of calculating the representative point distance ratio, if the representative point distance ratio is larger than 1, the representative point distance L2 is shorter than the representative point distance L1, and the vehicle can Since it is away from the feature B, the comparative image data group photographing direction determination unit 105 determines that the image data 140A and the image data 140B are image data obtained by photographing the direction opposite to the traveling direction of the vehicle.

  As described above, the comparison image data group photographing direction determination unit 105 compares the distance between two features in the matched range with respect to the two image data 140, so that the image data 140 can be easily obtained. It can be determined whether or not the image data is an image of the traveling direction.

  The shooting direction determination unit 106 finally determines the shooting direction of the image data based on the shooting direction determined by the comparison image data group shooting direction determination unit 105. Specifically, when there are a plurality of comparison image data groups, the shooting direction determination unit 106 has many shooting direction results for each comparison image data group determined by the comparison image data group shooting direction determination unit 105. The shooting direction result is finally determined as the shooting direction of the image data.

The shooting direction output unit 107 outputs the shooting direction result determined by the shooting direction determination unit 106. Specifically, the shooting direction output unit 107 outputs the shooting direction result in the shooting direction 154 of the detection result list 150.
Note that the travel log data 121 in this embodiment functions as shooting-related data.

[Shooting direction determination method and shooting direction determination processing procedure]
Here, the photographing direction determination method will be described. In the present embodiment, the shooting direction determination method is to determine a plurality of comparison image data groups each consisting of a set of image data whose shooting times are close, compare the image data of the comparison image data, and compare the image data. This is a method for determining the shooting direction based on the above, and finally determining the shooting direction of the largest number of determination results as the shooting direction of the image data.

  First, the comparative image data group determination unit 101 searches the travel log data 121 and acquires the image data 140 corresponding to the travel log data 121 having the same camera identifier in the order of the photographing time.

  The comparison image data group determination unit 101, when the back travel presence / absence information 127 of the corresponding travel log data 121 is “present” in the acquired image data 140, compares the image data 140 with the comparison image. Not subject to data group extraction. Specifically, as shown in FIG. 5A, the back travel presence / absence information 127 of the travel log data 121 corresponding to the image data 140-4, the image data 140-8, and the image data 140-10 is set to “present”. If so, the comparison image data group determination unit 101 excludes the image data 140-4, the image data 140-8, and the image data 140-10 from the comparison image data group extraction target.

  Then, as shown in FIG. 5B, the comparison image data group determination unit 101 determines a comparison image data group made up of a set of image data 140 whose shooting times are close.

  Then, as shown in FIG. 5C, the comparison image data group determination unit 101 avoids the bias of the scene (the comparison image data group including the image data 140 at the time of low speed running is not collected. ), The order of the comparison image data group is changed.

  According to this, the photographing direction determination apparatus 1 can prevent the erroneous recognition of the photographing direction from occurring by making the photographing direction determination order of the comparative image data group free from scene bias.

  Further, as a result of referring to the speed information 128 of the travel log data 121 corresponding to the image data 140 of the comparison image data group, the comparison image data group determination unit 101 determines that the image data is determined to be less than a predetermined threshold value. The determination order of the comparison image data group including 140 is changed. FIG. 5D shows an example in which the comparison image data group determination unit 101 changes the shooting direction determination order when the speed information 128 of the travel log data 121 corresponding to the image data 140-12 is a speed equal to or lower than the threshold. Indicates. In this case, the comparison image data group determination unit 101 changes the shooting direction determination order of the comparison image data group including the image data 140-12. As described above, the imaging direction determination device 1 travels on a curve in consideration of the fact that there are many cases in which image data 140 captured during curve travel cannot be matched even if shape pattern matching is performed using a template image. The determination order of the comparison image data group including the image data 140 photographed at a speed that can be estimated as follows is postponed. As a result, the photographing direction determination apparatus 1 can preferentially determine the comparative image data group that has a high possibility of shape pattern matching as compared to the image data 140 that is running on a curve.

  The template image extraction unit 102 extracts a template image from one of the image data 140 of the comparison image data group, and the shape pattern matching unit 104 uses the template image to perform shape pattern matching on the other image data 140. I do.

  Then, the comparison image data group shooting direction determination unit 105 matches the image data 140 after the shooting time in the comparison image data group when the template image related to the template of the image data 140 with the previous shooting time is enlarged. Then, it is determined that the image data 140 is the image of the traveling direction of the vehicle. Further, the comparison image data group shooting direction determination unit 105 shoots the reverse direction of the traveling direction of the vehicle when the template image related to the template of the previous image data 140 in the comparison image data group is reduced. It is determined that the image data 140 is the target. The comparison image data group shooting direction determination unit 105 determines that the shooting time is unknown when the template of the previous image data 140 is enlarged or reduced without matching. As described above, the comparison image data group shooting direction determination unit 105 determines the shooting direction by comparing the sizes of the matching regions in the image data 140 of the comparison image data group.

  A specific example will be described with reference to FIG. Assume that there are image data 140A and image data 140B, which are comparison image data groups as shown in FIG. 6, and the image data 140A is ahead of the image data 140B.

  The template image extraction unit 102 extracts a template image of the template area 161 of the image data 140A. Then, the shape pattern matching unit 104 enlarges or reduces the template 161 and performs shape pattern matching on the image data 140B. As shown in FIG. 6, the comparison image data group photographing direction determination unit 105 performs shape pattern matching. As a result, the region 162 corresponding to the template 161 in the image data 140B is larger than the template 161 in the image data 140A. It is determined that the traveling direction of the vehicle is being photographed.

  When the comparison image data group shooting direction determination unit 105 determines whether or not all the comparison image data groups are the image data 140 in the vehicle traveling direction, the shooting direction determination unit 106 determines whether the comparison image data group is the comparison image data. Among the shooting directions of the group of image data 140, the shooting direction with the largest number is set as the shooting direction of each image data 140.

  Then, the shooting direction output unit 107 outputs the result determined by the shooting direction determination unit 106 in the shooting direction 156 of the detection result list 150.

  As described above, the imaging direction determination apparatus 1 acquires the image data 140 in the order of the imaging time, determines a comparison image data group for the image data 140, and performs shape pattern matching on the image data 140 of the comparison image group. And the shooting direction is determined based on the pattern matching result. As a result, the shooting direction determination apparatus 1 can determine more easily than the case of determining the shooting direction by obtaining the optical flow of the image data.

  Hereinafter, the procedure of the shooting direction determination process will be described with reference to the flowchart shown in FIG. This process is realized by the control unit 20 in the photographing direction determination apparatus 1 executing a program prepared in advance.

  First, the comparative image data group determination unit 101 reads the travel log data 121 (step S1), refers to the shooting time 123 of the travel log data 121 and the camera identifier of the camera information 126, and in order of the shooting time, the same camera After acquiring the image data 140 corresponding to the travel log data 121 of the identifier, the image data 140 corresponding to the travel log data 121 whose back travel presence / absence information 127 is “present” is excluded from the comparison image data group. (Step S2).

  Then, the comparison image data group determination unit 101 determines a comparison image data group composed of a set of image data 140 having a close photographing time (step S3). Then, the comparison image data group determination unit 101 rearranges the comparison image data group randomly so that there is no bias in the scene (step S4). Then, the curve determination unit 103 refers to the speed information 128 of the travel log data 121. When the speed is equal to or less than a predetermined speed, the curve determination unit 103 determines the shooting direction of the comparison image data group including the image data 140 corresponding to the travel log data 121. The order of determination is changed (step S5).

  Then, the template image extraction unit 102 extracts a template image from one of the image data 140 of the comparison image data group, and the shape pattern matching unit 104 uses the template image to shape the other image data 140. Pattern matching is performed, and the comparative image data group photographing direction determination unit 105 determines the photographing direction of the image data 140 based on the result of the shape pattern matching (step S6).

  Then, the comparison image data group photographing direction determination unit 105 holds the photographing direction of the image data 140 in the comparison image data group in a memory or the like (step S7).

  If there is a comparison image data group for which the shooting direction has not been determined for the image data 140 of the comparison image data group (step S8; Yes), the process moves to step S6, and images of the comparison image data group that have not been determined When the shooting direction of the data 140 is determined and there is no comparison image data group for which the shooting direction is not determined (step S8; No), the shooting direction determination unit 106 captures the image data 140 in the comparison image data group. Regarding the direction, the most shooting direction is finally set as the shooting direction of the image data 140 (step S9).

  Then, the shooting direction output unit 107 outputs the shooting direction determined by the shooting direction determination unit 106 to the detection result list 150 related to the image data 140 (step S10), and ends the shooting direction determination process.

  As described above, the shooting direction determination device includes image data holding means for holding image data shot by a shooting camera mounted on a moving body, and shooting related data defining the relationship between the shooting time and the image data. A shooting-related data holding means for holding, a comparison image data group determining means for determining a comparison image data group composed of a plurality of image data based on the shooting time, and a template from the image data of the comparison image data group Template image extraction means for extracting images, shape pattern matching means for shape pattern matching using template images, and comparison results based on the results of comparison of scales of shape pattern matching results in comparison image data groups A comparison image data group shooting direction determining means for determining a shooting direction of image data of the image data group; Based on the shooting direction determined by the comparison image data group shooting direction determination unit, a shooting direction determination unit that determines a shooting direction of the image data, a shooting direction output unit that outputs a result determined by the shooting direction determination unit, Is provided.

  As described above, the imaging direction determination device 1 extracts a template image from one image data 140 in the comparison image data group, uses the template image to shape-match the other image data 140, and Based on the magnitude relationship between the template image and the matching portion in the matching result, it can be determined whether or not the image data 140 is image data obtained by photographing the traveling direction of the vehicle. Therefore, since the shooting direction determination apparatus 1 determines the shooting direction by comparing the two image data 140, the shooting is performed more efficiently than when the optical flow of the image data is calculated to determine the shooting direction. The direction can be determined.

  In addition, the shooting direction determination apparatus 1 finally sets the shooting direction of the image data 140 as the largest shooting direction among the shooting directions determined by comparing the image data 140 of each comparison image data group. Thereby, the imaging direction determination apparatus 1 can perform appropriate determination as a whole even if some determination results are incorrect.

  In the above-described embodiment, the case where the shooting direction output unit 107 outputs the result determined by the shooting direction determination unit 106 to a file such as the detection result list 150 has been described. However, the present invention is not limited to this. The results determined by the shooting direction determination unit 106 may be output by other various output methods such as screen display.

  In the above-described embodiment, the case where the comparison image data group determination unit 101 specifies the image data 140 to be compared based on the shooting time 123 and the camera information 126 has been described, but the present invention is not limited thereto. When the shooting direction determination apparatus 1 holds only the image data 140 shot by the same camera, the comparison image data group determination unit 101 extracts the image data 140 based on the shooting time 123. Also good.

  In the above-described embodiment, the case where the curve determination unit 103 detects a curve using an optical flow has been described. However, the present invention is not limited to this, and the gyro data 129 that is data capable of detecting a curve is used as the travel log data 121. In addition, the curve may be detected by referring to the travel log data 121.

  In the above-described embodiment, the case where the curve determination unit 103 estimates the image data during the curve traveling with reference to the speed information has been described. However, the present invention is not limited to this, and the estimation is performed using the optical flow. You may do it. Specifically, when the amount of vertical movement of the optical flow vector is larger than the threshold value, the curve determination unit 103 estimates that the image data 140 is image data 140 captured during traveling.

  In the above-described embodiment, the template image extraction unit 102 extracts only one template image, and the shape pattern matching unit 104 performs shape pattern matching using the template image. Without being limited thereto, the template image extraction unit 102 extracts a plurality of template images, the shape pattern matching unit 104 performs shape pattern matching using the plurality of template images, and performs a comparison image data group photographing direction determination unit. 105 may determine the photographing direction of the image data 140 of the comparison image data group based on the result of shape pattern matching using a plurality of template images.

  According to this, the photographing direction determination apparatus 1 can perform more accurate determination as compared to the case where shape pattern matching is performed using one template image. FIG. 8 shows the result when shape pattern matching is performed using a plurality of templates and the shooting direction is determined. Note that the image data 140 is taken from behind the vehicle.

  FIG. 8A shows the result when shape pattern matching is performed using a template image of only one template region 161A, and FIG. 8B shows the templates of three template regions 161A to 161C. FIG. 8C shows the result when the shape pattern matching is performed using the template images of the five template regions 161A to 161E. As shown in the figure, every time the number of template images increases, the correct number of results in the shooting direction increases.

  In the above-described embodiment, the template image extraction unit 102 extracts one template image from one image data 140 in the comparison image data group, and the shape pattern matching unit 104 uses the template image to obtain the other image data. If there is no portion that matches the template image as a result of performing shape pattern matching on 140, the template image extraction unit 102 may extract a template image other than the template image that has already been extracted. According to this, as a result of performing shape pattern matching on the other image data 140 using the template image extracted from one image data 140 of the comparison image data group, the imaging direction determination device 1 is “Unknown Even if it is determined, it is possible to make an appropriate determination by re-extracting another template image, performing shape pattern matching using the re-extracted template image, and determining the shooting direction based on the matching result. The possibility can be improved.

  An example in which the template image extraction unit 102 re-extracts the template image will be described with reference to FIG. First, the template image extraction unit 102 extracts the template image of the template region 161A in the central portion of the image data 140 that is enlarged with the best balance.

  Then, when the photographing direction determination device 1 determines that it is “unknown” as a result of the shape pattern matching using the template image, the template image extraction unit 102 has a center having more characteristic portions than the center upper portion. The template image of the template area 161B which is the lower part is re-extracted.

  When the photographing direction determination apparatus 1 determines that the pattern is matched with the template image corresponding to the template region 161B and determines that it is “unknown”, the template image extraction unit 102 determines that the center upper portion of the image data 140 If the template image of the template region 161C is re-extracted and the imaging direction determination apparatus 1 determines that the pattern pattern matching is performed using the template image of the template region 161C, the result is “unknown”. "

  As described above, the imaging direction device 1 extracts the template images step by step in the order of the template region 161 in which the image data 140 of the comparison image data group is easy to perform shape pattern matching, and performs shape pattern matching using the template image. If it carries out, the imaging | photography determination processing which balanced the hit rate of shape pattern matching and processing efficiency is realizable.

  In the above-described embodiment, the case where the shooting direction determination apparatus 1 determines the shooting direction for the image data 140 of all the comparison image data groups has been described. However, the present invention is not limited to this, and a part of the comparison images is used. The shooting direction of the image data 140 may be determined only for the data group.

It is a block diagram which shows schematic structure of an imaging | photography direction determination apparatus. It is a figure which shows the data structure of driving log data. It is a figure which shows the structure of a detection result list | wrist. It is a figure which shows the determination method of an imaging | photography direction. It is a figure which shows the extraction method of the image data group for a comparison. It is a figure which shows the specific example which determines an imaging | photography direction. It is a flowchart of a photographing direction determination process. It is an example of shape pattern matching using a plurality of template images. It is an example of the shape pattern matching which re-extracts a template image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Shooting direction determination apparatus 101 Comparison image data group determination part 102 Template image extraction part 103 Curve determination part 104 Shape pattern matching part 105 Comparison image data group shooting direction determination part 106 Shooting direction determination part 107 Shooting direction output part 120 Travel log DB
140 Image data 150 Detection result list

Claims (8)

Image data holding means for holding image data shot by a shooting camera mounted on a moving body;
Shooting related data holding means for holding shooting related data defining the relationship between the shooting time and the image data;
Comparison image data group determination means for determining a comparison image data group that is a set of image data that is composed of a plurality of image data based on the shooting time and that has the latest shooting time ;
Template image extraction means for extracting a template image from the image data of the comparison image data group;
Shape pattern matching means for performing shape pattern matching using the template image;
Based on the determination result of the size relationship between the template images detected in each of the image data included in the comparison image data group by shape pattern matching, the shooting direction of the image data in the comparison image data group is A comparison image data group photographing direction determining means for determining whether the traveling direction or the traveling direction of the moving body is opposite ;
Shooting direction determination for determining whether the shooting direction of the image data is the moving direction of the moving body or the opposite direction of the moving body based on the shooting direction determined by the comparison image data group shooting direction determining means Means,
Shooting direction output means for outputting the result determined by the shooting direction determination means;
An imaging direction determining apparatus comprising: There are a plurality of image data groups for comparison,
The comparison image data group determining means determines a shooting direction of the image data of the comparison image data group in a predetermined order,
2. The photographing according to claim 1, wherein the photographing direction determination unit determines the largest one of the photographing directions determined by the comparative image data group photographing direction determination unit as the photographing direction of the image data. Direction determination device. The comparison image data group determining means determines the comparison image data group by excluding image data captured while the moving body is traveling backward from image data captured by the imaging camera. The imaging direction determination apparatus according to claim 1 or 2.   The template image extraction unit re-extracts another template image when the comparison image data group shooting direction determination unit cannot determine the shooting direction using the extracted template image. 2. A photographing direction determining apparatus according to 2. It further comprises curve determination means for detecting image data taken during the curve running,
3. The photographing direction determination according to claim 2, wherein the comparison image data group extraction unit postpones the photographing direction determination order of the comparison image data group including the image data photographed during the curve travel. apparatus. An image data holding step for holding image data shot by a shooting camera mounted on a moving body;
A shooting-related data holding step for holding shooting-related data defining a relationship between the shooting time and the image data;
A comparison image data group determination step for determining a comparison image data group that is a set of image data that is composed of a plurality of image data and has the latest shooting time based on the shooting time ;
A template image extraction step of extracting a template image from the image data of the comparison image data group;
A shape pattern matching step for performing shape pattern matching using the template image;
Based on the determination result of the size relationship between the template images detected in each of the image data included in the comparison image data group by shape pattern matching, the shooting direction of the image data in the comparison image data group is A comparative image data group photographing direction determination step for determining whether the traveling direction or the traveling direction of the moving body is opposite ;
Shooting direction determination for determining whether the shooting direction of the image data is the moving direction of the moving object or the moving direction of the moving object based on the shooting direction determined in the comparison image data group shooting direction determining step Process,
A shooting direction output step of outputting a result determined by the shooting direction determination step;
An imaging direction determination method comprising: A shooting direction determination program executed by an apparatus including a computer,
Image data holding means for holding image data shot by a shooting camera mounted on a moving body;
Shooting related data holding means for holding shooting related data defining the relationship between the shooting time and the image data;
Comparison image data group determination means for determining a comparison image data group that is a set of image data that is composed of a plurality of image data based on the shooting time and that has the latest shooting time ;
Template image extraction means for extracting a template image from the image data of the comparison image data group;
Shape pattern matching means for performing shape pattern matching using the template image;
Based on the determination result of the size relationship between the template images detected in each of the image data included in the comparison image data group by shape pattern matching, the shooting direction of the image data in the comparison image data group is A comparison image data group photographing direction determining means for determining whether the traveling direction or the traveling direction of the moving body is opposite ;
Shooting direction determination for determining whether the shooting direction of the image data is the moving direction of the moving body or the opposite direction of the moving body based on the shooting direction determined by the comparison image data group shooting direction determining means Means,
An imaging direction determination program that causes the computer to function as an imaging direction output unit that outputs a result determined by the imaging direction determination unit.   A storage medium storing the shooting direction determination program according to claim 7.

Images