JP2006119591A - Map information generation method, map information generation program and map information collection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently generate map information, that is, information on stores or the like to be indicated on a map, which is necessary for creating the map, considering that surrounding information to be indicated on the map which is necessary for map creation etc. is not acquired efficiently in a conventional map creation apparatus. <P>SOLUTION: An image data which is imaged as a imaging means moves, and positioning information which is recorded according to the image data and measured according to the imaging position of the imaging means, are input. A detection object of the inputted image data is detected and relative position information between the detected detection object and the imaging means is calculated and based on the calculated relative position information and the positioning information, the position information of the detection object is generated. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for generating map information.

  Conventionally, as a technique for generating map information, Japanese Patent Application Laid-Open No. 9-269726 discloses a technique for creating a road map using a video camera and a high-precision positioning device. In this prior art, a device in which a video camera and a high-precision positioning device are mounted on a car is used. First, a white line indicating a road lane is detected from an image obtained by a video camera, and from this result, it is detected how far the vehicle is traveling from the center of the lane. Then, the position information at the center of the lane is obtained by using the position information of the vehicle obtained by the high-accuracy positioning device and the information on the deviation of the vehicle from the road center obtained by the camera image processing result. By processing this continuously, it is possible to calculate the position information of the road center and correct the map by comparing with the existing map information.

JP-A-9-269726

  However, what is disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 9-269726 is the centerline of the road to be detected, and it is possible to acquire the peripheral information described on the map necessary for map creation and the like. There wasn't.

  The present invention has been made to solve such problems, and has an object to efficiently generate information such as stores described on a map necessary for map creation, that is, map information. To do.

  In the present invention, video data to be imaged as the imaging unit moves and positioning information recorded corresponding to the video data and positioned in accordance with the imaging position of the imaging unit are input and input. Detecting the detection target in the video data, calculating relative position information between the detected detection target and the imaging means, and detecting the position information of the detection target based on the calculated relative position information and the positioning information. Is generated.

  According to the present invention, map information that does not exist in an existing map can be efficiently generated.

Embodiment 1 FIG.
Embodiments of the present invention will be described below.
FIG. 1 is a configuration diagram showing a configuration of a data collection apparatus that collects information necessary for generating map information in the present embodiment. In the figure, reference numeral 101 denotes GPS, inertial navigation or a combination of them, positioning means for measuring its own position, 102 a video camera for photographing the surrounding landscape, and 103 an orientation sensor for measuring the direction.
The positioning means 101, the video camera 102, and the azimuth sensor 103 are installed in a vehicle 104 as a moving body, and these devices are installed after their positional relationship is accurately measured. For example, the positioning means 101, the video camera 102, and the azimuth sensor 103 are arranged at the center on the same vertical line with respect to the horizon and horizontally with the vehicle traveling direction. In addition, the video camera and the orientation sensor are installed almost in contact. The direction sensor 103 measures the direction and orientation of the vehicle 104.

  Reference numeral 106 denotes a vehicle speed sensor that detects the vehicle speed of the vehicle 104, and 105 denotes a recording device that synchronously records information from the vehicle speed sensor 106 and the positioning means 101, the video camera 102, and the direction sensor 103. The recording device 105, the positioning means 101, the video camera 102, and the direction sensor 103 are connected by a signal line.

Next, the operation of generating map information in the present invention will be described. Here, as map information, information that accurately generates position information of a detection target (for example, a white line) is shown. Furthermore, the map is handled based on the position information.
FIG. 2 is a flowchart showing the operation of generating map information in this embodiment.
First, the information (data) of the information (data) collection target area is collectively collected as follows (step ST201). When the vehicle 104 travels on the road in the information collection target area and starts collecting data, the positioning means 101 sends position information (latitude, longitude, altitude, etc.) in the form of NMEA-0183, video data from the video camera 102, direction The sensor 103 outputs data such as the three-axis angle and angular acceleration of the vehicle 104, and these pieces of information (data) are input to the recording device 105 to start recording. The timing of each data signal is synchronized with the internal time of the recording apparatus at the time of input. Recording in the recording device 105 is stopped at a location that is not the information collection target area.
Thereafter, map information is generated based on the data recorded in the recording device 105.

  First, the video data recorded in the recording device 105 is analyzed to detect a subject, and the image coordinates are acquired (step ST202: input step, detection step). In the present embodiment, the subject is, for example, a white line on the road. The subject is detected by evaluating edge information and linearity from the video data. As shown in FIG. 3, the coordinates at a point on the boundary line between the white line and the road (hereinafter referred to as a target point) are detected. In this embodiment, the white line on the road is a detection target, but in addition to the white line, a road indicator such as an arrow indicating a pedestrian crossing or a dedicated lane, a signboard of a store existing around the road, a traffic sign, etc. To do.

  A known method is used as a white line detection method, and is not particularly mentioned in the present embodiment. In addition, the shape is recognized here, the area is extracted, the character string included in the area is grasped, the coordinates of the outline are calculated, and the patterns such as traffic signs and signboards of chain stores are pre-set. It is memorized as a model, and the type of traffic sign or store can be specified by matching with the model. The result obtained here is a coordinate value sequence in an image coordinate system as shown in FIG.

  Next, the relative position (three-dimensional) between the camera and the target point at the time of shooting is calculated from the obtained image coordinates (two-dimensional) of the target point (step ST203: calculation step). For example, when only one camera is used for an object existing on the road surface, the relative position can be calculated according to the following constraint conditions. The height of the camera installed on the vehicle is known by measurement. At this time, since the relationship between the height of the road surface where the target point exists and the camera can be assumed to be constant except in special cases, the height becomes a known parameter, and the element (Y) of the equation shown below is reduced by one, It can be in the form of simultaneous ternary linear equations including the scale factor s. x, y indicate coordinate points in the image coordinate system, X, Y, and Z indicate spatial coordinates in the camera coordinate system (shown in FIG. 5) centered on the camera, and f indicates the focal length of the camera.

x = sfX / Z (1)
y = sfY / Z (2)
1 = s / Z (3)

  When this equation is solved for X and Z by substituting the known image coordinates m (x, y), focal length f, and height Y from the camera, spatial coordinates (= relative position) M (X, Y, Z) Can be requested. Actually, the calculation is performed using the relationship of m = A [R, t] M using the internal matrix A, translation matrix t, and rotation matrix R of the camera. Here, a simple formula is used for the sake of simplicity of explanation, but it is of course possible to calculate with a relational formula of m = A [R, t] M. In this case, each value of A (3 × 3 matrix) is called an internal parameter of the camera, and a known value can be obtained by performing this measurement in advance. The rotation matrix R is also a 3 × 3 matrix, and the rotation angle of each of the three axes can be measured and input. By using this, it is possible to reduce the error of the calculation result even when the camera angle changes due to vehicle running. t is a 3 × 1 matrix indicating the amount of translation from the camera origin. Normally, as in the case of this embodiment, since the camera coordinates and the position of the origin coincide with each other, the parallel movement amount can be regarded as zero. If the camera installation position is deviated, data can be accurately calculated by adjusting this parameter and calibrating the data collection environment.

  Here, the detection of road indicators on the road is taken as an example, but for objects that do not exist on the road, such as road signs and signboards, two cameras taken simultaneously from two images are taken from the camera. Since a method for calculating a relative three-dimensional position is known, it can be used.

  Next, the geographical position of the target point is calculated using the relative distance information obtained in the process of step ST203 and the positioning information when the target frame is captured (step ST204: target position information generation step). FIG. 6 shows the relationship between the camera coordinate system and the geographic coordinate system. As shown in FIG. 6, the camera coordinate system has an angle with respect to the vertical axis of the geographic coordinate system (north direction in the case of the northern hemisphere). As a unit to be handled, a unit in the camera coordinate system is meter, and a unit in the geographic coordinate system is latitude and longitude.

  The direction of the Z axis in the camera coordinate system is the traveling direction of the vehicle. This can be acquired by an azimuth sensor as an azimuth angle. Since the positioning information of the camera at the time of shooting can be obtained as latitude and longitude, the origin O of the camera coordinates has a relationship as shown in the upper part of FIG. 7 in the geographic coordinate system. At this point, the camera coordinates and geographic coordinates are considered to be parallel. The latitude and longitude of the target point M can also be converted based on the latitude and longitude of the origin O. However, since there is actually an angle indicated by an azimuth angle between the camera coordinate system and the geographic coordinate system, the target point M is rotated around the origin O using the azimuth information. The image after rotation is the lower figure of FIG. Rotation can be obtained by multiplying a 2 × 2 matrix by a widely known rotation matrix. By this calculation, the coordinates of the target point indicated by the relative distance in the camera coordinate system can be converted into coordinate values in the geographic coordinate system. The geographical coordinates calculated in this way are recorded in a recording area such as a hard disk.

Next, it is determined whether the frame being processed is the final frame (step ST205 in FIG. 2). If it is not the last frame, the process returns to step ST202, and if it is the last frame, the process after step ST206, that is, the process for associating the detection target on the map (map correspondence step) is executed.
In the process of step ST206, the position information sequence accumulated in the recording area is plotted on the map. The plot result is as shown in FIG.

  Next, a process of line segmentation by connecting the plotted coordinates is performed (step ST207). Line differentiation can be performed by general linear interpolation, or quadratic or higher-order curve interpolation. If there is a part where there is no information such as a road break, it is checked whether the data is not more than a certain value, and if it is far away, the line segment is divided there.

  As a result, the white line information on the road can be vectorized by information such as the start point, end point, and curvature. At this time, instead of using all the calculated position information points equally as points to be vectorized, the priority of the points used by the vehicle speed information obtained simultaneously with the video by the vehicle speed sensor is changed. Thus, errors due to movement of the vehicle can be suppressed, and the accuracy of information to be added can be increased. For example, since the position and video recording blur does not occur in the stopped state, the priority at this time is set to, for example, the highest level 5 out of 5 levels. The acquired data up to 10 km / h is set to 4, 3 from 10 km / h to 30 km / h, 2 from 30 km / h to 60 km / h, and 1 from 60 km / h. By using this priority, the point sequence information can be vectorized with high accuracy.

As shown in the upper part of FIG. 9, if the result of simple plotting is used to perform vector interpolation by linear interpolation, a curved line may be obtained. On the other hand, when the priority is used, for example, since the confidence line of the plot point of priority 3 is low, the weighting coefficient for averaging is set to a small value 0.5, and the place of priority 5 is reliable. Since the data is used as it is, the weighting coefficient is set to 1, and the priority 4 is set to 0.75. By using this to vectorize the line segment, information on points with low reliability can be dropped and information on points with high reliability can be adopted, so point sequence information with high accuracy as shown in the lower part of FIG. Can be vectorized.
Then, the data vectorized in step ST207 is managed as one layer in step ST208. This can be done by using a method used in general map software.

  As described above, detailed information that does not exist in the existing map can be newly generated and added to the map, and information such as stores on the map necessary for map creation can be efficiently generated. .

  In addition, text information and feature information is read from an object (for example, traffic sign) recognized from video data, the type of object is determined, an attribute is added, and the restoration result is overlaid on the existing map together with the attribute. It is also possible to add information that is not in the map, and it is possible to generate and add more detailed information. In addition to traffic signs, the object may be a display board indicating a signboard or a building, and the store or building may be identified by feature information obtained from the object, and the store information may be added to the map.

The constituent elements described in the above embodiments can be replaced with other means within a range that does not change the gist thereof.
It is also possible to install a plurality of video cameras 102 on the vehicle 104 and acquire images from a plurality of angles at the same time. The video camera 102 may be replaced with another one as long as it can acquire an image such as a line sensor.

  The above description of the embodiment shows a method for generating map information, but it can also be realized by software operating on a computer.

Embodiment 2. FIG.
The second embodiment will be described below. In the first embodiment, the method of positioning the target object with one vehicle collection device has been described. In the second embodiment, a target object is detected from video data collected using a large number of vehicles and a plurality of sensors, and a threshold value process is performed on a value obtained by accumulating the detection results, thereby detecting the target object with high accuracy.・ Explain the method of positioning.

  Here, the object means map information to be collected. For example, a road sign, a white line on the road, a pedestrian crossing, a store signboard around the road, a traffic light, a telephone pole , Etc.

FIG. 10 is a configuration diagram of the map information collecting apparatus according to the second embodiment. The map information collection apparatus 1000 includes a video information / position information database 1001, an object information collection unit 1002, a map database 1003, and an object information accumulation unit 1004.
The video information / position information database 1001 is a storage area for storing video information and vehicle position information synchronized therewith, and is constructed in a storage device provided inside or outside the map information collection device 1000.
The target information collection unit 1002 includes a target detection unit 1005, a target positioning unit 1006, and a target storage unit 1007.
Further, the object detection unit 1005 includes an image scanning unit 1010, a similarity calculation unit 1011, a rectangular area merging unit 1012, and a threshold processing unit 1013.
The map database 1003 is a storage area for storing the three-dimensional coordinates of the latitude / longitude / elevation of an object and other information related to the object, and is constructed in a storage device provided inside or outside the map information collection apparatus 1000. Is done.
The object information accumulating unit 1004 includes an object merging unit 1008 and a threshold processing unit 1009.

  Here, the relationship between the components according to the present embodiment and the components according to the first embodiment will be described. First, the video information / position information database 1001 corresponds to the recording apparatus 105 described in the first embodiment. The object detection unit 1005 is a processing device corresponding to the detection step (step ST202) described in the first embodiment. The object positioning unit 1006 is a processing device corresponding to a calculation step (step ST203) and a target position information generation step (step ST204).

  Next, the operation will be described. The operation of the map information collection apparatus 1000 according to the present embodiment includes an object information collection phase in which an object is detected and measured from video information and position data synchronized therewith and stored in the map database 1003, and stored in the map database 1003. The object information accumulation phase for accumulating the information of the target object and two phases.

First, the object information collection phase will be described with reference to FIG. FIG. 11 is a flowchart showing the operation of the object information collection phase. First, using the configuration described in the first embodiment, using a large number of vehicles equipped with GPS and video cameras (or sensors), video data of roads and surrounding scenes, and vehicle position information synchronized with the video data Are stored in the video information / position information database 1001 (step ST1101). Note that the video data in each vehicle and the method of collecting position information synchronized with the video data are the same as those in the first embodiment, and thus description thereof is omitted.
Here, the video data and the position information data may be collected by changing the date in the same vehicle. Further, a plurality of video cameras and sensors may be mounted on one vehicle, and video information and positioning information data of roads in the same section may be collected from different positions and angles. By doing so, it is possible to collect data uniformly over a wide area of the city.
In addition, the video data collected in step ST1101 is not limited to a visible image captured by a video camera. For example, depth video data obtained by a line scan laser, thermal video data obtained by an infrared camera, and the like from sensors other than the camera. The obtained video data may be used.

  Next, the object detection unit 1005 selects one of a large number of video information stored in the video information / positioning information database 1001 (step ST1102).

  Next, the object detection unit 1005 acquires an image for only one frame from the video information selected in step ST1102 (step ST1103).

Next, the object detection unit 1005 analyzes the image acquired in step ST1103 to detect a rectangular area where the object appears, and obtains the similarity of the rectangular areas at the same time (step ST1104). For example, when road facility management is performed based on video information, the object detection unit 1005 detects road signs, traffic lights, utility poles, and the like that appear in the input image, and outputs a rectangular area surrounding them. The similarity of the area is calculated.
Here, the similarity is defined as “a numerical expression of the degree of similarity of an image of a certain rectangular area with an image of a target object”. Hereinafter, in the present embodiment, for example, the higher the possibility that the image in the rectangular area is the target object, the higher the positive value, and the lower the possibility that the image in the rectangular area is the target object. It shall take a negative value.
As a method of calculating the similarity, for example, it is possible to calculate the similarity between the template image and the rectangular area by using template matching. When the color of the object is specified, the similarity between the color of the rectangular area and the color specific to the object may be calculated. Further, for example, by using the algorithms described in Reference Documents 1 and 2, it is possible to detect a rectangular region of an object with high accuracy and high speed and calculate the similarity.
[Reference 1]
Paul A. Viola, Michael J. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), ISSN: 1063-6919, Vol. 1, pp. 511 -518, December 2001
[Reference 2]
Paul A. Viola, Michael J. Jones, “Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade.” Advances in Neural Information Processing Systems 14, p1311-1318, 2001, MIT Press,

  Here, a method of calculating the similarity using the object detection algorithm described in Reference Document 1 and Reference Document 2 will be described.

数式１において、confは判別関数の出力を累積した値であり、xは入力された矩形領域の画像であり、θは検出率を調整するための閾値であり、tは判別関数のインデックスであり、Tは加算する判別関数の総数である。
また、h_t(x)は判別関数であり、画像xが対象物の画像である可能性が高ければ正の値を返し、画像xが対象物である可能性が低ければ負の値を返す。例えば、道路標識を検出したい場合、ある判別関数は道路標識に固有の形を検出することで道路標識である可能性の大小を数値化して返し、また、ある判別関数は道路標識に固有の色を検出して目的の道路標識である可能性の大小を数値化して返す。
数式１において、xが目的の対象物の画像である可能性が高いほどconfは大きな正の値をとり、またxが対象物の画像である可能性が低いほどconfは負の値をとる。このため、対象物検出部１００５において、数式１に示すconfを類似度として使用することが可能である。
なお、上記の判別関数を求める方法や、閾値θを決定する方法については文献００２および文献００３に記載されているのでここでは言及しない。 In References 1 and 2, Viola & Jones et al. Used a learning algorithm called Adaboost to detect an object with high accuracy and high speed. This is done by accumulating the output of a function that discriminates whether or not the input image is a target object based on information about feature quantities such as edges (hereinafter referred to as discriminant function), and depending on the magnitude of the accumulated value. Thus, it is determined whether or not the input image is an object. An equation for accumulating the output of the discriminant function is given by Equation 1 shown below.

In Equation 1, conf is a value obtained by accumulating the output of the discriminant function, x is an image of the input rectangular area, θ is a threshold value for adjusting the detection rate, and t is an index of the discriminant function. , T is the total number of discriminant functions to be added.
H _t (x) is a discriminant function, which returns a positive value if the image x is likely to be an image of the object, and returns a negative value if the image x is unlikely to be an object. . For example, if you want to detect a road sign, a certain discriminant function quantifies and returns the possibility of being a road sign by detecting the shape unique to the road sign. Is detected, and the possibility of being the target road sign is converted into a numerical value and returned.
In Equation 1, conf takes a larger positive value as the possibility that x is an image of the target object is higher, and conf takes a negative value as the possibility that x is an image of the object is lower. For this reason, the object detection unit 1005 can use conf shown in Equation 1 as the similarity.
Note that the method for obtaining the discriminant function and the method for determining the threshold θ are described in Documents 002 and 003, and are not described here.

  Hereinafter, a method will be described in which the object detection unit 1005 calculates the rectangular area of the object and the similarity thereof using the object detection algorithms described in Reference Document 1 and Reference Document 2, for example. FIG. 12 is a flowchart showing the operation of the object detection unit 1005. FIG. 12 describes in detail the operation of the object detection unit 1005 in step ST1104 shown in FIG.

First, the image scanning unit 1010 receives the image acquired in step ST1103 as an input (step ST1201).
Next, the image scanning unit 1010 generates a plurality of images obtained by reducing the original image at different rates in order to detect objects of different sizes (step ST1202). FIG. 14 is an explanatory diagram illustrating an operation example of the image scanning unit 1010. For example, as shown in FIG. 14A, the original image is reduced to generate a plurality of images having different sizes.
Next, the image scanning unit 1010 scans the reduced image generated in step ST1202 by moving the rectangular area pixel by pixel (step ST1203).

Next, similarity calculation section 1011 calculates the similarity of the rectangular area scanned in step ST1203 (step ST1204). For example, Formula 1 is used for calculating the similarity.
Next, the similarity calculation unit 1011 determines whether or not the similarity of the input rectangular area is greater than 0 (step ST1205).
Next, if the similarity of the input rectangular area is greater than 0, the similarity calculation unit 1011 regards this rectangular area as an image of the object, and stores the coordinates and similarity of the rectangular area ( Step ST1206). FIG. 15 is an explanatory diagram illustrating an example of a rectangular region of an object. Here, the coordinates of the rectangular area refer to four coordinate values x1, x2, y1, and y2 that specify the position of the rectangular area in the image coordinate system XY, as shown in FIG. On the other hand, if the similarity of the input rectangular area is not greater than 0 in step ST1205, the process proceeds to step ST1207.

  Next, image scanning section 1010 determines whether or not all reduced images generated in step ST1202 have been scanned (step ST1207). If there is a reduced image that has not been scanned yet, the process returns to step ST1203 until all the images are scanned, and the process of detecting the rectangular region of the object is repeated. If all the reduced images have been scanned, the process proceeds to step ST1208.

  By the processing from step ST1203 to step ST1207, normally, as shown in FIG. 14B, a plurality of rectangular areas are detected overlapping one object. Therefore, the rectangular area merging unit 1012 merges these overlapping rectangular areas when the areas of the rectangular areas overlap by a certain ratio or more in order to combine the saved rectangular areas into one. Output (hereinafter, merge process) (step ST1208). At this time, the coordinates of the new rectangular area are weighted averages based on the similarity of the coordinates of the plurality of rectangular areas. The similarity of the new rectangular area is the sum of the two similarities.

FIG. 16 is an explanatory diagram illustrating an operation example of the rectangular area merging unit 1012. An example of the operation in step ST1208 will be described with reference to FIG. As shown in FIG. 16A, it is assumed that a rectangular area A, a rectangular area B, and a rectangular area C are detected in a two-dimensional image coordinate system. Here, it is assumed that the rectangular area A has a similarity of 5, the rectangular area B has a similarity of 10, and the rectangular area C has a similarity of 10. Since the areas of the rectangular area A and the rectangular area B overlap each other by a certain amount or more, these two areas are merged to generate a new rectangular area D (FIG. 16B). The coordinates of the rectangular area D are weighted averages of the coordinates based on the similarity between the rectangular area A and the rectangular area B. Further, the similarity of the rectangular area D is calculated as 10 + 5 = 15.
A specific calculation example of step ST1208 is shown below. In FIG. 16A, the coordinates of the rectangular area A are (a_x1, a_x2, a_y1, a_y2), and the coordinates of the rectangular area B are (b_x1, b_x2, b_y1, b_y2). Further, the similarity of the rectangular area A is a_conf, and the similarity of the rectangular area B is b_conf. At this time, the coordinates (d_x1, d_x2, d_y1, d_y2) of the rectangular area D in FIG.
d_x1 = (a_conf × a_x1 + b_conf × b_x1) / (a_conf + b_conf)
d_x2 = (a_conf × a_x2 + b_conf × b_x2) / (a_conf + b_conf)
d_y1 = (a_conf × a_y1 + b_conf × b_y1) / (a_conf + b_conf)
d_y2 = (a_conf × a_y2 + b_conf × b_y2) / (a_conf + b_conf)
... (Formula 2)
The similarity d_conf of the rectangular area D is given by Equation 3.
d_conf = a_conf + b_conf (Formula 3)

Finally, threshold processing section 1013 performs threshold processing on the similarity added in step ST1208, and rejects a rectangular area having a similarity equal to or lower than a certain threshold (step ST1209). An operation example of the threshold processing unit 1013 is shown in FIGS. In FIG. 16 (b), there are a rectangular area D having a similarity 15 and a rectangular area C having a similarity 10. Since no other rectangular area exists in the vicinity of the rectangular area C, the merging process is not performed, and the degree of similarity remains at 10. At this time, it is assumed that a rectangular process having a similarity of 10 or less is subjected to threshold processing for rejection. In this process, the rectangular area C is rejected. On the other hand, the rectangular region D has a similarity of 15, so it remains without being rejected. This rectangular area D is set as the output of the threshold processing unit 1013.
Usually, since a plurality of rectangular areas are detected in the vicinity of the object, the similarity can be increased by merging them. On the other hand, when erroneously detected due to the influence of noise or the like, a plurality of rectangular areas are rarely detected in an overlapping manner, and the merge process is not performed, and the similarity between the rectangular areas remains low. For this reason, it is possible to reduce false detections by performing threshold processing after performing merge processing and accumulating similarities.

  As described above, the object detection unit 1005 detects the object from the input image using, for example, the object detection algorithms described in Reference Document 1 and Reference Document 2, and calculates the rectangular area and similarity. can do.

  In addition, the method of calculating the similarity of the object in the object detection unit 1005 is not limited to the algorithm described in References 1 and 2, and the target object image and the image of the rectangular area are similar. Any algorithm may be used as long as the degree can be calculated numerically.

  The operation of the object information collection unit 1002 will be described again using FIG. The object positioning unit 1006 uses the coordinates (two-dimensional) of the rectangular area of the object detected in step ST1104 to calculate the relative position (three-dimensional) of the object around the camera at the time of photographing ( Step ST1105). Note that the method for calculating the relative position of the target object from the coordinates of the rectangular area of the target object is the same as the process in step ST203 in the first embodiment, and thus the description thereof is omitted in the second embodiment.

  Next, the target object positioning unit 1006 calculates the geographical coordinates of the target object using the relative position information obtained in the process of step ST1105 and the positioning information when the target frame currently being processed is photographed. (Step ST1106). Note that the calculation of the geographical coordinates of the object is the same as the process in step ST204 of the first embodiment, and thus the description thereof is omitted in the second embodiment.

  Next, the object storage unit 1007 stores information on the object detected from the video information in the map database 1003 (step ST1107). Here, the object information is, for example, the geographical coordinates (latitude, longitude, altitude) of the object obtained in step ST1106, the similarity of the object, the type of the object, the size of the object, and the extracted object. It refers to information about an object such as an image of an object.

  Next, the object information collection unit 1002 determines whether or not the image acquired in step ST1103 is the last frame of the video information acquired in step ST1102 (step ST1108). If the image acquired in step ST1103 is the final frame of the video information acquired in step ST1102, the process proceeds to the next step ST1109. Otherwise, the process returns to step ST1103 to repeat the process of acquiring the next frame from the video information and detecting the object.

  Next, the object information collection unit 1002 determines whether or not the video information currently being processed is the last video information stored in the video information / positioning information database 1001 (step ST1109). If video information that has not yet been subjected to the object detection process remains in the video information / positioning information database 1001, the process returns to step ST1102 to process the next video information. If the video information currently being processed is the last video information stored in the video information / positioning information database 1001, the processing of the object information collection phase is terminated.

  As described above, in the object information collection phase, the object information collection unit 1002 detects an object from video information collected by a large number of vehicles and sensors, and stores the position information in the map database 1003.

  Next, the object information accumulation phase will be described. In the above-described object information collection phase, for example, when information on the same object is collected by two vehicles, these are individually stored in the map database 1003. Therefore, in the object information accumulation phase, a plurality of objects at the same geographic coordinates are regarded as the same, and processing for combining the object information into one is performed. Below, the operation | movement procedure of a target object information accumulation phase is demonstrated using FIG. FIG. 13 is a flowchart showing the operation of the object information accumulation phase.

  First, the object information accumulating unit 1004 selects one object from the map database 1003 in numerical order, and acquires information on the object (step ST1301). It is assumed that the information on the target object stored in the map database 1003 is numbered according to characteristics such as the stored date and position.

  Next, the object merging unit 1008 searches the map database 1003 and checks whether there is another object in the vicinity of the geographical coordinates of the object selected in step ST1301 (step ST1302). If another object is found in the vicinity of the geographical coordinates of the object selected in step ST1301, the process proceeds to the next step ST1303. If no other detected object is found in the vicinity of the geographical coordinates of the object selected in step ST1301, the process returns to step ST1301 and information on the next object is acquired.

  Next, the object merging unit 1008 combines the information on the object selected in step ST1301 and the information on the object existing in the vicinity of the geographical coordinates of the object (step ST1303). Hereinafter, an operation example of Step ST1303 will be described with reference to FIGS. 17 and 18.

  First, a method for expressing an object will be described with reference to FIG. FIG. 17 is an explanatory diagram illustrating an example in which the object is a rectangular parallelepiped. As shown in FIG. 17, the object is represented by a rectangular parallelepiped having an appropriate size in a three-dimensional space of longitude, latitude, and altitude. Here, the size of the rectangular parallelepiped representing the object is determined by the actual object size and the positioning accuracy of the geographic coordinates in the object information collection phase. For example, when a 50 cm square road sign with a negligible thickness is measured with an accuracy of 2 cm of standard deviation, 6 cm, which is three times the standard deviation, is added to the size of the road sign as shown in FIG. The size of the rectangular parallelepiped is calculated.

Next, with reference to FIG. 18, a description will be given of a method for combining information on objects. FIG. 18 is an explanatory diagram illustrating an operation example of the object information accumulation unit 1004. As shown in FIG. 18A, it is assumed that there are three objects, that is, an object A, an object B, and an object C in a three-dimensional space of latitude, longitude, and altitude on the map database 1003. Each object also has similarity information calculated in the object information collection phase. In this example, it is assumed that the object A has a similarity of 5, the object B has a similarity of 10, and the object C has a similarity of 5.
Here, as a reference for determining whether or not the object B exists in the vicinity of the geographical coordinates of the object A, for example, it is conceivable to use a volume in which the rectangular parallelepipeds of the object overlap. For example, in FIG. 18A, it is assumed that the volumes of the object A and the object B overlap each other by a certain amount or more. At this time, the object A and the object B are combined into one, and a new object D is generated (hereinafter, merge processing). Further, the information on the object A and the object B in the map database 1003 is deleted, and the information on the object D is newly stored in the map database 1003.

A specific calculation method is shown below. The object A is assumed to be a rectangular parallelepiped surrounded by the six planes of Equation 4.
X = a_x1, X = a_x2, Y = a_y1, Y = a_y2, Z = a_z1, Z = a_z2 (Equation 4)
Similarly, it is assumed that the object B is a rectangular parallelepiped surrounded by six planes of Equation 5.
X = b_x1, X = b_x2, Y = b_y1, Y = b_y2, Z = b_z1, Z = b_z2 (Equation 5)
Further, the similarity of the object A is a_conf, and the similarity of the object B is b_conf.
At this time, the coordinates (d_x1, d_x2, d_y1, d_y2, d_z1, d_z2) of the object D are weighted averages based on the similarity between the coordinates of the object A and the object B, and are given by Equation 6.
d_x1 = (a_conf × a_x1 + b_conf × b_x1) / (a_conf + b_conf)
d_x2 = (a_conf × a_x2 + b_conf × b_x2) / (a_conf + b_conf)
d_y1 = (a_conf × a_y1 + b_conf × b_y1) / (a_conf + b_conf)
d_y2 = (a_conf × a_y2 + b_conf × b_y2) / (a_conf + b_conf)
d_z1 = (a_conf × a_z1 + b_conf × b_z1) / (a_conf + b_conf)
d_z2 = (a_conf × a_z2 + b_conf × b_z2) / (a_conf + b_conf)
... (Formula 6)
Further, the similarity d_conf of the object D is given by Equation 7.
d_conf = a_conf + b_conf (Formula 7)
In the example shown in FIG. 18B, the similarity of the newly generated object D is calculated as 15 by Equation 7. On the other hand, since the object C shown in FIG. 18A does not exist in the vicinity, the coordinate similarity information is not updated.
In the above example, the ratio of the volume of the rectangular parallelepiped of the object is used as a reference for determining whether or not the object B exists in the vicinity of the geographical coordinates of the object A. However, the present invention is not limited to this. Other criteria may be used.

  Next, the object merging unit 1008 determines whether or not the object selected in step ST1301 is the last object stored in the map database 1003 (step ST1304). If the target selected in step ST1301 is the last target stored in the map database 1003, the process proceeds to step ST1305. If not, that is, if there is an object not yet selected in the map database 1003, the next object is selected.

  Finally, threshold processing section 1009 performs threshold processing on the similarity of the objects added in step ST1303, and outputs an object having a similarity equal to or greater than a certain threshold (step ST1305). For example, in FIG. 18B, the object D generated by merging the object A and the object B has a similarity of 15. The object C has a similarity of 5. Here, if threshold processing is performed and an object having a similarity of 10 or more is output, only the object 15 with a similarity of 15 is output as the object, and the object C is not output.

  Note that the data of the object C whose similarity is lower than the threshold value is stored as it is without being deleted from the map database 1003. This is because new data may be added in the vicinity of the object C by information on another vehicle or information collected on another day. For example, it is assumed that an object E having a similarity of 10 is newly stored in the vicinity of the geographical coordinates of the object C. When the volume of the cube of the object C and the object E overlaps a certain ratio or more, the object C and the object E are merged, and a new object F is generated. At this time, the new object F has a similarity of 15. Here, when threshold processing is performed and an object having a similarity of 10 or less is rejected, both the object D and the object F having the similarity 15 are output.

  In general, an object detected by a certain vehicle is highly likely to be detected by another vehicle. For this reason, it is possible to further increase the similarity of an object by detecting the object from information collected by a large number of vehicles and sensors and accumulating the similarity. On the other hand, an object that is not originally an object but is erroneously detected as an object is unlikely to be detected again by another vehicle, and the number of opportunities for similarity accumulation is extremely small. For this reason, by performing threshold processing on the accumulated similarity, it is possible to reduce erroneous detection and detect an object with high accuracy.

It is also possible to create an interface for displaying the detected object using the similarity, and use it for facility management. For example, it is also possible to search for an object from the collected video and display the search results in descending order of similarity.
Furthermore, by displaying the search result of the target object in different colors according to the degree of similarity, it is possible to support human work for searching for the target object from the video.

The block diagram which shows the structure of the data collection device in embodiment of this invention The flowchart which shows the operation | movement of the map information generation in embodiment of this invention Explanatory drawing showing an example of subject detection Explanatory drawing explaining the coordinate value sequence of the image coordinate system Explanatory drawing of the space coordinates of the camera coordinate system centered on the camera Explanatory diagram showing the relationship between camera coordinate system and geographic coordinate system Explanatory diagram showing the relationship between camera coordinate system and geographic coordinate system Explanatory drawing showing an example of a location information string plotted on a map Explanatory drawing which shows the example which vectorizes the point of a positional information row | line | column Configuration diagram of map information collection device according to Embodiment 2 Flow chart showing the operation of the object information collection phase A flowchart showing the operation of the object detection unit 1005 Flow chart showing the operation of the object information accumulation phase Explanatory drawing which shows the operation example of the image scanning part 1010. Explanatory drawing which shows the example of the rectangular area of a target object Explanatory drawing which shows the operation example of the rectangular area merge part 1012 Explanatory drawing which shows the example whose object is a rectangular parallelepiped Explanatory drawing which shows the operation example of the target object information accumulation part 1004.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Positioning means, 102 Video camera, 103 Direction sensor, 104 Vehicle, 105 Recording apparatus, 910 Video information / position information database, 920 Similarity calculation unit, 930 Object region positioning unit, 940 Similarity / positioning information storage unit, 950 Detection result accumulation unit, 960 Object detection / positioning unit, 970 Map database, 1000 Map information collection device, 1001 Video information / position information database, 1002 Object information collection unit, 1003 Map database, 1004 Object information accumulation unit, 1005 Object detection unit, 1006 Object positioning unit, 1007 Object storage unit, 1008 Object merging unit, 1009 Threshold processing unit, 1010 Image scanning unit, 1011 Similarity calculation unit, 1012 Rectangular area merging unit, 1013 Threshold processing unit.

Claims (9)

An input step for inputting video data to be imaged in accordance with movement of the imaging means, and positioning information recorded in correspondence with the imaging position of the imaging means recorded in correspondence with the video data;
A detection step of detecting a detection target in the video data input in the input step;
A calculation step for calculating relative position information between the detection object detected in the detection step and the imaging means;
A target position information generation step for generating position information of the detection target based on the relative position information calculated in the calculation step and the positioning information input in the input step. Information generation method.   The map information generation method according to claim 1, further comprising a map correspondence step that associates the detection target with a map based on the position information of the detection target generated in the target position information generation step.   Input speed information indicating the moving speed of the image pickup means recorded in correspondence with the video data, and correcting the position information of the detection target based on the speed information, to the map of the detection target in the map corresponding step. The map information generation method according to claim 2, wherein correspondence is performed.   The map according to any one of claims 1 to 3, further comprising a feature information extraction step for extracting feature information detected from the video data corresponding to a detection target detected in the detection step. Information generation method. An input step for inputting video data to be imaged in accordance with movement of the imaging means, and positioning information recorded in correspondence with the imaging position of the imaging means recorded in correspondence with the video data;
A detection step of detecting a detection target in the video data input in the input step;
A calculation step for calculating relative position information between the detection object detected in the detection step and the imaging means;
Based on the relative position information calculated in the calculation step and the positioning information input in the input step, a map information generation method including a target position information generation step for generating position information of the detection target is provided to a computer. Map information generation program to be executed. A video information / position information database for storing video information and position information synchronized with the video information;
An object information collecting unit for detecting an object from the video information and collecting information on the object;
A map database for storing information on the object collected by the object information collecting unit;
A map information collecting apparatus comprising: an object information accumulating unit that accumulates information on the object stored in the map database. The object information collection unit
An object detection unit for detecting the object from the video information;
An object positioning unit that calculates geographic coordinates of the object detected by the object detection unit;
The map information collection device according to claim 6, further comprising: an object storage unit that stores the geographical coordinates of the object calculated by the object positioning unit in the map database. The object detection unit includes:
An image scanning unit that scans an image acquired from the video information and detects a rectangular region of the object;
A similarity calculation unit for calculating a similarity between the rectangular area detected by the image scanning unit and the object;
Among the rectangular areas detected by the image scanning unit, a rectangular area obtained by merging the rectangular areas existing in the vicinity is output, and the merging process is performed by accumulating the similarities of the rectangular areas existing in the vicinity. A rectangular area merging unit that outputs the similarity of the areas;
The threshold processing unit that outputs only a rectangular region having a similarity equal to or higher than a certain threshold among the rectangular regions subjected to the merging process that is output by the rectangular region merging unit. Map information collection device. The object information accumulating unit
Among the objects stored in the map database, information on the objects existing in the vicinity is merged, and the similarity of the objects existing in the vicinity is accumulated to merge the objects. An object merging unit to output as a similarity,
7. A threshold processing unit that outputs only objects having a degree of similarity equal to or greater than a certain threshold among the merged objects output by the object merging unit. Item 9. The map information collection device according to any one of items 8 to 9.

Images