JP2019144501A - Map generation system - Google Patents

Abstract

To efficiently transmit an image captured by an imaging unit from an on-vehicle device to a map generation device.SOLUTION: A map generation system 1 includes an on-vehicle device 100 and a map generation device 200. The on-vehicle device 100 includes: a camera 101 for imaging a front side of a vehicle V at a predetermined imaging period; a vehicle sensor 102 for acquiring a speed of a vehicle V; a track recognition unit 113 for recognizing a travel track; an extraction period setting unit 112 for setting an extraction condition of an image captured by the camera 101 on the basis of a speed of a vehicle V; an image extraction unit 111 for extracting an image satisfying an extraction condition; and a transmission unit 105 for transmitting a travel track and an extracted image to the map generation device 200 by communication. The map generation device 200 includes a map generation unit 202 for generating a map on the basis of a transmitted travel track and image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a map generation system.

  For example, Patent Document 1 describes a system that generates a map on the map generation device side based on an image captured by an imaging unit mounted on a vehicle. More specifically, the in-vehicle device transmits the image captured by the imaging unit and the travel locus of the vehicle to the map generation device. The map generation device generates a map by pasting an image along the received travel locus.

JP 2009-237901 A

  In the above system, the imaging unit captures an image at a constant cycle (frame rate) when capturing a moving image. When a map is generated based on such an image, for example, depending on the speed of the vehicle, a gap is generated when the image is pasted, or a plurality of redundant images in which most of the images overlap are generated by the map generation device Or sent to. For this reason, in this technical field, it is requested | required that the image imaged by the imaging part should be efficiently transmitted from a vehicle-mounted apparatus to a map generation apparatus.

  A map generation system according to the present invention is a map generation system that is mounted on a vehicle and includes a vehicle-mounted device that images the surroundings of the vehicle, and a map generation device that generates a map based on an image captured by the vehicle-mounted device. The vehicle-mounted device is based on an imaging unit that captures the surroundings of the vehicle at a predetermined imaging cycle, a traveling state acquisition unit that acquires the traveling state of the vehicle, a locus recognition unit that recognizes the traveling locus of the vehicle, and the traveling state. A condition setting unit that sets an extraction condition for an image captured by the imaging unit, an image extraction unit that extracts an image satisfying the extraction condition among the images captured by the imaging unit, a travel locus, and the extracted image A transmission unit that transmits to the map generation device by communication, and the map generation device receives the travel locus and image transmitted from the transmission unit as a reception unit that receives by communication, and the received travel locus and image. On the basis of And a map generating unit for generating a map.

  ADVANTAGE OF THE INVENTION According to this invention, the image imaged by the imaging part can be efficiently transmitted from a vehicle-mounted apparatus to a map generation apparatus.

1 is a diagram illustrating a schematic configuration of a map generation system according to a first embodiment. FIG. 2A shows a camera image and a road surface equivalent area. FIG. 2B is a diagram illustrating a road surface cut-out image. FIG. 2C shows a road surface image. FIG. 2D shows a road surface orthoimage. FIG. 3A and FIG. 3B are diagrams showing the overlap of road surface images depending on the speed of the vehicle. It is a graph which shows the change of a duplication rate. It is a flowchart which shows the flow of the transmission process of the image performed in a vehicle-mounted apparatus. It is a figure which shows schematic structure of the map generation system which concerns on 2nd Embodiment. It is a flowchart which shows the flow of the transmission process of the image performed in a vehicle-mounted apparatus. It is a figure which shows the changed exposure pattern. It is a figure which shows schematic structure of the map generation system which concerns on 3rd Embodiment. It is a flowchart which shows the flow of the transmission process of the image performed in a vehicle-mounted apparatus. It is a figure which shows schematic structure of the map generation system which concerns on 4th Embodiment. It is a figure which shows a mode that a road surface image is affixed and a road surface ortho image is produced | generated. It is a flowchart which shows the flow of the transmission process of the image performed in a vehicle-mounted apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
A first embodiment of a map generation system will be described. A map generation system 1 shown in FIG. 1 is a system that generates a map based on an image captured by a camera 101 mounted on a vehicle V (hereinafter referred to as “camera image”). The map generation system 1 includes an in-vehicle device 100 and a map generation device 200.

  Here, an outline of map generation processing performed in the map generation system 1 will be described. The in-vehicle device 100 cuts out a road surface cutout image C including the road surface of the traveling road L as shown in FIG. 2B from the camera image A captured by the camera 101 shown in FIG. When the map generation device 200 receives the road surface cut image C from the in-vehicle device 100, the map generation device 200 performs a bird's eye view conversion on the road surface cut image C and generates a road surface image D as shown in FIG. The map generation apparatus 200 generates a road surface ortho image E as a map by sequentially pasting road surface images D as shown in FIG. Hereinafter, the configuration of each part of the map generation system 1 will be described in detail.

(Details of in-vehicle devices)
The in-vehicle device 100 is mounted on the vehicle V. The in-vehicle device 100 includes a camera (imaging unit) 101, a vehicle sensor (running state acquisition unit) 102, a GPS reception unit 103, an ECU 104, and a transmission unit 105.

  The camera 101 is an imaging device that images the external environment of the vehicle V. The camera 101 is provided inside the windshield of the vehicle V. The camera 101 images the front of the vehicle V at a predetermined imaging cycle (frame rate). In addition, the camera 101 images the front of the vehicle V so as to include the road surface of the traveling path on which the vehicle V travels. The camera 101 outputs the captured camera image to the ECU 104.

  The vehicle sensor 102 is a sensor that acquires the traveling state of the vehicle V. In the present embodiment, the vehicle sensor 102 detects the speed of the vehicle V as the traveling state. That is, the vehicle sensor 102 includes a vehicle speed sensor that detects the speed of the vehicle V. As the vehicle speed sensor, for example, a wheel speed sensor that is provided for a wheel of the vehicle V or a drive shaft that rotates integrally with the wheel and detects the rotation speed of the wheel is used. The vehicle sensor 102 outputs the detected vehicle speed information to the ECU 104.

  The GPS receiving unit 103 measures the position of the vehicle V (for example, the latitude and longitude of the vehicle V) by receiving signals from three or more GPS satellites. The GPS receiving unit 103 outputs the measured position information of the vehicle V to the ECU 104.

  The ECU 104 is an electronic control unit having a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like. In the ECU 104, for example, various functions are realized by loading a program stored in the ROM into the RAM and executing the program loaded in the RAM by the CPU. The ECU 104 may be composed of a plurality of electronic units. Functionally, the ECU 104 includes an image extraction unit 111, an extraction cycle setting unit (condition setting unit) 112, and a locus recognition unit 113.

  As shown in FIG. 2A, the image extraction unit 111 is a camera that satisfies the extraction condition set by the extraction cycle setting unit 112 among a plurality of camera images A captured by the camera 101 at a predetermined imaging cycle. Image A is extracted sequentially. Details of the extraction conditions will be described later. The image extraction unit 111 extracts the camera image A based on the extraction condition every time the camera image A is captured at a predetermined imaging cycle. Hereinafter, when it is necessary to distinguish between the extracted camera image A and the non-extracted camera image A among the plurality of camera images A captured by the camera 101, the extracted camera image A, that is, the camera image satisfying the extraction condition. A is referred to as an extracted camera image A1.

  Further, the image extraction unit 111 corresponds to a road surface equivalent region B set in front of the vehicle V (see FIG. 2A) in the extracted camera image A1, as shown in FIG. Is extracted (extracted), and a road surface cut-out image C is generated. In the images shown in FIGS. 2A and 2B, the traveling path L on which the vehicle V travels and the white line L1 attached to both sides of the traveling path L are shown.

  Here, the road surface equivalent area B is a preset area in front of the vehicle V. For example, the road surface equivalent area B is an area in the range from 8 m to 16 m in front of the vehicle V from the installation position of the camera 101 (that is, an area having a depth of 8 m) and in the range of ± 3.5 m to the left and right (that is, an area having a width of 7 m). Is set. The mounting direction of the camera 101 is set so that the road surface equivalent region B is included in the camera image A. Assuming that the road surface corresponding to the road surface equivalent area B in the extracted camera image A1 is reflected in the road surface L, the image extraction unit 111 cuts out the image corresponding to the road surface equivalent area B and cuts the road surface. An output image C is generated.

  The image extraction unit 111 outputs the cut out road surface cut image C to the transmission unit 105. That is, the image extraction unit 111 sequentially cuts out the road surface cut image C from the extracted camera image A1 satisfying the extraction condition among the camera images A captured by the camera 101 with a predetermined imaging cycle, and outputs the cut image C to the transmission unit 105.

  Note that the image extraction unit 111 is not limited to extracting the road cut-out image C from the extracted camera image A1 after extracting the extracted camera image A1 based on the extraction condition. For example, the image extraction unit 111 cuts out the road surface cut image C from the camera image A every time it is captured by the camera 101, cuts out the road surface cut image C, and then extracts the camera image A (extracted camera image A1) that satisfies the extraction condition. ) May be extracted from the road surface cut image C.

  The extraction cycle setting unit 112 sets the extraction condition based on the speed of the vehicle V detected by the vehicle sensor 102. As described above, this extraction condition is a condition for the image extraction unit 111 to extract the extracted camera image A1 that extracts the road cut-out image C from the plurality of camera images A captured by the camera 101. In the present embodiment, this extraction condition indicates whether or not the camera image A captured at the current time (current imaging timing) is an extraction target. That is, the extraction condition includes a time at which the camera image A is extracted as the extracted camera image A1.

  Here, when the road surface image D is generated from the camera image A captured at each time, when the speed of the vehicle V is high, when the road surface orthoimage E is generated as shown in FIG. The overlapping area of the road surface image D becomes smaller. In this case, depending on the setting of each part, image omission may occur when the road surface image D is pasted. When the speed of the vehicle V is low, as shown in FIG. 3B, the overlapping area of the road surface image D becomes large when the road surface orthoimage E is generated. In this case, a road surface cut-out image C redundant for generating the road surface orthoimage E is transmitted from the in-vehicle device 100 to the map generation device 200. Therefore, the extraction cycle setting unit 112 in the present embodiment sets the extraction condition so that the size of the overlapping area of the road surface image D is appropriate when the map generation device 200 generates the road surface orthoimage E.

  Specifically, the extraction cycle setting unit 112 calculates the overlapping rate of the two road surface images D when generating the road surface orthoimage E based on the two road surface images D. One of the two road surface images D is the road surface image D generated based on the latest (most recent) extracted camera image A1 extracted by the image extraction unit 111. Further, the other road surface image D of the two road surface images is the road surface image D when it is assumed that the road image is generated based on the camera image A captured at the current time (current imaging timing). In this way, the extraction cycle setting unit 112 calculates the overlap rate between the most recently generated road surface image D and the assumed road surface image D generated based on the camera image A at the current time. Here, the overlapping rate varies depending on the speed of the vehicle V. For this reason, the extraction cycle setting unit 112 calculates the overlap rate in consideration of the speed of the vehicle V.

  In addition, the extraction cycle setting unit 112 determines whether or not the calculated duplication rate is less than a predetermined duplication rate threshold. When the calculated duplication rate is less than a predetermined duplication rate, the extraction cycle setting unit 112 determines that the camera image A taken at the current time (current imaging timing) is the extraction camera image A1 to be extracted. To do. The extraction cycle setting unit 112 sets, as an extraction condition, that the camera image A captured at the current time is to be extracted.

  On the other hand, if the calculated overlap rate is not less than the predetermined overlap rate, the extraction cycle setting unit 112 determines that the camera image A captured at the current time (current image capture timing) is not an extraction target. . After this determination, the extraction cycle setting unit 112 generates again based on the most recently generated road surface image D and the camera image A at the current time, as described above, after the elapse of a predetermined imaging cycle. The overlap rate with the road surface image D is calculated. As described above, the extraction cycle setting unit 112 has the most recently generated road surface image D and the road surface image D when it is generated based on the camera image A at the current time for each predetermined imaging cycle. The duplication rate of is calculated. Then, based on the calculated overlap rate, the extraction cycle setting unit 112 determines whether or not the camera image A captured at the current time is to be extracted, and sets extraction conditions.

  More specifically, the extraction cycle setting unit 112 is based on the amount of movement of the vehicle V in one cycle of the imaging cycle of the camera 101 and the road surface equivalent region B when the road surface cutout image C is cut out. Can be calculated. The extraction cycle setting unit 112 can calculate the amount of movement of the vehicle V based on the speed of the vehicle V detected by the vehicle sensor 102 and a preset imaging cycle (predetermined imaging cycle) of the camera 101. it can. The road surface equivalent area B here is, as an example, the range from 8 m to 16 m in front of the vehicle V from the installation position of the camera 101 (that is, an area having a depth of 8 m), and the range of ± 3.5 m to the left and right (that is, width 7 m). Area).

  For example, it is assumed that the camera image A captured at time t [s] is extracted as the extracted camera image A1, and the road surface image D is generated from the extracted camera image A1. Assume that the vehicle V travels 1 m forward at time t + 0.1 [s], which is the next imaging cycle of the camera 101. In this case, the road surface image D generated from the extracted camera image A1 captured at time t [s] and the road surface image D when assumed to be generated from the camera image A captured at time t + 0.1 [s]. As shown in FIG. 4, the overlap ratio with is about 87%. In this case, the duplication rate is equal to or higher than a preset duplication rate threshold (40%). For this reason, the extraction cycle setting unit 112 does not determine that the camera image A captured at time t + 0.1 [s] is an extraction target.

  For example, the road surface image D generated from the extracted camera image A1 captured at time t [s] and the image captured at time t + 0.5 [s], which is an imaging cycle five times after time t [s]. The overlap rate with the road surface image D when it is assumed that the image is generated from the camera image A is less than the overlap rate threshold (40%). Therefore, the extraction cycle setting unit 112 determines that the camera image A captured at time t + 0.5 [s] is an extraction target. Then, the extraction cycle setting unit 112 sets that the camera image A captured at time t + 0.5 [s] is to be extracted as an extraction condition.

  As described above, the extraction cycle setting unit 112 is generated based on the road surface image D generated based on the latest extracted camera image A1 and the camera image A captured at a timing later than the road surface image D. The overlapping rate with the road surface image D when assumed is calculated, and the extraction timing (extraction condition) of the camera image A is determined according to the calculated overlapping rate. The extraction cycle setting unit 112 outputs the set extraction condition to the image extraction unit 111. It should be noted that the overlap rate threshold value used when setting the extraction condition may be a constant value or may be changed according to the location where the vehicle V is traveling.

  The trajectory recognition unit 113 recognizes the travel trajectory and posture (the direction of the vehicle V) of the vehicle V. In the present embodiment, the trajectory recognition unit 113 recognizes the position and orientation of the vehicle V at each imaging timing when the camera 101 performs imaging as the travel trajectory and orientation. The trajectory recognition unit 113 recognizes the position of the vehicle V at each imaging timing based on the position information measured by the GPS reception unit 103. The trajectory recognition unit 113 recognizes the posture of the vehicle V at each imaging timing based on the change in position information. The trajectory recognition unit 113 may detect the attitude of the vehicle V based on a detection result of a gyro sensor or the like that detects a yaw rate (rotational angular velocity) around the vertical axis of the center of gravity of the vehicle V. The trajectory recognition unit 113 outputs the recognized position and orientation of the vehicle V to the transmission unit 105.

  When the road surface cut-out image C is input, the transmission unit 105 receives the input road surface cut-out image C and the vehicle at the time when the extracted camera image A1 that is the basis for generating the road surface cut-out image C is captured. The position and orientation of V are transmitted to the map generation apparatus 200 by communication. The transmission unit 105 transmits the road surface cut image C and the like to the map generation device 200 by various communication methods such as wireless communication or wired communication. That is, the transmission unit 105 transmits a part of the extracted camera image A1 extracted based on the extraction condition to the map generation device 200.

(Details of map generator)
The map generation device 200 is provided in a facility such as a center that manages map information, for example. The map generation device 200 includes a reception unit 201 and a map generation unit 202.

  The receiving unit 201 acquires the road surface cut image C and the position and posture of the vehicle V transmitted from the transmitting unit 105 through communication.

  Similar to the ECU 104, the map generation unit 202 is configured by an electronic control unit having a CPU, a ROM, a RAM, and the like. The map generation unit 202 generates a map based on the road surface cut image C received by the reception unit 201 and the position and orientation of the vehicle V. Specifically, the map generation unit 202 performs a bird's eye view conversion on the received road surface cutout image C to generate a road surface image D (see FIG. 2C). Then, the map generation unit 202 pastes the generated road surface image D on the map coordinate system. Note that when the map generation unit 202 pastes the road surface image D, the map generation unit 202 moves and rotates the road surface image D based on the position and orientation of the vehicle V and pastes them.

  The map generation unit 202 performs this pasting every time the road surface cut image C or the like is received by the reception unit 201, and generates a road surface orthoimage E as a map. By pasting on the basis of the road surface cut image C generated based on the extraction conditions, the overlap (overlap) between the road surface images D becomes less than the overlap rate threshold. The road surface orthoimage E generated by the map generation unit 202 is used for various purposes such as creation of map information.

(Flow of image transmission processing)
Next, the flow of image transmission processing performed in the in-vehicle device 100 will be described with reference to FIG. The camera 101 performs imaging at a predetermined imaging cycle. The process shown in FIG. 5 is started for each imaging cycle (every imaging time) when the camera 101 performs imaging.

  As shown in FIG. 5, the trajectory recognition unit 113 recognizes the position and posture of the vehicle V at the current time (S101). The vehicle sensor 102 detects the speed of the vehicle V at the current time (S102). When it is assumed that the extraction cycle setting unit 112 is generated based on the road surface image D generated based on the latest extracted camera image A1 and the camera image A at the current time while considering the speed of the vehicle V The overlap rate with the road surface image D is calculated (S103).

  The extraction cycle setting unit 112 determines whether or not the calculated duplication rate is less than the duplication rate threshold (S104). If the overlap rate is not less than the overlap rate threshold (S104: NO), the in-vehicle device 100 ends the current process, and starts the process illustrated in FIG. 5 again after a predetermined imaging period has elapsed.

  On the other hand, when the duplication rate is less than the duplication rate threshold (S104: YES), the extraction cycle setting unit 112 sets the camera image A taken at the current time (current imaging timing) as the extraction camera image A1 to be extracted. Determination is made, and an extraction condition for setting the camera image A as an extraction target is set. Based on the extraction condition, the image extraction unit 111 extracts the camera image A captured at the current time as an extracted camera image A1, and cuts out the extracted extracted camera image A1 to generate a road surface cutout image C (S105). ). The transmission part 105 transmits the produced | generated road surface cut-out image C and the position and attitude | position of the vehicle V in the present time recognized by S101 to the map production | generation apparatus 200 by communication (S106).

  As described above, the map generation system 1 according to the present embodiment transmits the road surface from the in-vehicle device 100 to the map generation device 200 so that the overlap rate of the road surface image D when generating the road surface orthoimage E is less than the overlap rate determination threshold. The cut image C can be transmitted. Thereby, the map generation system 1 can efficiently transmit the image (road surface cut image C) captured by the camera 101 from the in-vehicle device 100 to the map generation device 200.

(Second Embodiment)
A second embodiment of the map generation system will be described. The in-vehicle device 100 in the first embodiment determines whether or not to transmit the road surface cut image C or the like while checking the overlap rate at each time of the imaging cycle of the camera 101. In the in-vehicle device in the second embodiment, the next transmission timing of the road surface cut image C or the like is estimated based on the speed and acceleration of the vehicle V at the time when the road surface cut image C or the like is transmitted.

  Specifically, as illustrated in FIG. 6, the map generation system 1 </ b> A according to the second embodiment includes an in-vehicle device 100 </ b> A and a map generation device 200. In addition, in the following description, about the component similar to the map generation system 1 of 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted. The in-vehicle device 100A includes a camera (imaging unit) 101, a vehicle sensor (running state acquisition unit) 102A, a GPS reception unit 103, an ECU 104A, and a transmission unit 105.

  The ECU 104A is an electronic control unit similar to the ECU 104 in the first embodiment. The ECU 104A functionally includes an image extraction unit 111, an extraction cycle setting unit (condition setting unit) 112A, and a locus recognition unit 113. That is, the in-vehicle device 100A includes a vehicle sensor 102A and an extraction cycle setting unit 112A instead of the vehicle sensor 102 and the extraction cycle setting unit 112 of the in-vehicle device 100 in the first embodiment.

  The vehicle sensor 102A is a sensor that acquires the traveling state of the vehicle V. In the present embodiment, the vehicle sensor 102A detects the speed and acceleration of the vehicle V as the traveling state. That is, the vehicle sensor 102 </ b> A includes a vehicle speed sensor that detects the speed of the vehicle V and an acceleration sensor that detects the acceleration of the vehicle V. The vehicle sensor 102A inputs the detected speed and acceleration of the vehicle V to the ECU 104A.

  The extraction cycle setting unit 112A sets the extraction condition based on the speed and acceleration of the vehicle V detected by the vehicle sensor 102A. This extraction condition includes the time when the image extraction unit 111 next extracts the camera image A. Specifically, when the extraction camera image A1 is extracted by the image extraction unit 111, the extraction cycle setting unit 112A is based on the speed and acceleration of the vehicle V at the time when the extraction camera image A1 is extracted. The overlapping rate between the road surface image D generated based on the image A1 and the road surface image D assumed to be generated based on the camera image A taken at a time later than the road surface image D is less than the overlap threshold. Is calculated. The extraction cycle setting unit 112 sets the time when the duplication rate is less than the duplication threshold as the extraction condition as the time when the image extraction unit 111 should extract the extracted camera image A1.

  The image extraction unit 111 extracts the extracted camera image A1 based on the extraction condition set by the extraction cycle setting unit 112. Specifically, the image extraction unit 111 refers to the extraction condition, and when the current time is the time when the extracted camera image A1 is extracted next after the extraction condition is set, the camera imaged by the camera 101 is captured. An image A is extracted as an extracted camera image A1, and a road surface cutout image C is generated from the extracted camera image A1.

(Flow of image transmission processing)
Next, the flow of image transmission processing performed in the in-vehicle device 100A will be described with reference to FIG. The camera 101 performs imaging at a predetermined imaging cycle. The process shown in FIG. 7 is started for each imaging cycle (every imaging time) when the camera 101 performs imaging.

  As shown in FIG. 7, the trajectory recognition unit 113 recognizes the position and posture of the vehicle V at the current time (S201). The image extraction unit 111 determines whether or not the current time is the time when the extracted camera image A1 is extracted next after the extraction condition is set (S202). When the current time is not the next time to extract the extracted camera image A1 (S202: NO), the in-vehicle device 100A ends the current process and starts the process illustrated in FIG. 7 again after a predetermined imaging period has elapsed. To do. The case where the current time is not the time when the extracted camera image A1 is extracted next is a case where the current time is before the time when the extracted camera image A1 is extracted next.

  On the other hand, when the current time is the time when the extracted camera image A1 is extracted next (S202: YES), the image extracting unit 111 sets the camera image A captured by the camera 101 at the current time as the extracted camera image A1. Extraction is performed, and a road surface cutout image C is generated from the extracted camera image A1 (S203). The vehicle sensor 102A detects the speed and acceleration of the vehicle V at the current time (S204).

  Based on the speed and acceleration of the vehicle V, the extraction cycle setting unit 112A calculates a time for extracting the extracted camera image A1 next. Specifically, as described above, the extraction cycle setting unit 112A calculates a time when the overlap rate of the road surface image D is less than the overlap threshold, and sets it as an extraction condition (S205). The transmission unit 105 transmits the road surface cut image C generated in S203 and the position and posture of the vehicle V recognized at S201 to the map generation device 200 by communication (S206).

  As described above, in the map generation system 1A of the present embodiment, as in the first embodiment, the image (road surface cut image C) captured by the camera 101 is efficiently transmitted from the in-vehicle device 100A to the map generation device 200. it can. Further, the in-vehicle device 100A can grasp the timing at which the camera image A is extracted as the extracted camera image A1 after extracting the camera image A captured by the camera 101 as the extracted camera image A1. For this reason, the in-vehicle device 100A can use the camera 101 for other purposes until the next time of extracting the extracted camera image A1.

  For example, as shown in FIG. 8, it is assumed that the camera 101 is capturing four images with different exposure patterns every 0.1 second. Here, the camera 101 captures images of the exposure patterns A to D, respectively. The exposure pattern A is an exposure pattern for capturing an image used to generate the road surface orthoimage E. In this case, if the timing at which the image extraction unit 111 extracts the camera image A next is 0.3 seconds later, it is not necessary to capture the exposure pattern A during that time. For this reason, for example, the camera 101 can capture an image of the exposure pattern C at intervals of 0.05 seconds by capturing with an exposure pattern C for capturing an image used for object recognition instead of the exposure pattern A. . Thereby, for example, the cycle of object recognition can be accelerated.

(Third embodiment)
A third embodiment of the map generation system will be described. In the in-vehicle device according to the third embodiment, by changing the range to be cut out from the camera image A as the road surface cut image C, the overlap rate when the road surface image D is pasted becomes the desired overlap rate. Note that, in this embodiment, instead of extracting the extracted camera image A1 from the plurality of camera images A to generate the road surface cut image C as in the first embodiment, all the camera images captured by the camera 101 are used. A road cut-out image C is generated from A.

  Specifically, as illustrated in FIG. 9, the map generation system 1 </ b> B according to the third embodiment includes an in-vehicle device 100 </ b> B and a map generation device 200. The in-vehicle device 100B includes a camera (imaging unit) 101, a vehicle sensor (running state acquisition unit) 102, a GPS reception unit 103, an ECU 104B, and a transmission unit 105.

  The ECU 104B is an electronic control unit similar to the ECU 104 in the first embodiment. The ECU 104B functionally includes an image extraction unit 111B, an image cut-out range setting unit (condition setting unit) 112B, and a trajectory recognition unit 113. That is, the in-vehicle device 100B includes an image extraction unit 111B and an image cut-out range setting unit 112B instead of the image extraction unit 111 and the extraction cycle setting unit 112 of the in-vehicle device 100 in the first embodiment.

  The image cutout range setting unit 112B calculates a range (cutout range) to be cut out as a road surface cutout image C from the camera image A based on the speed of the vehicle V detected by the vehicle sensor 102. Specifically, for example, the imaging cycle of the camera 101 is 0.1 seconds, and the current speed of the vehicle V is 40 km / h. In this case, the vehicle V is expected to travel approximately 1 m before the next imaging timing of the camera 101. For this reason, the image cutout range setting unit 112B includes a road surface image D generated based on the camera image A at the next imaging time, and a road surface image D generated based on the camera image A at the current time. The range to be cut out from the camera image A as the road cut-out image C is calculated so that the overlap ratio of the image becomes approximately 50%, for example. In this case, the range to be extracted from the camera image A as the road surface cut image C is a range from 8 m to 10 m in front of the vehicle V from the installation position of the camera 101 (that is, a region having a depth of 2 m) and a range of ± 3.5 m to the left and right ( That is, the image range corresponds to a region having a width of 7 m.

  The image cutout range setting unit 112B sets a range to be cut out as the calculated road cutout image C as an extraction condition. The image cutout range setting unit 112B sets extraction conditions for each imaging cycle of the camera 101. The image cutout range setting unit 112B outputs the set extraction condition to the image extraction unit 111B.

  The image extraction unit 111B generates a road surface cut image C by cutting out a part of the camera image A corresponding to the range set as the extraction condition. The image extracting unit 111B cuts out the road surface cut image C based on the extraction condition input from the image cutout range setting unit 112B every time the camera 101 captures the camera image A. The image extraction unit 111B outputs the cut out road surface cut image C to the transmission unit 105.

(Flow of image transmission processing)
Next, the flow of image transmission processing performed in the in-vehicle device 100B will be described with reference to FIG. The camera 101 performs imaging at a predetermined imaging cycle. The process illustrated in FIG. 10 is started for each imaging cycle (every imaging time) when the camera 101 performs imaging.

  As shown in FIG. 9, the trajectory recognition unit 113 recognizes the position and posture of the vehicle V at the current time (S301). The vehicle sensor 102 detects the speed of the vehicle V at the current time (S302). Based on the speed of the vehicle V, the image cutout range setting unit 112B calculates a range cut out from the camera image A as a road surface cutout image C (S303). The image extraction unit 111B cuts out the camera image A and generates a road surface cutout image C (S304). The transmission unit 105 transmits the generated road surface cut image C and the position and posture of the vehicle V at the current time recognized in S301 to the map generation device 200 by communication (S305).

  As described above, in the map generation system 1B of the present embodiment, the image extraction unit 111B captures the camera image A so that the overlapping rate of the road surface image D when generating the road surface orthoimage E becomes a desired overlapping rate. A road surface cutout image C is generated by cutting out. Thereby, the map generation system 1B suppresses a redundant portion for generation of the road surface orthoimage E in the road surface cut image C, and the image (road surface cut image C) captured by the camera 101 from the in-vehicle device 100B. It can be efficiently transmitted to the map generation device 200.

  When the vehicle V is traveling at a constant speed, the road surface image D is set even when the overlap rate of the road surface image D used when calculating the range to be cut out from the camera image A as the road surface cut image C is set low. There is a low possibility that an image will be lost when the image is pasted. However, during acceleration / deceleration, if the overlap rate is set low, there is a possibility that image omission occurs when the road surface image D is pasted. For this reason, the image cutout range setting unit 112B may set a range to be cut out as the road surface cutout image C in consideration of not only the speed of the vehicle V but also the acceleration of the vehicle V. In this case, the map generation system 1B can more efficiently transmit the road surface cut image C and the like from the in-vehicle device 100B to the map generation device 200.

(Fourth embodiment)
A fourth embodiment of the map generation system will be described. As shown in FIG. 11, a map generation system 1C according to the fourth embodiment includes an in-vehicle device 100C and a map generation device 200C.

  Here, an overview of map generation processing performed in the map generation system 1C will be described. Many portions of the road surface of the traveling path of the vehicle V are configured only by lanes (white lines), and there are only some characteristic portions having road markings or the like. For this reason, the map generation system 1C duplicates and pastes a road surface image of only a lane generated from a camera image captured at a certain time for a portion composed of only a lane. The map generation system 1C generates and pastes a road surface image including a road surface sign generated from a captured camera image for a characteristic part having a road surface sign or the like. Thereby, map generation system 1C can control the amount of communication of image data between in-vehicle device 100C and map generation device 200C.

  As a specific example of the map generation process, a case where the vehicle V is traveling from the left side to the right side in the drawing in the traveling path L described in the upper part of FIG. 12 will be described. First, it is assumed that the vehicle V is traveling in the leftmost position of the traveling path L shown in the upper part of FIG. The in-vehicle device 100C performs the recognition process of the road surface of the traveling road L, determines that the road marking M does not exist in front of the vehicle V (immediately before), and cuts only the lane (white line L1) where the road marking M does not exist. The output image C is cut out from the camera image A. Then, the in-vehicle device 100C transmits the cut-out road surface cut image C and the position and posture of the vehicle V at that time to the map generation device 200C. The map generation device 200C generates a road surface image DA from the received road surface cut-out image C, and pastes the road surface image DA on the map coordinate system based on the position and orientation of the vehicle V (see the lower part of FIG. 12).

  Thereafter, since the vehicle V travels only in the lane for a while, the in-vehicle device 100C does not transmit the road surface cut image C or the like. When the vehicle V reaches the position of the road marking M and the road marking M is recognized by the road surface recognition processing, the in-vehicle device 100C cuts out the road cut image C from the camera image A including the road marking M. Then, the in-vehicle device 100C includes a road surface cut-out image C including the road marking M and a travel locus of the vehicle V from the previous transmission of the road surface cut-out image C (history of changes in the position of the vehicle V). And the attitude | position in each position on a driving | running | working locus | trajectory is transmitted to 200C of map production | generation apparatuses. The map generation device 200C generates a road surface image DB from the road surface cut image C including the received road surface marking M. As a result of the recognition processing of the road surface of the traveling road in the in-vehicle device 100C, a road surface having only a lane continues between the road surface image DA and the road surface image DB as in the road surface image DA. For this reason, the map generation device 200C generates the road surface image Da by duplicating the road surface image DA generated based on the road surface cut image C of the lane only received last time. The map generation apparatus 200C sequentially pastes the road surface image Da on the section between the road surface image DA and the road surface image DB based on the travel locus and direction of the vehicle V. Thereafter, the map generation device 200C pastes the road surface image DB including the road surface marking M. In the lower part of FIG. 12, the road surface image Da generated by duplication is indicated by a broken line in order to facilitate the distinction between the road surface image D and the road surface image Da. Similarly, a road surface image Dc described later is indicated by a broken line.

  When the vehicle V further travels, the in-vehicle device 100C cannot recognize the road marking M by the road surface recognition process. When the road marking M cannot be recognized in this manner, the in-vehicle device 100C cuts out the road cut image C only from the lane in which the road marking M does not exist from the camera image A again. Then, the in-vehicle device 100C transmits the cut-out road surface cut image C and the travel locus of the vehicle V from the previous transmission of the road surface cut-out image C and the posture at each position to the map generation device 200C. To do. The map generation device 200 </ b> C generates a road surface image DC from the received road surface cut-out image C, and pastes the road surface image DC to the map coordinate system based on the position and orientation of the vehicle V.

  Thereafter, since the vehicle V travels only in the lane for a while, the in-vehicle device 100C does not transmit the road surface cut image C or the like. Then, when the vehicle V reaches the end point of the area for generating the map, the in-vehicle device 100C transmits the road surface cut image C last time to the current travel locus of the vehicle V and each position on the travel locus. The attitude is transmitted to the map generation device 200C. From the result of the recognition processing of the road surface of the traveling road in the in-vehicle device 100C, the road surface of only the lane continues like the road image DC from the road image DC to the end point of the area for generating the map. For this reason, the map generation device 200C generates the road surface image Dc by duplicating the road surface image DC generated based on the road surface cut image C of only the lane received last time. The map generation device 200C sequentially pastes the road surface image Dc on a section between the road surface image DC and the end point of the area where the map is generated based on the travel locus and direction of the vehicle V.

  As described above, the map generation apparatus 200 </ b> C side transmits images of several points in the area where the map is generated, the travel locus, and the direction. The map generation device 200C duplicates and pastes the previously received image for positions where no image is received. In this way, the map generation device 200C pastes the road surface image, thereby generating a road surface orthoimage E as a map.

(Configuration of in-vehicle device)
The in-vehicle device 100C includes a camera (imaging unit) 101, a GPS reception unit 103, an ECU 104C, and a transmission unit 105C. The ECU 104C is an electronic control unit similar to the ECU 104 in the first embodiment. Functionally, the ECU 104C includes a trajectory recognition unit 113, a travel path recognition unit (travel situation acquisition unit) 114, an extraction condition setting unit (condition setting unit) 115, and an image extraction unit 116.

  The travel path recognition unit 114 acquires the travel status of the vehicle V. The traveling state here refers to the state of the traveling path on which the vehicle V travels. The travel path recognition unit 114 recognizes the presence or absence of the road marking M attached on the road surface as the situation of the travel path. Specifically, the travel path recognition unit 114 performs a road surface recognition process based on the camera image A of the camera 101. The travel path recognition unit 114 performs a road surface recognition process every time the camera image A is captured by the camera 101. Here, the traveling road recognition unit 114 recognizes the presence or absence of the road marking M attached to the road surface and the type of the white line L1 (broken line, practice, double line, etc.). For example, the traveling road recognition unit 114 can recognize the presence or absence of the road marking M based on a known image processing technique. The traveling road recognition unit 114 sets the road marking flag to ON when the road marking M is present on the road surface. When the road marking M does not exist, the traveling road recognition unit 114 sets the road marking flag to OFF.

  The extraction condition setting unit 115 sets the extraction condition based on the traveling state of the vehicle V. Specifically, the extraction condition setting unit 115 sets the extraction condition based on the presence / absence of a road marking M attached to the road surface of the traveling road as the traveling state. This extraction condition is a condition for the image extraction unit 116 to extract the extracted camera image A1 for extracting the road surface cut image C from the plurality of camera images A captured by the camera 101. More specifically, the extraction condition setting unit 115 uses the camera 101 as an extraction condition when the road surface flag is set to ON by the traveling road recognition unit 114 and when the road surface flag changes from ON to OFF. The fact that the captured camera image A is to be extracted is set. That is, the extraction condition setting unit 115 sets the time when the image extraction unit 116 extracts the extracted camera image A1 based on the recognition result of the travel route recognized by the travel route recognition unit 114.

  The image extraction unit 116 extracts, as an extracted camera image A1, a camera image A that satisfies the extraction condition set by the extraction cycle setting unit 112 from among a plurality of camera images A captured by the camera 101 at a predetermined imaging cycle. Further, the image extraction unit 116 sets a range (cutout range) to be cut out from the extracted camera image A1 as the road surface cutout image C. The image extraction unit 116 uses, for example, the image range of the portion corresponding to the road surface equivalent region B described in the first embodiment as the basic range of the range cut out from the extracted camera image A1 as the road surface cut image C. This basic range is preset. Note that the range to be extracted as the road surface cut image C from the extracted camera image A1 may be variable with a size that does not exceed the basic range, based on the result of the road surface recognition process of the traveling road.

  For example, when the center line or boundary line of the road is a broken line, the map is appropriately set based on the result of the road surface recognition process, taking into account the period between the white line part of the broken line and the road surface part. When a road surface image including a broken center line or the like is duplicated and sequentially pasted in the generation device 200C, the generation can be pasted without any contradiction in the period between the broken white line portion and the road surface portion.

  The image extraction unit 116 cuts out the extracted camera image A1 based on the set range and generates a road surface cutout image C. The image extraction unit 116 outputs the generated road surface cutout image C to the transmission unit 105C. Note that the image extracting unit 116 extracts the camera image A captured by the camera 101 as the extracted camera image A1 even when the road surface flag is OFF at the start of the map generation process, and the road surface cutting is performed from the extracted camera image A1. An output image C is generated.

  When the road surface cut-out image C is input, the transmission unit 105C transmits the input road surface cut-out image C and the travel locus and each position of the vehicle V from the previous transmission of the road surface cut-out image C to the present. Are transmitted to the map generation apparatus 200C. At the start of the map generation process, the transmission unit 105C sends the road surface cut image C generated by the image extraction unit 116 at the start of the generation process and the position and posture of the vehicle V at that time to the map generation apparatus 200C. Send. In addition, when the vehicle V reaches the end point of the area where the map is generated, the transmission unit 105C displays the travel locus of the vehicle V from the previous transmission of the road surface cut image C and the posture at each position, It transmits to the map generation apparatus 200C.

(Details of map generator)
The map generation device 200C includes a reception unit 201 and a map generation unit 202C. The map generation unit 202C is an electronic control unit similar to the ECU 104 in the first embodiment. As described above, when the map generation unit 202C receives the road surface cut image C, the map generation unit 202C converts the road surface cut image C into the road surface image D, and converts the road surface image D into the map coordinate system based on the travel locus and direction. paste. Further, the map generation unit 202C duplicates the road surface image D generated based on the road surface cut image C transmitted last time (transmitted in the past) for the section where the road surface cut image C is not transmitted. The copied road surface image D is pasted on the coordinate system of the map based on the travel locus and the direction. Thereby, the road surface orthoimage E as a map is produced | generated.

(Flow of image transmission processing)
Next, the flow of image transmission processing performed in the in-vehicle device 100C will be described with reference to FIG. The camera 101 performs imaging at a predetermined imaging cycle. The process illustrated in FIG. 13 is started for each imaging cycle (every imaging time) when the camera 101 performs imaging.

  The trajectory recognition unit 113 determines whether or not it is immediately after the start of map generation processing in the map generation system 1C (S401). If it is immediately after the start of the map generation process (S401: YES), the trajectory recognition unit 113 recognizes the position and orientation of the vehicle V at the current time (S402). The travel path recognition unit 114 acquires the camera image A captured by the camera 101 at the current time (S403). Based on the camera image A, the traveling road recognition unit 114 recognizes the presence / absence of the road marking M attached on the road surface and the type of the white line L1. The traveling road recognition unit 114 sets the road marking flag to ON when the road marking M exists on the road surface, and sets the road marking flag to OFF when the road marking M does not exist (S404).

  The image extraction unit 116 sets a basic range as a range from which the road surface cut image C is cut out (S405). The image extraction unit 116 extracts the camera image A captured by the camera 101 at the current time as the extracted camera image A1, and cuts out the road surface cut image C from the extracted camera image A1 based on the set basic range (S406). ). The transmission unit 105C transmits the road surface cut image C cut by the image extraction unit 116 and the position and posture of the vehicle V at the current time to the map generation device 200C by communication (S407). After transmitting the road surface cut-out image C or the like, the in-vehicle device 100C ends the current process and starts the process illustrated in FIG. 13 again after a predetermined imaging period has elapsed.

  On the other hand, when it is not immediately after the start of the map generation process (S401: NO), the trajectory recognition unit 113 recognizes the position and orientation of the vehicle V at the current time (S408). The traveling path recognition unit 114 acquires the camera image A captured by the camera 101 at the current time (S409). Based on the camera image A, the traveling road recognition unit 114 recognizes the presence / absence of the road marking M attached on the road surface and the type of the white line L1. The traveling road recognition unit 114 sets the road marking flag to ON when the road marking M exists on the road surface, and sets the road marking flag to OFF when the road marking M does not exist (S410).

  The extraction condition setting unit 115 sets the extraction condition based on ON and OFF of the road surface flag set by the traveling road recognition unit 114 (S411). The image extraction unit 116 determines whether the camera image A captured at the current time satisfies the extraction condition (S412). If the extraction condition is not satisfied (S412: NO), the in-vehicle device 100C ends the current process, and starts the process illustrated in FIG. 13 again after a predetermined imaging period has elapsed.

  When the extraction condition is satisfied (S412: YES), the image extraction unit 116 sets a range for cutting out the road surface cut image C (S413). Here, the image extraction unit 116 may set a basic range as a range for cutting out the road surface cut image C, or may set a range having a size that does not exceed the basic range. The image extraction unit 116 extracts the camera image A captured by the camera 101 at the current time as the extracted camera image A1, and cuts out the road surface cut image C from the extracted camera image A1 based on the set range (S414). . The transmission unit 105C transmits the road surface cut-out image C cut out by the image extraction unit 116, the trajectory of the vehicle V from the previous transmission of the road surface cut-out image C and the like in the processing in S407 or S415, and the The posture at the position is transmitted by communication to the map generation device 200C (S415). After transmitting the road surface cut-out image C or the like, the in-vehicle device 100C ends the current process and starts the process illustrated in FIG. 13 again after a predetermined imaging period has elapsed.

  Thus, immediately after the start of the map generation process, the in-vehicle apparatus 100C performs the processes of S402 to S407, and transmits the road surface cut image C and the like to the map generation apparatus 200C. If it is not immediately after the start of the map generation processing, the in-vehicle device 100C performs the processing of S408 to S415, and transmits the road surface cutout image C and the like to the map generation device 200C according to the extraction condition (presence or absence of the road marking M).

  As described above, in the map generation system 1C of the present embodiment, the in-vehicle device 100C transmits the road surface cut image C and the like to the map generation device 200C according to the presence or absence of the road marking M. The map generation device 200C generates a road surface orthoimage E using the previously transmitted road surface cut image C for a section in which the road surface cut image C is not transmitted. Thereby, map generation system 1C can control the amount of communication of image data between in-vehicle device 100C and map generation device 200C, and can perform transmission of image data efficiently.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, the camera 101 images the front of the vehicle V, but may image the rear of the vehicle V, for example, as long as it is around the vehicle V. Even in this case, the map generation system 1 or the like can generate a map as in the case where the camera image in front of the vehicle V is used. Further, the imaging cycle of the camera 101 may be constant or variable.

  DESCRIPTION OF SYMBOLS 1, 1A, 1B, 1C ... Map generation system, 100, 100A, 100B, 100C ... In-vehicle apparatus, 101 ... Camera (imaging part), 102, 102A ... Vehicle sensor (running condition acquisition part), 105, 105C ... Transmission part 111, 111B, 116 ... image extraction unit, 112, 112A ... extraction cycle setting unit (condition setting unit), 112B ... image cutout range setting unit (condition setting unit), 113 ... trajectory recognition unit, 114 ... travel path recognition Part (running condition acquisition part), 115 ... extraction condition setting part (condition setting part), 200, 200C ... map generation device, 201 ... reception part, 202C ... map generation part, V ... vehicle.

Claims (5)

A map generation system comprising: an in-vehicle device that is mounted on a vehicle and images the surroundings of the vehicle; and a map generation device that generates a map based on an image captured by the in-vehicle device,
The in-vehicle device is
An imaging unit for imaging the periphery of the vehicle at a predetermined imaging cycle;
A travel status acquisition unit for acquiring the travel status of the vehicle;
A trajectory recognition unit for recognizing a travel trajectory of the vehicle;
A condition setting unit that sets an extraction condition for the image captured by the imaging unit based on the traveling state;
An image extraction unit that extracts the image that satisfies the extraction condition from among the images captured by the imaging unit;
A transmission unit that transmits the travel locus and the extracted image to the map generation device by communication;
With
The map generation device includes:
A receiving unit that receives the travel locus and the image transmitted from the transmitting unit by communication; and
A map generation unit that generates a map based on the received travel locus and the image;
A map generation system comprising: The travel status acquisition unit acquires the speed of the vehicle as the travel status,
The map generation system according to claim 1, wherein the condition setting unit sets a time at which the image extraction unit extracts the image as an extraction target based on a speed of the vehicle as the extraction condition. The travel status acquisition unit acquires the speed of the vehicle as the travel status,
The map generation system according to claim 1, wherein the condition setting unit sets, as the extraction condition, a cutout range of the image when the image extraction unit extracts the image based on the speed of the vehicle. The travel status acquisition unit acquires a recognition result of a travel route as the travel status,
The condition setting unit sets, as the extraction condition, a time at which the image extraction unit extracts the image as an extraction target based on the recognition result of the travel path,
2. The map according to claim 1, wherein when extracting the image, the image extraction unit sets a cutout range of the image based on a recognition result of the travel route, and extracts the image within the set cutout range. Generation system.   The map generation system according to claim 4, wherein the map generation unit generates the map based on the image transmitted in the past for a section in which the image is not transmitted from the in-vehicle device.