JP6529420B2 - Road monitoring device and road monitoring program - Google Patents

Description

  The present invention relates to a road and a technology for monitoring a road.

  2. Description of the Related Art MMS (Mobil Mapping System) is known which performs three-dimensional measurement using a measurement vehicle around a road and a road periphery.

  Patent Document 1 discloses a method of correcting map data using time-series position data measured by a plurality of moving objects.

Patent No. 4297626

  An object of the present invention is to make it possible to notify a supervisor of an event that hinders the passage of traffic, such as a depression of a road or a tree that has come off the road.

The road monitoring device of the present invention is
A packet transmitted from a communication device mounted on a vehicle traveling on a road, the packet including event position information indicating a position at which the event is occurring for the event occurring on the road or around the road A receiver for receiving an event packet;
A map image which is an image displayed on a display device and which is an image of a map in which an event diagram representing the event is added to a location corresponding to the position indicated by the event position information included in the event packet And a map image generation unit.

  According to the present invention, it is possible to display an image of a map to which an event map is added at a location corresponding to a position where an event has occurred for a road or an event occurring around the road. Therefore, it is possible to notify the supervisor of an event that hinders passage, such as a depression in a road or a tree that has run out on the road.

BRIEF DESCRIPTION OF THE DRAWINGS The block diagram of the road monitoring system 500 in Embodiment 1. FIG. FIG. 2 is a block diagram of a road monitoring device 510 according to the first embodiment. FIG. 5 is a block diagram of an event packet 531 according to the first embodiment. FIG. 2 is a block diagram of a vehicle packet 532 according to the first embodiment. 5 is a configuration diagram of a viewpoint packet 534 in Embodiment 1. FIG. FIG. 6 is a configuration diagram of a measurement point packet 533 in Embodiment 1. 3 is a flowchart of a road monitoring method according to Embodiment 1. FIG. 5 is a schematic diagram of a monitoring screen 503 in the first embodiment. FIG. 7 is a diagram showing a point cloud image 310 (the direction from the rear side to the front side in the direction of the line of sight) in the first embodiment. FIG. 6 is a diagram showing a point cloud image 310 (the direction from the front side to the back side of the sight line direction) in the first embodiment. FIG. 7 is a diagram showing a point cloud image 310 (the direction from the right to the left of the line of sight direction) in the first embodiment. FIG. 6 is a diagram showing a point cloud image 310 (the direction from the left to the right) in the first embodiment. FIG. 7 is a diagram showing a point cloud image 310 (the direction from the upper side to the lower side of the sight line direction) in the first embodiment. The figure which looked at the event mark 315 in Embodiment 1 from each direction. FIG. 7 is a diagram showing a window (map scale is large) for the map image 560 in the first embodiment. FIG. 10 is a diagram showing a window (small map scale) for the map image 560 in the first embodiment. 10 is a flowchart of initial screen processing (S210) in the first embodiment. FIG. 2 is a configuration diagram of a point cloud image generation device 100 according to Embodiment 1. FIG. 1 is an external view of a measurement vehicle 110 according to Embodiment 1. FIG. 6 shows a display example of the display device 300 in Embodiment 1. FIG. 2 is an explanatory view of a viewpoint position in the first embodiment. FIG. 2 is a configuration diagram of a point cloud image 310 according to Embodiment 1. FIG. 2 is a block diagram of an interface image 320 according to the first embodiment. 6 is a flowchart of a point cloud image generation method in Embodiment 1. FIG. 6 is a diagram showing point cloud images 310 according to gaze directions in the first embodiment. FIG. 6 shows point cloud images 310 according to moving offsets in the upper, lower, left, and right directions in the first embodiment. FIG. 6 shows point cloud images 310 according to movement offsets before and after in Embodiment 1. FIG. 6 shows point cloud images 310 according to rotation offsets in the upper, lower, left, and right in the first embodiment. FIG. 2 is a hardware configuration diagram of a point cloud image generation device 100 according to Embodiment 1. FIG. 6 shows another example of the point cloud image 310 according to the first embodiment. FIG. 7 is a block diagram of a road monitoring device 510 according to a second embodiment. 15 is a flowchart showing an operation of <Example 1> of the event position calculation unit 515 in the second embodiment. 15 is a flowchart showing an operation of <Example 2> of the event position calculation unit 515 in Embodiment 2;

Embodiment 1
A road monitoring system 500 for monitoring a road or an event occurring around the road will be described based on FIGS. 1 to 30. FIG.

*** Brief Description ***
The configuration of the road monitoring system 500 will be described based on FIG.
The road monitoring system 500 includes a measurement vehicle 110 and a road monitoring device 510.

The measurement vehicle 110 is a vehicle used in MMS (mobile mapping system).
Various measuring devices are attached to the measuring vehicle 110, and when the measuring vehicle 110 travels on the road, three-dimensional measurement of the road and the surroundings of the road is performed. Then, an event such as a road depression or a tree that has run out on the road is detected, and information on the detected event is transmitted to the road monitoring device 510 in real time via the network 501. Details of the measurement vehicle 110 will be described later.
The road monitor 510 receives information regarding the event. Then, the road monitoring device 510 displays on the display device 519 a map to which a diagram representing the event is added at a location corresponding to the position where the event has occurred.
As a result, the supervisor 502 can know in real time an event that causes a problem in traffic. And, it is possible to respond quickly to events and secure safe and smooth traffic.

*** Description of the configuration ***
The configuration of the road monitoring device 510 will be described based on FIG.
The road monitoring device 510 is a computer including hardware such as a processor 901, a memory 903, an auxiliary storage device 902, and a communication device 904. The processor 901 is connected to other hardware via a signal line.

The processor 901 is an integrated circuit (IC) that performs processing, and controls other hardware. Specifically, the processor 901 is a CPU, a DSP or a GPU. CPU is an abbreviation for Central Processing Unit, DSP is an abbreviation for Digital Signal Processor, and GPU is an abbreviation for Graphics Processing Unit.
The memory 903 is a volatile storage device. The memory 903 is also called a main storage device or a main memory. Specifically, the memory 903 is a random access memory (RAM).
The auxiliary storage device 902 is a non-volatile storage device. Specifically, the auxiliary storage device 902 is a ROM, an HDD or a flash memory. ROM is an abbreviation for Read Only Memory, and HDD is an abbreviation for Hard Disk Drive.
The communication device 904 comprises a receiver 9041 and a transmitter 9042. Specifically, the communication device 904 is a communication chip or a NIC (Network Interface Card).

  The road monitoring device 510 includes “parts” such as a map image generation unit 512, a point cloud image generation unit 513, and an interface image generation unit 514 as elements of functional configuration. The function of "section" is realized by software. The function of "section" will be described later.

The auxiliary storage device 902 stores a program for realizing the function of “section”. A program for realizing the function of “part” is loaded into the memory 903 and executed by the processor 901.
Furthermore, an OS (Operating System) is stored in the auxiliary storage device 902. At least a portion of the OS is loaded into the memory 903 and executed by the processor 901.
That is, the processor 901 executes a program that implements the function of “section” while executing the OS.
Data obtained by executing a program for realizing the function of “part” is stored in a storage device such as a memory 903, an auxiliary storage device 902, a register in the processor 901 or a cache memory in the processor 901. These storage devices function as storage units that store data.
The road monitoring device 510 may include a plurality of processors 901, and the plurality of processors 901 may cooperatively execute a program that realizes the function of “section”.

The memory 903 stores data used, generated, input / output or transmission / reception of the road monitoring device 510.
Specifically, the memory 903 stores map data 520, an event packet 531, a vehicle packet 532, a measurement point packet 533, a viewpoint packet 534, and the like. The contents of each data stored in the memory 903 will be described later.

  The communication device 904 functions as a communication unit that communicates data, the receiver 9041 functions as a reception unit 511 that receives data, and the transmitter 9042 functions as a transmission unit that transmits data.

Hardware combining the processor 901, the memory 903 and the auxiliary storage device 902 is called "processing circuitry".
"Part" may be read as "process" or "process". The function of “part” may be realized by firmware.
The program for realizing the function of “section” can be stored in a non-volatile storage medium such as a magnetic disk, an optical disk or a flash memory.

  Next, the event packet 531, the vehicle packet 532, the measurement point packet 533, and the viewpoint packet 534 will be described. These packets are packets transmitted from the communication device mounted on the measurement vehicle 110 to the road monitoring device 510.

The configuration of the event packet 531 will be described based on FIG.
The event packet 531 includes a header 541 in which a vehicle identifier 542 is set. Further, event packet 531 includes information identifier 543, size 544, event identifier 551 and event location information 552.
The vehicle identifier 542 is an identifier that identifies the measurement vehicle 110 of the transmission source.
The information identifier 543 is an identifier that identifies the type of information included in the packet.
The size 544 is the data size of the packet.
The event identifier 551 is an identifier for identifying an event. Specifically, the event identifier 551 is a combination of a type identifier for identifying the type of event and a serial number according to the type of event.
Event position information 552 is information indicating a position where an event has occurred. Specifically, the event position information 552 is a three-dimensional coordinate value indicating the position where the event has occurred.

The configuration of the vehicle packet 532 will be described based on FIG.
Vehicle packet 532 includes a header 541 in which a vehicle identifier 542 is set. Furthermore, the vehicle packet 532 includes an information identifier 543, a size 544, a time 545, and a vehicle position information 553. The same type of information as the information contained in the event packet 531 is assigned the same reference numeral and the description is omitted.
The time 545 is a time associated with the information contained in the packet.
The vehicle position information 553 is information indicating the position of the measurement vehicle 110. Specifically, the vehicle position information 553 is a three-dimensional coordinate value indicating the position of the measurement vehicle 110.

The configuration of the viewpoint packet 534 will be described based on FIG.
The viewpoint packet 534 includes a header 541 in which a vehicle identifier 542 is set. Furthermore, the viewpoint packet 534 includes an information identifier 543, a size 544, a time 545, viewpoint information 554, and line-of-sight information 559. The same type of information as the information contained in the event packet 531 or the vehicle packet 532 is assigned the same reference numeral and the description is omitted.
The viewpoint information 554 is information indicating the position of the viewpoint of the point cloud image displayed on the display device mounted on the measurement vehicle 110. The viewpoint of the point cloud image will be described later.
The line-of-sight information 559 is information indicating the line-of-sight direction of the point cloud image displayed on the display device mounted on the measurement vehicle 110. The viewing direction of the point cloud image will be described later.

The configuration of the measurement point packet 533 will be described based on FIG.
The measurement point packet 533 includes a header 541 in which a vehicle identifier 542 is set. Furthermore, the measurement point packet 533 includes an information identifier 543, a size 544, a time 545, and three-dimensional point cloud data 555. The same type of information as the information contained in the event packet 531 or the vehicle packet 532 is assigned the same reference numeral and the description is omitted.
The three-dimensional point cloud data 555 includes a road and measurement point information 556 of each measurement point located around the road.
The measurement point information 556 is information indicating the position of each measurement point. Specifically, the measurement point information 556 is three-dimensional coordinate values of the ENU coordinate system and three-dimensional coordinate values of the BLH coordinate system.
The coordinate value E is a coordinate value that becomes positive east of the origin with respect to the east-west direction and becomes negative west of the origin. The coordinate value N is a coordinate value which is positive north of the origin and negative at the south of the origin with the north-south direction as an axis. The coordinate value U is a coordinate value which becomes positive above the origin and becomes negative below the origin with the vertical direction as an axis. The origin is a point at which the measurement by the measurement vehicle 110 is started.
The coordinate value B is latitude, the coordinate value C is longitude, and the coordinate value H is an ellipsoidal height.

*** Description of operation ***
The operation of the road monitoring device 510 corresponds to a road monitoring method. Further, the procedure of the road monitoring method corresponds to the procedure of the road monitoring program.

The road monitoring method of the road monitoring device 510 will be described based on FIG. 7.
Step S210 is an initial screen process.
In step S210, the road monitoring device 510 causes the display device 519 to display the monitoring screen 503 in the initial state.

As shown in FIG. 8, the monitoring screen 503 includes a point cloud image 310, an interface image 320 and a map image 560.
The point cloud image 310 is an image in which measurement points located in a range viewed in a specified gaze direction from a specified viewpoint are drawn.
The interface image 320 is an image used as a user interface for specifying a viewpoint and a gaze direction. Interface image 320 is displayed along with point cloud image 310.
The map image 560 is an image of a road and a map representing the road. The map image 560 is displayed together with the point cloud image 310.

FIG. 9 shows a point cloud image 310 in the case where the viewpoint is located behind the measurement vehicle 110 and the direction of sight is from the rear side of the measurement vehicle 110 to the front side of the measurement vehicle 110.
FIG. 10 shows a point cloud image 310 in the case where the viewpoint is located in front of the measurement vehicle 110 and the sight line direction is a direction from the front side of the measurement vehicle 110 to the rear side of the measurement vehicle 110.
FIG. 11 shows a point cloud image 310 in the case where the viewpoint is located to the right of the measurement vehicle 110 and the sight line direction is from the right side of the measurement vehicle 110 to the left side of the measurement vehicle 110.
FIG. 12 shows a point cloud image 310 in the case where the viewpoint is located to the left of the measurement vehicle 110 and the sight line direction is a direction from the left side of the measurement vehicle 110 to the right side of the measurement vehicle 110.
FIG. 13 shows a point cloud image 310 in the case where the viewpoint is located above the measurement vehicle 110 and the sight line direction is a direction from the upper side of the measurement vehicle 110 to the lower side of the measurement vehicle 110.

In FIGS. 9 to 13, in the point cloud image 310, a three-dimensional point cloud 311, a vehicle mark 319, a route mark 314, and an event mark 315 are drawn. However, although the vehicle mark 319 and a part of the three-dimensional point group 311 are drawn in the point cloud image 310 in the initial state, the route mark 314 and the event mark 315 are not drawn.
The three-dimensional point group 311 is a set of measurement points. The measurement points are also referred to as three-dimensional points or irradiation points. Specifically, the measurement points are represented by small triangles.
The vehicle mark 319 is a mark representing the measurement vehicle 110, and corresponds to a measurement position mark 312 or an estimated position mark 313 described later. Specifically, the vehicle mark 319 is a white triangle.
The route mark 314 is a mark representing the route of the measurement vehicle 110. That is, the route mark 314 corresponds to a set of past vehicle marks 319. Specifically, the route mark 314 is a black triangle arranged on the route.
The event mark 315 is a mark representing an event. Specifically, event mark 315 is a diagram in which a solid having an event identifier marked at a plurality of locations and projected on a point cloud image 310 is a solid. More specifically, the event mark 315 is a diagram in which a double triangular pyramid in which two triangular pyramids are combined up and down is projected onto the point group image 310.

FIG. 14 shows event marks 315 viewed from each direction.
(1) of FIG. 14 is the event mark 315 seen from the front, and is mainly drawn on the point cloud image 310 of FIG.
(2) of FIG. 14 is an event mark 315 viewed from the back, and is mainly drawn on the point cloud image 310 of FIG.
(3) of FIG. 14 is an event mark 315 viewed from the right, and is mainly drawn on the point cloud image 310 of FIG.
(4) of FIG. 14 is an event mark 315 viewed from the left, and is mainly drawn on the point cloud image 310 of FIG.
(5) of FIG. 14 is the event mark 315 seen from the top, and is mainly drawn on the point cloud image 310 of FIG.

The triangular pyramid above the event mark 315 has three sides 316 on which the event identifier is marked. Specifically, the three sides 316 are numbered according to the type of event.
The triangular pyramid below the event mark 315 has a top 317 that points to a location corresponding to the location where the event is occurring.
The color of the event mark 315 differs for each type of event.

FIG. 15 shows a map image 560 when the map scale is large.
FIG. 16 shows a map image 560 when the map scale is small.
The map scale is a scale designated as the scale of the map drawn on the map image 560.
The scale is expressed in fractions. The smaller the denominator of the scale, the larger the scale, and the larger the denominator of the scale, the smaller the scale. That is, the large scale map is a detailed map representing a narrow range, and the small scale map is a schematic map representing a wide range.

In FIG. 15, the map image 560 includes a vehicle view 561, a viewpoint view 562, a route view 563 and an event view 564. In FIG. 16, the map image 560 includes a vehicle view 561, a route view 563 and an event view 564. However, although the vehicle view 561 and the view view 562 are drawn on the map image 560 in the initial state, the route view 563 and the event view 564 are not drawn.
The vehicle diagram 561 is a diagram showing the measurement vehicle 110. Specifically, the vehicle diagram 561 is a black triangle.
The viewpoint view 562 is a view representing a viewpoint and a viewpoint direction. The perspective view 562 is displayed when the map scale is large, but is not displayed when the map scale is small.
The route diagram 563 is a diagram showing the route of the measurement vehicle 110. That is, the route map 563 corresponds to a set of past vehicle maps 561. The route diagram 563 when the map scale is large is a white triangle arranged on the route. The route diagram 563 when the map scale is small is a line indicating a route.
An event diagram 564 is a diagram representing an event. Specifically, the event diagram 564 is a diagram including an event identifier identifying an event and event position information indicating a position where the event is occurring.

In FIGS. 15 and 16, the window for the map image 560 includes the following user interface.
The scale designation unit 571 is a user interface for designating a map scale. Specifically, the scale designation unit 571 is a button for raising and lowering the map scale.
The vehicle position display unit 572 is a user interface that indicates the position of the measurement vehicle 110. Specifically, the vehicle position display unit 572 is a text box indicating latitude, longitude, and ellipsoidal height.
The viewpoint display designation unit 573 is a user interface for designating display of the viewpoint view 562 or non-display of the viewpoint view 562. Specifically, the viewpoint display designation unit 573 is a check box checked when the viewpoint view 562 is displayed.
The path deletion unit 574 is a user interface for specifying deletion of the path diagram 563. Specifically, the path deletion unit 574 is a button pressed when the path diagram 563 is deleted.
The manual scroll designating unit 575 is a user interface for designating manual scrolling of the map drawn on the map image 560. Specifically, the manual scroll designation unit 575 is a button pressed when the map is manually scrolled.
The vehicle center designation unit 576 is a user interface for designating the position of the measurement vehicle 110 as the center of the map drawn on the map image 560. Specifically, the vehicle center designating unit 576 is a button that is pressed when the position of the measurement vehicle 110 is at the center of the map.
The update designating unit 577 is a user interface for designating the stop of the update of the map image 560 or the start of the update of the map image 560. Specifically, the update designation unit 577 is a button that is pressed when updating of the map image 560 is stopped and when updating of the map image 560 is started.

The initial screen processing (S210) will be described based on FIG.
In step S211, the road monitoring device 510 causes the display device 519 to display a window for the point cloud image 310, a window for the interface image 320, and a window for the map image 560. The data of these windows are stored in advance in the memory 903.
Specifically, the map image generation unit 512 inputs window data for the map image 560 to the display device 519. Then, the display device 519 displays a window for the map image 560 using the input data.
The point cloud image generation unit 513 also inputs window data for the point cloud image 310 to the display device 519. Then, the display device 519 displays a window for the point cloud image 310 using the input data.
Furthermore, the interface image generation unit 514 inputs window data for the interface image 320 into the display device 519. Then, the display device 519 displays a window for the interface image 320 on the display device 519 using the input data.

In step S212, the interface image generation unit 514 causes the display device 519 to display the interface image 320. The interface image 320 is stored in advance in the memory 903.
Specifically, the interface image generation unit 514 inputs the interface image 320 to the display device 519. The display device 519 then displays the interface image 320 in a window for the interface image 320.

  In step S 213, the reception unit 511 receives the vehicle packet 532, the measurement point packet 533, and the viewpoint packet 534 transmitted from the communication device of the measurement vehicle 110, and stores the received packet in the memory 903.

  In step S 214, the point cloud image generation unit 513 causes the display device 519 to display the point cloud image 310.

Specifically, the point cloud image generation unit 513 displays the point cloud image 310 as follows.
First, the point cloud image generation unit 513 acquires the vehicle position information 553 from the vehicle packet 532, and acquires the measurement point information 556 from the measurement point packet 533. Further, the point cloud image generation unit 513 acquires the viewpoint information 554 and the line-of-sight information 559 from the viewpoint packet 534.
Next, the point cloud image generation unit 513 is equivalent to a point cloud image displayed on a display device mounted on the measurement vehicle 110 based on the vehicle position information 553, the measurement point information 556, the viewpoint information 554, and the line of sight information 559. An image to be generated is generated as a point cloud image 310. The point cloud image 310 is an image in which a measurement point is drawn at a location corresponding to the position indicated by the measurement point information 556. The method of generating the point cloud image 310 is the same as the method of generating the projection point cloud data 235 and the projection mark data 236 by the point cloud image generating device 100 mounted on the measurement vehicle 110. The method of generating the projection point cloud data 235 and the projection mark data 236 will be described later.
Then, the point cloud image generation unit 513 inputs the point cloud image 310 to the display device 519, and the display device 519 displays the point cloud image 310 in the window for the point cloud image 310.

  In step S215, the map image generation unit 512 causes the display device 519 to display the map image 560 to which the vehicle view 561 and the viewpoint view 562 are added.

Specifically, the map image generation unit 512 displays the map image 560 as follows.
First, the map image generation unit 512 acquires the vehicle position information 553 from the vehicle packet 532. Also, the map image generation unit 512 acquires the viewpoint information 554 from the viewpoint packet 534.
Next, the map image generation unit 512 calculates the size of the area drawn on the map image 560 based on the map scale and the size of the window for the map image 560. The map scale and the window size for the map image 560 are stored in advance in the memory 903.
Next, the map image generation unit 512 calculates the range of the area drawn on the map image 560 based on the calculated size of the area and the vehicle position information 553. This area is an area including the position indicated by the vehicle position information 553.
Next, the map image generation unit 512 generates the map image 560 using the map data 520 of the area included in the calculated range.
Next, the map image generation unit 512 calculates a portion corresponding to the position indicated by the vehicle position information 553 among the portions in the map drawn on the map image 560, and adds the vehicle diagram 561 to the calculated portion.
Furthermore, the map image generation unit 512 calculates a part corresponding to the position indicated by the viewpoint information 554 among the parts in the map drawn on the map image 560, and adds the viewpoint map 562 to the calculated part.
Then, the map image generation unit 512 inputs the map image 560 to which the vehicle view 561 and the viewpoint view 562 have been added to the display device 519, and the display device 519 uses the input map image 560 as a window for the map image 560. indicate.

However, when the map scale is small, the map image generation unit 512 does not add the viewpoint 562.
Specifically, the map image generation unit 512 compares the map scale with the view scale threshold. Then, when the map scale is larger than the view scale threshold, the map image generation unit 512 adds the view 562. However, when the map scale is smaller than the view scale threshold, the map image generation unit 512 does not add the view 562.

Returning to FIG. 7, the description will continue from step S220.
In step S220, the reception unit 511 receives the event packet 531, the vehicle packet 532, the measurement point packet 533 or the viewpoint packet 534 transmitted from the communication device of the measurement vehicle 110, and stores the received packet in the memory 903.
If the receiving unit 511 receives the event packet 531, the process proceeds to step S230.
If the receiving unit 511 receives the vehicle packet 532 or the viewpoint packet 534, the process proceeds to step S240.
If the receiving unit 511 receives the measurement point packet 533, the process proceeds to step S250.

In step S230, the point cloud image generation unit 513 adds an event mark 315 to a point in the point cloud image 310 corresponding to the position indicated by the event position information 552 included in the event packet 531.
Then, the point cloud image generation unit 513 inputs the point cloud image 310 to which the event mark 315 is added to the display device 519, and the display device 519 uses the input point cloud image 310 as a window for the point cloud image 310. indicate.

Specifically, the point cloud image generation unit 513 adds an event mark 315 as follows.
First, the point cloud image generation unit 513 acquires the event identifier 551 and the event position information 552 from the received event packet 531.
Next, the point cloud image generation unit 513 generates a three-dimensional event mark 315 using the event identifier 551. The three-dimensional event mark 315 is three-dimensional data of a double triangular pyramid in which an event identifier 551 is written on each side of the upper triangular pyramid.
Next, the point cloud image generation unit 513 calculates a point in the point cloud image 310 corresponding to the position indicated by the event position information 552.
Then, the point cloud image generation unit 513 projects the three-dimensional event mark 315 on the point cloud image 310 so that the top 317 of the lower part of the event mark 315 points to the calculated point in the point cloud image 310. Projection means converting three-dimensional coordinate values into two-dimensional coordinate values in the point cloud image 310. In the method of projecting the three-dimensional event mark 315 on the point cloud image 310, the point group image generation device 100 mounted on the measurement vehicle 110 is used to generate the three-dimensional point group 311, the measurement position mark 312 and the estimated position mark 313 as a point cloud image. It is the same as the two-dimensional projection processing to project to 310. The two-dimensional projection process will be described later.

Furthermore, in step S230, the map image generation unit 512 adds an event diagram 564 to a location corresponding to the position indicated by the event location information 552 included in the event packet 531 among the locations in the map image 560.
Then, the map image generation unit 512 inputs the map image 560 to which the event diagram 564 is added to the display device 519, and the display device 519 displays the input map image 560 in the window for the map image 560.

Specifically, the map image generation unit 512 adds the event diagram 564 as follows.
First, the map image generation unit 512 acquires the event identifier 551 and the event position information 552 from the received event packet 531.
Next, the map image generation unit 512 generates an event diagram 564 in which the event identifier 551 and the event location information 552 are described.
Then, the map image generation unit 512 calculates a part corresponding to the position indicated by the event position information 552 among the parts in the map drawn on the map image 560, and adds the event diagram 564 to the calculated part.
The map image generation unit 512 may change the color of the event mark 315 for each type of event based on the event identifier 551.

  In step S240, the point cloud image generation unit 513 updates the point cloud image 310.

Specifically, the point cloud image generation unit 513 updates the point cloud image 310 as follows.
First, the point cloud image generation unit 513 acquires vehicle position information 553 from each vehicle packet 532, and acquires viewpoint information 554 and line-of-sight information 559 from the latest viewpoint packet 534. Further, the point cloud image generation unit 513 acquires three-dimensional point cloud data 555 from each measurement point packet 533. Further, the point cloud image generation unit 513 acquires an event identifier 551 and event position information 552 from each event packet 531.
Next, the point group image generation unit 513 generates a point group image 310 based on the vehicle position information 553, the viewpoint information 554, the line of sight information 559, and the three-dimensional point group data 555. The method of generating the point cloud image 310 is the same as the method of generating the projection point cloud data 235 and the projection mark data 236 by the point cloud image generating device 100 mounted on the measurement vehicle 110. The method of generating the projection point cloud data 235 and the projection mark data 236 will be described later.
Next, the point cloud image generation unit 513 adds an event mark 315 to the generated point cloud image 310 based on the event identifier 551 and the event position information 552. The method of adding the event mark 315 is the same as the method described in step S230.
Then, the point cloud image generation unit 513 inputs the point cloud image 310 to which the event mark 315 is added to the display device 519, and the display device 519 uses the input point cloud image 310 as a window for the point cloud image 310. indicate.

  Furthermore, in step S240, the map image generation unit 512 updates the map image 560.

Specifically, the map image generation unit 512 updates the map image 560 as follows.
First, the map image generation unit 512 acquires the vehicle position information 553 from the latest vehicle packet 532 and the past vehicle packet 532, and acquires the viewpoint information 554 from the latest viewpoint packet 534. Furthermore, the map image generation unit 512 acquires an event identifier 551 and event position information 552 from each event packet 531.
Next, based on the vehicle position information 553 acquired from the latest vehicle packet 532 and the viewpoint information 554 acquired from the latest viewpoint packet 534, the map image generation unit 512 adds the vehicle diagram 561 and the viewpoint diagram 562 A map image 560 is generated. The method of generating this map image 560 is the same as the method described in step S215.
In addition, the map image generation unit 512 calculates the route of the measurement vehicle 110 based on the vehicle position information 553 acquired from the vehicle packet 532 in the past, and adds a route diagram 563 representing the calculated route to the map image 560.
Furthermore, the map image generation unit 512 adds an event diagram 564 to the map image 560 based on the event identifier 551 and the event position information 552. The method of adding the event diagram 564 is the same as the method described in step S230.
Then, the map image generation unit 512 inputs the map image 560 to which the vehicle diagram 561, the viewpoint diagram 562, the route diagram 563 and the event diagram 564 are added to the display device 519, and the display device 519 receives the input map image 560 Is displayed in the window for the map image 560.

  However, as described in step S215, when the map scale is small, the map image generation unit 512 does not add the viewpoint 562 to the map image 560.

Further, the map image generation unit 512 changes the form of the route diagram 563 to be added to the map image 560 according to the map scale.
Specifically, the map image generation unit 512 compares the map scale with the route scale threshold.
Then, when the map scale is larger than the scale scale threshold for the route, the map image generation unit 512 sets a mark representing the position where the measurement vehicle 110 has passed at a location corresponding to the position indicated by the vehicle position information 553 for each time. Add as 563
When the map scale is smaller than the route scale threshold, the map image generation unit 512 adds a line passing through a location corresponding to the position indicated by the vehicle position information 553 at each time as the route diagram 563.

  In step S250, the point cloud image generation unit 513 updates the point cloud image 310. The method of updating the point cloud image 310 is the same as the method described in step S240.

*** Description of the configuration ***
The configuration of the point cloud image generation device 100 will be described based on FIG.
The point cloud image generation device 100 includes a measurement data processing unit 200, a display device 300, an operation unit 400, and a communication device 301.
Point cloud image generation device 100 is mounted on measurement vehicle 110.

The measurement vehicle 110 is a vehicle that performs various measurements while traveling on a road.
Positioning reinforcement data 101, positioning data 102, inertial measurement data 103, traveling distance data 104, and azimuth distance data 105 are obtained by various measurements.

The configuration of the measurement vehicle 110 will be described based on FIG.
The measuring vehicle 110 is provided with a top plate 119 provided on the roof of the vehicle body. The measurement vehicle 110 also includes an odometer 114.
A positioning reinforcing signal receiver 111, a positioning signal receiver 112, an inertial measurement device 113, and a laser scanner 115 are attached to the top 119.

The positioning reinforcement signal receiver 111 receives positioning reinforcement data 101 used for positioning the position of the measurement vehicle 110 with high accuracy from a GNSS satellite or a wireless LAN or mobile telephone line on the ground. GNSS is an abbreviation of Global Positioning System.
The positioning reinforcement data 101 may be distributed from GNSS satellites or distributed from a GNSS correction information distribution service provider via a mobile telephone network.
When the GNSS satellite is a Quasi-Zenith satellite that delivers positioning augmentation data 101, positioning-compensation data 101 with centimeter accuracy is included in the L6 band signal transmitted from the Quasi-Zenith satellite. In this case, the positioning reinforcement data 101 indicates the error of the satellite clock of the GPS satellite, the orbit error of the GPS satellite, the inter-frequency bias, the ionospheric propagation delay error, and the tropospheric delay error.

The positioning signal receiver 112 receives the positioning signal transmitted from the positioning satellite, and positions the position of the measurement vehicle 110 using the reception result of the positioning signal. An example of a positioning satellite is a GPS satellite.
The positioning data 102 is data indicating the position of the measurement vehicle 110 measured by the positioning signal receiver 112.

The inertial measurement device 113 includes a gyro that measures angles and angular velocities of the measurement vehicle 110 in three axial directions, and an acceleration sensor that measures acceleration in the three axial directions of the measurement vehicle 110. The angles in the three axis directions of the measurement vehicle 110 are an elevation angle, a rotation angle, and an azimuth angle.
The inertial measurement data 103 is data indicating information obtained by the inertial measurement device 113.

The odometer 114 detects a vehicle speed pulse output each time the wheel of the measurement vehicle 110 rotates, and calculates a travel distance by integral calculation using the radius of gyration of the tire and the amount of rotation of the wheel.
The travel distance data 104 is data indicating the travel distance calculated by the odometer 114.

The laser scanner 115 emits a laser while rotating at a speed of about 100 rotations per second, and scans a feature located around the measurement vehicle 110.
At that time, the laser scanner 115 measures the time difference between the time when the laser was emitted and the time when the laser reflected from the feature was received for each point (irradiation point) where the laser was irradiated, and the measured time difference Use to determine the distance to the point where the laser was illuminated. Further, the laser scanner 115 obtains an azimuth indicating a direction in which the laser is emitted.
The azimuth distance data 105 is data in which measurement time, distance, and azimuth are associated with each other. The measurement time is a time when the measurement by the laser scanner 115 is performed. An example of the measurement time is the time when the laser was emitted or the time when the laser was received.

Returning to FIG. 18, the description of the point cloud image generation device 100 will be continued.
The measurement data processing unit 200 includes a position calculation unit 121, an attitude calculation unit 122, a three-dimensional point group generation unit 123, a position estimation unit 220, a two-dimensional projection unit 230, an event position calculation unit 240, and packet generation. And a unit 250.

The position calculation unit 121 calculates three-dimensional coordinate values of the measurement position, which is the position of the measurement vehicle 110 at the measurement time, using the positioning reinforcement data 101, the positioning data 102, the inertia measurement data 103, and the travel distance data 104. The measurement position is also referred to as self position or vehicle position.
Then, the position calculation unit 121 generates measurement position data 201 indicating the calculated measurement position.

The posture calculation unit 122 calculates a measurement posture that is the posture of the measurement vehicle 110 (or the laser scanner 115) at the measurement time using the positioning data 102, the inertia measurement data 103, and the travel distance data 104.
Then, the posture calculation unit 122 generates measurement posture data 202 indicating the calculated measurement posture.

The three-dimensional point group generation unit 123 uses the azimuth distance data 105, the measurement position data 201, and the measurement attitude data 202 to represent three-dimensional point group data representing a feature measured by the laser scanner 115 by a three-dimensional point group. Generate 203.
At this time, the three-dimensional point group generation unit 123 corrects the azimuth indicated by the azimuth distance data 105 using the measurement attitude indicated by the measurement attitude data 202, and the corrected azimuth and the measurement position and azimuth distance indicated by the measurement position data 201. The three-dimensional coordinate value of the point irradiated with the laser is calculated using the distance indicated by the data 105.
The three-dimensional point cloud data 203 is azimuth distance data 105 obtained by measuring the azimuth and distance to a plurality of laser irradiation points by irradiating the laser scanner 115 attached to the measurement vehicle 110 to surrounding features. The data is generated using the data and indicates three-dimensional coordinate values of each of a plurality of laser irradiation points.

The position estimation unit 220 estimates the three-dimensional coordinate value of the estimated position, which is the position of the measurement vehicle 110 when the point group image 310 is displayed, using the measurement position data 201 indicating the three-dimensional coordinate value of the measurement position. . The estimated position is also referred to as a current position or an estimated self position.
Then, the position estimation unit 220 generates estimated position data 229 indicating the estimated three-dimensional coordinate values.

The two-dimensional projection unit 230 uses the measurement position data 201, the measurement posture data 202, the three-dimensional point group data 203, and the estimated position data 229 to project points according to information indicated by the operation data 204 output from the operation unit 400. Group data 235 and projection mark data 236 are generated. Although not shown, the measurement position data 201 is input from the position calculation unit 121 to the two-dimensional projection unit 230, and the measurement posture data 202 is input from the posture calculation unit 122 to the two-dimensional projection unit 230.
The projection point group data 235 is data obtained by converting the three-dimensional point group data 203 into two-dimensional coordinate values. The generated projection point group data 235 and projection mark data 236 are output to the display device 300.
The coordinate conversion process (projection process) in the two-dimensional projection unit 230 will be described later.

FIG. 22 is a view showing an example of a point group image 310 displayed on the display device 300 based on the projection point group data 235 and the projection mark data 236. As shown in FIG.
The point cloud image 310 includes a three-dimensional point cloud 311, a measurement position mark 312, and an estimated position mark 313.
The three-dimensional point group 311 represents a plurality of irradiation points obtained by irradiating the peripheral feature with the laser scanner 115 attached to the measurement vehicle 110, and is displayed based on the three-dimensional point group data 203. Each point constituting the three-dimensional point group 311 is represented by a triangle.
The measurement position mark 312 represents a measurement position which is the position of the measurement vehicle 110 when the measurement vehicle 110 acquires various measurement data. The measurement position is also referred to as self position or vehicle position.
The estimated position mark 313 represents an estimated position which is the position of the measurement vehicle 110 when the point cloud image 310 is displayed. The estimated position is also referred to as the current position or the vehicle position.
Measurement position mark 312 and estimated position mark 313 are displayed based on projection mark data 236.

The point cloud image 310 displayed on the measuring vehicle 110 may include a route mark 314 and an event mark 315, as with the point cloud image 310 displayed on the road monitoring device 510. That is, the projection mark data 236 may include two-dimensional coordinate values in the point cloud image 310 of the path mark 314 and the event mark 315, respectively.
The projection mark data 236 is data including two-dimensional coordinate values of various marks in the point cloud image 310.
The projection point group data 235 is data including two-dimensional coordinate values of each point of the three-dimensional point group 311 in the point group image 310.

Returning to FIG. 18, the description of the point cloud image generation device 100 will be continued.
The operation unit 400 outputs operation data 204 indicating the viewpoint position and the gaze direction designated by the user.
The operation unit 400 is software or hardware for operating the interface image 320 or the user interface represented by the interface image 320.
The interface image 320 is an image representing a user interface for changing the viewpoint position and the gaze direction.
The user specifies the viewpoint position and the gaze direction by operating the user interface represented by the interface image 320 using an input device such as a keyboard and a mouse. The user may specify the viewpoint position and the gaze direction using an input device such as a touch pad or a touch panel.

A display example of the display device 300 will be described based on FIG.
The display device 300 displays the point cloud image 310 and the interface image 320 corresponding to the operation unit 400. The display device 300 may include a touch panel operated by the user.

FIG. 21 is a diagram for explaining the viewpoint position of the point cloud image 310 displayed on the display device 300. As shown in FIG.
The reference viewpoint position 411 is set at a position separated by a predetermined distance from the position of the measurement vehicle 110 (the vehicle position 401). In the initial state, a point cloud image 310 of a feature which can be seen in the direction of the line of sight from the reference viewpoint position 411 toward the traveling direction (forward) of the measurement vehicle 110 is displayed. The reference viewpoint position 411 is also referred to as a reference position or a viewpoint reference position.
The user can change the viewing direction from the reference viewpoint position 411 to a desired direction via the operation unit 400, and can set a viewpoint position where a predetermined offset value is provided to the reference viewpoint position 411.

The configuration of the interface image 320 will be described based on FIG.
The interface image 320 includes a front button 321F, a rear button 321B, a left button 321L, a right button 321R, and a lower button 321D.
The front button 321F, the rear button 321B, the left button 321L, and the right button 321R are buttons having an arrow shape, and the lower button 321D is a button having a square shape.
The forward button 321F is disposed above the downward button 321D, the rearward button 321B is disposed below the downward button 321D, the left button 321L is disposed to the left of the downward button 321D, and the right button 321R is disposed to the right of the downward button 321D It is done.

The front button 321F is a button for turning the sight line direction to the traveling direction of the measurement vehicle 110.
The rear button 321B is a button for turning the sight line direction in the opposite direction to the traveling direction of the measurement vehicle 110.
The left button 321L is a button for turning the sight line direction to the left with respect to the traveling direction of the measurement vehicle 110.
The right button 321R is a button for turning the sight line direction to the right with respect to the traveling direction of the measurement vehicle 110.
The lower button 321D is a button for turning the line of sight downward.

The interface image 320 includes left and right widgets 322X, front and rear widgets 322Y, upper and lower widgets 322Z, elevation widget 323P, azimuth widget 323Y, and cancel button 329.
The left and right widgets 322 </ b> X are user interfaces for designating a movement amount for moving the viewpoint position in the left and right direction of the measurement vehicle 110 with respect to the reference viewpoint position 411.
The front and rear widget 322Y is a user interface for specifying a movement amount for moving the viewpoint position in the front and rear direction of the measurement vehicle 110 with respect to the reference viewpoint position 411.
The upper and lower widgets 322 </ b> Z are a user interface for specifying a movement amount for moving the viewpoint position in the up and down direction of the measurement viewpoint 110 with respect to the reference viewpoint position 411.
The elevation widget 323P is a user interface for setting an offset amount for shifting the viewing direction from the reference viewpoint position 411 by a predetermined angle in the vertical direction.
The azimuth angle widget 323 </ b> Y is a user interface for setting an offset amount for shifting the viewing direction from the reference viewpoint position 411 by a predetermined angle in the horizontal direction.
The cancel button 329 is a button for canceling the designated movement amount and the designated offset amount.

The processing of the two-dimensional projection unit 230 will be described below.
The two-dimensional projection unit 230 calculates the reference viewpoint position 411 using the measurement position data 201 and the measurement posture data 202, and the user designates using the reference viewpoint position 411 and the operation data 204 from the operation unit 400. Find the viewpoint position and viewpoint direction
The two-dimensional projection unit 230 performs two-dimensional projection processing to convert the three-dimensional point group data 203 into two-dimensional coordinate data based on the viewpoint position and the gaze direction specified by the user, and generates projection point group data 235.
Similarly, the two-dimensional projection unit 230 performs two-dimensional projection processing for converting the measurement position data 201 and the estimated position data 229 into two-dimensional coordinate data, respectively, based on the viewpoint position and the viewpoint direction specified by the user. The mark data 236 is generated.

Returning to FIG. 18, the description of the point cloud image generation device 100 will be continued.
The event position calculation unit 240 uses the projection point group data 235 and the three-dimensional point group data 203 to calculate an event position where an event has occurred.
The method for calculating the event position will be described in the second embodiment.

  The packet generator 250 generates various packets to be transmitted to the road monitoring device 510.

Specifically, the packet generator 250 generates the following packet.
The packet generation unit 250 generates an event packet 531 including event position information 552 indicating the event position calculated by the event position calculation unit 240.
The packet generation unit 250 generates a vehicle packet 532 including the measured position data 201 or the estimated position data 229 as the vehicle position information 553.
The packet generation unit 250 generates a viewpoint packet 534 including the operation data 204 as the viewpoint information 554 and the gaze information 559.
The packet generation unit 250 generates a measurement point packet 533 including the three-dimensional point cloud data 203 as the three-dimensional point cloud data 555.

  The communication device 301 transmits various packets generated by the packet generation unit 250 to the road monitoring device 510.

*** Description of operation ***
The operation of the point cloud image generation device 100 corresponds to a point cloud image generation method. The point cloud image generation method corresponds to the processing procedure of the point cloud image generation program.

A point cloud image generation method will be described based on FIG.
Step S110 is measurement processing.
In step S110, each device of the measurement vehicle 110 performs measurement, and outputs data obtained by the measurement. The data to be output are positioning reinforcement data 101, positioning data 102, inertia measurement data 103, traveling distance data 104, and azimuth distance data 105.

Step S120 is position calculation processing.
In step S120, the position calculation unit 121 calculates three-dimensional coordinate values of the measurement position using the positioning reinforcement data 101, the positioning data 102, the inertia measurement data 103, and the travel distance data 104. The measurement position is the position of the measurement vehicle 110 at the time of measurement.
Then, the position calculation unit 121 generates measurement position data 201 indicating the three-dimensional coordinate value of the measurement position, and outputs the generated measurement position data 201.

Step S130 is attitude calculation processing.
In step S130, the posture calculation unit 122 calculates a measured posture using the positioning data 102, the inertia measurement data 103, and the traveling distance data 104. The measurement attitude is the attitude of the measurement vehicle 110 at the time of measurement.
Then, the posture calculation unit 122 generates measurement posture data 202 indicating the measured posture, and outputs the generated measurement posture data 202.

Step S140 is three-dimensional point group generation processing.
In step S140, the three-dimensional point group generation unit 123 calculates three-dimensional coordinate values of each irradiation point irradiated with the laser, using the measurement position data 201, the measurement posture data 202, and the azimuth distance data 105.
Then, the three-dimensional point group generation unit 123 generates three-dimensional point group data 203 indicating the three-dimensional coordinate value of each irradiation point, and outputs the generated three-dimensional point group data 203.

Step S150 is position estimation processing.
In step S <b> 150, the position estimation unit 220 uses the measured position data 201 to calculate three-dimensional coordinate values of the estimated position. The estimated position is a position estimated as the position of the measurement vehicle 110 when the point cloud image 310 is displayed.
Then, the position estimation unit 220 generates estimated position data 229 indicating three-dimensional coordinate values of the estimated position, and outputs the generated estimated position data 229.

The position estimation unit 220 calculates the estimated position as follows.
The position estimation unit 220 obtains the difference between the time when the latest measurement position data 201 is acquired and the current time, and the self position at the current time from the traveling locus, the angular velocity, and the acceleration when the latest measurement position data 201 is acquired. Is calculated as the estimated position. Here, the current time is the time at which the display device 300 displays the point cloud image 310 based on the projection point group data 235 and the projection mark data 236 output from the two-dimensional projection unit 230. This current time takes into consideration the processing time required for the display device 300 to display the point cloud image 310 acquired at the same time as the measurement position data 201 is acquired after the measurement position data 201 is acquired. Is required.

  For example, it is assumed that, at time T0, the measurement vehicle 110 present at the point P0 performs uniform linear motion at a speed of 50 km per hour in the X-axis direction. In this case, at the current time T1, the measuring vehicle 110 is estimated to be present at the point P1 which is advanced in the X-axis direction from the point P0 by (50 * 1000/3600) * (T1-T0) [meters] .

Step S160 is a two-dimensional projection process.
In step S160, the two-dimensional projection unit 230 generates projection point group data 235 according to the information indicated by the operation data 204, using the three-dimensional point cloud database, the measurement position data 201, and the measurement posture data 202.
The two-dimensional projection unit 230 generates the projection mark data 236 according to the information indicated by the operation data 204 using the measurement position data 201.
Then, the two-dimensional projection unit 230 outputs the projection point group data 235 and the projection mark data 236 to the display device 300 as data of the point cloud image 310.

Step S170 is point cloud image display processing.
In step S170, the display device 300 displays the point cloud image 310 using the projection point cloud data 235 and the projection mark data 236.
The event position calculation unit 240 may output and display on the display device 300 an image obtained by adding the event mark 315 to the point cloud image 310, as in the point cloud image generation unit 310.

FIG. 25 shows point cloud images 310 according to the viewing direction.
(F) of FIG. 25 shows a point cloud image 310 when the forward button 321F is pressed, that is, when the sight line direction is the traveling direction of the measurement vehicle 110.
(B) of FIG. 25 shows a point cloud image 310 when the back button 321 B is pressed, that is, when the sight line direction is reverse to the traveling direction of the measurement vehicle 110.
(L) in FIG. 25 shows a point cloud image 310 when the left button 321L is pressed, that is, when the sight line direction is the left direction with respect to the traveling direction of the measurement vehicle 110.
(R) of FIG. 25 shows a point cloud image 310 when the right button 321R is pressed, that is, when the sight line direction is the right direction with respect to the traveling direction of the measurement vehicle 110.
FIG. 25D shows a point cloud image 310 when the lower button 321D is pressed, that is, when the line of sight direction is downward.

FIG. 26 shows point cloud images 310 according to moving offsets in the vertical and horizontal directions.
(0) in FIG. 26 is a point group when the movement amount specified by the upper and lower widgets 322Z and the movement amount specified by the left and right widgets 322X are zero, that is, when the viewpoint position is the same as the reference viewpoint position 411. An image 310 is shown.
(U) of FIG. 26 shows a point cloud image 310 when +5.0 m is specified by the upper and lower widgets 322Z, that is, when the viewpoint position moves upward from the reference viewpoint position 411.
(D) of FIG. 26 shows the point cloud image 310 when the viewpoint position has moved downward from the reference viewpoint position 411 when -5.0 m is specified by the upper and lower widgets 322Z.
(L) in FIG. 26 shows the point cloud image 310 when +5.0 m is specified by the left and right widgets 322X, that is, when the viewpoint position has moved from the reference viewpoint position 411 to the left.
(R) of FIG. 26 shows a point cloud image 310 in the case where -5.0 meters is designated by the left and right widgets 322X, that is, the viewpoint position is moved from the reference viewpoint position 411 to the right.

FIG. 27 shows point cloud images 310 according to moving offsets before and after.
(O) of FIG. 27 shows the point cloud image 310 when the movement amount designated by the front and rear widget 322 Y is zero, that is, when the viewpoint position is the same as the reference viewpoint position 411.
FIG. 27F shows the point cloud image 310 when +5.0 m is specified by the front and rear widgets 322Y, that is, when the viewpoint position has moved forward from the reference viewpoint position 411.
(B) of FIG. 27 shows the point cloud image 310 when the viewpoint position has moved backward from the reference viewpoint position 411 when -5.0 m is specified by the front and rear widget 322Y.

FIG. 28 shows point cloud images 310 according to upper, lower, left, and right rotational offsets.
(O) of FIG. 28 is a point cloud image when the offset amount designated by the elevation widget 323P and the offset amount designated by the azimuth widget 323Y are zero, that is, when the two-dimensional projection plane is not rotated. 310 is shown.
(U) of FIG. 28 shows a point cloud image 310 when +10 degrees is designated by the elevation angle widget 323P, that is, when the two-dimensional projection surface is inclined upward.
FIG. 28D shows the point cloud image 310 when the elevation widget 323P specifies -10 degrees, that is, when the two-dimensional projection plane is inclined downward.
(L) in FIG. 28 shows a point cloud image 310 when +10 degrees is designated by the azimuth angle widget 323 Y, that is, when the two-dimensional projection plane is inclined leftward.
(R) of FIG. 28 shows a point cloud image 310 when the azimuth widget 323Y specifies -10 degrees, that is, when the two-dimensional projection plane is inclined to the right.

*** Description of hardware configuration ***
A hardware configuration example of the point cloud image generation device 100 will be described based on FIG.
The point cloud image generation apparatus 100 is a computer including hardware such as a processor 901, an auxiliary storage device 902, a memory 903, a communication device 904, an input interface 905, and an output interface 906.
The processor 901 is connected to other hardware via a signal line 910. The input interface 905 is connected to the input device 907 via the cable 911. Output interface 906 is connected to output device 908 via cable 912.

The processor 901 is an IC that performs processing and controls other hardware. An example of the processor 901 is a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit, DSP is an abbreviation for Digital Signal Processor, and GPU is an abbreviation for Graphics Processing Unit.
The auxiliary storage unit 902 stores data. An example of the auxiliary storage device 902 is a ROM, a flash memory, or an HDD. ROM is an abbreviation for Read Only Memory, and HDD is an abbreviation for Hard Disk Drive.
The memory 903 stores data. An example of the memory 903 is a RAM. RAM is an abbreviation of Random Access Memory.
The communication device 904 comprises a receiver 9041 for receiving data and a transmitter 9042 for transmitting data. An example of the communication device 904 is a communication chip, NIC. NIC is an abbreviation of Network Interface Card.
The input interface 905 is a port to which the cable 911 is connected, and an example of the port is a USB terminal. USB is an abbreviation for Universal Serial Bus.
The output interface 906 is a port to which the cable 912 is connected, and the USB terminal and the HDMI terminal are an example of a port. HDMI (registered trademark) is an abbreviation of High Definition Multimedia Interface.
An input device 907 inputs data, instructions and requests. An example of the input device 907 is a mouse, a keyboard, a touch panel, and a touch pad.
An output device 908 outputs data, results and responses. An example of the output device 908 is a display or a printer. An example of a display is an LCD. LCD is Liquid
Acronym for Crystal Display.

An OS is stored in the auxiliary storage device 902. OS is an abbreviation of Operating System.
In addition, the auxiliary storage device 902 stores programs for realizing functions of “parts” such as the position calculation unit 121, the posture calculation unit 122, the three-dimensional point group generation unit 123, the position estimation unit 220, and the two-dimensional projection unit 230. ing. A program for realizing the function of “section” can be stored in a storage medium.
At least a part of the OS is loaded into the memory 903, and the processor 901 executes a program that implements the function of “unit” while executing the OS. A program for realizing the function of “part” is loaded into the memory 903, read into the processor 901, and executed by the processor 901.
Note that the point cloud image generation device 100 may include a plurality of processors 901, and the plurality of processors 901 may cooperatively execute programs that realize the function of “section”.

  Data, information, signal values, variable values and the like indicating the result of the processing of “part” are stored in the memory 903, the auxiliary storage device 902, a register in the processor 901, or a cache memory in the processor 901.

The "part" may be implemented by "circuitry". "Part" may be read as "circuit", "process", "procedure" or "treatment".
The “circuit” and “circuitry” are concepts including processing circuits such as the processor 901, logic IC, GA, ASIC, and FPGA. GA is an abbreviation for Gate Array, and ASIC is Application Specific Integrated
It is an abbreviation for Circuit, and FPGA is an abbreviation for Field-Programmable Gate Array.

*** Effect of the embodiment ***
The road monitoring device 510 displays a map image 560 to which an event diagram 564 is added at a location corresponding to a location where an event has occurred for a road or an event occurring around the road. Therefore, the road monitoring device 510 can notify the supervisor 502 of an event that may cause a road depression or a tree or the like that has run out of the road.
The road monitoring device 510 adds a vehicle diagram 561 to a location in the map image 560 that corresponds to the position of the measurement vehicle 110. In addition, the road monitoring device 510 adds a route diagram 563 representing the route on which the measuring vehicle 110 has traveled to the map image 560. Therefore, the road monitoring device 510 can notify the supervisor 502 of the road measured by the measuring vehicle 110.
The road monitoring device 510 switches the form of the route map 563 according to the map scale. Further, the road monitoring device 510 switches display or non-display of the perspective view 562 according to the map scale. Therefore, the road monitoring device 510 can provide the supervisor 502 with an easy-to-see map image 560 according to the map scale.

The road monitoring device 510 displays a point cloud image 310 corresponding to the point cloud image 310 displayed on the measuring vehicle 110. Therefore, the road monitoring device 510 can provide the surveillance person 502 with the point cloud image 310 corresponding to the point cloud image 310 displayed in the measuring vehicle 110.
The road monitoring device 510 adds an event mark 315 to a location in the point cloud image 310 that corresponds to the location where the event has occurred. Accordingly, the road monitoring device 510 can notify the supervisor 502 of the situation where the event has occurred.
The event mark 315 is a solid having an event identifier marked at a plurality of places. Therefore, the observer 502 can be notified of the event identifier of the event represented by the event mark 315 regardless of the viewing direction of the point cloud image 310.

*** Other configuration ***
The point group image generation unit 513 may generate a point group image 310 according to the viewpoint and the line of sight direction specified by the observer 502 using the interface image 320.

The measurement of the road and the surroundings of the road may be performed by multiple measurement vehicles 110.
In that case, the map image generation unit 512 generates a map image 560 to which a viewpoint view 562 and an event view 564 for each measurement vehicle 110 are added.
In FIG. 30, in the map image 560, the viewpoint 562 and the event diagram 564 of each of the two measurement vehicles 110 are added. A vehicle identifier 542 is described in each view 562.

The vehicle packet 532 may include information such as visible information, accuracy information, and measurement distance. That is, these pieces of information may be transmitted from the communication device of the measurement vehicle 110 to the road monitoring device 510.
The visible information is information indicating the number of visible satellites when the measurement vehicle 110 is positioned. A visible satellite is a positioning satellite of the transmission source of the positioning signal which the positioning signal receiver 112 received. The visible information is obtained by the positioning signal receiver 112.
The accuracy information is information indicating the accuracy of the positioning result. The accuracy information is obtained by the positioning signal receiver 112.
The measurement distance is the distance of the route measured by the measurement vehicle 110.
The road monitoring device 510 displays such information on the display device 519. For example, the map image generation unit 512 adds a diagram showing these pieces of information to the map image 560. Thus, the road monitoring device 510 can notify the observer 502 of the measurement status or the progress status of the measurement operation.

The function of the road monitoring device 510 may be realized by hardware.
Specifically, the road monitoring device 510 may include processing circuitry in place of the processor 901, the auxiliary storage device 902, and the memory 903. The processing circuit is a dedicated electronic circuit that realizes the function of "part". The "part" also includes a storage part.
The road monitoring device 510 may be provided with a plurality of processing circuits, and the plurality of processing circuits may be realized in cooperation with the function of “section”.

  The functions of the road monitoring device 510 may be realized by a combination of software and hardware. That is, part of the "part" may be realized by software, and the rest of the "part" may be realized by hardware.

  In the measurement vehicle 110, the point cloud image generation device 100 may display the point cloud image 310 including the event mark 315 on the display device 300. Furthermore, the point cloud image generation device 100 may include a map image generation unit 512 and map data 520. That is, the point cloud image generation device 100 may display the map image 560 on the display device 300. The point cloud image generation device 100 functions as a road monitoring device mounted on the measurement vehicle 110. The two-dimensional projection unit 230 of the point group image generation device 100 functions as a point group image generation unit 513 of the road monitoring device 510, and the operation unit 400 of the point group image generation device 100 is an interface image generation unit of the road monitoring device 510. It functions as 514.

Second Embodiment
The form in which the road monitoring device 510 calculates the position where the event has occurred will be described based on FIG. 31 to FIG. However, the description overlapping with that of the first embodiment is omitted.

*** Description of the configuration ***
The configuration of the road monitoring device 510 will be described based on FIG.
The road monitoring device 510 further includes an event position calculation unit 515 as a function of “part”. The event position calculation unit 515 will be described below.

*** Description of operation ***
Example 1
The operation of the event position calculation unit 515 will be described based on FIG.
In step S311, the observer 502 refers to the displayed point cloud image 310 and designates a point in the point cloud image 310 representing an event using an input device such as a mouse or a keyboard.
Then, the event position calculation unit 515 receives the designated portion as the designated portion corresponding to the event position.

In step S312, the event position calculation unit 515 selects a measurement point corresponding to the designated part.
Specifically, the measurement points corresponding to the designated place are the designated places or the measurement points drawn around the designated place. More specifically, the measurement points corresponding to the designated part are one measurement point drawn in the designated part or a plurality of measurement points drawn in the vicinity of the designated part. When the distance from the designated part to the measurement point is shorter than the distance threshold, the measurement point is a measurement point drawn in the vicinity of the designated part.
The measurement points depicted in the point cloud image 310 are identified by measurement point identifiers.

  In step S313, the event position calculation unit 515 uses the three-dimensional coordinate values of the selected measurement point to calculate the three-dimensional coordinate value corresponding to the designated part as the event position.

Specifically, the event position calculation unit 515 acquires measurement point information 556 indicating the three-dimensional coordinate value of the selected measurement point as follows.
Each measurement point information 556 in the three-dimensional point cloud data 555 included in the measurement point packet 533 includes a measurement point identifier.
The event position calculation unit 515 selects measurement point information 556 including the measurement point identifier of the selected measurement point from the three-dimensional point cloud data 555 included in each measurement point packet 533.

When one measurement point is selected as the measurement point corresponding to the designated part, the three-dimensional coordinate value of the selected measurement point is the event position.
When a plurality of measurement points are selected as measurement points corresponding to the designated part, event positions are calculated using three-dimensional coordinate values of each of the selected measurement points. For example, three-dimensional coordinate values of an intermediate point located in the middle of each measurement point are calculated as the event position.

In step S314, the observer 502 designates, using the input device, a type identifier for identifying the type of event.
Then, the event position calculation unit 515 receives the designated type identifier.

  In step S315, the event position calculation unit 515 generates an event identifier 551 using the received type identifier. The event identifier 551 is a combination of a type identifier and a serial number for each type of event.

  In step S316, the event position calculation unit 515 stores the event identifier 551 and the event position information 552 indicating the event position in the memory 903.

Example 2
The operation of the event position calculation unit 515 will be described based on FIG.
In step S 321, the event position calculation unit 515 calculates the road surface height using the vehicle position information 553 included in the vehicle packet 532. The road surface height is the height of the road surface of the portion where the tire of the measurement vehicle 110 is in contact with the ground.
Specifically, the event position calculation unit 515 calculates the road surface height by subtracting the height from the coordinate center of the measurement vehicle 110 to the lower end of the tire of the measurement vehicle 110 from the ellipsoidal height indicated by the vehicle position information 553. For example, when the center of the roof of the measurement vehicle 110 is the coordinate center, the event position calculation unit 515 calculates the road surface height by subtracting the vehicle height including the tire from the ellipsoid height indicated by the vehicle position information 553.

In step S322, the event position calculation unit 515 compares the road surface height with the height of each measurement point.
Specifically, the event position calculation unit 515 compares the ellipsoid height indicated by the measurement point information 556 included in the three-dimensional point cloud data 555 with the road surface height.

In step S323, the event position calculation unit 515 selects a measurement point corresponding to the event based on the comparison result.
Specifically, the event position calculation unit 515 sets measurement point information 556, which indicates an ellipsoidal height lower than the road surface height and whose difference with the road surface height is larger than the first height threshold, to a portion where the road is depressed. It selects as measurement point information 556 of the measurement point to be located.
Further, the event position calculation unit 515 locates measurement point information 556 indicating an ellipsoidal height higher than the road surface height and the difference between the road surface height and the road surface height being larger than the second height threshold at a portion where the tree protrudes from the road. It is selected as measurement point information 556 of the measurement point.

  In step S324, the event position calculation unit 515 generates an event identifier 551 identifying an event corresponding to the selected measurement point.

  In step S 325, the event position calculation unit 515 stores the event identifier 551 and the event position information 552 in the memory 903. The event position information 552 is the same information as the measurement point information 556 of the selected measurement point.

*** Effect of the embodiment ***
The road monitoring device 510 uses the three-dimensional point cloud data 555 to calculate the event position. Therefore, the road monitoring device 510 can notify the observer 502 of the event position even if the event position is not calculated in the measuring vehicle 110.

Each embodiment is an illustration of a preferred embodiment, and is not intended to limit the technical scope of the present invention. Each embodiment may be implemented partially or in combination with other embodiments.
The procedure described using a flowchart or the like is an example of a method and program procedure.

  100 point group image generation apparatus, 101 positioning reinforcement data, 102 positioning data, 103 inertia measurement data, 104 traveling distance data, 105 azimuth distance data, 110 measurement vehicles, 111 positioning reinforcement signal receivers, 112 positioning signal receivers, 113 inertia Measuring device, 114 odometer, 115 laser scanner, 119 top plate, 121 position calculation unit, 122 attitude calculation unit, 123 three-dimensional point group generation unit, 200 measurement data processing unit, 201 measurement position data, 202 measurement position data, 203 tertiary Original point cloud data, 204 operation data, 220 position estimation unit, 229 estimated position data, 230 two-dimensional projection unit, 231 viewpoint position calculation unit, 232 two-dimensional projection surface calculation unit, 233 point group projection unit, 234 mark projection unit, 235 Projection Point Cloud Data, 236 Projection Mark Data 240 event position calculation unit 250 packet generation unit 300 display device 301 communication device 310 point group image 311 three-dimensional point group 312 measurement position mark 313 estimated position mark 320 interface image 321B backward button 321D lower Button, 321F forward button, 321L left button, 321R right button, 322X right and left widget, 322Y front and back widget, 322Z up and down widget, 323P elevation widget, 323Y azimuth widget, 329 cancel button, 401 own vehicle position, 411 reference viewpoint position, 431 Viewpoint position data, 433 two-dimensional projection plane data, 500 road surveillance system, 501 network, 502 observers, 503 surveillance screen, 510 road surveillance device, 511 reception unit, 512 Map image generator, 513 point cloud image generator, 514 interface image generator, 515 event position calculator, 519 display device, 520 map data, 531 event packet, 532 vehicle packet, 533 measurement point packet, 534 viewpoint packet, 541 Header, 542 vehicle identifier, 543 information identifier, 544 size, 545 time, 551 event identifier, 552 event location information, 553 vehicle location information, 554 viewpoint information, 555 three-dimensional point cloud data, 556 measurement point information, 559 gaze information, 560 map image, 561 vehicle diagram, 562 viewpoint diagram, 563 route diagram, 564 event diagram, 571 scale designation unit, 572 vehicle position display unit, 573 viewpoint display designation unit, 574 route erasure unit, 575 manual scroll designation unit, 576 vehicle Center designation section, 577 Update designation unit, 901 processor, 902 auxiliary storage device, 903 memory, 904 communication device, 9041 receiver, 9042 transmitter, 905 input interface, 906 output interface, 907 input device, 908 output device, 910 signal line, 911 cable, 912 cable .

Claims (16)

A packet transmitted from a communication device mounted on a vehicle traveling on a road, the packet including event position information indicating a position at which the event is occurring for the event occurring on the road or around the road A measurement point packet which is a packet transmitted from the communication device of the vehicle which receives an event packet and which includes measurement point information indicating the position of the road and a measurement point located around the road a receiver for receiving,
A map image which is an image displayed on a display device and which is an image of a map in which an event diagram representing the event is added to a location corresponding to the position indicated by the event position information included in the event packet and a map image generating unit that,
Generating a point group image which is an image displayed together with the map image and in which a measurement point is drawn at a position corresponding to a position indicated by the measurement point information included in the measurement point packet; A road monitoring apparatus comprising: a point cloud image generation unit that adds an event mark representing the event to a location corresponding to the position indicated by the event location information among the locations in the inside . The road monitoring device according to claim 1 , wherein the event mark is a diagram in which a solid having side portions in which an event identifier identifying the event is described at a plurality of places is projected onto the point cloud image. 3. The road monitoring device according to claim 2 , wherein the event mark is a diagram in which a solid including a triangular pyramid having three sides on which the event identifier is written is projected onto the point cloud image. The event mark is a diagram in which a double triangular pyramid obtained by connecting two triangular pyramids up and down is projected on the point group image,
The triangular pyramid above the event mark has three sides on which the event identifier is written,
4. The road monitoring device according to claim 3 , wherein the triangular pyramid below the event mark has a top pointing to a point corresponding to the position where the event is occurring. A packet transmitted from a communication device mounted on a vehicle traveling on a road, the packet including event position information indicating a position at which the event is occurring for the event occurring on the road or around the road A measurement point packet which is a packet transmitted from the communication device of the vehicle which receives an event packet and which includes measurement point information indicating the position of the road and a measurement point located around the road a receiver for receiving,
A map image which is an image displayed on a display device and which is an image of a map in which an event diagram representing the event is added to a location corresponding to the position indicated by the event position information included in the event packet and a map image generating unit that,
Point group image generation unit that generates a point group image that is an image displayed together with the map image and in which a measurement point is drawn at a location corresponding to the position indicated by the measurement point information included in the measurement point packet Equipped with
The vehicle is equipped with a display device for displaying a point cloud image on which measurement points located in a range viewed in a specified gaze direction from a designated viewpoint are drawn.
The receiving unit receives a viewpoint packet, which is a packet transmitted from the communication device of the vehicle, and is a packet including viewpoint information indicating the position of the viewpoint and gaze information indicating the gaze direction.
The point group image generation unit displays an image corresponding to a point group image displayed on the display device of the vehicle together with the map image, based on the viewpoint information and the line-of-sight information included in the viewpoint packet. Generate as a point cloud image
The road monitor according to claim 1, wherein the map image generation unit adds a viewpoint view representing the viewpoint to a location in the map corresponding to a position indicated by the viewpoint information . The map image generation unit adds the viewpoint view when the scale designated as the scale of the map is larger than the scale threshold for the viewpoint, but the specified scale is smaller than the scale threshold for the viewpoint The road monitoring device according to claim 5 , wherein the perspective view is not added to the road surface. 7. The road monitoring device according to claim 6, wherein the point cloud image generation unit adds an event mark representing the event to a location in the point cloud image corresponding to the position indicated by the event location information. The road monitoring device according to claim 7, wherein the event mark is a diagram in which a solid having side portions in which an event identifier identifying the event is described at a plurality of points is projected on the point cloud image. 9. The road monitoring device according to claim 8, wherein the event mark is a diagram in which a solid including a triangular pyramid having three sides on which the event identifier is written is projected onto the point cloud image. The event mark is a diagram in which a double triangular pyramid obtained by connecting two triangular pyramids up and down is projected on the point group image,
The triangular pyramid above the event mark has three sides on which the event identifier is written,
10. The road monitoring device according to claim 9, wherein the triangular pyramid below the event mark has a top pointing to a point corresponding to the position where the event occurs. The receiving unit receives a vehicle packet that is a packet transmitted from the communication device of the vehicle and is a packet including vehicle position information indicating the position of the vehicle.
The said map image generation part adds the vehicle figure showing the said vehicle to the location corresponding to the position which the said vehicle positional information included in the said vehicle packet shows among the locations in the said map. Road monitoring device according to any one of the above. The receiving unit receives a vehicle packet for each time including vehicle position information for each time,
The road monitoring device according to claim 11, wherein the map image generation unit adds a route diagram representing a route of the vehicle to the map based on vehicle position information for each time included in a vehicle packet for each time. The map image generation unit
When the scale designated as the scale of the map is larger than the scale threshold for the route, a mark representing the position where the vehicle has passed is added as the route diagram to the location corresponding to the position indicated by the vehicle position information for each time. ,
The road monitoring device according to claim 12, wherein if the designated scale is smaller than the scale threshold for a route, a line passing through a location corresponding to a position indicated by vehicle position information at each time is added as the route diagram. The receiving unit receives a vehicle packet for each vehicle including vehicle position information for each vehicle,
The road monitoring device according to claim 11, wherein the map image generation unit adds a vehicle diagram of each vehicle to the map based on vehicle position information of each vehicle included in a vehicle packet of each vehicle. A packet transmitted from a communication device mounted on a vehicle traveling on a road, the packet including event position information indicating a position at which the event is occurring for the event occurring on the road or around the road A measurement point packet which is a packet transmitted from the communication device of the vehicle which receives an event packet and which includes measurement point information indicating the position of the road and a measurement point located around the road an information receiving process of receiving,
A map image which is an image displayed on a display device and which is an image of a map in which an event diagram representing the event is added to a location corresponding to the position indicated by the event position information included in the event packet Map image generation processing ,
Generating a point group image which is an image displayed together with the map image and in which a measurement point is drawn at a position corresponding to a position indicated by the measurement point information included in the measurement point packet; A road monitoring program for causing a computer to execute a point cloud image generation process of adding an event mark representing the event to a location corresponding to a position indicated by the event location information among locations in the inside . A packet transmitted from a communication device mounted on a vehicle traveling on a road, the packet including event position information indicating a position at which the event is occurring for the event occurring on the road or around the road A measurement point packet which is a packet transmitted from the communication device of the vehicle which receives an event packet and which includes measurement point information indicating the position of the road and a measurement point located around the road an information receiving process of receiving,
A map image which is an image displayed on a display device and which is an image of a map in which an event diagram representing the event is added to a location corresponding to the position indicated by the event position information included in the event packet Map image generation processing ,
Point group image generation processing for generating a point group image that is an image displayed together with the map image and in which a measurement point is drawn at a location corresponding to the position indicated by the measurement point information included in the measurement point packet When
A road surveillance program to make a computer execute
The vehicle is equipped with a display device for displaying a point cloud image on which measurement points located in a range viewed in a specified gaze direction from a designated viewpoint are drawn.
The information receiving process receives a viewpoint packet which is a packet transmitted from the communication device of the vehicle, and is a packet including viewpoint information indicating the position of the viewpoint and gaze information indicating the gaze direction.
The point cloud image generation process displays an image corresponding to a point cloud image displayed on the display device of the vehicle together with the map image based on the viewpoint information and the line-of-sight information included in the viewpoint packet. Generate as a point cloud image
The road monitoring program according to claim 1, wherein the map image generation processing adds a viewpoint view representing the viewpoint to a location in the map corresponding to a position indicated by the viewpoint information .

Images