JP2019117214A - Object data structure - Google Patents

Abstract

To provide an object data structure which allows for enhancing entertainability using point cloud information.SOLUTION: Object data 5a contains display styles 5a4 that are used when displaying an image according to point cloud information comprising a plurality of points representing objects around a vehicle, each display style corresponding to an object included in the image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a feature data structure indicating features.

  Conventionally, a laser radar is used to obtain positions in a three-dimensional space with respect to a plurality of points on an object present in a detection area such as the front of a vehicle (that is, an area to be an object for detecting an object). , The technology which recognizes the object is proposed. For example, according to Patent Document 1, cluster points are obtained by clustering the points at which the position is acquired, and a cluster point group is acquired, and an object corresponding to the cluster point group is identified based on temporal variation of the measurement amount of the cluster point group It is supposed to be done.

Unexamined-Japanese-Patent No. 2017-151043

  As described above, laser radar (hereinafter also referred to as LiDAR (Light Detection And Ranging)) is used to recognize an object present in the vicinity of a vehicle or the like as a moving body. However, information acquired by laser radar (so-called point cloud information) is not used except for recognition of such an object.

  On the other hand, in vehicles equipped with an automatic driving function in the future, it is expected to improve entertainment in the traveling time during automatic driving. However, the idea of utilizing the information acquired by the laser radar for the improvement of entertainment has not existed conventionally.

  One of the problems to be solved by the present invention is to improve entertainment by using so-called point cloud information.

  In order to solve the above problems, the invention according to claim 1 includes feature data including information used when displaying an image based on point cloud information in which a feature around a moving object is represented by a plurality of points. The structure is characterized in that the information used when displaying the image includes information indicating a display mode of a portion corresponding to the feature included in the image.

  The invention according to claim 3 is a storage device for storing feature data including information used when displaying an image based on point cloud information in which a feature around the moving body is represented by a plurality of points, The feature data includes, as information used when displaying the image, information indicating a display mode of a portion corresponding to the feature included in the image.

FIG. 1 is a block diagram of a display control apparatus according to a first embodiment of the present invention. It is the table which showed the example of the feature data structure which the display control shown in Drawing 1 memorizes. It is a flowchart of the display operation of the display control apparatus shown by FIG. It is an example of the display mode of the display control apparatus shown by FIG. It is a schematic block diagram of the system which consists of the display control apparatus concerning 2nd Example of this invention, and a memory | storage device. It is a functional block diagram of the server apparatus shown by FIG. It is a block block diagram of the display control apparatus shown by FIG. It is a block block diagram of the display control apparatus concerning the 3rd Example of this invention. It is an example of a setting of the color for every distance in the display control apparatus shown by FIG. It is an example of a setting of the color for every height in the display control apparatus shown by FIG. It is a flowchart of the display operation of the display control apparatus shown by FIG. It is a block block diagram of the modification of this invention.

  Hereinafter, a feature data structure according to an embodiment of the present invention will be described. A feature data structure according to an embodiment of the present invention is a feature data structure including information used when displaying an image based on point cloud information in which a feature around a moving object is represented by a plurality of points. The information used when displaying the image includes information indicating the display mode of the portion corresponding to the feature included in the image. In this way, by referring to the feature data structure, it is possible to easily determine the display mode of the feature when displaying the image based on the point cloud information. Therefore, the display mode can be changed according to the feature included in the point cloud information, and the entertainment property can be improved using the point cloud information.

  Also, the feature data structure may further include feature attribute information. By doing this, the display mode can be determined based on the attribute of the feature.

  In addition, the storage device according to the embodiment of the present invention stores feature data including information used when displaying an image based on point cloud information in which a feature around the moving object is represented by a plurality of points. It is a storage device, and the feature data includes, as information used when displaying an image, information indicating a display mode of a portion corresponding to the feature included in the image. By doing this, it is possible to easily determine the display mode of the feature when displaying the image based on the point cloud information by referring to the feature data stored in the storage device. Therefore, the display mode can be changed according to the feature included in the point cloud information, and the entertainment property can be improved using the point cloud information.

  Moreover, you may provide the output part which outputs feature data to the terminal device provided in the mobile body. By doing so, feature data can be distributed to a mobile object such as an autonomous driving vehicle, for example. Therefore, there is no need for the autonomous driving vehicle etc. to have all the feature data, and the capacity of the storage device possessed by the autonomous driving vehicle etc can be reduced. In addition, feature data can be easily updated.

  A display control apparatus according to a first embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 1, the display control device 1 includes a lidar 2, a display control unit 3, a display unit 4, and a storage unit 5. Moreover, the display control apparatus 1 shown in FIG. 1 shall be mounted in vehicles, such as a motor vehicle, as an example of a mobile body. Further, the vehicle described in the present embodiment is an autonomous driving vehicle capable of always traveling autonomously or a vehicle capable of switching between automatic driving and manual driving (in the present embodiment, these are generically referred to as an autonomous driving vehicle ).

  The lidar 2 as a point cloud information acquisition unit is a sensor for recognizing an object present around the vehicle, and is also described as LiDAR (Light Detection And Ranging). The lidar 2 irradiates light such as laser light and discretely measures the distance to an object present in the outside by reflected light of the light, and recognizes the position and shape of the object as a three-dimensional point group Known sensors. Therefore, the point cloud acquired by the rider 2 is information (point cloud information) in which a feature including a structure such as a building or a road existing around the vehicle is three-dimensionally represented by a plurality of points. In addition to the landmarks on the road on which the vehicle is traveling and buildings and road features such as road trees, the rider 2 recognizes people and other vehicles existing around the vehicle. May be

  The display control unit 3 is a coordinate for displaying the point cloud information acquired by the lidar 2 on the display unit 4 that causes the point cloud information to be displayed from the coordinate system (vehicle coordinate system) in a state where the lidar 2 acquires the point cloud information. Convert to a system (display coordinate system). The vehicle coordinate system is a coordinate system based on the vehicle (lider 2). The display coordinate system is a two-dimensional coordinate system defined on the display unit 4. The display control unit 3 converts the three-dimensional vehicle coordinate system in the point group information into a two-dimensional display coordinate system. As this coordinate conversion, a known coordinate conversion method such as conversion from a so-called view coordinate system to a screen coordinate system can be used.

  In addition, the display control unit 3 determines the attribute of the feature included in the image in which the coordinate system has been converted, and performs processing so that the display mode corresponds to the determined attribute of the feature. Output to

  The display unit 4 displays an image based on the point cloud information processed by the display control unit 3. The display unit 4 can be configured by, for example, a monitor screen installed on a center console or the like in a vehicle, a HUD (Head-Up Display), or the like. Further, the display unit 4 is not limited to one installed in the vehicle, and may be displayed on a portable terminal or the like owned by a passenger, for example. In addition, the display unit 4 may be a glass portion of a vehicle such as a windshield, a door glass, or a sun roof glass of a vehicle provided with the display control device 1. As described above, the display unit 4 is not limited to the display device fixed in the vehicle or a portion that can be displayed by projection or the like, and may be a terminal or the like brought in the vehicle, that is, as long as it is disposed in the vehicle Good.

  The storage unit 5 as a storage device stores map data and feature data 5a including detailed information to the extent that the vehicle on which the display control device 1 is mounted can autonomously travel. An example of the feature data 5a (feature data structure) is shown in FIG.

  In the table shown in FIG. 2, a feature ID 5a1, a position 5a2, a name 5a3, and a display mode 5a4 are set for each feature. The feature ID 5a1 is a unique number set for each feature. The position 5a2 indicates the position of the feature in terms of latitude and longitude. The name 5a3 indicates a unique name or a general name of the feature such as a building name, a street tree, a telephone pole, a white line, a guardrail, a curb or the like. The display mode 5a4 shows the display mode for each feature. Here, the feature ID 5a1, the position 5a2, and the name 5a3 are attributes of the feature.

  Next, the operation (display control method) in the display control apparatus 1 having the above-described configuration will be described with reference to the flowchart in FIG. The flowchart shown in FIG. 3 is executed by the display control unit 3. Therefore, by configuring the display control unit 3 with a microcomputer or the like having a CPU or the like, it is possible to provide a display control program that executes the display control method.

  First, in step S11, it is determined whether or not the vehicle on which the display control device 1 is mounted is in automatic driving. If automatic driving is in progress, the process proceeds to step S12. finish. That is, this flowchart is assumed to be executed during automatic operation. However, when displaying on a portion such as a display unit provided on the center console or the like that does not interfere with operation, steps S12 and after may be performed even during manual operation.

  Next, the display control unit 3 acquires point cloud information detected by the rider 2 (step S12), and based on the feature data 5a stored in the storage unit 5, attributes of features included in the point cloud information Is determined (step S13). For the determination of the attribute, for example, the map data stored in the storage unit 5 is matched with the point cloud information acquired in step S12, and the feature for the point cloud information is specified. Then, with respect to the specified feature, by comparing the position or name of the feature included in the map data with the position or name of the feature included in the feature data 5a, the attribute of the feature is obtained. judge.

  Next, the display control unit 3 determines a display mode according to the attribute of each feature determined in step S13 based on the feature data 5a shown in FIG. 2 (step S14). Then, the display control unit 3 causes the display unit 4 to display the image based on the point cloud information on the feature in the image in the display mode determined in step S14 (step S15). That is, the display mode of the part corresponding to the said feature is changed among the images based on point cloud information based on the attribute defined beforehand with respect to the predetermined | prescribed feature contained in point cloud information. In addition, as the information used when displaying the image based on the point cloud information, the display mode of the portion corresponding to the feature included in the image is the display mode 5a4 set in the feature data 5a shown in FIG. It is information that indicates

  As is clear from the above description, step S12 is a point cloud information acquisition step, and steps S13 and S14 are display control steps.

  Here, an example of a display mode in the display control device 1 having the above-described configuration will be described with reference to FIG. The upper part of FIG. 4 is an example of point cloud information acquired by the rider 2 about the front of the vehicle. In the upper part of FIG. 4, a curb is provided along the lane as a feature T1. In addition, a roadside tree (tree) is provided along the lane as a feature T2. Furthermore, there are buildings as features T3 and T4 at the back of the traveling direction. This building shall be more than a predetermined height.

  The lower part of FIG. 4 is an example in which the display mode of the feature detected in the upper part of FIG. 4 is changed. In the lower part of FIG. 4, the portion of the feature T1 which is a curb in the upper part of FIG. 4 is such a display mode P1 that a predetermined color or a figure expressing a color or light moves along the traveling direction of the road (arrow AR). Become. In addition, a portion of the feature T2 which is a tree is a display mode P2 of a shape in which a tree is deformed by arranging figures such as a convex shape or a rectangle in the height direction. In addition, portions of the features T3 and T4 that are buildings become display modes P3 in which color or a figure representing color or light moves in the height direction at the edge portion.

  In addition, as what is provided along the road, not only a curb but a guardrail and a white line may be sufficient. Further, the feature data 5a of FIG. 2 may be given a name of "provided along a road" without distinction such as curb or white line. Moreover, as what is more than predetermined height, not only buildings, such as a building, but a tree and a telephone pole may be sufficient. Moreover, you may give the name "what is more than predetermined height", without making a distinction, such as a building and a tree. The predetermined height may be determined as appropriate.

  According to the present embodiment, the display control device 1 includes point cloud information when the lidar 2 acquires point cloud information and the display control unit 3 causes the display unit 4 to display an image based on the point cloud information. The display mode of the portion corresponding to the feature is changed in the image based on the point cloud information based on the attribute predetermined for the feature. By doing this, the display mode can be changed according to the feature included in the point cloud information. Therefore, the entertainment property can be improved by using the point cloud information.

  In addition, when the display control unit 3 determines that the attribute is provided along a road such as a curb or the like, the portion corresponding to the feature represents movement of color, color, light, or the like along the traveling direction. It is assumed that the displayed figure is displayed so as to move. By doing this, it is possible to provide a display mode in which a traveling state is imaged or a sense of depth is imaged in regard to what is provided along the road.

  In addition, when the display control unit 3 determines that the attribute has a predetermined height or more, such as a building, the color corresponding to the feature changes along the height direction of the feature or the color The display mode may be such that a graphic representing light or light is displayed to move. By doing this, it is possible to set a display mode that makes use of the height of a feature having a predetermined height or more such as a high-rise building.

  The feature data 5a also includes a display mode 5a4 of a portion corresponding to the feature included in the image when displaying an image based on point cloud information in which the feature around the vehicle is represented by a plurality of points. It is. In this way, by referring to the feature data 5a, it is possible to easily determine the display mode of the feature when displaying the image based on the point cloud information. Therefore, the display mode can be changed according to the feature included in the point cloud information, and the entertainment property can be improved using the point cloud information.

  The feature data 5a may further include the feature name 5a3. By doing this, the display mode can be determined based on the name of the feature.

  In the above-described embodiment, as attributes of features, those provided along roads (white lines, curbs, guardrails, etc.) or those having a predetermined height or more (buildings, street trees (trees), utility poles) However, the present invention is not limited thereto. Information indicating whether the feature is a static feature or a dynamic feature may be set as an attribute. As a static feature, a white line, a building, etc. are mentioned, for example. As a dynamic feature, a surrounding vehicle, a person, etc. are mentioned. In the identification of static features and dynamic features, the features matched when map data and point cloud information are matched are static features because they are on the map, and those that do not match are dynamic features It may be determined as

  Next, a display control apparatus according to a second embodiment of the present invention will be described with reference to FIGS. The same parts as those of the first embodiment described above are designated by the same reference numerals and the description thereof will be omitted.

  In the first embodiment, the feature data 5a is stored in advance in the storage unit 5 of the display control device 1. However, in the present embodiment, feature data is distributed from the server device 10. . FIG. 5 shows a schematic block diagram of this embodiment. That is, in the present embodiment, the server device 10 functions as a storage device.

  As shown in FIG. 5, the server device 10 as a storage device according to the present embodiment is a display control device 1A as a terminal device mounted on a vehicle C which is an automatic driving vehicle via a network N such as the Internet. It is possible to communicate with The functional configuration of the server device 10 is shown in FIG. The server device 10 includes a control unit 11, a communication unit 12, and a storage unit 13.

  The control unit 11 functions as a CPU (Central Processing Unit) of the server device 10 and controls the entire server device 10. Control unit 11 reads feature data of a necessary area from feature data 13a stored in storage unit 13 in response to a request from display control device 1A, and displays control device 1A via communication unit 12 Deliver to.

  The communication unit 12 functions as a network interface of the server device 10 and receives request information and the like output from the display control device 1A. The control unit 11 also transmits feature data read from the feature data 13a to the display control device 1A.

  The storage unit 13 functions as a storage device such as a hard disk of the server device 10, and stores feature data 13a. The server device 10 may have map data used for automatic driving as well as distribution of the feature data 13a, and may distribute the map data to the vehicle.

  FIG. 7 shows a block configuration of the display control device 1A according to this embodiment. The block configuration shown in FIG. 7 is different from FIG. 1 in that the communication unit 6 is added and the display control unit 3 is changed to the display control unit 3A.

  The communication unit 6 transmits, to the server device 10, request information and the like output by the display control unit 3A. Also, the feature data distributed from the server device 10 is received.

  In the display control unit 3A, processes such as coordinate conversion, attribute determination, and processing of a display mode are the same as in the first embodiment. In addition to the above, the display control unit 3A performs a request for feature data via the communication unit 6 and storing feature data received by the communication unit 6 in the storage unit 5. The feature data request may be made based on, for example, the current position acquired by a GPS (Global Positioning System) receiver (not shown) mounted on the vehicle C.

  According to the present embodiment, the server device 10 includes the communication control unit 12 that outputs feature data to the display control device 1A provided in the vehicle C. By doing this, feature data can be distributed to the vehicle C. Therefore, the vehicle C does not need to have feature data in advance, and the capacity of the storage device of the vehicle C can be reduced. In addition, the feature data 5a can be easily updated.

  Next, a display control apparatus according to a third embodiment of the present invention will be described with reference to FIGS. The same parts as those of the first embodiment described above are designated by the same reference numerals and the description thereof will be omitted.

  In this embodiment, the display mode is changed based on the information of the distance and height of the point group detected by the lidar 2 regardless of the attribute of the feature. FIG. 8 shows a block configuration of the display control device 1B according to the present embodiment. The block configuration shown in FIG. 8 differs from FIG. 1 in that the display control unit 3 is changed to a display control unit 3B.

  The display control unit 3B changes the display mode of the point cloud based on the distance information and the height information among the point cloud information acquired by the lidar 2. Specifically, the color or the like is changed according to the distance and the height of the point group acquired by the rider 2. The lidar 2 can measure the distance to the object as described above, but can also acquire the angle (horizontal angle, vertical angle) at which the laser light was irradiated at that time. Therefore, it is possible to determine the height (ground height) of the object (point group) from the distance and the angle.

  FIG. 9 shows an example of setting a color for each distance. In the case of FIG. 9, the range from 0 to 10 m from the vehicle is color A, the range from 10 to 30 m from the vehicle is color B, the range from 30 to 50 m from the vehicle is color C, 50 m from the vehicle The range is color D.

  FIG. 10 shows an example of setting the color for each height. In the case of FIG. 10, the height is in the range of 0 m to 10 m above ground color A, in the range of 10 m to 50 m below ground color B, in the range of 50 m to 100 m below ground C, and over 100 m above ground color D It has become.

  The display mode is not limited to changing the color, and the contrast may be changed. Also, the color and the contrast may be changed by combining the distance and the height.

  Also, the display mode is not limited to changing the color and contrast of the point group, and for example, after performing deformation as shown in the lower part of FIG. 4, the color and contrast may be changed according to the above distance and height. May be Furthermore, the type of effect may be changed according to distance or height, not color or contrast.

  FIG. 11 shows an operation (display control method) in the display control device 1B of the present embodiment. The flowchart shown in FIG. 11 is executed by the display control unit 3B. Therefore, by configuring the display control unit 3B with a microcomputer or the like having a CPU or the like, it is possible to provide a display control program that executes the display control method.

  In the flowchart of FIG. 11, steps S11, S12 and S15 are the same as the flowchart shown in FIG. In the flowchart of FIG. 11, step S13 shown in FIG. 3 is omitted, and step S14 is changed to step S14A.

  Step S14A determines the display mode of the point group based on the distance information and the height information among the point group information acquired by the rider 2 as described above. That is, step S12 is a point cloud information acquisition process, and step S14A is a display control process.

  According to this embodiment, the display control device 1B acquires the point cloud information including the distance information to the feature and the height information of the feature, and the display control unit 3B obtains the distance information and the height information. The display mode of the image based on the point cloud information is changed based on. By doing this, it is possible to change the display mode based on the information such as the distance included in the point cloud information. For example, the effect can be changed according to the distance. Therefore, the entertainment property can be improved by using the point cloud information.

  Further, the display control unit 3B may obtain angle information based on the distance information and the height information, and change the display mode of the image based on the point cloud information based on the angle information. By doing this, it is possible to change the display mode such as the effect based on the angle of the point cloud included in the point cloud information.

  The change in display mode may include a change in color or a change in contrast, and the display control unit may change the color or contrast of the image based on the point cloud information. By doing this, color and contrast can be changed and displayed according to the distance and height.

  In the above-described three embodiments, the point cloud information acquired by the lidar 2 is processed in real time by the display control unit 3 or the like, but as shown in FIG. 12, the point cloud information is stored as point cloud information 5b. The display control unit 3C may store in advance the point cloud information corresponding to the current position estimated by the current position estimation unit 7 from the storage unit 5 and perform predetermined processing. That is, in the case of FIG. 12, the display control unit 3C functions as a point cloud information acquisition unit. In the case of the configuration of FIG. 12, since the point cloud information is stored in advance, the dynamic features such as the surrounding vehicles are different from the actual situation, and for example, only the static features may be displayed.

  The current position estimation unit 7 estimates the current position of the vehicle on which the display control device 1C is mounted. The current position estimation unit 7 is formed of, for example, a GPS receiver, and may estimate position information of latitude and longitude representing the current position from radio waves received from GPS satellites.

  In FIG. 12, the display control device 1C includes the storage unit 5. However, since such 3D data has a large amount of data, it is stored in an external server or the like and includes a communication device for communicating with the external server. The control unit 3C may acquire point cloud information via the communication device.

  In addition, the point cloud information acquired by the rider 2 includes information indicating an object present around the vehicle, for example, information indicating a pedestrian to be noted by the driver who is the user, and other vehicles. Therefore, the driver can monitor an object or the like to be watched while enjoying the image by displaying the image having the entertainment property using the point cloud information. Along with this, it is possible to improve the sense of security the driver has.

  Further, the present invention is not limited to the above embodiment. That is, those skilled in the art can carry out various modifications without departing from the gist of the present invention in accordance with conventionally known findings. As long as the configuration of the display control apparatus of the present invention is provided even by such a modification, it is of course included in the scope of the present invention.

1, 1A, 1B, 1C Display control device 2 lidar (point cloud information acquisition unit)
3, 3A, 3B, 3C Display control unit 4 Display unit 5 Storage unit 5a Feature data (map data structure)
10 server device 13a feature data (map data structure)

Claims (4)

A feature data structure including information used when displaying an image based on point cloud information in which a feature around a moving object is represented by a plurality of points,
A feature data structure comprising information indicating a display mode of a portion corresponding to a feature included in the image as the information used when displaying the image.   The feature data structure according to claim 1, further comprising attribute information of the feature. A storage device for storing feature data including information used when displaying an image based on point cloud information in which a feature around a moving object is represented by a plurality of points,
The storage device, wherein the feature data includes information indicating a display mode of a portion corresponding to the feature included in the image as the information used when displaying the image.   The storage device according to claim 3, further comprising an output unit configured to output the feature data to a display control device provided in the moving body.

Images