JP2019117053A - Display control device - Google Patents

Abstract

To provide a display control device capable of enhancing entertainability using point cloud information.SOLUTION: A display control device 1 has a LiDAR 2 configured to acquire point cloud information comprising a plurality of points representing objects around a vehicle, and a display control unit 3 configured to display a point cloud information-based image in one of a plurality of display modes with different display styles on a display unit 4.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control apparatus that displays an image on a display unit.

  Conventionally, a laser radar is used to obtain positions in a three-dimensional space with respect to a plurality of points on an object present in a detection area such as the front of a vehicle (that is, an area to be an object for detecting an object). , The technology which recognizes the object is proposed. For example, according to Patent Document 1, cluster points are obtained by clustering the points at which the position is acquired, and a cluster point group is acquired, and an object corresponding to the cluster point group is identified based on temporal variation of the measurement amount of the cluster point group It is supposed to be done.

Unexamined-Japanese-Patent No. 2017-151043

  As described above, laser radar (hereinafter also referred to as LiDAR (Light Detection And Ranging)) is used to recognize an object present in the vicinity of a vehicle or the like as a moving body. However, information acquired by laser radar (so-called point cloud information) is not used except for recognition of such an object.

  On the other hand, in vehicles equipped with an automatic driving function in the future, it is expected to improve entertainment in the traveling time during automatic driving. However, the idea of utilizing the information acquired by the laser radar for the improvement of entertainment has not existed conventionally.

  One of the problems to be solved by the present invention is to improve entertainment by using so-called point cloud information.

  In order to solve the above problems, the invention according to claim 1 is a display control device for displaying an image on a display unit arranged on a moving body, and the features around the moving body are represented by a plurality of points. Providing a point cloud information acquisition unit for obtaining the point cloud information represented, and a display control unit for causing the display unit to display an image based on the point cloud information in any of a plurality of display modes having different display modes. It is characterized.

  The invention according to claim 6 is a display control method to be executed by a display control apparatus for controlling display of a display unit arranged on a moving body, wherein a feature around the moving body is represented by a plurality of points. Characterized by including a point cloud information acquiring step of acquiring the point cloud information, and a display control step of causing the display unit to display an image based on the point cloud information in any of a plurality of display modes having different display modes. There is.

  The invention according to a seventh aspect is characterized in that the display control method according to the sixth aspect is executed by a computer.

FIG. 1 is a block diagram of a display control apparatus according to an embodiment of the present invention. It is a flowchart of the display operation of the display control apparatus shown by FIG. It is an example of the operation image in the display control apparatus shown by FIG. It is an example of the display mode of a 1st mode and the display mode of a 2nd mode. It is an example of the other display mode of 2nd mode. It is an example of the display mode which mixed the display mode of 1st mode and 2nd mode. It is a modification of the display control apparatus shown by FIG.

  Hereinafter, a display control apparatus according to an embodiment of the present invention will be described. In the display control device according to an embodiment of the present invention, the point cloud information acquisition unit acquires point cloud information in which the feature around the moving object is represented by a plurality of points, and the display control unit is an image based on the point cloud information Is displayed on the display unit in any of a plurality of display modes having different display modes. By doing this, it is possible to display in a plurality of display modes based on the point cloud information. Therefore, the entertainment property can be improved by using the point cloud information.

  In addition, the display control unit may switch between a plurality of display modes and cause the display unit to display. By doing this, it is possible to display an image in which the display mode is changed by switching the plurality of display modes as appropriate.

  In addition, the display control unit may display a plurality of display modes in an overlapping manner (or in combination) on the display unit. By doing this, it is possible to display such that the display modes of a plurality of display modes are superimposed, and it is possible to increase the variation of the display. Such display includes, for example, gradual transition from one display mode to another display mode.

  In addition, the display control unit may include a setting unit that sets music in each of a plurality of display modes, and the display control unit may change the display mode of the image displayed in one display mode according to the music to be reproduced. . By doing this, it is possible to display an image in a display manner that matches the image of the music.

  In addition, one display mode of the plurality of display modes may be a display mode in which the display mode in another display mode is deformed. By doing this, it is possible to display in a display mode deformed based on the display modes in the other display modes.

  Further, in the display control method according to the embodiment of the present invention, point cloud information in which a feature around the moving object is represented by a plurality of points is obtained in the point cloud information obtaining step, and the point group information is obtained in the display control step. There are a plurality of display modes for displaying different images based on each other. By doing this, it is possible to display in a plurality of display modes based on the point cloud information. Therefore, the entertainment property can be improved by using the point cloud information.

  In addition, a display control program that causes a computer to execute the display control method described above may be used. By doing this, it is possible to display in a plurality of display modes based on point cloud information using a computer. Therefore, the entertainment property can be improved by using the point cloud information.

  A display control apparatus according to a first embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 1, the display control device 1 includes a rider 2, a display control unit 3, a display unit 4, a music acquisition unit 5, a music analysis unit 6, and an operation unit 7. . Moreover, the display control apparatus 1 shown in FIG. 1 shall be mounted in vehicles, such as a motor vehicle, as an example of a mobile body.

  The lidar 2 as a point cloud information acquisition unit is a sensor for recognizing an object present around the vehicle, and is also described as LiDAR (Light Detection And Ranging). The lidar 2 irradiates light such as laser light and discretely measures the distance to an object present in the outside by reflected light of the light, and recognizes the position and shape of the object as a three-dimensional point group Known sensors. Therefore, the point cloud acquired by the lidar 2 is information (point cloud information) in which a feature including a structure such as a building or a road existing around the vehicle is three-dimensionally represented by a plurality of points. In addition to the landmarks on the road on which the vehicle is traveling and buildings and road features such as road trees, the rider 2 recognizes people and other vehicles existing around the vehicle. May be

  The display control unit 3 is a coordinate for displaying the point cloud information acquired by the lidar 2 on the display unit 4 that causes the point cloud information to be displayed from the coordinate system (vehicle coordinate system) in a state where the lidar 2 acquires the point cloud information. Convert to a system (display coordinate system). The vehicle coordinate system is a coordinate system based on the vehicle (lider 2). The display coordinate system is a two-dimensional coordinate system defined on the display unit 4. The display control unit 3 converts the three-dimensional vehicle coordinate system in the point group information into a two-dimensional display coordinate system. For this coordinate conversion, a known coordinate conversion method such as conversion from a so-called view coordinate system to a screen coordinate system can be used.

  Further, the display control unit 3 has two modes of a first mode and a second mode as a display mode for displaying an image based on point cloud information on the display unit 4. The point cloud information displayed on the display unit 4 is subjected to processing such as effects based on the result of analysis by the music analysis unit 6 and output to the display unit 4 as an image based on the point cloud information. That is, the display control unit 3 has a plurality of display modes as display modes for causing the display unit 4 to display an image based on point cloud information.

  The display unit 4 displays an image based on the point cloud information processed by the display control unit 3. The display unit 4 can be configured, for example, by a monitor screen installed on a center console of a vehicle or the like, a HUD (Head-Up Display), or the like. Further, the display unit 4 is not limited to one installed in the vehicle, and may display an image on a portable terminal or the like owned by a passenger, for example. In addition, the display unit 4 may be a glass portion of a vehicle such as a windshield, a door glass, or a sun roof glass of a vehicle provided with the display control device 1. As described above, the display unit 4 is not limited to the display device fixed in the vehicle or a portion that can be displayed by projection or the like, and may be a terminal or the like brought in the vehicle, that is, as long as it is disposed in the vehicle Good.

  The music acquisition unit 5 acquires, for example, music data reproduced in a car by a car audio or the like, or recorded in a car audio or the like. The music acquisition unit 5 acquires music data not limited to car audio, but may be a portable music reproduction device connected by short distance wireless communication or the like, a smartphone having a music reproduction function, or the like. Alternatively, it may be music data distributed from an external network.

  The music analysis unit 6 as the music information acquisition unit analyzes the feature amount of the music data acquired by the music acquisition unit 5. The feature amount includes tempo (BPM: Beats Per Minute), rhythm (beat, beat) and the like. The analysis of these feature quantities may be performed by a known method. In addition, information of a genre included in the music data itself may be acquired. Therefore, this feature amount and genre information become music information related to music.

  The operation unit 7 is configured of, for example, a touch panel or a push button. The operation unit 7 can switch between the first mode and the second mode described above.

  Next, the operation (display control method) in the display control apparatus 1 having the above-described configuration will be described with reference to the flowchart in FIG. The flowchart shown in FIG. 2 is executed by the display control unit 3. Therefore, by configuring the display control unit 3 with a microcomputer or the like having a CPU or the like, it is possible to provide a display control program that executes the display control method.

  First, the display control unit 3 acquires point cloud information detected by the rider 2 (step S11), and acquires the analysis result (tempo, rhythm, genre, etc.) analyzed by the music analysis unit 6 (step S12).

  Next, the display mode selected by the operation unit 7 or the like is determined (step S13). Here, an example of the operation unit 7 is shown in FIG. FIG. 3 shows, for example, an operation image 40 displayed on the display unit of the center console. Moreover, this display part shall have a touch panel. Alternatively, when the display unit 4 is configured as a display unit of the center console, the display area of the display unit 4 may be divided into a display area of an image for operation and a display area of an image based on point cloud information.

  In FIG. 3, the image 40 includes a music display unit 41, a slide unit 42, and a button 44. In the music display section 41, titles and jacket images of two musics (may be a collection of a plurality of music such as an album and a playlist) may be arranged side by side. In the example of FIG. 3, a title "AAAAA" and its jacket image J1 are displayed on the left, and a title "BBBBB" and its jacket image J2 are displayed on the right.

  The slide portion 42 has a slider 43, and the slider 43 can slide in the lateral direction in the slide portion 42 by dragging with a finger or the like. The button 44 includes a button 44L and a button 44R, and the button 44L slides the slider 43 in the left direction. The button 44R slides the slider 43 in the right direction. That is, the slider 43 can be slid directly by the drag operation, and can also be slid by the operation of the button 44.

  In FIG. 3, the first mode is used when the slider 43 is slid to the left end, and the second mode is used when slid to the right end, and “MIX” is used when the slider 43 is stopped between the left end and the right end. Therefore, when the slider 43 is slid to the left end in step S13, it is determined that the mode is the first mode, and the point cloud information is processed into the display mode of the first mode (step S14). When the slider 43 is slid to the right end in step S13, it is determined that the second mode is set, and the point cloud information is processed into the display mode of the second mode (step S15).

  Further, in FIG. 3, the music played at the position of the slider corresponds to the mode of the display mode. That is, music is set in each of the first mode and the second mode. For example, when the slider 43 is slid to the left end, the title "AAAAA" is reproduced in the display mode of the first mode, and when the slider 43 is slid to the right end, the title "BBBBB" is displayed in the second mode Played with. In addition, when the slider 43 is stopped between the left end and the right end, "MIX" is set, and both music pieces are mixed and reproduced, and two display modes are also superimposed (combined).

  Therefore, when the slider 43 is slid to the left end in step S13, it is determined that the mode is the first mode, and the point cloud information is processed into the display mode of the first mode (step S14). When the slider 43 is slid to the right end in step S13, it is determined that the second mode is set, and the point cloud information is processed into the display mode of the second mode (step S15).

  Here, an example of the display mode of the first mode and the display mode of the second mode is shown in FIG. The upper part of FIG. 4 is an example of the display mode of the first mode. The first mode of the present embodiment is a display mode in which the point group is displayed as it is based on the point group information acquired by the rider 2. That is, it is an image in which each feature is reproduced in a point cloud. However, the points may be colored according to each feature or distance.

  The lower part of FIG. 4 is an example of the display mode of the second mode. The second mode of the present embodiment is an image that is a display mode in which deformation is performed, such as, for example, emphasizing a contour portion of a feature based on point cloud information acquired by the rider 2.

  In the present embodiment, the display mode has two modes, but may have three or more modes. Moreover, in the present embodiment, in the display control unit 3, music is set in each of the first mode and the second mode, and displayed on the display unit 4 in one display mode according to the music being reproduced. Although the display mode of the image is changed according to the content described later, the display mode of each mode may be selected from a plurality of display modes according to the music. Alternatively, it may be selected according to the genre of the music.

  FIG. 5 shows another example of the display mode of the second mode. Specifically, using the music information of the music set in each of the first mode and the second mode analyzed by the music analysis unit 6, an example of the change in the display mode of the image displayed in the second mode is described. Show. In FIG. 5, for example, a circular effect E1 on the wall of a building such as a building, an effect E2 in which a graphic representing color or light is moved in the traveling direction of a moving object along a curb or white line of a road An effect E3 is displayed that displays as it flows by changing color, light, etc. along the height direction of. These effects E1 to E3 are circular, color, blinking speed of light or the like, movement speed of a figure expressing color or light, etc. by the feature amount of music (tempo (BPM), rhythm (beat, beat), genre, etc.) The flow rate of color or light changes. Moreover, the circular effect E4 displayed on the wall surface of a building such as a building as in the effect E1 changes the size of the circle depending on the rhythm of the music. That is, the timing of the effect applied to the image changes based on the feature amount of the music. The lower part of FIG. 4 and FIG. 5 are, for example, images as if projection mapping was applied to the entire town in which the vehicle is traveling.

  In addition, when the slider 43 is stopped between the left end and the right end in step S13, the image of the display mode of the first mode and the image of the display mode of the second mode are superimposed as “MIX” (combined ) Make an image (step S16). An example of the display mode in the case of MIX is shown in FIG. The ratio of overlapping (combining) the two display modes may be determined by the position of the slider 43. For example, when the position of the slider 43 is close to the left end, the ratio of the display mode in the first mode is increased. This "MIX" display also includes, for example, the case where the display changes when the slider 43 is moved from the display mode in the first mode to gradually shift to the display mode in the second mode.

  That is, when the slider 43 is positioned from the left end to the right end, or from the right end to the left end, the display mode of one display mode is selected from the plurality of display modes, and the image is displayed on the display unit 4. When the slider 43 is positioned between the left end and the right end, the overlapping (combination) ratio of the display modes in the plurality of display modes is changed and displayed on the display unit 4. That is, when the slider 43 is at a position closer to the left end from the center of the slide portion 42, an image in which the image of the display mode of the first mode and the image of the display mode of the second mode are superimposed is It will be displayed on the display part 4 in the state to which the ratio of the image of a display mode was higher than the ratio of the image of the display mode of 2nd mode.

  Then, in step S17, the image generated in any one of the display modes in steps S14 to S16 is displayed on the display unit 4.

  According to the present embodiment, the display control device 1 acquires point cloud information in which the lidar 2 represents features around the vehicle by a plurality of points, and the display control unit 3 displays two different display modes. The display unit 4 displays an image based on the point cloud information. By doing this, an image based on point cloud information can be displayed in two display modes. Therefore, the entertainment property can be improved by using the point cloud information.

  For example, an image as shown in FIGS. 4 to 6 can be generated by processing based on an image taken by a camera, but an image taken by a camera does not include distance information. It is necessary to estimate the distance by image processing or the like, which increases the processing load as a display control device. On the other hand, since point cloud information has distance information at each point, it is possible to obtain correct depth information without the need for analysis such as image processing. Therefore, the processing load is lower than that of the camera image, and it is possible to express finely. Therefore, it is possible to display an image with improved entertainment properties with a low processing load.

  Further, the display control unit 3 switches the image according to the two display modes and causes the display unit 4 to display the image. By doing this, it is possible to display an image in which the display mode is changed by switching between the two display modes as appropriate.

  Further, the display control unit 3 changes the overlapping (combination) ratio of the images in the two display modes and causes the display unit 4 to display the change. By doing this, it is possible to display such that the display modes of the two display modes are superimposed (combined), and it is possible to increase the variation of the display. In the present embodiment, two display modes (first mode, second mode) and two display mode images are superimposed (combined), but the present invention is not limited to this. The three display modes may be switched by setting the superposition (combination) of the images of the two display modes as the third mode.

  In addition, music is set in each of the two display modes, and the display control unit 3 changes the display mode of the image displayed on the display unit 4 in one display mode according to the music to be reproduced. . By doing this, it is possible to display an image in a display manner that matches the image of the music.

  In the second mode of the two display modes, the display mode of the first mode is displayed in a deformed display mode. By doing this, it is possible to display in the display mode deformed based on the display mode of the first mode.

  The music information analyzed or acquired by the music analysis unit 6 includes the feature amount of the music and genre information. By doing this, it is possible to change the display mode based on the feature amount of music such as tempo and rhythm, and the genre of music.

  Further, the display control unit 3 changes the timing of the effect applied to the image based on the feature amount of the music. By doing this, for example, effects can be applied or changed in accordance with the tempo or rhythm of the music.

  Note that, when the music data includes a jacket image, the music analysis unit 6 may obtain the jacket image and change the display mode based on the jacket image. For example, the color tone may be determined based on the jacket image of the music. In this case, when the jacket of the music being reproduced uses many red colors, the display mode can be set to a color centered on the red system. Alternatively, a jacket image may be included in the display mode. For example, instead of the circular image of the effects E1 and E4 of FIG. 5, a jacket image may be used. Some characters, shapes, logos, etc. of the jacket image may be extracted and displayed. By doing this, it is possible to visually sense the world view of the music being played.

  Further, as the feature amount of music, the effect may be changed according to the structure of music such as A melody, B melody, and rust, as well as tempo and rhythm. For example, it may be directed that a flashy effect is applied to the rising portion of the rust.

  In the configuration shown in FIG. 1, the point cloud information acquired by the lidar 2 is processed in real time by the display control unit 3. However, as shown in FIG. 7, the point cloud information is stored in advance in the storage unit 9. The display control unit 3A may read point cloud information corresponding to the current position estimated by the current position estimation unit 8 from the storage unit 9 and perform predetermined processing. That is, in the case of FIG. 7, the display control device 3A functions as a point cloud information acquisition unit.

  The current position estimation unit 8 estimates the current position of the vehicle on which the display control device 1A is mounted. The current position estimation unit 8 is formed of, for example, a GPS (Global Positioning System) receiver, and may estimate position information of latitude and longitude indicating the current position from radio waves received from GPS satellites.

  In FIG. 7, the display control device 1A includes the storage unit 9. However, since such 3D data has a large amount of data, it is stored in an external server or the like and includes a communication device for communicating with the external server. The control unit 3A may acquire point cloud information via the communication device.

  In addition, the point cloud information acquired by the rider 2 includes information indicating an object present around the vehicle, for example, information indicating a pedestrian to be noted by the driver who is the user, and other vehicles. Therefore, the driver can monitor an object or the like to be watched while enjoying the image by displaying the image having the entertainment property using the point cloud information. Along with this, it is possible to improve the sense of security the driver has.

  Further, the present invention is not limited to the above embodiment. That is, those skilled in the art can carry out various modifications without departing from the gist of the present invention in accordance with conventionally known findings. As long as the configuration of the display control apparatus of the present invention is provided even by such a modification, it is of course included in the scope of the present invention.

1, 1A Display control device 2 lidar (point cloud information acquisition unit)
3, 3A Display processing unit (point cloud information acquisition unit, setting unit)
4 display unit 5 music acquisition unit 6 music analysis unit (music information acquisition unit)

Claims (7)

A display control apparatus for displaying an image on a display unit disposed on a mobile body, the display control apparatus comprising:
A point cloud information acquisition unit that acquires point cloud information in which features around the moving body are represented by a plurality of points;
A display control unit that causes the display unit to display an image based on the point cloud information in any of a plurality of display modes having different display modes;
A display control apparatus comprising:   The display control device according to claim 1, wherein the display control unit switches the plurality of display modes and causes the display unit to display.   The display control device according to claim 1, wherein the display control unit causes the display unit to display the plurality of display modes in an overlapping manner. A setting unit configured to set a music in each of the plurality of display modes;
The display according to any one of claims 1 to 3, wherein the display control unit changes the display mode of the image displayed in one display mode according to the music to be reproduced. Control device.   The display control apparatus according to any one of claims 1 to 4, wherein one display mode in the plurality of display modes is a display mode in which a display mode in another display mode is deformed. A display control method executed by a display control device that controls display of a display unit arranged on a mobile object, comprising:
A point cloud information acquiring step of acquiring point cloud information in which features around the moving body are represented by a plurality of points;
A display control step of causing the display unit to display an image based on the point cloud information in any of a plurality of display modes having different display modes;
A display control method comprising:   A display control program causing a computer to execute the display control method according to claim 6.

Images