JP2022106885A - Image generation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image generation device capable of improving entertainment by using so-called point cloud information.

SOLUTION: In an information processing device 1, a processing unit 12 acquires point cloud information from an own vehicle Ca and other vehicle Cb1, Cb2, or a road side device 5 and a traffic signal 6, and generates a three-dimensional map image around the own vehicle Ca based on the acquired point cloud information to display a three-dimensional map image on a display unit 14.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

Patent Law Article 30, Paragraph 2 Application Applicable Announced at the 45th Tokyo Motor Show 2017 held at the Tokyo International Exhibition Center on October 25, 2017

The present invention relates to an image generator including a generator that generates an external image of an object.

Conventionally, a laser radar is used to acquire positions in three-dimensional space with respect to a plurality of points on an object existing in a detection area such as the front of a vehicle (that is, a target area for detecting an object). , A technique for recognizing the object has been proposed. For example, in Patent Document 1, the points from which the positions have been acquired are clustered to acquire a cluster point cloud, and an object corresponding to the cluster point cloud is identified based on the temporal variation of the measured amount related to the cluster point cloud. I'm supposed to do it.

Japanese Unexamined Patent Publication No. 2017-151043

As described above, the laser radar (hereinafter, also referred to as LiDAR (Light Detection And Ranging)) is used for recognizing an object existing in the vicinity of a vehicle or the like as a moving body. However, the information acquired by the laser radar (so-called point cloud information) is not used for anything other than recognition of such an object.

On the other hand, in the future, vehicles equipped with an automatic driving function are expected to improve entertainment during travel time during automatic driving. However, the idea of using the information acquired by the laser radar to improve entertainment has not existed in the past.

As an example of the problem to be solved by the present invention, it is possible to improve the entertainment property by using the so-called point cloud information.

In order to solve the above problem, the invention according to claim 1 is an image generation device including a generation unit that generates an image displayed on a display unit of a moving body, and the image generation device is arranged. An acquisition unit that acquires point group information representing an object around each of the plurality of devices from a plurality of devices including an external device outside the moving body by a plurality of points, and an acquisition unit acquired by the acquisition unit. It is characterized by including a generation unit that generates a three-dimensional space image showing a three-dimensional space including at least the periphery of the moving body based on the point group information.

The invention according to claim 8 is an image generation method executed by an image generation device that generates an image displayed on a display unit of the moving body, and is outside the moving body in which the image generating device is arranged. In the acquisition step of acquiring the point group information in which the objects around each of the plurality of devices are represented by a plurality of points from the plurality of devices including the external device in the above, and the point group information acquired in the acquisition step. Based on this, it is characterized by including, at least, a generation step of generating a three-dimensional space image around the moving body showing a three-dimensional space including the periphery of the moving body.

The invention according to claim 9 is characterized in that the image generation method according to claim 8 is executed by a computer.

It is a schematic block diagram of the vehicle and the roadside machine equipped with the information processing apparatus which concerns on 1st Embodiment of this invention. It is a block block diagram of the information processing apparatus shown in FIG. It is a flowchart of the operation of the generation and display of the 3D model in the information processing apparatus shown in FIG. It is explanatory drawing of the situation which is the premise of the concrete example of the display of the three-dimensional model image in this embodiment. This is a specific example of displaying a three-dimensional model image at time t1 in FIG. This is a specific example of displaying a three-dimensional model image at time t2 in FIG. This is a specific example of displaying a three-dimensional model image at time t3 in FIG. It is a flowchart of the operation of the generation and display of the 3D model in the information processing apparatus which concerns on 2nd Embodiment of this invention. It is explanatory drawing of the situation which is the premise of the concrete example of the display of the three-dimensional map image in this embodiment. This is a specific example of displaying a three-dimensional map image at the angle A1 of FIG. This is a specific example of displaying a three-dimensional map image at the angle A2 of FIG. This is a specific example of displaying a three-dimensional map image at the angle A3 of FIG. This is a configuration example when a server device is used.

Hereinafter, an image generator according to an embodiment of the present invention will be described. In the image generation device according to the embodiment of the present invention, point group information in which an acquisition unit represents an object around the external device by a plurality of points from an external device outside the moving body in which the image generation device is arranged. Is acquired, and the generation unit generates an appearance image based on the point group information acquired by the acquisition unit. By doing so, it is possible to generate an image of the appearance of a vehicle, a feature, or the like based on the point cloud information acquired by another moving body or the like, which does not depend on the point cloud information acquired by itself. Such an image includes information that is originally invisible to the passengers of the moving body, and can be displayed and produced as if the direction of the moving body is being viewed from the outside. Therefore, the point cloud information can be used to improve the entertainment property.

Further, the surrounding object may include a moving body, and the generation unit may generate an appearance image including at least a moving body image showing the moving body. By doing so, it is possible to generate an image showing a state in which the moving body is currently traveling, which is normally invisible to the passengers of the moving body, as an external appearance image.

Further, the external device may be mounted on another moving body different from the moving body, and the acquisition unit may acquire the point cloud information from the other moving body. By doing so, it is possible to generate an appearance image based on the point cloud information acquired by another moving body traveling around the moving body.

Further, the external device may be mounted on the roadside machine installed on the traveling path of the moving body, and the acquisition unit may acquire the point cloud information from the roadside machine. By doing so, it is possible to generate an appearance image based on the point cloud information acquired by the roadside machine even when there are few other moving objects.

In addition, the generation unit may stop the acquisition of the point cloud information when the degree of generation of the appearance image of the object to be generated exceeds a predetermined value. By doing so, when the appearance image is completed to some extent, it is possible to stop the acquisition of the point cloud information and reduce the amount of communication with the external device and the processing load of the own device.

Further, an operation unit capable of adjusting the angle of the appearance image may be provided, and the display unit may display the appearance image by changing the angle according to the angle adjustment by the operation unit. By doing so, for example, it is possible to display the appearance image of the moving body so as to be visually recognized from various angles. Therefore, since the angle can be changed according to the preference of the passenger or the like, the display can be displayed without getting bored.

Further, the image generation method according to the embodiment of the present invention is a point in which an object around the external device is represented by a plurality of points from an external device outside the moving body in which the image generation device is arranged in the acquisition step. Group information is acquired, and an appearance image is generated based on the point group information acquired by the acquisition unit in the generation process. By doing so, it is possible to generate an image of the appearance of a vehicle, a feature, or the like based on the point cloud information acquired by another moving body or the like, which does not depend on the point cloud information acquired by itself. Such an image includes information that is originally invisible to the passengers of the moving body, and can be displayed and produced as if the direction of the moving body is being viewed from the outside. Therefore, the point cloud information can be used to improve the entertainment property.

Further, it may be an image generation program in which the above-mentioned image generation method is executed by a computer. By doing so, it is possible to generate an image of the appearance of a vehicle, a feature, etc. based on the point cloud information acquired by another moving body, etc., without relying on the point cloud information acquired by itself using a computer. Can be done. Such an image includes information that is originally invisible to the passengers of the moving body, and can be displayed and produced as if the direction of the moving body is being viewed from the outside. Therefore, the point cloud information can be used to improve the entertainment property.

The information processing device 1 as the image generation device according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 7. As shown in FIG. 1, the information processing device 1 is mounted on a vehicle C (Ca, Cb) as a moving body. In this embodiment, the vehicle C will be described as a vehicle that can always travel autonomously (automated driving vehicle) or a vehicle that can switch between automatic driving and manual driving. Further, in FIG. 1, the vehicle Ca is treated as a own vehicle (own vehicle) as necessary in the following description, and the vehicle Cb is treated as another vehicle (other vehicle).

Further, the rider 2 is mounted on the vehicle C. The lidar 2 is a sensor for recognizing an object existing around the lidar 2 mounted on the lidar 2, and is also referred to as LiDAR (Light Detection And Ranging). The rider 2 irradiates light such as laser light, measures the distance to an object existing in the outside world discretely by the reflected light of the light, and recognizes the position and shape of the object as a three-dimensional point group. It is a known sensor. Therefore, the point cloud acquired by the rider 2 is information (point cloud information) in which features including structures such as buildings and roads existing around the vehicle C move are represented three-dimensionally by a plurality of points. .. In addition to the signs on the road on which the vehicle is traveling and features such as buildings and roadside trees that exist along the road, the rider 2 includes pedestrians and other vehicles that exist in the vicinity of the vehicle. May be recognized.

In FIG. 1, a rider 2a that mainly detects an object in front of the vehicle C, a rider 2b that mainly detects an object behind the vehicle C, a rider 2c that mainly detects an object on the left side of the vehicle C, and mainly a vehicle C. A rider 2d that detects an object on the right side of is described. The mounting position and number of riders 2 are not limited to FIG. 1, and may be other arrangements and numbers.

Further, in this embodiment, the rider is installed as a roadside machine 5 in addition to the vehicle C. Examples of the roadside unit 5 installed include a confluence point in a traffic light, an expressway, or the like, various road signs, signs, and the like. In the case of FIG. 1, the roadside machine 5 is installed as a roadside machine 5a and a roadside machine 5b on the upper and lower lines, respectively.

FIG. 2 shows a functional configuration diagram of the information processing apparatus 1 according to the present embodiment. The information processing device 1 includes a communication unit 11, a processing unit 12, a storage unit 13, a display unit 14, an operation unit 15, and an I / F 16.

The communication unit 11 receives point cloud information and the like from the vehicle Cb, which is another vehicle, and the roadside unit 5. The communication unit 11 can receive the point cloud information by, for example, vehicle-to-vehicle communication or road-to-vehicle communication. Alternatively, the point cloud information may be received by existing wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark).

The processing unit 12 is composed of, for example, a CPU (Central Processing Unit), a microcontroller having a memory, and the like. The processing unit 12 receives the point group information based on the point group information received by the communication unit 11, the current position of the vehicle Ca detected by the GPS receiver 3, and the map information stored in the storage unit 13. A portion related to the vehicle Ca, which is the own vehicle, is extracted from the information, and a three-dimensional model image as an external image of the vehicle Ca is generated based on the extracted point group information. That is, the processing unit 12 functions as a generation unit that generates an appearance image of the object. Details of generating a three-dimensional model image of the vehicle Ca will be described later.

The storage unit 13 is a large-capacity storage device composed of a hard disk drive, an SSD (Solid State Drive), or the like. The storage unit 13 stores map data including detailed information to the extent that the vehicle C can autonomously travel. The map data is three-dimensional map data, and the features and the like included in the map data include absolute position information (latitude and longitude information).

The display unit 14 is composed of, for example, a display device installed on a center console, a HUD (Head-Up Display), or the like. The display unit 14 displays a three-dimensional model image or the like of the vehicle Ca generated by the processing unit 12. In the following description, it is assumed that the output unit outputs to the display unit 14, but the communication unit 11 functions as the output unit, and the communication unit 11 processes the terminal equipment or the like possessed by the passenger of the own vehicle Ca. The three-dimensional model image of the vehicle Ca generated by the unit 12 may be output wirelessly. In this case, the terminal device or the like possessed by the passenger becomes a display unit arranged in the moving body. Further, the display unit 14 may be a glass portion of the vehicle Ca, such as a windshield, a door glass, a sunroof glass, etc. of the vehicle Ca. In this way, the display unit 4 is not limited to a display device fixed in the vehicle or a part that can be displayed by projection or the like, or may be a terminal or the like brought into the vehicle, that is, if the vehicle has it. good.

The operation unit 15 is composed of, for example, a touch panel, a push button, or the like. The operation unit 15 can perform operations such as adjusting the angle of the three-dimensional model image of the vehicle Ca displayed on the display unit 14.

The I / F 16 is a connection interface (I / F) with a rider 2, a GPS (Global Positioning System) receiver 3, and other sensors 4 installed in the vehicle C. The I / F 16 converts signals, data, and the like output from the rider 2, GPS receiver 3, and the like into a data format and signal level to be output to the processing unit 12.

As is well known, the GPS receiver 3 detects the position information of latitude and longitude representing the current position by radio waves received from a plurality of GPS satellites.

Other sensors 4 include an inertial measurement unit (IMU) for recognizing the posture (orientation, etc.) of the vehicle and correcting the acquired data of other sensors, a gyro sensor, and the external situation of the vehicle C. Examples thereof include an in-vehicle camera that generates a colored image representing the above, an acceleration sensor that detects the acceleration of the vehicle C, a speed sensor that detects the speed of the vehicle C, and the like.

Next, the operation of generating and displaying the three-dimensional model (image generation method) in the information processing apparatus 1 having the above-described configuration will be described with reference to the flowchart of FIG. The flowchart of FIG. 3 is configured as, for example, a computer program executed by the processing unit 12.

First, in step S11, the processing unit 12 acquires point cloud information from the other vehicle Cb and the roadside machines 5a and 5b via the communication unit 11. That is, the processing unit 12 points a plurality of objects around the other vehicle Cb or the like (external device) from the other vehicle Cb or the like (external device) outside the own vehicle Ca (moving body) provided with the information processing device 1. It functions as an acquisition unit that acquires the point group information represented by. This point cloud information is, for example, point cloud information acquired by a rider possessed by an external device. The point cloud information is acquired, for example, by another vehicle Cb or roadside units 5a and 5b transmitting the point cloud information by broadcasting, and the communication unit 11 receiving and acquiring the point cloud information transmitted by the broadcast. The point cloud information transmitted by the other vehicle Cb or the like includes not only the point cloud information (distance, direction, etc.) but also the source identification information and the source position information.

Next, in step S12, the processing unit 12 acquires the position information of the own vehicle Ca from the GPS receiver 3 via the I / F 16.

Next, in step S13, the processing unit 12 generates a three-dimensional model image. In the generation of the three-dimensional model image, first, the point cloud related to the portion of the own vehicle Ca is extracted from the received point cloud information. From the position information of the transmission source included in the point cloud information acquired in step S11 and the position information of the own vehicle Ca acquired in step S12, the direction and distance from the transmission source to the own vehicle Ca can be known. Since the point cloud information originally includes information on the distance and direction at each point, it is sufficient to extract the point cloud that matches the direction and distance of the own vehicle Ca. Further, when there are a plurality of sources, this process is performed for the point cloud information of each source. Then, the extracted point cloud is modeled by a polygon or the like by a well-known method to generate a three-dimensional model image. When there are a plurality of sources, the point cloud may be combined at the time of compositing or modeling to generate a three-dimensional model image. Further, in the process, at least the point cloud related to the portion of the own vehicle Ca may be extracted, and for example, the point cloud related to the background including the own vehicle Ca and the features existing around it may be extracted.

Next, in step S14, the processing unit 12 outputs the three-dimensional model image generated in step S13 to the display unit 14 and displays it. That is, the processing unit 12 functions as an output unit that outputs a three-dimensional model image (appearance image) to the display unit 14 arranged in the own vehicle Ca (moving body).

Then, by executing the flowchart of FIG. 3 sequentially and repeatedly, the three-dimensional model image displayed on the display unit 14 is updated in substantially real time, and for example, an effect or display such that the own vehicle Ca runs in the town is performed. Can be done.

As is clear from the above description, step S11 functions as an acquisition step, step S13 functions as a generation step, and step S14 functions as an output step.

Next, a specific example of displaying the above-mentioned three-dimensional model image will be described with reference to FIGS. 4 to 7. FIG. 4 is a diagram illustrating a situation of a specific example. In FIG. 4, the own vehicle Ca is traveling in the left lane, and the other vehicle Cb is traveling in the right lane. Further, the detection range of the rider 2 of the other vehicle Cb is set to A. At this time, FIGS. 5 to 7 show three-dimensional model images of each time t1 to t3 when the own vehicle Ca overtakes the other vehicle Cb. The three-dimensional model images illustrated in FIGS. 5 to 7 show images as if the vehicle Ca was viewed from different angles (viewpoints).

At time t1, the own vehicle Ca is traveling to the left rear of the other vehicle Cb. Therefore, as shown in FIG. 5, since the point cloud from the diagonally right front direction of the own vehicle Ca is acquired as the point cloud information, the three-dimensional model image is obtained as if the own vehicle Ca was viewed from the diagonally right front.

At time t2, the own vehicle Ca is traveling on the left side when viewed from the other vehicle Cb. Therefore, as shown in FIG. 6, since the point cloud from the right direction of the own vehicle Ca is acquired as the point cloud information, the three-dimensional model image as if the own vehicle Ca is viewed from the right is obtained. At time t2, a three-dimensional model image may be generated with reference to the point cloud information acquired at time t1, which is the previous time.

At time t3, the own vehicle Ca is traveling to the left front when viewed from the other vehicle Cb. Therefore, as shown in FIG. 5, since the point cloud from the diagonally right front direction of the own vehicle Ca is acquired as the point cloud information, the three-dimensional model image as if the own vehicle Ca is viewed from the diagonally right rear direction is obtained. At time t3, the three-dimensional model image may be generated by referring to the point cloud information acquired at the previous times t1 and t2.

In this embodiment, as shown in FIGS. 5 to 7, the external image is a three-dimensional model image in which the own vehicle Ca is modeled from the point cloud information, but the point cloud is used as it is as the shape of the own vehicle Ca. It may be an image in which the image is displayed, or it may be a wire frame or the like. That is, the appearance image in this embodiment may be an image in which the appearance shape of the object can be identified.

Further, in FIGS. 5 to 7, the background portion other than the own vehicle Ca may be generated from a portion other than the portion extracted as the own vehicle Ca from the point cloud information acquired at each time, or is based on the acquired position information. The feature such as the corresponding building may be extracted from the map data stored in the storage unit 13 and displayed. Further, the background portion is not limited to a three-dimensional image modeled like the vehicle Ca, and may be an image in which a point cloud is displayed as it is, or may be a wire frame or the like.

Then, the user can change the angle of the displayed three-dimensional model image by the operation by the operation unit 15. For example, the display unit 14 switches and displays the three-dimensional model images shown in FIGS. 5 to 7 based on the information regarding the angle received from the user by the operation unit 15. That is, the display unit 14 is controlled to display the three-dimensional model image corresponding to the angle designated by the user by operating the operation unit 15. Therefore, since the angle of the displayed image can be changed according to the preference of the passenger of the own vehicle Ca, it is possible to display the image without getting bored. The angle of the image displayed by the display unit 14 may be determined by an operation other than the operation by the operation unit 15 described above. For example, when the vehicle Ca approaches an external device (for example, a roadside device) equipped with a rider (when the vehicle Ca enters within a predetermined range from the external device), the angle of the image displayed on the display unit 14 is The angle from the external device (image from the viewpoint of the external device) may be automatically switched.

According to this embodiment, the communication unit 11 acquires the point cloud information from the other vehicle Cb, the processing unit 12 generates a three-dimensional model image of the own vehicle Ca based on the acquired point cloud information, and the display unit 14 displays the point cloud information. Display a 3D model image. By doing so, it is possible to generate an image of the appearance of a vehicle, a feature, or the like based on the point cloud information acquired by the other vehicle Cb, which does not depend on the point cloud information acquired by the own vehicle Ca itself. Such an image includes information that is originally invisible to the passengers of the own vehicle Ca, and can be displayed and produced as if the direction of the own vehicle Ca is being viewed from the outside. For example, an image showing the state in which the own vehicle Ca is currently traveling can be generated as a three-dimensional model image and displayed inside the own vehicle Ca. Therefore, the point cloud information acquired by the rider can be used to improve the entertainment property. In addition, the point cloud information includes information indicating an object existing around the rider, and includes information indicating, for example, a pedestrian or another vehicle that the driver who is a user should pay attention to. Therefore, by displaying an entertaining three-dimensional model image using the point cloud information, the driver can monitor an object or the like to be noted while enjoying the image. Along with this, the sense of security that the driver has can be improved.

Further, the processing unit 12 may acquire point cloud information from the roadside unit 5 installed on the traveling path of the own vehicle Ca via the communication unit 11. By doing so, it is possible to generate a three-dimensional model image based on the point cloud information acquired by the roadside machine 5 even when the number of other vehicles Cb is small. When generating a three-dimensional model image based on the point cloud information acquired by the roadside machine 5, for example, the vehicle is not limited to the roadside machine 5a corresponding to the lane in which the own vehicle Ca shown in FIG. 1 is traveling, but is in the opposite lane. The point cloud information acquired by the roadside machine 5b corresponding to the above may also be used. This is because it is possible to acquire the point cloud information of the vehicle traveling in the lane in which the own vehicle Ca is traveling, depending on the installation position and the angle of view of the roadside machine 5b.

In the above-described embodiment, the point cloud information is always acquired when it can be acquired. However, for example, when the three-dimensional model of the own vehicle Ca is generated with a certain degree of perfection, the point cloud is further acquired. The communication unit 11 may stop the acquisition of the point cloud information because the acquisition of the information is unnecessary. That is, when the degree of generation of the appearance image exceeds a predetermined value, the acquisition of the point cloud information may be stopped. This degree of perfection is, for example, a case where point cloud information can be acquired from all directions of front, back, left, and right. By doing so, when the three-dimensional model image is completed to some extent, it is possible to stop the acquisition of the point cloud information and reduce the amount of communication with the external device and the processing load of the own device.

Next, the information processing device 1 as the image generation device according to the second embodiment of the present invention will be described with reference to FIGS. 8 to 13. The same parts as those in the first embodiment described above are designated by the same reference numerals, and the description thereof will be omitted.

In this embodiment, the configuration of the information processing device 1 is the same as that in FIG. In this embodiment, a point cloud acquired from a device (rider) that not only generates and displays a three-dimensional model of a specific object but also acquires a point cloud of a plurality of vehicles including the own vehicle and a roadside machine. Based on the above, a three-dimensional map image which is an image reproducing at least the three-dimensional space around the traveling path of the own vehicle is generated.

The operation of generating and displaying a three-dimensional map image (image generation method) in the information processing apparatus 1 according to this embodiment will be described with reference to the flowchart of FIG. The flowchart of FIG. 8 is configured as, for example, a computer program executed by the processing unit 12.

First, in step S21, the processing unit 12 acquires the point cloud information acquired by the rider 2 of the own vehicle Ca via the I / F, and communicates the point cloud information from the other vehicles Cb1, Cb2, the roadside machine 5, and the like. Obtained via 11.

Next, in step S22, the processing unit 12 generates a three-dimensional map image. In the generation of the three-dimensional map image, the three-dimensional map image in the detection range is generated based on the point cloud information received from each device, and they are combined to generate the entire three-dimensional map image. Alternatively, the point cloud information received from each device may be combined, and the entire three-dimensional map image may be generated from the combined point cloud information. The three-dimensional map image generated at this time does not have to be at a level where the appearance of a feature such as a building is reproduced, and the shape may be reproduced in a state such as a point cloud or a wire frame. Further, the three-dimensional map image generated in step S22 also includes vehicles, pedestrians, and the like acquired by each rider. That is, the processing unit 12 functions as a generation unit that generates a three-dimensional map image (three-dimensional space image) showing a three-dimensional space including at least the periphery of the own vehicle Ca (moving body) based on the acquired point cloud information. .. Further, the processing unit 12 combines (in other words, synthesizes) the point cloud information received from the external device and the point cloud information acquired by the rider 2 of the own vehicle Ca to generate the three-dimensional map image. You may do so.

Next, in step S23, the processing unit 12 causes the display unit 14 to display the three-dimensional map image generated in step S22. That is, the processing unit 12 functions as an output unit that outputs a three-dimensional map image (three-dimensional space image) to the display unit 14 arranged in the own vehicle Ca (moving body).

As is clear from the above description, step S21 functions as an acquisition step, step S22 functions as a generation step, and step S23 functions as an output step.

Next, a specific example of displaying the three-dimensional map image in this embodiment will be described with reference to FIGS. 9 to 12. FIG. 9 is a diagram illustrating a situation of a specific example. In FIG. 9, the own vehicle Ca is traveling on the road R1. The detection range of the rider 2 of the own vehicle Ca is L1. In addition, the other vehicle Cb1 is traveling on the road R1. In addition, the detection range of the rider 2 of the other vehicle Cb1 is L2. In addition, the other vehicle Cb2 is traveling on the road R2. In addition, the detection range of the rider 2 of the other vehicle Cb2 is L3.

The roadside machine 5 is installed along the road R2. The detection range of the roadside machine 5 (rider) is L4. The traffic light 6 is installed at an intersection of road R1 and road R2. A rider is also provided in the traffic light 6, and the detection range of this rider is set to L5.

Further, along the road R2, there is a feature BL1 and a feature BL2 which are features such as a predetermined building, and along the road R1 there is a feature BL3 which is a feature such as a predetermined building.

In the case of FIG. 9, since it is possible to obtain information on the features along the roads R1 and R2 shown in FIG. 9 from the detection range of each rider, it is generated based on the point group information detected by each rider. The generated three-dimensional map information is combined to generate a three-dimensional map image of roads R1 and R2.

The three-dimensional map image generated in this way does not only include features BL1 to BL3 such as buildings, but also includes own vehicle Ca in the detection range L2 of the rider 2 of the other vehicle Cb1. Further, although not shown, pedestrians detected in each detection range L1 to L5 may be included. Moreover, since this three-dimensional map image is updated sequentially, if each vehicle moves, it moves in the three-dimensional map image. For example, if the own vehicle Ca continues to receive point cloud information from other vehicles Cb1, Cb2, etc., the objects detected by the other vehicles Cb1 and Cb2 are reflected in the three-dimensional map image in substantially real time. Therefore, the three-dimensional map image of this embodiment can be a moving image instead of a still image.

Further, the three-dimensional map image can be displayed in a field of view viewed from an arbitrary viewpoint according to the operation of the passenger of the own vehicle Ca, for example. For example, the field of view A1 in FIG. 9 is a field of view seen from the front viewpoint from the own vehicle Ca, and is displayed on the display unit 14 as shown in FIG. 10, for example.

Further, the field of view A2 in FIG. 9 is a field of view seen from the right viewpoint of the own vehicle Ca, and is displayed on the display unit 14 as shown in FIG. 11, for example. Further, the field of view A3 in FIG. 9 is a field of view seen from the right rear viewpoint of the own vehicle Ca, and is displayed on the display unit 14 as shown in FIG. 12, for example.

The three-dimensional map image is not limited to being generated by the information processing device 1 mounted on the vehicle C, and may be generated by, for example, an external server device or the like and distributed to the vehicle C. A configuration example is shown in FIG.

In FIG. 13, the server device 10 is connected to a network N such as the Internet. Then, the server device 10 acquires the point cloud information from the own vehicle Ca, the other vehicle Cb, the roadside unit 5, and the signal 6, and generates a three-dimensional map image as shown in the flowchart of FIG.

Then, the three-dimensional map image is distributed in response to the request from the information processing device 1 of the own vehicle Ca, and the three-dimensional map image is displayed on the display unit 14 of the information processing device 1.

Although the viewpoint can be freely changed even in the case of FIG. 13, if the processing capacity of the information processing device 1 is not high, the information of the viewpoint is transmitted to the server device 10 and an image corresponding to the viewpoint is generated. It may be delivered to the information processing device 1.

According to this embodiment, in the information processing device 1, the processing unit 12 acquires point cloud information from the own vehicle Ca, other vehicles Cb1, Cb2, the roadside unit 5, and the signal 6, and based on the acquired point cloud information. , At least a three-dimensional map image around the own vehicle Ca is generated, and the three-dimensional map image is displayed on the display unit 14. By doing so, based on the point cloud information acquired from a plurality of devices, for example, a map image of the cityscape including other vehicles Cb1, Cb2, pedestrians, etc. that reproduces the three-dimensional space around the own vehicle Ca is generated. can do. Then, it is possible to display and produce such that the own vehicle Ca travels in the reproduced three-dimensional space.

Further, the processing unit 12 acquires the sequentially acquired point cloud information and updates the three-dimensional map image based on the sequentially acquired point cloud information. By doing so, the three-dimensional map image can be updated in real time, and it is possible to display and produce the image according to the actual running state. In addition, by using a three-dimensional map image as a moving image, more realistic display and production can be performed.

Further, the processing unit 12 obtains point cloud information from other vehicles Cb1 and Cb2, roadside units 5, and signal 6 outside the own vehicle Ca, and point cloud information acquired by the rider 2 arranged in the own vehicle Ca. A three-dimensional map image is generated based on this. By doing so, it is possible to acquire point cloud information outside the detection range of the sensor or the like installed in the sensor. Therefore, a wider range of 3D map images can be generated.

Further, the operation unit 15 capable of adjusting the position of the viewpoint of the three-dimensional map image is provided, and the display unit 14 changes the position of the viewpoint of the three-dimensional map image according to the adjustment of the position of the viewpoint by the operation unit 15. It may be displayed. By doing so, it is possible to display images viewed from various viewpoints in the space represented by, for example, a three-dimensional map. Therefore, since the viewpoint can be changed according to the preference of the passenger or the like, the display can be displayed without getting bored.

Further, the present invention is not limited to the above examples. That is, those skilled in the art can carry out various modifications according to conventionally known knowledge within a range that does not deviate from the gist of the present invention. Of course, such a modification is included in the category of the present invention as long as it still has the configuration of the image generating apparatus of the present invention.

1 Information processing device (image generator)
2 Rider 3 GPS receiver 4 Other sensors 5 Roadside unit (external device)
6 Traffic light (external device)
11 Communication unit 12 Processing unit (acquisition unit, generation unit, output unit)
13 Storage unit 14 Display unit 15 Operation unit 16 I / F

Claims (9)

An image generation device including a generation unit that generates an image displayed on a display unit of a moving body.
An acquisition unit that acquires point cloud information representing an object around each of the plurality of devices by a plurality of points from a plurality of devices including an external device outside the moving body in which the image generation device is arranged. ,
Based on the point cloud information acquired by the acquisition unit, a generation unit that generates a three-dimensional space image showing a three-dimensional space including at least the periphery of the moving body, and a generation unit.
An image generator characterized by comprising. The acquisition unit sequentially acquires the point cloud information and obtains the point cloud information.
The image generation device according to claim 1, wherein the generation unit updates the three-dimensional spatial image based on the point cloud information acquired sequentially. The image generation device according to claim 2, wherein the generation unit generates the three-dimensional space image as a moving image. The generation unit generates the three-dimensional spatial image based on the point cloud information acquired from the external device and the point cloud information acquired by the sensor arranged on the moving body. The image generator according to any one of items 1 to 3. The external device includes a device for acquiring the point cloud information mounted on another moving body.
The image generation device according to claim 4, wherein the acquisition unit acquires the point cloud information from the other moving body. The external device includes a roadside machine installed on the road.
The image generation device according to claim 4 or 5, wherein the acquisition unit acquires the point cloud information from the roadside machine. It is equipped with an operation unit that can adjust the position of the viewpoint of the three-dimensional space image.
The method according to any one of claims 1 to 6, wherein the generation unit generates the three-dimensional spatial image in which the position of the viewpoint is changed according to the adjustment of the position of the viewpoint by the operation unit. Image generator. It is an image generation method executed by an image generation device that generates an image displayed on a display unit of a moving body.
An acquisition step of acquiring point cloud information representing an object around each of the plurality of devices by a plurality of points from a plurality of devices including an external device outside the moving body in which the image generation device is arranged. ,
Based on the point cloud information acquired in the acquisition step, a generation step of generating a three-dimensional space image around the moving body showing at least the three-dimensional space including the periphery of the moving body, and a generation step.
An image generation method comprising. An image generation program characterized in that the image generation method according to claim 8 is executed by a computer.

Images