JP5771540B2 - Transmission device, display device, and display system - Google Patents

Description

  The present invention relates to a transmission device, a display device, and a display system in remote control of a moving body.

  When the operator drives the vehicle by remote operation, the display device on the operation side needs to display an image in which the surrounding environment of the vehicle is captured. Here, when an image of the surrounding environment of the vehicle is transmitted from the vehicle-side imaging device to the operation-side display device with a large amount of transmission data, the image displayed on the operation-side display device includes Display delay occurs as compared with the actual surrounding environment. Patent Document 1 discloses a remote control device that reduces the display delay of an image of the surrounding environment by compressing the captured image and transmitting it with a small amount of data.

JP 2009-118072 A

  However, although the remote operation device (transmission device) disclosed in Patent Document 1 can provide an operator with an image of the surrounding environment of the vehicle with a small amount of transmission data, the device configuration on the operation side There is a problem that it becomes complicated.

  The present invention has been made in view of the above points, and reduces the amount of transmission data by reducing the image of the surrounding environment of the moving body without complicating the device configuration on the operator's side for remotely operating the moving body. An object of the present invention is to provide a transmission device, a display device, and a display system that can be provided to the operator.

  The present invention has been made to solve the above-described problem, and includes an imaging unit that images the surrounding environment, a measurement unit that measures three-dimensional information of the surrounding environment, and a predetermined height or more in the surrounding environment. An extraction unit that extracts a position based on the three-dimensional information, a cutout unit that cuts out an image of the position extracted by the extraction unit from an image captured by the imaging unit, and a cutout unit And a transmission unit that transmits the difference information of the three-dimensional information.

  In addition, the present invention includes a positioning unit that measures the position of the own device, and the transmission unit pastes the clipped image on an overhead image according to the position of the own device and displays the image on the display device. It is a transmission device characterized by transmitting the position of its own device.

  Further, the present invention is the transmission device, wherein the transmission unit transmits a texture map that is an image in which the extracted images are arranged.

  Further, the present invention provides a receiving unit that receives an image cut out by a transmission device and difference information of the three-dimensional information, reproduces the three-dimensional information by accumulating the difference information, and reproduces the reproduced information. A generating unit that extracts a display area from an area indicated by three-dimensional information; a pasting unit that pastes an overhead image and the clipped image on the display area extracted by the generating unit; and the overhead image; And a display unit that displays the extracted display area to which the clipped image is pasted.

  Further, the present invention is characterized in that the reception unit receives a position of a transmission device, and the generation unit extracts the display area according to the position of the transmission device received by the reception unit. Display device.

  Further, according to the present invention, the receiving unit receives a texture map that is an image in which the cut out images are arranged, and the pasting unit extracts the cut out image arranged in the texture map. The display device is characterized by being attached to the displayed area.

  According to another aspect of the present invention, there is provided a display system comprising a transmission device and a display device.

  According to the present invention, the transmission unit transmits an image of the clipped subject. Thereby, the transmission device can provide the operator with an image of the surrounding environment of the moving body with a small amount of transmission data without complicating the configuration of the apparatus on the operator side who remotely operates the moving body. .

It is a block diagram which shows the structural example of the display system in 1st Embodiment of this invention. It is a bird's-eye view which shows the example of the mobile body provided with the transmitter in 1st Embodiment of this invention, and its surrounding environment. It is a figure which shows the example of the height grid map in 1st Embodiment of this invention. It is a figure for demonstrating the cutting-out process of a model image in 1st Embodiment of this invention. It is a figure which shows the example of the height grid map in which the corrected model range was defined in 1st Embodiment of this invention. It is a figure which shows the example of the bird's-eye view of the reproduction | regeneration 3D map in 1st Embodiment of this invention. It is a flowchart which shows the operation | movement procedure of the transmitter in 1st Embodiment of this invention. It is a figure which shows the example of the texture map in 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement procedure of the transmitter in 2nd Embodiment of this invention.

[First Embodiment]
A first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of a display system. The display system compares the image of the surrounding environment of the moving body with the case where the image is simply transmitted without complicating the configuration of the operator's device for remotely operating the moving body (for example, a vehicle). And providing the operator with a small amount of transmission data.

  The display system includes a transmission device 100 and a display device 200. The transmitting device 100 is a device on the moving body side, and moves with the moving body while imaging the surrounding environment of the moving body by being installed on the moving body. The display device 200 is a device on the operator side, receives an image of the surrounding environment of the moving body from the transmission device 100 wirelessly, and provides the received image to the operator.

First, a configuration example of the transmission device 100 will be described.
The transmission device 100 includes an imaging unit 110, a measurement unit 120, a positioning unit 130, an extraction unit 140, a cutout unit 150, and a transmission unit 160.

  Hereinafter, the description will be continued assuming that the three-dimensional information is represented by three-dimensional coordinates (x, y, z) in a predetermined coordinate system. Here, the coordinate component z is a height component. In addition, the relative position of the positioning unit 130 and the measuring unit 120 is determined so that the correspondence between the three-dimensional coordinates and orientation of the moving body measured by the positioning unit 130 and the three-dimensional coordinates of the surrounding environment measured by the measuring unit 120 can be understood. It is assumed that the distance and posture are calibrated in advance. In addition, the relative relationship between the imaging unit 110 and the measurement unit 120 so that the correspondence between the two-dimensional coordinates in the entire image of the surrounding environment captured by the imaging unit 110 and the three-dimensional coordinates of the ambient environment measured by the measurement unit 120 can be understood. It is assumed that the correct distance and posture are calibrated in advance.

  The imaging unit 110 captures an environment around the moving body, and an image including an image of a subject (for example, a structure, a person, a road, a white line, a curb) (hereinafter referred to as a “model image”) to the clipping unit 150. Output. Here, the imaging unit 110 may capture the subject in any format of a moving image or a still image.

  The measurement unit 120 is a laser radar device that measures the three-dimensional coordinates of the environment around the moving body. The measurement unit 120 irradiates the surrounding environment of the moving body with laser light. The measurement unit 120 determines the ambient environment of the moving object based on the time until the irradiated laser light is reflected back to the surrounding environment and returns to the measurement unit 120 and the irradiation direction of the laser light (attitude of the measurement unit 120). Dimensional information (three-dimensional coordinates) is repeatedly measured while scanning the surrounding environment.

  In addition, about an example of a laser radar apparatus, Unexamined-Japanese-Patent No. 2011-99742 can be referred, for example.

  FIG. 2 is an overhead view showing an example of a mobile object including a transmission device and its surrounding environment. In FIG. 2, the transmission apparatus 100 is installed on the moving body 900, so that the imaging unit 110 (see FIG. 1) causes the subject 500, the subject 510, the subject 520, the road 400, and the like in the surrounding environment of the moving body 900. And moving on the road 400 together with the moving body 900. The measuring unit 120 (see FIG. 1) measures the three-dimensional coordinates of the subject 500, the subject 510, the subject 520, the road 400, and the like in the surrounding environment of the moving body 900, along with the moving body 900 on the road 400. Have moved.

  In FIG. 2, measurement points 300-1 to 300-19 indicate positions (hereinafter referred to as “measurement points”) where the laser light is reflected in the surrounding environment. Among these, the measurement points 300-1 to 300-9 and 300-19 are measurement points in which each coordinate component z (height component) is less than a predetermined height threshold. On the other hand, the measurement points 300-10 to 300-18 are measurement points at which the coordinate component z of the measurement point is not less than the height threshold.

  That is, since the subject 520 is three-dimensional, the measurement points 300-10 and 300-11 are measurement points whose coordinate component z is higher than that of the surrounding measurement points. The same applies to each measurement point of the subject 510 and each measurement point of the subject 500. The measurement unit 120 outputs information indicating the three-dimensional coordinates of each measurement point to the extraction unit 140.

  In FIG. 2, since the distance between the subject 530 and the moving body 900 is too far, the subject 530 is not irradiated with laser light, and the measurement unit 120 does not measure the three-dimensional coordinates of the subject 530. It is shown. When the moving body 900 further moves on the road 400 and the subject 530 approaches the moving body 900, the measuring unit 120 can measure the three-dimensional coordinates of the subject 530. Similarly, for the imaging unit 110, the moving body 900 further moves on the road 400, and the subject 530 and the moving body 900 approach each other, so that the imaging unit 110 can capture the subject 530. .

  Returning to FIG. 1, the description of the configuration example of the transmission device 100 is continued. The positioning unit 130 repeatedly measures the three-dimensional coordinates and orientation of the transmission device 100, that is, the three-dimensional coordinates and orientation of the moving body 900, and repeatedly outputs information indicating the three-dimensional coordinates and orientation of the moving body 900 to the extracting unit 140. Output. Here, the positioning unit 130 may measure the three-dimensional coordinates and orientation of the moving body 900 by receiving radio waves from an artificial satellite (for example, a Global Positioning System (GPS) satellite). In addition, the positioning unit 130 may measure the direction (moving direction) of the moving body 900 based on a change in coordinate components for each time.

  The positioning unit 130 includes a speed sensor, a three-axis acceleration sensor, and a three-axis gyro (rotation angle sensor) as an IMU (Internal Measurement Unit), and is based on at least one of the sensing results of these sensors. Thus, the three-dimensional coordinates and orientation of the moving body 900 may be measured. Here, the positioning unit 130 detects, for example, the moving speed of the moving body 900 by integrating the acceleration detected by the acceleration sensor, and further integrates the detected moving speed to obtain the three-dimensional coordinates of the moving body 900. You may measure.

  Information indicating the three-dimensional coordinates of each measurement point is input from the measurement unit 120 to the extraction unit 140. In addition, information indicating the three-dimensional coordinates and orientation of the moving body 900 is input from the positioning unit 130 to the extraction unit 140.

  Based on the information indicating the three-dimensional coordinates of each measurement point and the information indicating the three-dimensional coordinates and the orientation of the moving body 900, the extraction unit 140 arranges the measurement points represented by the three-dimensional coordinates in a grid pattern on a plane. The generated map (hereinafter referred to as “height grid map”) is generated.

  In addition, the extraction unit 140 selects measurement points having a coordinate component z (height component) equal to or higher than a height threshold among measurement points constituting the height grid map based on information indicating the three-dimensional coordinates of each measurement point. To height grid map. The extraction unit 140 outputs information indicating the height grid map from which the measurement points having the coordinate component z equal to or higher than the height threshold are extracted to the cutout unit 150. Here, the information indicating the height grid map may be represented by an array of three-dimensional coordinates, for example.

  FIG. 3 shows an example of the height grid map. The extraction unit 140 virtually connects a plurality of extracted measurement points, and tentatively determines a range connecting the measurement points as a range (hereinafter referred to as “model range”) associated with the model image. Here, the model range may be defined as a straight line or a curved line (fan shape) on the ground plane (xy plane) of the height grid map.

  In FIG. 3, a model range 620a is provisionally defined in a range where measurement points 300-10 and 300-11 are connected by a straight line. In addition, a model range 610a is provisionally defined in a range where measurement points 300-12 and 300-13 are connected by a straight line. In addition, a model range 600a is provisionally defined in a range where the measurement points 300-14 and 300-18 are connected by a curve.

  An image including a model image is input to the cutout unit 150 from the imaging unit 110 at a predetermined period. In addition, information indicating a height grid map from which measurement points whose coordinate component z is equal to or greater than the height threshold is extracted from the extraction unit 140.

  The cutout unit 150 is based on the correspondence between the two-dimensional coordinates in the image input from the imaging unit 110 (hereinafter referred to as “captured image”) and the three-dimensional coordinates of the measurement point whose coordinate component z is greater than or equal to the height threshold. Then, a provisionally determined model range is determined for the captured image. Here, when it is assumed that the frame indicating the tentatively defined model range actually exists in the surrounding environment, the cutout unit 150 uses the range in which the frame is captured in the captured image as the captured image. Determine.

  In addition, the cutout unit 150 roughly corrects the model range so as to include the model image. In addition, the cutting unit 150 determines the strength of the contour (edge) of the model image with respect to the model range defined in the captured image and a predetermined range around the model range, so that the model range is included so as to include the model image. Further correction. Here, the model range may be represented by three-dimensional coordinates of each vertex of the model range. Further, the cutout unit 150 cuts out the model image included in the corrected model range from the captured image.

  FIG. 4 is a diagram for explaining a model image cut-out process. FIG. 4A shows a captured image. In FIG. 4A, the model image 700, the model image 710, the model image 720, and the image of the road 400 are included in the captured image. Here, the model image 700 is an image of the subject 500. The model image 710 is an image of the subject 510. The model image 720 is an image of the subject 520.

  FIG. 4B shows a captured image including a model image and a frame image indicating a provisionally determined model range. In FIG. 4B, a frame image showing the model range 600a, a frame image showing the model range 610a, and a frame image showing the model range 620a are further included in the captured image shown in FIG. ing.

  FIG. 4C shows a captured image including a model image and a frame image indicating a model range corrected based on the contour of the model image. In FIG. 4C, a frame image showing the corrected model range 600a, a frame image showing the corrected model range 610a, and a frame image showing the corrected model range 620a are captured images shown in FIG. Is further included.

  4C, the cutout unit 150 cuts out the model image 700 included in the model range 600a corrected based on the contour of the model image 700. In addition, the cutout unit 150 cuts out the model image 710 included in the model range 610 a corrected based on the contour of the model image 710. In addition, the cutout unit 150 cuts out the model image 720 included in the model range 620a corrected based on the contour of the model image 720.

  The cutout unit 150 outputs information indicating the corrected model range (hereinafter referred to as “corrected model range information”), the cut out model image, and information indicating the height grid map to the transmission unit 160. . Here, the model range and the extracted model image are associated with each other by identification information.

  FIG. 5 shows an example of a height grid map in which the corrected model range is defined. In FIG. 5, the model range 600a is corrected to be a straight line parallel to the x-axis on the ground plane (xy plane) of the height grid map. The model range 610a is corrected to be parallel to the model range 600a on the ground plane (xy plane) of the height grid map. Similarly, the model range 620a is corrected to be parallel to the model range 600a on the ground plane (xy plane) of the height grid map.

  Returning to FIG. 1, the description of the configuration example of the transmission device 100 is continued. The transmission unit 160 receives the corrected model range information and the cut model image from the cut unit 150 in association with the identification information. In addition, information indicating the height grid map is input from the cutout unit 150 to the transmission unit 160. The transmission unit 160 wirelessly transmits the corrected model range information and the extracted model image in association with the identification information.

  Moreover, the transmission part 160 transmits the information which shows a height grid map by radio | wireless. Here, the transmission unit 160 wirelessly transmits only updated information or untransmitted information (hereinafter referred to as “difference information”) among the information indicating the height grid map.

Next, a configuration example of the display device 200 will be described.
The display device 200 includes a reception unit 210, a storage unit 220, a generation unit 230, a pasting unit 240, and a display unit 250.

  The receiving unit 210 wirelessly receives the corrected model range information, the cut out model image, and the identification information that associates them with each other from the transmitting unit 160 of the transmission device 100. The receiving unit 210 transfers the corrected model range information and the extracted model image to the pasting unit 240 in association with the identification information. In addition, the reception unit 210 receives difference information wirelessly from the transmission unit 160 of the transmission device 100. The reception unit 210 transfers information indicating the height grid map to the generation unit 230.

  The storage unit 220 stores in advance an overhead image (for example, an aerial image, an aerial photograph, a satellite image) associated with the two-dimensional coordinates (x, y) of the ground plane of the height grid map.

  The generation unit 230 adds the difference information input from the reception unit 210 to the height grid map that has already been input and accumulated, so that the height grid map transmitted from the transmission unit 160 of the transmission device 100 is added. Reproduce. Further, the generation unit 230 extracts a region (display region) to be displayed on the display unit 250 from the height grid map to which the difference information is added. The generation unit 230 outputs information indicating the extracted height grid map to the pasting unit 240.

  Here, the area to be displayed on the display unit 250 is determined in the height grid map so as to include, for example, only the areas added up to a predetermined number of times before the difference information added one after another. It is an area. Further, for example, the area to be displayed on the display unit 250 may be an area defined in the height grid map so as to include the two-dimensional coordinates (x, y) of the moving body 900. In this case, the reception unit 210 receives the two-dimensional coordinates (x, y) of the moving body 900 measured by the positioning unit 130 of the transmission device 100 from the transmission unit 160 of the transmission device 100 and transfers it to the generation unit 230. .

  The correction model range information and the cut model image are input to the pasting unit 240 from the receiving unit 210 in association with the identification information. In addition, information indicating the extracted height grid map is input from the generation unit 230 to the pasting unit 240. The pasting unit 240 stores an overhead image associated with the two-dimensional coordinates (x, y) of the ground plane of the extracted height grid map based on the information indicating the extracted height grid map, Obtain from 220.

  In addition, the pasting unit 240 pastes an overhead image on the extracted height grid map, and generates an overhead view of the height grid map. Hereinafter, the height grid map to which the overhead image is pasted is referred to as a “reproduced three-dimensional map”. In addition, the pasting unit 240 pastes contour lines on the reproduced three-dimensional map by connecting the measurement points having the same coordinate component z (height component) based on the information indicating the extracted height grid map. wear.

  In addition, the pasting unit 240 determines the corrected model range in the overhead view of the reproduced three-dimensional map based on the corrected model range information. In addition, the pasting unit 240 pastes a model image associated with the identification information in the corrected model range. In addition, the pasting unit 240 pastes the bird's-eye view image of the moving object 900 on the predetermined coordinates of the bird's-eye view of the reproduced three-dimensional map. In addition, the pasting unit 240 outputs an overhead view of the reproduced three-dimensional map to the display unit 250.

  FIG. 6 shows an example of an overhead view of the reproduction three-dimensional map. In FIG. 6, the model image 700 previously associated with the model range 600a is pasted on the model range 600a (see FIG. 5) in the overhead view of the reproduction three-dimensional map. In addition, the model image 710 previously associated with the model range 610a is pasted on the model range 610a (see FIG. 5) in the overhead view of the reproduction three-dimensional map. Further, the model image 720 previously associated with the model range 620a is pasted on the model range 620a (see FIG. 5) in the overhead view of the reproduction three-dimensional map. Further, the bird's-eye view image of the moving object 900 is pasted at predetermined coordinates in the bird's-eye view of the reproduced three-dimensional map. Also, the images showing the contour lines 800-1 to 800-5 are pasted on the overhead view of the reproduced three-dimensional map so as to overlap the contour lines defined in the reproduced three-dimensional map.

  Returning to FIG. 1, the description of the configuration example of the display device 200 is continued. The display unit 250 displays an overhead view (see FIG. 6) of the reproduced three-dimensional map in which the model image, the image showing the contour lines, and the overhead view image of the moving object are pasted. The display unit 250 may further display various information necessary for remote operation (for example, the three-dimensional coordinates of the moving object 900). The display unit 250 is, for example, a liquid crystal display device.

Next, the operation procedure of the transmission apparatus 100 will be described.
FIG. 7 is a flowchart illustrating an operation procedure of the transmission apparatus. The cutout unit 150 acquires an image including a model image and the three-dimensional coordinates of each measurement point (step S1). And the extraction part 140 produces | generates a height grid map (step S2). In addition, the extraction unit 140 provisionally determines a range connecting measurement points whose height components are greater than or equal to a height threshold in the height grid map as a model range (step S3).

  The cutout unit 150 corrects the model range so as to include the model image (step S4). Further, the cutout unit 150 determines the contour of the model image, and cuts out the corrected model range from the image. That is, the cutout unit 150 cuts out a model image based on the corrected model range. (Step S5). Then, the transmission unit 160 transmits the height grid map difference information and the like, the corrected model range information, and the extracted model image (step S6).

  Each unit of the transmission device 100 determines whether or not an end instruction from the user is accepted (step S7). When the end instruction is received (step S7—Yes), each unit of the transmission device 100 ends the process. On the other hand, when the termination instruction has not been received (step S7—No), each unit of the transmission device 100 returns the process to step S1.

  As described above, the transmission device 100 includes the imaging unit 110 that images the surrounding environment, the measurement unit 120 that measures the three-dimensional information of the surrounding environment, and the position in the surrounding environment that is equal to or higher than a predetermined height. An extraction unit 140 that extracts based on the information, a cutout unit 150 that cuts out the image at the position extracted by the extraction unit 140 from the image picked up by the image pickup unit 110, and an image cut out by the cutout unit 150 ( And a transmission unit 160 that transmits difference information of the three-dimensional information (height grid map).

  With this configuration, the transmission unit 160 transmits the cut image and the difference information of the three-dimensional information. Thus, the transmission device 100 can provide the operator with an image of the surrounding environment of the moving body with a small amount of transmission data without complicating the configuration of the operator side who remotely operates the moving body. it can. In addition, the transmission device 100 can display an overhead image (for example, an aerial image) on the display device 200 without delay. In addition, unlike the case where the overhead image is simply pasted on the height grid map, the model image of the side surface of the subject is displayed on the display device 200, so that the operator feels unnaturalness or discomfort in the displayed image. The mobile object 900 can be remotely operated without any trouble.

In addition, the transmission device 100 includes a positioning unit 130 that measures the position of the transmission device 100. The transmission unit 160 transmits the position of the transmission device 100 to the display device 200 that pastes and displays the clipped image on an overhead image corresponding to the position of the transmission device 100.
Thereby, the transmission apparatus 100 can display the overhead image according to the position of the transmission apparatus 100 on the display apparatus 200.

  Further, the display device 200 reproduces the three-dimensional information by accumulating the difference information, a receiving unit 210 that receives the clipped image (model image) and the difference information of the three-dimensional information, A generation unit 230 that extracts a display area from an area indicated by the reproduced three-dimensional information (height grid map), and an overhead image and the clipped image in the display area extracted by the generation unit 230. A pasting unit 240 to be pasted, and a display unit 250 that displays the extracted display area (reproduction three-dimensional map) to which the overhead image and the clipped image are pasted.

With this configuration, the display unit 250 displays the overhead image to which the overhead image and the clipped image are pasted.
As a result, the display device 200 can provide the operator with an image of the surrounding environment of the moving body with a small amount of transmission data without complicating the apparatus configuration on the side of the operator who remotely operates the moving body. it can. Further, the display device 200 can display the overhead view image without delay. In addition, unlike the case where the overhead image is simply pasted on the height grid map, the model image of the side surface of the subject is displayed on the display device 200, so that the operator feels unnaturalness or discomfort in the displayed image. The mobile object 900 can be remotely operated without any trouble.

In addition, the reception unit 210 receives the position of the transmission device 100. The generation unit 230 extracts the display area according to the position of the transmission device 100.
Thereby, the display device 200 can display an overhead image according to the position of the transmission device 100.

[Second Embodiment]
The second embodiment is different from the first embodiment in that an image in which model images are arranged is transmitted. Only the differences from the first embodiment will be described below.

First, a configuration example of the transmission device 100 will be described.
The transmission unit 160 generates an image in which the cut model images are arranged (hereinafter referred to as “texture map”). The transmission unit 160 performs a predetermined data compression process (for example, a data compression process that can be decoded by a decompression process compliant with the Moving Picture Experts Group (MPEG) standard) on the texture map, and transmits the texture map that has been compressed. . Here, a plurality of texture maps may be used.

  FIG. 8 shows an example of a texture map. The texture map 1000 has a predetermined coordinate system (u, v). Here, the coordinate (u, v) = (0, 1) is associated with a model range 600a (see FIG. 5) in advance, and the coordinate (u, v) = (1, 1) has a model range. Assume that 610a is associated in advance and the model range 620a is associated in advance with the coordinates (u, v) = (2, 1).

  In FIG. 8, the model image 700 is pasted at coordinates (u, v) = (0, 1) previously associated with the model range 600 a including the model image 700. The model image 710 is pasted at coordinates (u, v) = (1, 1) associated in advance with the model range 610a including the model image 710. The model image 720 is pasted at coordinates (u, v) = (2, 1) associated in advance with the model range 620a including the model image 720.

  The reception unit 210 (see FIG. 1) receives information indicating the height grid map, the correction model range information, and the texture map 1000 from the transmission unit 160 of the transmission device 100 by radio. The receiving unit 210 decodes the model image included in the texture map 1000 by performing processing on the texture map 1000 to expand the data compressed by the transmitting unit 160. The receiving unit 210 transfers the corrected model range information and the decoded model image to the pasting unit 240 in association with the identification information.

Next, a configuration example of the display device 200 will be described.
The pasting unit 240 may reassemble the model image into a frame model image (for example, a cube or a rectangular parallelepiped) so that the decoded model image looks three-dimensional. The pasting unit 240 pastes the frame model image associated with the model range to the model range in the overhead view of the reproduction three-dimensional map.

Next, the operation procedure of the transmission apparatus 100 will be described.
FIG. 9 is a flowchart illustrating an operation procedure of the transmission apparatus. Steps Sa1 to Sa5 are the same as steps S1 to S5 in the flowchart shown in FIG. The transmission unit 160 updates the texture map 1000 based on the corrected model range information and the extracted model image (step Sa6). The transmission unit 160 transmits the difference information of the height grid map, the correction model range information, and the texture map 1000 (step Sa7).

  Each unit of the transmission device 100 determines whether or not an end instruction from the user is received (step Sa8). When the end instruction is received (step Sa8-Yes), each unit of the transmission device 100 ends the process. On the other hand, when the termination instruction has not been received (step Sa8-No), each unit of the transmission device 100 returns the process to step Sa1.

As described above, the transmission unit 160 transmits the texture map 1000 that is an image in which the extracted images are arranged.
Thereby, the transmission apparatus 100 provides the operator with an image of the surrounding environment of the moving body with a smaller amount of transmission data without complicating the apparatus configuration on the side of the operator who remotely operates the moving body. be able to.

  The receiving unit 210 receives a texture map 1000 that is an image in which the cut out images are arranged. The pasting unit 240 pastes the clipped images arranged in the texture map 1000 to the extracted display area.

  Thereby, the display device 200 provides the operator with an image of the surrounding environment of the moving body with a smaller amount of transmission data without complicating the configuration of the operator side who remotely operates the moving body. be able to.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the positioning unit 130 may measure only the coordinates (x, y) of the ground plane among the three-dimensional coordinates (x, y, z) of the moving body 900. In addition, for example, the extraction unit 140 may extract a measurement point having a coordinate component z higher than the measurement point closer to the moving body 900 from the plurality of measurement points to the height grid map. Good.

  For example, the transmission unit 160 of the transmission device 100 may transmit information to the reception unit 210 of the display device 200 by wired communication. For example, the pasting unit 240 may deform the model image according to the motion parallax of the moving body 900. Here, the pasting unit 240 may paste the pre-read model image so that the display delay is corrected on the overhead view of the reproduction three-dimensional map.

  A program for realizing the transmission device, the display device, and the display system described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed. Thus, execution processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Transmission apparatus, 110 ... Imaging part, 120 ... Measurement part, 130 ... Positioning part, 140 ... Extraction part, 150 ... Extraction part, 160 ... Transmission part, 200 ... Display apparatus, 210 ... Reception part, 220 ... Storage part , 230 ... generating unit, 240 ... pasting unit, 250 ... display unit, 300 ... measuring point, 400 ... road, 500 ... subject, 510 ... subject, 520 ... subject, 530 ... subject, 600a ... model range, 610a ... model range 620a ... Model range, 700 ... Model image, 710 ... Model image, 720 ... Model image, 800 ... Contour, 900 ... Moving object, 1000 ... Texture map

Claims (7)

An imaging unit for imaging the surrounding environment;
A measurement unit for measuring three-dimensional information of the surrounding environment;
An extraction unit for extracting a position of a predetermined height or more in the surrounding environment based on the three-dimensional information;
A cutout unit that cuts out the image of the position extracted by the extraction unit from the image captured by the imaging unit;
A transmission unit that transmits the image cut out by the cut-out unit and the difference information of the three-dimensional information;
A transmission device comprising: It has a positioning unit that measures the position of its own device,
The said transmission part transmits the position of the said own apparatus to the display apparatus which pastes and displays the cut-out image on the bird's-eye view image according to the position of the said own apparatus. Transmitter device.   The transmission device according to claim 1, wherein the transmission unit transmits a texture map that is an image in which the cut-out images are arranged. A receiving unit that receives an image cut out by the transmission device according to any one of claims 1 to 3 and difference information of the three-dimensional information;
A generator that reproduces the three-dimensional information by accumulating the difference information and extracts a display area from an area indicated by the reproduced three-dimensional information;
An affixing unit that pastes the overhead image and the clipped image on the display area extracted by the generation unit;
A display unit for displaying the extracted display area to which the overhead image and the clipped image are pasted;
A display device comprising: The reception unit receives the position of the transmission device according to any one of claims 1 to 3,
The display device according to claim 4, wherein the generation unit extracts the display area according to a position of the transmission device received by the reception unit. The receiving unit receives a texture map that is an image in which the cut out images are arranged,
The display device according to claim 4, wherein the pasting unit pastes the cut out images arranged in the texture map to the extracted display area. The transmission device according to any one of claims 1 to 3,
A display device according to any one of claims 4 to 6,
A display system comprising:

Images