JP2007088660A - Image display apparatus and image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display apparatus and an image display method capable of obtaining a video output in an optional direction, fixing a target video image directed in the optional direction at all times and providing the fixed image to a pilot/a driver over an omnidirectional range in three-dimensions. <P>SOLUTION: The image display apparatus for displaying video images received from a plurality of cameras is characterized in to include: a video processing section for carrying out arithmetic operations for mapping the video images on a regular polyhedron; a memory for storing results of the mapping; a display processing section for reading a mapped image in an optional direction from the memory to produce a video image for display; and a display section for displaying the display video image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image display device and an image display method that are mounted on a moving body such as an aircraft or a vehicle and display an omnidirectional image based on the moving body, and more specifically, an arbitrary display in a three-dimensional omnidirectional range. The present invention relates to an image display device and an image display method for outputting an external image in a direction.

2. Description of the Related Art Conventionally, in order to take appropriate navigation measures, a camera is mounted on a moving body such as an aircraft or a vehicle to acquire video information around the aircraft and present it to the operator.
In addition, in remote operation of an unmanned helicopter, there is a type that receives an image captured by mounting an omnidirectional camera on an unmanned helicopter, and cuts out an image in an arbitrary direction by a pilot-side pilot system and presents it to the pilot (for example, , See Patent Document 1).

JP 2003-267295 A

  FIG. 8 is a diagram showing a schematic arrangement example of cameras on an aircraft in a conventional image display apparatus. FIG. 8A shows a state in which a movable camera is arranged on the nose. In FIG. 8A, the periphery of the body 111 is photographed within the movable range of the movable camera 112. FIG. 8B shows a plurality of cameras arranged on the fuselage 111. In FIG. 8B, the camera 113 is placed at the nose and photographs the front of the fuselage, and the camera 114 is placed at the rear of the fuselage and the rear of the fuselage. Take a picture.

FIG. 9 is a configuration diagram illustrating an example of a conventional image display device described in Patent Document 1. In FIG.
In FIG. 9, the omnidirectional camera 121 mounted on the unmanned helicopter 120 acquires 360 ° omnidirectional video, and the video transmitter 122 transmits this video signal to the video receiver 124 in the control system 123. The video receiver 124 sends the video signal to the video cutout unit 125. The video cutout unit 125 uses the conversion table based on the signal from the angle sensor 126 that detects the direction of the driver's line of sight in any direction within the video signal. Cut out the video part. A display unit 127 such as a head mounted display presents the clipped video to the operator.

  However, when the above-described movable camera is used, there are problems that the response speed is slow, a mechanical mechanism is required, and the movable range is limited. In addition, when a plurality of fixed cameras are used, since the video is a separate system, there is a disadvantage that switching is necessary and it is difficult to use. Furthermore, using an omnidirectional camera to cut out an image of an arbitrary angle and send it to the display system is effective for viewing a limited range of field of view such as the ground or landscape. There was a problem that the video could not be seen.

  The present invention intends to solve the problems of the conventional image display apparatus and image display method, and obtains a video output in an arbitrary direction in a three-dimensional omnidirectional range and always. An object of the present invention is to provide an image display device and an image display method capable of fixing a target image in an arbitrary direction and presenting it to a pilot.

In order to achieve the above object, according to claim 1 of the present invention, in an image display device that displays images input from a plurality of cameras,
A video processing unit that performs an operation of mapping the video onto a regular polyhedron;
A memory for storing the mapped result, and
A display processing unit that reads a mapping in an arbitrary direction from the memory and generates a display image;
A display unit for displaying the display image;
It is characterized by having.

  According to a second aspect of the present invention, in the image display device according to the first aspect, the video processing unit maps the video input from the plurality of cameras onto a regular icosahedron.

  According to a third aspect of the present invention, in the image display device according to the first or second aspect, a plurality of the display processing unit and the display unit are provided.

  According to a fourth aspect of the present invention, in the image display device according to any one of the first to third aspects, the plurality of cameras are arranged so as to output a three-dimensional omnidirectional video.

  According to a fifth aspect of the present invention, the image display device according to any one of the first to fourth aspects is mounted on a moving body.

  According to a sixth aspect of the present invention, in the image display device according to the fifth aspect, the moving body is an aircraft.

  According to a seventh aspect of the present invention, in the image display device according to the sixth aspect, the display processing unit generates a video for display based on an output signal of a sensor that detects a direction of a fuselage mounted on an aircraft. And

  The image display device according to claim 6, wherein the display processing unit is an image that displays a preset target, and is always based on an output signal of a sensor that detects the direction of the aircraft. An image displaying a target is generated.

In an image display method for displaying video input from a plurality of cameras arranged so as to photograph all three-dimensional directions in claim 9,
Mapping the images of the plurality of cameras onto a regular polyhedron;
Storing the mapped result in memory;
Reading a map in any direction from the memory to generate a video for display;
Displaying the video for display;
It is characterized by having.

  In the image display method according to claim 9, in the image display method according to claim 9, the step of mapping the images of the plurality of cameras onto a regular polyhedron includes mapping the images of the plurality of cameras onto a regular icosahedron. Features.

  According to the present invention, an image display device and an image display method capable of obtaining a video output in an arbitrary direction within a three-dimensional omnidirectional range without requiring a mechanical mechanism or a camera to be switched are realized. it can.

  In order to take appropriate navigation measures, it is possible to acquire video information around a moving body such as an aircraft or a vehicle, and present a video in an arbitrary direction to the operator. Furthermore, by generating a display image using the output signal of the sensor that detects the direction of the aircraft, it is possible to obtain a camera image corresponding to the orientation of the aircraft, and always on the ground or in space regardless of the orientation of the aircraft. A locked image can be output while facing one direction.

  In recent years, general-purpose 3D graphic technology for dividing a three-dimensional model into polygons and pasting video information (texture) has been advanced. This 3D graphic technology can be used by normalizing the image captured by the camera to a regular polyhedron, which is advantageous in terms of development efficiency and cost.

  In the 3D graphic technology, a triangle is generally used as a polygon for ease of calculation. The regular icosahedron has a regular triangular shape on each surface, and by adopting a regular icosahedron as a regular polyhedron for normalization, a texture having a unit of a triangular shape can be used. With the recent development of GDP (Graphic Data Processor), a function to generate a picture when disassembling a solid into a polygon based on a triangle, pasting video information as a texture, and viewing it from any direction It can be realized easily and at a low price. By modeling the space around the airframe into a shape like a regular icosahedron and using this general-purpose 3D graphic technology, a display processing device can be easily manufactured at low cost.

  A regular polyhedron to be used for normalization may be a regular hexahedron, but the regular icosahedron can reduce the area required to display a certain direction at the same time. The amount of data transmission with the display processing unit is suppressed, and the processing load can be reduced. For example, a regular hexahedron requires one to three planes at the same time to display an image in one direction, that is, 1/6 to 3/6 of the entire mapping data in all directions. It is necessary to simultaneously display 1 to 6 images in one to six planes, that is, 1/20 to 6/20 of the total area, and the advantage that the amount of data can be reduced when using a regular icosahedron. There is.

  In this way, by generating the display image by normalizing the image captured by the camera on the regular polyhedron, the image output in an arbitrary direction is obtained in the three-dimensional omnidirectional range and the target is always in the arbitrary direction. It is possible to provide an image display device and an image display method that can fix the video of the above and present it to the operator.

  Hereinafter, an image display device and an image display method of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of an image display device according to the present invention. In FIG. 1, the video processing device 2 processes video from the cameras 1 a to 1 f to generate three-dimensional mapping data described later, and stores the generated mapping data in the mapping memory 3. The display processing device 4 inputs mapping data of a portion necessary for display via the transmission path 6, generates video information to be displayed, and outputs the video information to the monitor 5a or HMD (Head Mount Display) 5b.

  FIG. 2 is a diagram showing a detailed configuration of the video processing device 2 and the mapping memory 3. A / D converters 21a to 21f for converting signals from the camera into digital signals, memories 22a to 22f for storing output signals of the A / D converters 21a to 21f, a CPU 23 for calculating mapping data, and a mapping calculated by the CPU 23 A mapping memory 3 for storing data, a communication I / F 24 for outputting the mapping data to the outside of the video processing device 2, an aircraft I / F 25 for inputting posture information from the aircraft, and a bus 26 are included.

  FIG. 3 is a diagram showing a detailed configuration of the display processing device 4. In the display processing device 4, the mapping data is input from the video processing device 2 via the communication I / F 41 and stored in the memory 42. The CPU 43 controls the communication I / F 41 and the memory 42. Based on the mapping data, the GDP 44 generates a video viewed in an arbitrary direction from the center, stores it in the memory 45, and outputs it to the monitor 5a and HMD 5b via the video I / F 46.

  FIG. 4 is a diagram illustrating an arrangement example when the cameras 1a to 1f are attached to the aircraft body. As shown in FIG. 4, the cameras 1 a to 1 f are wide-angle cameras attached in the six-axis direction of the aircraft, and output video signals in the six-axis direction.

  FIG. 5 is a diagram schematically showing the arrangement of the camera shown in FIG. FIG. 5 is obtained by replacing the aircraft of FIG. 4 with a regular hexahedron, and the cameras 1a to 1f are attached to the centers of the respective surfaces and project images in the normal direction. The video of each camera can be considered as information on a regular hexahedron shown in FIG.

  FIG. 6 is an explanatory diagram relating to the generation of the three-dimensional mapping data. FIG. 6 shows a model in which the position of the camera (which is regarded as equivalent to the aircraft) is the center point, and the outside world is a regular polyhedron (regular icosahedron). An image viewed from the camera is projected onto a regular icosahedron shown in FIG. 6 to normalize the image.

  If all the images on the regular icosahedron centered on the own device are known, the image projected in an arbitrary direction can be determined by virtually determining the direction of the camera. The display processing device 4 determines the orientation of the virtual camera, transmits mapping data necessary for generating display data from the video processing device 2, and generates a video to be output to the monitor 5a or the like.

  When calculating the map data and generating the display data, the calculation is performed using the machine information obtained via the machine I / F 25 in FIG. 2 or the machine I / F 47 in FIG. Specifically, attitude information is input from a sensor such as an inertial navigation system installed in the aircraft through the aircraft I / F 25 and the aircraft I / F 47 and used for calculation by the CPU 23 and the GDP 44.

  Even if the aircraft changes its attitude relative to the outside world (Earth), it locks while always pointing in one direction on the ground or space by adding and calculating its own attitude information when determining the camera direction. It is also possible to output the processed image.

  In this embodiment, the mapping data is normalized with a regular icosahedron, but the shape is not limited to a regular icosahedron. However, a triangle is suitable for polygon processing, a regular polyhedron consisting of the same triangle has only a regular octahedron and a regular icosahedron, and a regular polyhedron in which each surface has the same shape in terms of normalization is used. It is considered better to use a regular icosahedron from the viewpoint of simpler processing. Other shapes are possible, but the processing such as normalization and display data generation becomes complicated, and the development efficiency is lowered and the development cost is increased.

  FIG. 7 is a block diagram showing another embodiment of the image display device according to the present invention. The case where a plurality of display processing devices 4 are connected to one video processing device 2 is shown.

  Since the display processing device 4 transmits necessary data from the video processing device 2 to generate display data, a plurality of display processing devices 4 can be connected to one video processing device 2, and each display processing device 4 can be individually connected. It is possible to generate a video in any direction.

  As a result, when a plurality of people in the aircraft want to view the video outside the aircraft, even if there is only one data source (video processing device 2), each person can see a separate video. Here, as multiple people, pilots and crews with other missions, passengers in civil aircraft, etc. can be considered, and it is possible to see the external world video you want to see on your monitor Become.

FIG. 1 is a block diagram showing an embodiment of an image display device according to the present invention. FIG. 2 is a diagram showing a detailed configuration of the video processing device 2 and the mapping memory 3. FIG. 3 is a diagram showing a detailed configuration of the display processing device 4. FIG. 4 is a view showing an arrangement example of cameras attached to the aircraft body. FIG. 5 is a diagram schematically showing the arrangement of the camera shown in FIG. FIG. 6 is an explanatory diagram related to normalization of mapping data. FIG. 7 is a diagram showing a configuration example in which a plurality of display processing devices 4 are connected to one video processing device 2. FIG. 8 is a diagram showing a schematic arrangement example of cameras on an aircraft in a conventional image display apparatus. FIG. 9 is a block diagram showing an example of a conventional image display apparatus.

Explanation of symbols

1a to 1f camera 2 video processing device 3 mapping memory 4 display processing device 5a monitor 5b HMD
6 Transmission path

Claims (10)

In an image display device that displays video input from multiple cameras,
A video processing unit that performs an operation of mapping the video onto a regular polyhedron;
A memory for storing the mapped result, and
A display processing unit that reads a mapping in an arbitrary direction from the memory and generates a display image;
A display unit for displaying the display image;
An image display device comprising:   The image display device according to claim 1, wherein the video processing unit maps video input from the plurality of cameras onto a regular icosahedron.   The image display device according to claim 1, comprising a plurality of display processing units and a plurality of display units.   The image display device according to claim 1, wherein the plurality of cameras are arranged to output a three-dimensional omnidirectional video.   The image display device according to claim 1, wherein the image display device is mounted on a moving body.   The image display apparatus according to claim 5, wherein the moving body is an aircraft.   The image display device according to claim 6, wherein the display processing unit generates a display image based on an output signal of a sensor that detects a direction of an airframe mounted on an aircraft.   The display processing unit generates an image that displays a preset target and that always displays the target based on an output signal of a sensor that detects a direction of the aircraft. 6. The image display device according to 6. In an image display method for displaying images input from a plurality of cameras arranged to shoot all directions in three dimensions,
Mapping the images of the plurality of cameras onto a regular polyhedron;
Storing the mapped result in memory;
Reading a map in any direction from the memory to generate a video for display;
Displaying the video for display;
An image display method characterized by comprising: 10. The image display method according to claim 9, wherein the step of mapping the images of the plurality of cameras onto a regular polyhedron maps the images of the plurality of cameras onto a regular icosahedron.

Images