JP2012019452A - Image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that can prevent a user from feeling sick under display of an image of the outside of a moving object in the moving object because of discontinuity between the displayed image of the outside of the moving object and the actual scene of the outside of the moving object due to delay caused by the display of the image, and enables the user to naturally recognize the movement of the moving object on the basis of the displayed image of the outside of the moving object and fit the user's body to the displacement of the moving object.SOLUTION: A displacement after a predetermined time lapse of a moving object is estimated, and an image of the outside of the moving object is cut out and output as a display image on the basis of the estimated displacement, whereby the image can be seen as being continuous with the actual scene. Therefore, a user is made to have a visually-induced self-kinesthetic sense, and the user can suppressed from feeling sick.

Description

  The present invention relates to an image processing apparatus and an image processing method for processing an image viewed by a passenger on a moving body.

2. Description of the Related Art In recent years, people in a passenger seat and a rear seat are watching images such as a DVD with an image processing apparatus mounted in a moving body such as a vehicle. However, when viewing a DVD or the like in a moving body with vibration or shaking, it is said that the line of sight is fixed and motion sickness (motion sickness) occurs.
There is a video display device that can reduce the burden of motion sickness of a viewer by matching visual information with vestibule information and somatosensory information by movement of a vehicle against such a motion sickness. (For example, Patent Document 1)
In this video display device, a background video is generated based on the acceleration of the moving body, and the background video and a video such as a DVD are synthesized. This combined video gives the passengers in the passenger seat and rear seats a visual sense of moving their bodies (visually induced self-motion sensation). The occurrence of motion sickness can be reduced by matching the vestibule information and the somatosensory information.

JP 2008-242251 A

In the conventional technique, a background video corresponding to acceleration is generated and combined and displayed as a background of content such as a DVD. However, when an image captured by a camera or the like mounted in front of the moving body is displayed, a delay due to image processing is multiplied by a time required for at least one drawing rather than a real scene seen from the window of the moving body.
For this reason, the image from the front camera and the actual scene seen from the window appear discontinuous, and there is a problem that the vestibule information and the somatosensory information do not match and cause sickness.

  The present invention has been made to solve the above-described problems. When an image outside the moving body is displayed inside the moving body, the displayed moving body is displayed with a delay caused by displaying the image. In order to avoid the intoxication caused by the discontinuity between the outside image and the actual scene outside the moving body, the movement of the moving body can be naturally recognized by the displayed image outside the moving body, and the body is adjusted to the displacement. It is an object of the present invention to obtain an image processing apparatus that is easy to handle.

  An image processing apparatus according to the present invention includes an image input unit that inputs an image captured outside the moving body, a sensor that detects a physical state of the moving body corresponding to the time when the image is captured, and the sensor. Based on the detected physical state, a displacement prediction unit that predicts the displacement of the moving body after a predetermined time from the time when the image was captured, and the image input based on the displacement predicted by the displacement prediction unit An image cutout unit that cuts out an image input from the image processing unit, and an image processing unit that generates a display image based on the image cut out by the image cutout unit.

  This invention predicts the displacement of the moving body after a predetermined time, cuts out an image of the outside of the moving body based on the predicted displacement, and outputs it as a display image. Gives a sense of guided self-motion and can reduce sickness caused by moving objects.

1 is a system configuration diagram of an image processing apparatus 1 illustrating Embodiment 1 of the present invention. It is a flowchart which shows operation | movement of the image processing apparatus 1 which shows Embodiment 1 of this invention. It is. It is a figure which shows the calculation method of the synchronous information in Embodiment 1 of this invention. In Embodiment 1 of this invention, it is a figure which shows the example in which the image is displayed on the display 6 in a vehicle. In Embodiment 1 of this invention, it is a figure which shows the example of the image image | photographed with the front camera 9 when the vehicle drive | worked at high speed. In Embodiment 1 of this invention, it is a figure which shows the example of the image image | photographed with the front camera 9 while the vehicle drive | worked at low speed. It is a figure which shows the example of a display of the content image in Embodiment 1 of this invention. In Embodiment 1 of this invention, it is a figure which shows the example of a display while the vehicle is drive | working the left curve. It is a figure which shows the image process of the image outside a vehicle image | photographed with the content image and the front camera 9 in Embodiment 1 of this invention. It is the schematic which shows the inside of the moving body in the straight running in Embodiment 1 of this invention. It is the schematic which shows the inside of the moving body in the left curve driving | running | working in Embodiment 1 of this invention.

Embodiment 1 FIG.
FIG. 1 is a system configuration diagram showing an image processing apparatus 1 according to Embodiment 1 of the present invention.
In FIG. 1, an image processing apparatus 1 detects a physical state such as vehicle inclination, vibration, speed, acceleration, rotation, and the like by a sensor (hereinafter referred to as a tilt vibration sensor) 2, and the detected information is a sensor information input unit 3. Is input to the displacement prediction unit 4.
In addition, the external system communication unit 5 inputs position-related information such as vehicle position information, information on a road to be traveled, and map information from the global positioning system (GPS) or car navigation system to the displacement prediction unit 4. The

Then, the displacement prediction unit 4 predicts the displacement of the vehicle after a predetermined time from the physical state information such as tilt, vibration, speed, acceleration, and rotation input from the sensor information input unit 3.
On the other hand, a display 6 is installed in the vehicle (inside the vehicle), and a passenger on the back seat of the vehicle views content such as a DVD displayed on the display 6. Then, the viewpoint of the viewer who views the content is captured by the in-vehicle camera 7, and the viewpoint detection unit 8 detects the viewpoint position of the viewer from the captured image.

Further, the outside camera (outside the vehicle) is photographed by the front camera 9 installed in front of the vehicle, and the photographed image is converted into an image horizontal detection unit (hereinafter referred to as camera) by an image input unit (hereinafter referred to as camera image input unit) 10. The image is input to an image cutout unit (hereinafter, camera image cutout unit) 12 via an image horizontal detection unit) 11.
The camera image level detection unit 11 detects how much the image outside the vehicle is inclined from the horizontal plane (inclination with respect to the horizontal plane).

Then, based on the displacement predicted by the displacement prediction unit 4 and the viewpoint position detected by the viewpoint detection unit 8, the synchronization information calculation unit 13 calculates synchronization information.
The synchronization information calculated by the synchronization information calculation unit 13 synchronizes the actual scene outside the vehicle viewed by the viewer and the image outside the vehicle captured by the front camera 9 displayed on the display 6 (in order to make it visible continuously). Information.
Details of the calculation of the synchronization information calculated by the synchronization information calculation unit 13 will be described later.

Next, the camera image cutout unit 12 cuts out and corrects the image of the front camera 9 into an image 1/30 second ahead based on the synchronization information of the synchronization information calculation unit 13 and sends the image to the image processing unit 15.
In addition, the camera angle-of-view control unit 14 controls the angle of view of the front camera 9 based on the synchronization information from the synchronization information calculation unit 13.
Details of the camera image cutout unit 12 and the camera view angle control unit 14 will be described later.

Then, the image processing unit 15 performs projective transformation on the image outside the vehicle that has been cut out and corrected based on the synchronization information. Further, image processing for rotating the content image (content image) input from the content input unit 16 based on the synchronization information is performed.
At this time, based on the inclination with respect to the horizontal plane detected by the camera image level detection unit 11, the image outside the vehicle and the content image can be rotated and moved so as to be always parallel to the horizontal plane.
In this way, the image outside the vehicle that has undergone the projective transformation is synthesized with a three-dimensional (3D) depth image on the periphery of the rotated content image and displayed on the display 6.

Further, the image processing unit 15 acquires a past vehicle outside image (image processed vehicle image based on synchronization information) recorded in the recording unit 17 instead of the image processed vehicle outside image, It can also be displayed with a content image. The recording unit 17 records an image outside the vehicle that has been image-processed based on the synchronization information, together with the displacement predicted by the displacement prediction unit 4 and the current position acquired from the external system communication unit 5, and the external system communication unit 5 Based on the current position obtained from the above, a recorded image processed outside image is acquired.
Further, the additional information generation unit 18 acquires the current position and map information from the external system communication unit 5 and generates additional information (for example, names of buildings, mountains, rivers, etc.) of images outside the vehicle as character information.
The image processing unit 15 can also display the character information on the display 6 according to the image outside the vehicle.

Next, the operation of the image processing apparatus 1 will be described based on the flowchart of FIG.
While the vehicle is running, physical state displacements such as vibration, tilt, speed, acceleration, and rotation occur.
First, the physical state of these vehicles is detected by the tilt vibration sensor 2 (step (hereinafter S) 1).
The sensor information input unit 3 inputs information on the physical state of the vehicle detected by the tilt vibration sensor 2 to the displacement prediction unit 4. (S2)

Next, the displacement prediction unit 4 predicts a displacement, for example, 1/30 second ahead based on the physical state of the vehicle measured in real time by the tilt vibration sensor 2 (S3).
1/30 second is an estimated time when the frame rate of the display 6 is 30 fps (frame per second). When drawing is performed 30 times per second, that is, 1/30 seconds required for one drawing, the actual scene outside the vehicle and the image taken outside the vehicle displayed on the display 6 are synchronized (so that they appear continuously) Time).
Therefore, as a displacement prediction, for example, the distance and direction of movement in 1/30 seconds are calculated.

On the other hand, the viewpoint detection unit 8 detects the viewpoint position by photographing the viewpoint of the viewer who gets on the vehicle and views the content video such as a DVD by the in-vehicle camera 7 or the like. When there are a plurality of viewers, the center of the plurality of viewers is set as the viewpoint position.
Then, the synchronization information calculation unit 13 calculates synchronization information based on the displacement predicted by the displacement prediction unit 4 and the viewpoint position detected by the viewpoint detection unit 8 (S4).

Here, the calculation method of the synchronization information of the synchronization information calculation unit 13 will be described in detail with reference to FIG.
The calculation of the synchronization information is calculated based on the change in the viewpoint of the viewer of the content and the displacement of the physical state such as the inclination, vibration, speed, acceleration, rotation, etc. of the vehicle detected by the inclination vibration sensor 2, Here, the distance h of the displacement after 1/30 seconds predicted when the velocity v is detected and the half field angle θ of the front camera 9 controlled by the camera field angle control unit 14 are calculated as synchronization information. An example will be described.

The following formula (2) holds.
In FIG.
½ width of display 6 is w,
The distance between the viewpoint and the display 6 is d,
The distance between the viewpoint and the front camera 9 is l,
D, the synchronization target distance of the captured image (distance of the range in which the actual scene outside the vehicle and the captured image are synchronized)
Let W be 1/2 of the horizontal width of the imaging range.

At this time, the following equation (1) holds.
W = D × w / d (1)
h = v × (1/30) (2)
In the above formula (2),
When the speed is 20 km / h, h = 19 cm,
When the speed is 60 km / h, h = 56 cm,
When the speed is 100 km / h, h = 93 cm.

Also, when 1/2 of the angle of view of the front camera 9 controlled by the camera angle-of-view control unit 14 based on the synchronization information is θ, the following expression (3) is established.
θ = Arctan (W / (D−l + h) (3)
Here, when the above equation (1) is applied to W, the following equation (4) is established.
θ = Arctan ((D × w / d) / (D−1 + h)) (4)
By adjusting the angle of view based on the value of θ, the imaging range after 1/30 second can be taken with the front camera 9.
Further, the synchronization target distance D of the captured image is considered to be 5 m to 20 m in an urban area and 20 m to 50 m in a suburb. This is because the urban area has a narrow field of view and the vehicle speed is slow.

Returning to the flowchart of FIG. 2, the camera angle-of-view control unit 14 controls the angle of view of the front camera 9 based on the synchronization information calculated by the synchronization information calculation unit 13 as described above (S5).
Based on the synchronization information of the synchronization information calculation unit 13, the camera angle-of-view control unit 14 moves the vehicle to the telephoto side when the displacement of the vehicle is large (speed is high) or when the viewer's viewpoint moves forward. When the displacement is small (slow speed) or the viewer's viewpoint moves backward, the angle of view of the front camera 9 is controlled to the wide angle side.
In this way, ½ of the angle at which the camera angle-of-view control unit 14 controls the angle of view of the front camera 9 based on the displacement of the vehicle and the movement of the viewer's viewpoint is the value θ described above.

On the other hand, the camera image input unit 10 inputs image data outside the vehicle photographed by the front camera 9 whose angle of view is controlled by the camera angle of view control unit 14 to the image processing apparatus 1 and transmits it to the camera image horizontal detection unit 11. (S6).
The camera image horizontal detection unit 11 detects the edges of the horizontal part of the road and the vertical part of the building or wall from the image data outside the vehicle input by the camera image input unit 10. Then, it is detected how much the image outside the vehicle photographed by the front camera 9 is tilted from the horizontal plane (S7).

The image outside the vehicle whose inclination is detected by the camera image level detection unit 11 is sent to the camera image cutout unit 12 (S8).
The inclination from the horizontal plane detected by the camera image level detection unit 11 is information used instead of the inclination information of the synchronization information, and the process can be performed only with the synchronization information. Therefore, image data outside the vehicle input by the camera image input unit 10 may be sent directly to the camera image cutout unit 12.

Next, the camera image cutout unit 12 cuts out an image outside the vehicle of the front camera 9 based on the displacement of the vehicle and the change in the viewer's viewpoint position, which are the synchronization information of the synchronization information calculation unit 13, and performs correction (S9). ).
For example, when the vehicle is displaced upward and the viewer's viewpoint moves to the right, correction is performed to cut out the lower left from the image outside the vehicle of the front camera based on the displacement of the synchronization information in the upper right direction. Do. Also, when the vehicle is moving forward at a certain speed, the surrounding image toward the center is removed from the image outside the vehicle at the time of shooting so that the previous image will be the distance moved by 1/30 second. To crop the image.

The synchronization information may be based only on the displacement predicted by the displacement prediction unit 4 or may be based only on the viewpoint position detected by the viewpoint detection unit 8.
Further, based on the inclination from the horizontal plane detected by the camera image level detection unit 11 and the viewpoint position of the viewpoint detection unit 8, an image outside the vehicle can be cut out.
Here, how the camera image cutout unit 12 cuts out an image taken outside the vehicle photographed by the front camera 9 whose angle of view is controlled will be described with reference to FIGS.

FIG. 4 shows an example in which a display 6 for displaying a content image such as a DVD is installed for a viewer at the back seat of the vehicle. Reference numeral 20 denotes a content image displayed in the center of the display 6. Reference numeral 21 denotes an outside image taken by the front camera 9.
A viewer who views the content image 20 can enjoy the content while feeling the displacement of the vehicle by viewing the image 21 outside the vehicle captured by the front camera 9 continuously with the actual scene while viewing the content image 20.

FIG. 5 is a photographed image obtained by controlling the angle of view of the front camera 9 to the angle of view on the telephoto side while the vehicle is traveling at a high speed. If the outside camera is photographed with the front camera 9 controlled to the telephoto side, it is possible to photograph a distant view.
The camera image cutout unit 12 cuts out only the area of the content image 20 from the image 21 outside the vehicle that has been taken to a distant view in this way, and transmits it to the image processing unit 15.

FIG. 6 is a photographed image obtained by controlling the angle of view of the front camera 9 to the angle of view on the wide angle side while the vehicle is traveling at a low speed. When the outside of the vehicle is photographed by the front camera 9 controlled to the wide angle side, the nearby scenery can be captured in more detail.
The camera image cutout unit 12 cuts out the image 21 outside the vehicle in which the nearby scenery is captured in more detail as described above, only the region of the content image 20, and transmits it to the image processing unit 15.
The angle θ calculated by the above-described synchronization information calculation unit 13 is ½ of the angle of view to be controlled to the telephoto side or the wide angle side described here.

  When the angle of view control by the camera angle of view control unit 10 is not performed, the camera image cutout unit 12 performs correction based on the synchronization information by changing the cutout range. As an image after 1/30 second, the region of the content image 20 is cut out from the imaging range in consideration of the angle of view θ. Details will be described later.

Then, the image processing unit 15 maintains the content image such as a DVD input from the content input unit 16 so that the content image is horizontal based on the synchronization information, and rotates the image based on the displacement or the movement of the viewpoint position. I do.
For example, if the vehicle has a left curve and the displacement moves to the right, causing a clockwise roll, move the content image to the left and rotate it counterclockwise to keep the content image parallel to the horizontal plane. Perform image processing.

The content image can also be rotated based on the inclination from the horizontal plane detected by the camera image level detection unit 11 and the viewpoint position detected by the viewpoint detection unit 8.
Then, the rotated content image is arranged at the center of the screen.
Next, the image outside the vehicle that has been cut out and corrected so as to be synchronized with the actual scene is subjected to projective transformation around the content image that has been rotated and moved, image processing is performed so as to express the depth by the 3D effect, and the image is displayed on the display 6. (S10).

Here, the image processing performed in the image processing unit 15 will be described in detail with reference to FIGS.
First, content image processing will be described.
FIG. 7 is a diagram illustrating a display example of a content image displayed on the display 6.
In FIG. 7, the display 6 is the same as the display 6 of FIG. 1, and 20 is a content video (content image) displayed at the center of the display 6.

Based on the synchronization information, the image processing unit 15 moves the content image 20 up, down, left, and right, and performs image processing to rotate the content image 20 so that it is always parallel to the horizontal plane.
At this time, based on the inclination from the horizontal plane detected by the camera image level detection unit 11 and the viewpoint position detected by the viewpoint detection unit 8, the content image 20 can be rotated.
Further, a process of reducing the content image 20 from the display range of the display 7 is performed so that the display content is not lost even if the image obtained by moving or rotating the content image 20 up, down, left and right is displayed. .

Next, a display example in which content images and images outside the vehicle are processed and displayed will be described.
FIG. 8 is a diagram illustrating a display example when the vehicle is traveling on the left curve.
In FIG. 8, 19 is the head of the viewer who is viewing the content image, 20 is the content image displayed in the center of the display 6, and 21 is an image outside the vehicle taken by the front camera 9.
On the left curve, the vehicle rolls clockwise in the traveling direction, and the display 6 is also tilted clockwise.

An image 21 outside the vehicle photographed by the front camera 9 is an image photographed by the front camera 9 whose angle of view is adjusted based on the synchronization information, and is an image cut out by the camera image cutout unit 12. When the angle of view of the front camera 9 is not adjusted, it is cut out and corrected within the imaging range of the angle based on the synchronization information.
In addition, the image 21 outside the vehicle is displayed in the display 6 by performing image processing that is rotated counterclockwise in the display 6 so as to be parallel to the horizontal plane.

On the other hand, the content image 20 is displayed tilted together with the display 6 if no image processing is performed. However, the image processing unit 15 rotates and displays the content image 20 so as to be kept parallel to the horizontal plane. In this case, the image processing is performed so as to rotate in the reverse direction of the clockwise direction, and the parallelism with the horizontal plane is maintained in the same manner as the outside image 21 photographed by the front camera 9.
Then, the corrected image 21 outside the vehicle is arranged on the periphery of the content image 20 kept parallel to the horizontal plane by rotation or the like and displayed on the display 6.

  The viewer of the display 6 can obtain a visually induced self-motion sensation by seeing the content image 20 displayed in the center of the screen rotating so as to be always parallel to the horizontal plane. Furthermore, since the image outside the vehicle 21 corrected around the content image 20 is viewed in continuity with the actual scene viewed from the window in conjunction with the displacement, it is easier to obtain a visually induced self-motion sensation.

The visually induced self-motion sensation obtained at this time is due to the fact that the driver naturally performs the same action as the resistance to the centrifugal force that the driver naturally sees in front of the scenery.
Thus, by reducing the deviation of the stimulus between the visual sense and the sense of balance, the effect of the visually induced self-motion sensation is increased, and car sickness can be further reduced.

Next, how the image is processed and displayed by the image processing unit 15 will be described in detail with reference to FIG.
FIG. 9 is a diagram showing image processing for displaying the content image 20 and the image 21 outside the vehicle photographed by the front camera 9 on the display 6.
First, the display position and the rotation angle of the content image 20 are determined from the predicted vehicle displacement obtained from the synchronization information of the synchronization information calculation unit 13. The display position of the content image 20 is shifted in the direction of vehicle displacement (for example, in the case of FIG. 8, the left side that is the exit direction of the curve), and the content image 20 is rotated to the determined rotation angle.

Next, the image 21 outside the vehicle photographed by the front camera 9 is subjected to projective transformation, and is superimposed and displayed around the content image 20 with a depth in 3D.
The projective transformation referred to here is a transformation used when a three-dimensional picture is projected on a plane photograph or when a shadow of an object is made to correspond by fixing a light source.
A gradation that darkens toward the back side (side of the content screen 20) is imprinted on each of the images outside the vehicle divided into four images by projective conversion.

Then, by arranging these as surrounding images of the content image 20, it can be displayed on the display 6 with a depth in 3D.
In this way, the content image 20 that the viewer is viewing rotates and moves in accordance with the displacement, and is always displayed parallel to the horizontal plane, and the surrounding image 21 outside the vehicle is not delayed from the actual scene outside the vehicle. By displaying in 3D with a depth, the viewer can easily tilt the head naturally in accordance with the image, and can reduce the deviation of the stimulus to the sense of vision and balance.

Next, the recording unit 17 and the additional information generation unit 18 used for the additional function of the image processing apparatus 1 will be described.
First, an image outside the vehicle which is cut out and corrected based on the synchronization information by the camera image cutout unit 12 and image-processed by the image processing unit 15 is predicted by the current position obtained from the external system communication unit 5 and the displacement prediction unit 4. Is recorded in the recording unit 17 together with the displacement.
Then, based on the current position of the vehicle obtained from the external system communication unit 5, the image processing unit 15 extracts an image outside the vehicle that has undergone image processing from the recording unit 17 and displays the image on the display 6 as a peripheral image of the content image.
The image processed image recorded by the recording unit 17 is used when it is difficult to feel the movement of the vehicle in an image taken by the front camera 9, such as when it is raining.

Further, the additional information generation unit 18 uses the position information of the vehicle obtained from the external system communication unit 5 and the map information of the car navigation system as information on what is reflected in the image outside the vehicle image-processed by the image processing unit 15. Generate as information. For example, additional information such as the names of buildings, mountains, and rivers is generated and sent to the image processing unit 15.
Then, the image processing unit 15 displays the character information generated by the additional information generation unit 18 on the display 6 so as to overlap the positions of buildings, mountains, and rivers.

Finally, the effect of reducing car sickness according to the present invention will be described with reference to FIGS. 10 and 11 of image examples processed by the image processing unit 15.
FIG. 10 is a conceptual diagram illustrating the inside of a moving body that is traveling in a straight line.
In FIG. 10, the display 6 is the same as the display 6 of FIG. Reference numeral 19 denotes a viewer's head. The display 6 is leveled by being affected by the displacement of the moving body during straight running. The viewer's head 19 is vertical and not tilted.

On the other hand, FIG. 11 is a conceptual diagram showing the inside of the moving body that is traveling on the left curve.
In FIG. 11, the display 6 is affected by the displacement of the vehicle running on the curve, and rolls (the vehicle rotates about the front and rear axes) are generated. On the display 6 in such a situation, an image of the same configuration as that of FIG. 8, that is, an image outside the vehicle that is viewed continuously with the actual scene outside the vehicle by being corrected based on the synchronization information is displayed on the periphery of the content. .
Since the viewer sits perpendicularly to the vehicle, car sickness occurs when the head is shaken outside the curve by centrifugal force.

However, in the situation as shown in FIG. 11, an image as shown in FIG. 8 is displayed on the display 6, and when the viewer views it, it is the same as the resistance to the centrifugal force that the driver naturally sees the scenery in front. Operation is natural.
Like the driver's unconscious behavior, the head is actively tilted to the inside of the curve in opposition to the centrifugal force. The illusion of illusion is obtained, and there is an effect of preventing car sickness.
Viewers other than the driver have a problem that it is difficult to see the scenery in front. For this reason, in the present invention, the image 21 outside the vehicle corrected so that it can be viewed continuously with the actual scene outside the vehicle is displayed together with the content image 20 and viewed by the viewer, so that it is possible to reduce car sickness.

  In the first embodiment, the vehicle has been described as an example. However, a train, a ship, an airplane, an elevator, and the like may be another moving body that generates a physical state displacement such as vibration, speed, acceleration, inclination, and rotation. .

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 2 Tilt vibration sensor, 3 Sensor information input part, 4 Displacement prediction part, 5 External system communication part, 6 Display, 7 Car interior camera, 8 View point detection part, 9 Front camera, 10 Camera image input part, 11 Camera image level detection unit, 12 Camera image cutout unit, 13 Synchronization information calculation unit, 14 Camera view angle control unit, 15 Image processing unit, 16 Content input unit, 17 Recording unit, 18 Additional information generation unit

Claims (6)

An image input unit for inputting an image taken outside the moving body;
A sensor for detecting a physical state of the moving body corresponding to the time when the image is captured;
Based on the physical state detected by the sensor, a displacement prediction unit that predicts the displacement of the moving body after a predetermined time from the time when the image is captured;
An image cutout unit that cuts out an image input from the image input unit based on the displacement predicted by the displacement prediction unit;
An image processing unit that generates a display image based on the image cut out by the image cutout unit;
An image processing apparatus comprising: Based on the displacement predicted by the displacement prediction unit, a camera angle of view control unit that controls the angle of view of the camera that captures the outside of the moving body,
The image processing apparatus according to claim 1, wherein the image cutout unit cuts out an image shot at an angle of view controlled by the camera angle of view control unit. An in-car camera that captures the viewpoint of the viewer who views the content image on a mobile object,
A viewpoint detection unit that detects a viewer's viewpoint position from an image captured by the vehicle camera;
Synchronization that calculates synchronization information that synchronizes the display of an image taken outside the moving body with the actual scene outside the moving body, based on the displacement predicted by the displacement prediction section and the viewpoint position detected by the viewpoint detecting section. With an information calculator,
The camera angle-of-view control unit controls the angle of view of the camera that captures the outside of the moving body based on the synchronization information calculated by the synchronization information calculation unit,
The image cutout unit cuts out an image shot at an angle of view controlled by the camera angle of view control unit based on the synchronization information,
The image processing apparatus according to claim 2, wherein the image processing unit performs image processing on the image cut out by the image cutout unit based on the synchronization information, and generates a display image together with a content image. The image processing unit performs projective transformation on the image cut out by the image cutout unit based on the synchronization information of the synchronization information calculation unit, and a content image as a surrounding image of the content rotated and moved based on the synchronization information. The image processing apparatus according to claim 3, wherein a display image is generated together with the image. An external system communication unit that obtains position-related information of the moving object;
A recording unit that records position-related information obtained by the external system communication unit, a displacement predicted by the displacement prediction unit, and an image output from the image processing unit;
The said image processing part produces | generates a display image with the image of the content recorded on the said recording part which corresponds to the positional relevant information of the moving body acquired from the said external system communication part, The image of Claim 3 characterized by the above-mentioned. The display device according to claim 4. An image input step for inputting an image taken outside the moving body;
A state detection step of detecting with a sensor a physical state of the moving body corresponding to the time when the image was captured;
Based on the physical state of the moving body detected in the state detecting step, a displacement predicting step of predicting the displacement of the moving body after a predetermined time from the time when the image is captured;
Based on the displacement predicted by the displacement prediction step, an image cutout step of cutting out the image input in the image input step;
An image processing step for generating a display image based on the image cut out in the image cutting step;
An image processing method comprising:

Images