JP2005117193A - Imaging terminal, image display terminal, and image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display system displaying a stereoscopic image with high reality in matching with photographing side conditions. <P>SOLUTION: The image display system 1 includes: an imaging terminal 10 including an imaging means 101 configured to include a pair of photographing lenses 101a, 101b, an image processing section 102 for generating left side image data and right side image data, and a transmission section 108 for transmitting lens arrangement information, the left side image data and right side image data, and focal distance information; and an image display terminal 20 including a reception section 201 for receiving the lens arrangement information, the left side image data and right side image data, and the focal distance information, a display position calculation section 203 for calculating a display position so as to cause a both-eye parallax between a left side image and a right side image on the basis of the lens arrangement information and the focal distance information, and an image display section 204 for displaying the left side image and the right side image at the calculated display position. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging terminal, an image display terminal, and an image display system.

2. Description of the Related Art Conventionally, an apparatus for displaying an image captured by a camera or the like as a stereoscopic image has been provided. As such a stereoscopic image display device, for example, there is a device that displays an image captured by an imaging unit on a display screen mounted in front of the eye so that the display optical axis and the imaging optical axis coincide ( For example, see Patent Document 1.)
JP-A-5-344541

  On the other hand, recently, images taken by cameras of various shapes such as digital cameras and camera-equipped mobile phones are frequently transmitted and received via a network. In such a case, if the captured image can be displayed as a stereoscopic image, it is possible to give information with higher reality to the observer.

  However, in the above-described conventional technology, it is not realized to display a stereoscopic image in accordance with shooting conditions that vary for each imaging side terminal.

  Therefore, the present invention has been made in view of such circumstances, and provides an imaging terminal, an image display terminal, and an image display system capable of displaying a stereoscopic image with high reality in accordance with conditions on the photographing side. For the purpose.

  An imaging terminal according to the present invention includes an imaging unit configured to include a pair of imaging lenses, an image processing unit that generates left image data and right image data from images captured by the pair of imaging lenses, and a pair. Transmission means for transmitting lens arrangement information for specifying the relative position and direction of the photographing lens, left image data and right image data generated by the image processing means, and focal length information relating to the focal length of the photographing lens. It is characterized by providing.

  In such an imaging terminal, the image processing unit generates the left image data and the right image data from the images photographed by the pair of photographing lenses provided in the imaging unit, and the transmission unit performs relative processing of the pair of photographing lenses. Since information for specifying the position and direction, left image data, right image data, and focal length information of the photographing lens are transmitted, it is possible to transmit information on photographing conditions at the time of photographing together with left and right image data. It becomes.

  Further, the image display terminal of the present invention includes lens arrangement information for specifying the relative position and direction of a pair of photographing lenses of the imaging terminal, left image data and right image data representing an image photographed by the pair of photographing lenses, A receiving unit that receives focal length information related to a focal length of the photographing lens, and a left image and a right image data represented by the left image data based on the lens arrangement information and the focal length information received by the receiving unit. Display position calculation means for calculating the display position of the left image and the right image so as to generate binocular parallax with the right image represented, and the left image at the display position calculated by the display position calculation means And image display means for displaying an image on the right side.

  In such an image display terminal, the receiving means receives the information for specifying the relative position and direction of the pair of photographing lenses, the left image data, the right image data, and the focal length information of the photographing lens, and displays the display position. Since the calculation means calculates the display position of the image so as to generate binocular parallax based on the lens arrangement information and the focal length information, and the image display means displays the left and right images at the calculated display position. The left and right image data captured in (3) can be reproduced as a stereoscopic image that produces near-real binocular parallax in accordance with the imaging conditions.

  The image display system of the present invention is an image display system that displays an image captured by an imaging terminal as a stereoscopic image on the image display terminal, and the imaging terminal includes an imaging unit including a pair of imaging lenses. Image processing means for generating left image data and right image data from the images captured by the pair of photographing lenses, lens arrangement information for specifying the relative position and direction of the pair of photographing lenses, and image processing A transmission means for transmitting left image data and right image data generated by the means and focal length information relating to a focal length of the photographing lens, and the image display terminal includes a relative position of the pair of photographing lenses of the imaging terminal and Lens arrangement information for specifying a direction, left image data and right image data representing an image captured by a pair of imaging lenses, Receiving means for receiving focal length information relating to the focal length of the lens, and based on the lens arrangement information and focal length information received by the receiving means, the left image represented by the left image data and the right image data are represented. Display position calculating means for calculating the display position of the left image and the right image so as to generate binocular parallax with the right image, and the left image and the right image at the display position calculated by the display position calculating means. And an image display means for displaying the image.

  In such an image display system, the imaging terminal generates left image data and right image data from images captured by the pair of imaging lenses provided in the imaging means, and determines the relative positions and directions of the pair of imaging lenses. Since the information for specifying, the left image data, the right image data, and the focal length information of the photographing lens are transmitted, it is possible to transmit the left and right image data together with the information regarding the photographing condition at the time of photographing. The image display terminal receives information for specifying the relative position and direction of the pair of photographing lenses, left image data, right image data, and focal length information of the photographing lens, and lens arrangement information and focal length information. The display position of the image is calculated so as to generate binocular parallax, and the left and right images are displayed at the calculated display position, so the left and right image data captured by the imaging terminal is actually matched to the shooting conditions. It can be reproduced as a stereoscopic image that produces near binocular parallax.

  The imaging terminal further includes a focal length detection unit that detects a focal length of the pair of photographing lenses, and the transmission unit includes focal length information regarding the focal length detected by the focal length detection unit and a relative relationship between the pair of photographing lenses. It is preferable to transmit lens arrangement information for specifying the target position and direction, and left image data and right image data generated by the image processing means.

  In this case, since the focal length of the imaging lens is detected and the focal length information related to the detected focal length is transmitted, for example, even when the focal length of the imaging means varies due to the automatic focus adjustment function of the imaging terminal, etc. Information on the focal length at the time of shooting can be transmitted reliably.

  According to the image display system of the present invention, the imaging terminal generates the left image data and the right image data from the images photographed by the pair of photographing lenses provided in the imaging unit, and the relative positions of the pair of photographing lenses and Since information for specifying the direction, left image data, right image data, and focal length information of the photographing lens are transmitted, it is possible to transmit the left and right image data together with information related to the photographing conditions at the time of photographing. . The image display terminal receives information for specifying the relative position and direction of the pair of photographing lenses, left image data, right image data, and focal length information of the photographing lens, and lens arrangement information and focal length information. The display position of the image is calculated so as to generate binocular parallax, and the left and right images are displayed at the calculated display position, so the left and right image data captured by the imaging terminal is actually matched to the shooting conditions. It can be reproduced as a stereoscopic image that produces near binocular parallax. As a result, it is possible to provide an image display system capable of displaying a stereoscopic image with high reality in accordance with conditions on the photographing side.

  An image display system according to an embodiment of the present invention will be described with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the same element and the overlapping description is abbreviate | omitted.

  First, the configuration of the image display system according to the present embodiment will be described. FIG. 1 is a schematic configuration diagram of an image display system showing a preferred embodiment of the present invention. The image display system 1 includes an imaging terminal 10, an image display terminal 20, and a mobile communication network 30. In FIG. 1, the imaging terminal 10 and the image display terminal 20 are illustrated one by one, but a plurality of each may be provided. Hereinafter, each component will be described.

(Imaging terminal)
The imaging terminal 10 is a so-called camera-equipped mobile communication terminal for capturing an object, a person, or the like to generate image data. The imaging terminal 10 is a mobile communication terminal that can transmit and receive image data together with the image display terminal 20 via the mobile communication network 30. As functional components of the imaging terminal 10, an imaging unit 101 including a left imaging lens 101 a and a right imaging lens 101 b as a pair of imaging lenses, an image processing unit (image processing unit) 102, a data compression unit 103, and the like. The focal length adjustment unit 104, the focal length detection unit (focal length detection unit) 105, the lens position detection unit 106, and the transmission unit (transmission unit) 108 are configured.

  The left photographic lens 101a and the right photographic lens 101b are optical lenses for linking light emitted from a photographing object such as an object or a person into the imaging unit 101 as an image. The left photographic lens 101 a and the right photographic lens 101 b are attached to predetermined positions on the surface of the imaging terminal 10. At the time of photographing, the left photographing lens 101a is attached so that the relative position with respect to the right photographing lens 101b is on the left side with respect to the direction of the photographing object. Further, the focal lengths of the left photographing lens 101 a and the right photographing lens are variable, and the focal lengths can be adjusted by the focal length adjusting unit 104.

  The imaging means 101 including such photographing lenses 101a and 101b further incorporates an imaging element (not shown) such as a CCD. The imaging means 101 converts the images photographed by the left photographing lens 101a and the right photographing lens 101b into electric signals, respectively, and outputs them to the image processing unit 102.

  The image processing unit 102 is a part that generates left image data and right image data representing respective images photographed by the left photographing lens 101a and the right photographing lens 101b. Specifically, the image processing unit 102 converts the left image data and the right image data, which are digital data, from the electrical signals representing the images photographed by the left photographing lens 101a and the right photographing lens 101b output from the imaging unit 101. Generate. The image processing unit 102 outputs the generated left image data and right image data to the data compression unit 103. In addition, the image processing unit 102 can continuously process the images captured by the photographing lenses 101a and 101b to generate left image data and right image data as moving image data. .

  The data compression unit 103 is a part that combines the left image data and the right image data output from the image processing unit 102 and performs data compression. Examples of the data composition method include a method in which the left image data and the right image data are compressed in the horizontal direction and combined as one image data. Examples of data compression formats include JPEG (Joint Photographic Coding Experts Group) format for still image data, and MPEG (Moving Picture Coding Experts Group) format for moving image data. The data compression unit 103 compresses the image data obtained by combining the left image data and the right image data, and outputs the compressed data to the transmission unit 108 as combined image data.

  The focal length adjustment unit 104 is a part that automatically adjusts the focal lengths of the pair of photographing lenses 101a and 101b so that the photographing object is in focus. Specifically, the focal length adjustment unit 104 measures the distance between the imaging terminal 10 and the subject to be photographed at the time of photographing, and adjusts the focal position of the photographing lenses 101a and 101b.

  The focal length detection unit 105 is a part that detects the focal lengths of the photographing lenses 101 a and 101 b adjusted by the focal length adjustment unit 104. The focal length detection unit 105 outputs the detected focal length information to the transmission unit 108. Here, the method of measuring the detected focal length differs for each shape of the imaging terminal 10 (details will be described later).

  The lens position detection unit 106 is a part that detects lens arrangement information that specifies the relative position and direction of the photographing lens 101a and the photographing lens 101b. The content of the detected lens arrangement information differs for each shape of the imaging terminal 10 as in the focal length measurement method (details will be described later).

  Hereinafter, the focal length information detected by the focal length detection unit 105 and the lens arrangement information detected by the lens position detection unit 106 will be described more specifically for each shape of the imaging terminal 10.

  First, the focal length information and lens arrangement information detected when the imaging terminal 10 is a so-called straight type mobile communication terminal will be described with reference to FIG. FIG. 2 is a plan view of a straight type imaging terminal and an object to be photographed as seen from a direction perpendicular to a plane including the optical axis. As shown in FIG. 2, the left photographic lens 101a and the right photographic lens 101b are arranged on one surface 10b of the plate-shaped main body 10a of the imaging terminal 10 so that their optical axes V and W are parallel to each other. Is provided. In this case, the distance between the center point of the taking lens 101a and the center point of the taking lens 101b is r. At the time of shooting, the photographer adjusts the direction of the imaging terminal 10 so that the shooting target A is on a plane including both optical axes V and W and is equidistant from the optical axis V and the optical axis W. Shall. The focal length detected in this case is the distance d between the surface 10b and the shooting object A. In the case of such a straight type, the focal length detection unit 105 detects the focal length d as focal length information. Further, the lens position detection unit 106 detects the lens distance r and the terminal shape information “straight type” indicating the shape of the imaging terminal 10 from the data storage unit such as a ROM, and detects them as lens arrangement information.

  Next, the case where the imaging terminal 10 is a so-called rotational mobile communication terminal will be described with reference to FIGS. 3 and 4. 3 is a side view of the rotary imaging terminal viewed from the optical axis direction, and FIG. 4 is a plan view of the rotary imaging terminal and the object to be photographed viewed from a direction perpendicular to the plane including the optical axis. It is. As shown in FIG. 3, the imaging terminal 10 includes two plate-like main body portions 12a and 12b. The main body 12a and the main body 12b are connected by a shaft 12c so that their surfaces are parallel to each other and overlap each other at the end. With such a configuration, the main body 12a can be rotated about the shaft 12c with respect to the main body 12b. Further, a left photographing lens 101a and a right photographing lens 101b are provided on the upper surface of the main body 12a and the main body 12b in FIG. 3 so that the optical axis is perpendicular to the surface. In this case, the distance between the axis 12c and the center point of the left photographing lens 101a and the right photographing lens 101b is L. In addition, an angle between a straight line connecting the axis 12c and the center point of the left photographic lens 101a and a straight line connecting the axis 12c and the center point of the right photographic lens 101b is θ. The lens position detection unit 106 detects the distance L between the shaft lenses, which is the distance between the shaft 12c and the center point of the left photographing lens 101a and the right photographing lens 101b, by reading from a data storage unit such as a ROM. At the same time, the lens position detection unit 106 detects a rotation angle θ that is an angle formed by two straight lines connecting the axis 12c and the center point of the left photographic lens 101a and the right photographic lens 101b. The lens position detection unit 106 outputs the inter-lens distance L, the rotation angle θ, and terminal shape information “rotation type” representing the shape of the imaging terminal to the transmission unit 108 as lens arrangement information.

  When such a rotating imaging terminal 10 is used, the photographer performs photographing in a state as shown in FIG. That is, the photographer adjusts the direction of the imaging terminal 10 so that the photographing object A is on a plane including both the optical axes V and W and is equidistant from the optical axis V and the optical axis W. Shall. The focal length detected in this case is the distance d between the upper surface 12e of the main body 12b and the subject A. The focal length detection unit 105 outputs the distance d to the transmission unit 108 as focal length information.

  Next, a case where the imaging terminal 10 is a so-called foldable mobile communication terminal will be described with reference to FIG. FIG. 5 is a plan view of the folding imaging terminal and the object to be photographed as seen from a direction perpendicular to the plane including the optical axis. As shown in FIG. 5, the imaging terminal 10 includes two plate-like main body portions 14a and 14b. The main body part 14a and the main body part 14b are hingedly connected by a shaft 14c. With such a configuration, the main body portion 14a is rotatable with respect to the main body portion 14b at a maximum opening degree of 180 degrees. Further, the left photographic lens 101a and the right photographic lens 101b are provided on the surface 14d of the main body 14a and the surface 14e of the main body 14b so that the optical axes are perpendicular to the respective surfaces. In this case, the distance between the axis 14c and the center point of the left photographing lens 101a and the right photographing lens 101b is k. Further, an opening degree of the main body portion 14a with respect to the main body portion 14b is set as δ. The lens position detection unit 106 first detects an inter-lens distance k, which is the distance between the axis 14c and the center point of the left photographic lens 101a and the right photographic lens 101b, by reading it from a data storage means such as a ROM. At the same time, the lens position detector 106 detects the opening δ of the main body 14a relative to the main body 14b. The lens position detection unit 106 outputs the inter-lens distance k, the opening δ, and terminal shape information “folding type” indicating the shape of the imaging terminal to the transmission unit 108 as lens arrangement information.

  When such a foldable imaging terminal 10 is used, the photographer performs shooting in a state as shown in FIG. That is, the photographer adjusts the direction of the imaging terminal 10 so that the photographing object A is on a plane including both the optical axes V and W and is equidistant from the optical axis V and the optical axis W. Shall. The focal length detected in this case is the distance d between the axis 14c and the subject A. The focal length detection unit 105 outputs the distance d to the transmission unit 108 as focal length information.

  Returning to FIG. 1, the transmission unit 108 includes the focal length information regarding the focal length detected by the focal length detection unit 105, the relative position between the pair of photographing lenses 101 a and 101 b detected by the lens position detection unit 106, and This is a part for transmitting the lens arrangement information for specifying the direction and the left image data and the right image data generated by the image processing unit 102 to the image display terminal 20. In this case, the transmission unit 108 transmits the composite image data generated by the data compression unit 103. The transmission unit 108 transmits the focal length information and the lens arrangement information detected when the image represented by the image data is captured together with the composite image data. The lens arrangement information includes parameters corresponding to the shape of the imaging terminal 10 as described above. When the left image data and the right image data are generated as moving image data, the transmission unit 108 edits the focal length information and the lens arrangement information into time series data. This time series data includes the focal length information and the lens arrangement information detected at predetermined time intervals by the focal length detection unit 105 and the lens position detection unit 106 in time series. The transmitting unit 108 transmits these time series data to the image display terminal 20 together with the corresponding composite image data.

(Image display terminal)
The image display terminal 20 is a mobile communication terminal for displaying an image captured by the imaging terminal 10 as a stereoscopic image. The image display terminal 20 is a mobile communication terminal capable of transmitting and receiving image data as well as voice calls with the imaging terminal 10 via the mobile communication network 30. Functional components of the image display terminal 20 include a receiving unit (receiving unit) 201, a data separating unit 202, a display position calculating unit (display position calculating unit) 203, and an image display unit (image display unit) 204. A left video memory 205, a right video memory 206, and a stereoscopic image display 207.

  The receiving unit 201 is a part that receives lens arrangement information, composite image data, and focal length information from the imaging terminal 10. As described above, the lens arrangement information includes parameters with different contents depending on the shape of the imaging terminal 10. That is, when the shape of the imaging terminal 10 is a so-called straight type, the lens arrangement information includes the inter-lens distance r and the terminal shape information “straight type” indicating the shape of the imaging terminal 10. Further, when the shape of the imaging terminal 10 is a so-called rotation type, the lens arrangement information includes the inter-lens distance L, the rotation angle θ, and terminal shape information “rotation type” representing the shape of the imaging terminal. Further, when the shape of the imaging terminal 10 is a so-called folding type, the lens arrangement information includes the inter-lens distance k, the opening δ, and terminal shape information “folding type” indicating the shape of the imaging terminal. The receiving unit 201 separates the received composite image data, lens arrangement information, and focal length information. The receiving unit 201 outputs the separated composite image data to the data separating unit 202 and simultaneously outputs lens arrangement information and focal length information to the display position calculating unit 203. Here, when the received composite image data is moving image data, the reception unit 201 outputs the composite image data to the data separation unit 202, and at the same time, sets the focal length information and the lens arrangement information as time series data. The data is output to the display position calculation unit 203.

  The data separation unit 202 is a part that separates the composite image data received by the reception unit 201 into left image data and right image data. In this case, the data separation unit 202 first converts the composite image data compressed in a predetermined data compression format into a data format reproducible by the image display terminal 20 and then separates the left image data and the right image data. Thereafter, the data separation unit 202 outputs the left image data and the right image data to the image display unit 204.

  The display position calculation unit 203 determines the display position of the left image represented by the left image data and the right image represented by the right image data based on the lens arrangement information and the focal length information received by the reception unit 201. This is the part to calculate. At this time, the display position calculation unit 203 calculates the display position so that a binocular parallax equivalent to that when the observer observes the photographing object A with the naked eye is generated between the left image and the right image. More specifically, the display position calculation unit 203 determines the shape of the imaging terminal 10 based on the terminal shape information included in the lens arrangement information, and calculates the display position by a calculation method determined for each shape.

（上記式中、ｒは、撮影レンズのレンズ間距離、ｄは、撮影レンズの焦点距離を表す。）
さらに、像Ａ１の立体画像表示用ディスプレイ２０７の表示画面中央部から水平方向の変位ＸＡ１は、下記式（２）により求められる。

また、立体画像表示用ディスプレイ２０７の表示画面の水平方向の幅をｙとすると、像Ａ１が立体画像表示用ディスプレイ２０７の表示画面上に収まるための条件は、下記式（３）により与えられる。

表示位置算出部２０３は、上記条件を満たさない場合には、観察者の両目２Ｌ，２Ｒと立体画像表示用ディスプレイ２０７との距離ｄ１を変更した後、再度ＸＡ１を算出して、前記式（３）の条件を満たすようにする。そして、表示位置算出部２０３は、像Ａ１がディスプレイの中央からＸＡ１だけ左方向に変位するように、右側画像データによって表される右側の像の表示位置を算出する。同様にして、表示位置算出部２０３は、左側画像データの表示位置を算出する。 First, a method of calculating the display position when the terminal shape information is “straight type” will be described. FIG. 6 is a diagram of an observer's eyes and an image displayed on the display as viewed from above the observer. An image A1 in FIG. 6 shows an image of the photographing object A displayed on the display screen of the stereoscopic image display 207 so that the image A2 can be visually recognized only by the observer's right eye 2R. The image of the photographic subject A displayed on the display screen of the stereoscopic image display 207 so as to be visible only is shown. The image A1 is a right image represented by right image data, and the image A2 is a left image represented by left image data. Further, it is assumed that the average distance between the eyes of the observer is x, and the distance between the eyes 2L and 2R of the observer and the stereoscopic image display display 207 is set to d1 as an initial value. Further, in this case, the angle formed by the straight line connecting the right eye 2R and the center point of the image A1 and the display screen of the stereoscopic image display 207 is φ. Further, in FIG. 2, an angle formed by a straight line connecting the center point of the photographing object A and the right photographing lens 101b and a straight line orthogonal to the optical axes V and W and passing through the center point of the photographing object A is ψ1. And In such a case, the condition for causing binocular parallax equivalent to the case where the observer observes the photographing object A with the naked eye is φ = ψ1. That is, the direction in which the image A1 is viewed from the right eye 2R of the observer may be the same as the direction in which the subject A is viewed from the right photographing lens 101b. Therefore, in FIG. 6, when the distance between the perpendicular line drawn from the right eye 2R to the display screen of the stereoscopic image display 207 and the center point of the image A1 is r1, r1 is obtained by the following equation (1).

(In the above formula, r represents the distance between the lenses of the photographing lens, and d represents the focal length of the photographing lens.)
Further, the displacement XA1 in the horizontal direction from the center of the display screen of the stereoscopic image display display 207 of the image A1 is obtained by the following equation (2).

If the horizontal width of the display screen of the stereoscopic image display 207 is y, a condition for the image A1 to fit on the display screen of the stereoscopic image display 207 is given by the following formula (3).

If the above condition is not satisfied, the display position calculation unit 203 changes the distance d1 between the observer's eyes 2L and 2R and the stereoscopic image display 207, calculates XA1 again, and calculates the equation (3 ). Then, the display position calculation unit 203 calculates the display position of the right image represented by the right image data so that the image A1 is displaced leftward by XA1 from the center of the display. Similarly, the display position calculation unit 203 calculates the display position of the left image data.

（上記式中、Ｌは、撮像端末の軸レンズ間距離、θは、撮像端末の回転角、ｄは、撮影レンズの焦点距離を表す。）
以下ストレート型の場合と同様にして、像Ａ１の変位ＸＡ１を前記式（２）により求め、前記式（３）の条件を満たさない場合には、距離ｄ１を変更した後、変位ＸＡ１を再度算出する。表示位置算出部２０３は、算出された変位ＸＡ１に基づいて右側画像データの表示位置を算出する。さらに、表示位置算出部２０３は、右側画像データの表示位置の算出方法と同様にして、左側画像データの表示位置も算出する。 Next, a calculation method when the terminal shape information is “rotation type” will be described. Similarly, in FIG. 6, the angle formed by the straight line connecting the right eye 2R and the center point of the image A1 and the display screen of the stereoscopic image display 207 is φ. In FIG. 4, an angle formed by a straight line connecting the center point of the photographing object A and the right photographing lens 101b and a straight line orthogonal to the optical axes V and W and passing through the center point of the photographing object A is ψ2. And In such a case, the condition for causing binocular parallax equivalent to the case where the observer observes the photographing object A with the naked eye is φ = ψ2. Therefore, in FIG. 6, when the distance between the perpendicular line drawn from the right eye 2R to the display screen of the stereoscopic image display 207 and the center point of the image A1 is r1, r1 is obtained by the following equation (4). In this case, as shown in FIG. 3, the inter-lens distance r of the imaging terminal 10 is r = 2Lsin (θ / 2) from the inter-lens distance L and the rotation angle θ.

(In the above formula, L represents the distance between the axial lenses of the imaging terminal, θ represents the rotation angle of the imaging terminal, and d represents the focal length of the photographing lens.)
Thereafter, similarly to the straight type, the displacement XA1 of the image A1 is obtained by the equation (2). When the condition of the equation (3) is not satisfied, the distance d1 is changed, and then the displacement XA1 is calculated again. To do. The display position calculation unit 203 calculates the display position of the right image data based on the calculated displacement XA1. Further, the display position calculation unit 203 calculates the display position of the left image data in the same manner as the calculation method of the display position of the right image data.

（上記式中、ｋは、撮像端末の軸レンズ間距離、δは、撮像端末の開度、ｄは、撮影レンズの焦点距離を表す。）
次に、像Ａ１の変位ＸＡ１を式（２）により求め、式（３）の条件を満たさない場合には、距離ｄ１を変更した後、変位ＸＡ１を再度算出する。さらに、表示位置算出部２０３は、右側画像データの表示位置を算出した後、同様にして、左側画像データの表示位置も算出する。 Next, a calculation method when the terminal shape information is “folding type” will be described. Similarly, in FIG. 6, the angle formed by the straight line connecting the right eye 2R and the center point of the image A1 and the display screen of the stereoscopic image display 207 is φ. In FIG. 5, an angle formed by a straight line connecting the center point of the photographing object A and the right photographing lens 101b and a straight line orthogonal to the optical axis W and passing through the center point of the photographing object A is ψ3. . In such a case, the condition for causing binocular parallax similar to the case where the observer observes the photographing object A with the naked eye is φ = ψ3. Accordingly, in FIG. 6, when the distance between the perpendicular line drawn from the right eye 2R to the display screen of the stereoscopic image display 207 and the center point of the image A1 is r1, r1 is obtained by the following equation (5).

(In the above formula, k represents the distance between the axial lenses of the imaging terminal, δ represents the opening of the imaging terminal, and d represents the focal length of the photographing lens.)
Next, the displacement XA1 of the image A1 is obtained by Expression (2), and when the condition of Expression (3) is not satisfied, the distance X1 is changed, and then the displacement XA1 is calculated again. Furthermore, after calculating the display position of the right image data, the display position calculation unit 203 similarly calculates the display position of the left image data.

  Returning to FIG. 1, finally, the display position calculation unit 203 outputs the display position information and the observation distance information d1 of the left image data and the right image data calculated as described above to the image display unit 204.

  The image display unit 204 is a part that controls to display the left image and the right image at the display position calculated by the display position calculation unit 203 in the stereoscopic image display display 207. Specifically, the left side image data output from the data separation unit 202 and the image represented by the right side image data are displayed at the position indicated by the display position information output from the display position calculation unit 203. The display coordinates of the image data and the right image data are converted. The image display unit 204 outputs the converted left image data to the left video memory 205 and outputs the converted right image data to the right video memory 206. The left video memory 205 and the right video memory 206 are buffers for temporarily storing image data. At the same time, the image display unit 204 displays additional image data for displaying the observation distance information d1 on the left video memory in order to display the observation distance information d1 output from the display position calculation unit 203 on the stereoscopic image display display 207. Output to 205 and the right video memory 206 is also performed.

  The stereoscopic image display 207 is a display for displaying the left image data and the right image data converted by the image display unit 204 as a stereoscopic image. Examples of types of stereoscopic image display include a barrier type and a lenticular type. More specifically, as shown in FIG. 6, the stereoscopic image display 207 allows the image represented by the left image data stored in the left video memory 205 to be visible only from the left eye 2 </ b> L of the observer. indicate. At the same time, the stereoscopic image display 207 displays the image represented by the right image data stored in the right video memory 206 so that it can be viewed only from the right eye 2R of the observer.

  Next, the operation of the image display system 1 according to the present embodiment will be described. FIG. 7 is a first flowchart showing the operation of the imaging terminal, FIG. 8 is a second flowchart showing the operation of the imaging terminal, and FIG. 9 is a flowchart showing the operation of the image display terminal. First, the operation of the imaging terminal will be described with reference to FIGS.

  First, when shooting is started by key input or the like in the imaging terminal 10, the imaging unit 101 starts processing for images shot by the shooting lenses 101a and 101b (step S01). Next, the imaging terminal 10 and the image display terminal 20 are connected via the mobile communication network 30 (step S02). Then, the image processing unit 102 inquires of the image display terminal 20 whether a stereoscopic image can be displayed (step S03). As a result, when the image display terminal 20 cannot display a stereoscopic image (step S03; NO), the process proceeds to FIG. 8 to inquire the resolution and data compression format of the image data that can be displayed to the image display terminal 20. (Step S11). Thereafter, after the reply from the image display terminal 20, the image processing unit 102 generates left image data with the returned resolution (step S12). Similarly, the image processing unit 102 generates right image data (step S13). Then, the image processing unit 102 generates any one of the generated left image data and right image data as plane image data representing a plane image, and outputs it to the data compression unit 103 (step S14). The data compression unit 103 compresses the planar image data in the data compression format returned from the image display terminal 20 (step S15). Thereafter, the process proceeds to step S25.

  Returning to FIG. 7, on the other hand, if the result of the inquiry in step S03 is that the image display terminal 20 can display a stereoscopic image (step S03; YES), the image processing unit 102 is configured so that the shape of the imaging terminal 10 is straight. It is determined whether or not it is a mold (step S04). This determination is performed by referring to the parameter relating to the attribute of the imaging terminal 10 stored in the data storage means such as the ROM. If the result of determination is that the shape of the imaging terminal 10 is a straight type (step S04; YES), the lens position detector 106 reads the inter-lens distance r (step S05), and the process proceeds to step S16.

  On the other hand, when the shape of the imaging terminal 10 is not a straight type (step S04; NO), the image processing unit 102 determines whether or not the shape of the imaging terminal 10 is a rotation type (step S06). As a result, when the shape of the imaging terminal 10 is a rotation type (step S06; YES), the lens position detection unit 106 detects the rotation angle θ (step S07). At the same time, after detecting the inter-lens distance L (step S08), the lens position detection unit 106 proceeds to step S16.

  On the other hand, when the shape of the imaging terminal 10 is not the rotation type (step S06; NO), the image processing unit 102 determines that the shape of the imaging terminal 10 is the folding type, and performs the following processing. First, the lens position detection unit 106 detects the opening δ (step S09). Next, the lens position detection unit 106 detects the inter-lens distance k (step S10), and the process proceeds to step S16.

  Moving to FIG. 8, the transmission unit 108 edits the parameter detected as described above and the terminal shape information into lens arrangement information (step S16). At the same time, it is determined whether or not the imaging terminal 10 has a focal length adjustment function (step S17). If the imaging terminal 10 has a focal length adjustment function (step S17; YES), the focal length detection unit 105 A focal length d corresponding to the shape of the imaging terminal 10 is detected (step S19). On the other hand, when the focal length adjustment function is not provided (step S17; NO), that is, when the photographing lenses 101a and 101b are fixed focus lenses, the fixed focal length d is read from the ROM or the like (step). S18). Then, the transmission unit 108 edits the focal length d into the focal length information (Step S20).

  Next, the imaging terminal 10 inquires of the image display terminal 20 about the resolution and data compression format of image data that can be displayed (step S21). Thereafter, after the reply from the image display terminal 20, the image processing unit 102 generates left image data from the image photographed by the left photographing lens 101a with the replied resolution (step S22). Similarly, the image processing unit 102 generates right image data from an image photographed by the right photographing lens 101b (step S23). Thereafter, the data compression unit 103 combines the left image data and the right image data and performs data compression (step S24).

  Then, when the image display terminal can display a stereoscopic image, the transmission unit 108 transmits the composite image data, the focal length information, and the lens arrangement information to the image display terminal 20. On the other hand, when the image display terminal cannot display a stereoscopic image, the transmission unit 108 transmits planar image data to the image display terminal 20 (step S25).

  When the imaging terminal 10 finishes shooting by key input or the like, the imaging terminal 10 and the image display terminal 20 are disconnected, and the processing related to the images shot by the shooting lenses 101a and 101b is ended (step S26). .

  Next, the operation of the image display terminal will be described with reference to FIG.

  First, when shooting is started at the imaging terminal 10, the imaging terminal 10 and the image display terminal 20 are connected via the mobile communication network 30 (step S30). Next, it is determined by the image display terminal 20 whether a stereoscopic image can be displayed (step S31). This determination is made by referring to the parameter relating to the attribute of the image display terminal 20 stored in the data storage means such as ROM.

  As a result, when the image display terminal 20 cannot display a stereoscopic image (step S31; NO), the fact is returned to the imaging terminal 10, and the resolution and data compression format of the displayable image data are the imaging terminal. 10 (step S32). On the other hand, the planar image data transmitted from the imaging terminal is received by the receiving unit 201 (step S33). Then, the data separation unit 202 converts the planar image data into a data format that can be reproduced on the image display terminal 20 (step S34). Next, the converted planar image data is output to the left video memory 205 and the right video memory 206 by the image display unit 204 (step S35). Then, the planar image represented by the planar image data is displayed on the stereoscopic image display 207 (step S36).

  On the other hand, when the image display terminal 20 can display a stereoscopic image (step S31; YES), the fact is returned to the imaging terminal 10, and the resolution and data compression format of the displayable image data are the imaging terminal. 10 (step S37). Next, the receiving unit 201 receives data including image data transmitted from the imaging terminal 10 (step S38). Then, the reception unit 201 separates the composite image data, the lens arrangement information, and the focal length information from the received data (step S39). Thereafter, the data separation unit 202 separates the right image data and the left image data from the composite image data, and converts these image data into a data format that can be displayed on the image display terminal 20 (step S40). Next, the recommended recommended observation distance information d1 is read by the display position calculation unit 203 (step S41). Further, the display position calculation unit 203 calculates the display positions of the left image and the right image based on the lens arrangement information and the focal length information (step S42). This calculation method is selected according to the terminal shape information included in the lens arrangement information. The initial value of the distance between the observer's eyes and the stereoscopic image display 207 is calculated as d1. Then, the display coordinates of the left image data and the right image data are converted by the image display unit 204 based on the calculated display position (step S43). The converted left image data and right image data are output to the left video memory 205 and the right video memory 206, respectively (step S44). As a result, the left image and the right image are displayed on the stereoscopic image display 207 so as to generate binocular parallax (step S45).

  Hereinafter, the operation and effect of the image display system 1 according to the embodiment of the present invention will be described. According to the image display system 1 configured as described above, the imaging terminal 10 generates left image data and right image data from images captured by the pair of imaging lenses 101a and 101b provided in the imaging unit 101. Since the information for specifying the relative position and direction between the pair of photographing lenses 101a and 101b, the left image data and the right image data, and the focal length information of the photographing lenses 101a and 101b are transmitted, the left and right image data are transmitted. It is possible to transmit information related to shooting conditions at the time of shooting together. Further, the image display terminal 20 receives information for specifying the relative position and direction between the pair of photographing lenses 101a and 101b, left image data and right image data, and focal length information of the photographing lenses 101a and 101b. Since the display positions of the images are calculated based on the lens arrangement information and the focal length information so as to generate binocular parallax, and the left and right images are displayed at the calculated display positions, the left and right image data captured by the imaging terminal 10 are displayed. Can be reproduced as a stereoscopic image that produces binocular parallax close to reality in accordance with the shooting conditions.

The present invention is not limited to the above-described embodiment. For example, the photographing lenses 101a and 101b may have fixed focal lengths. In this case, the image display system 1 may not include the focal length adjustment unit 104 and the focal length detection unit 105.
Further, although the transmission unit 108 transmits the composite image data including the left image data and the right image data, the left image data and the right image data may be transmitted as separate image data.

1 is a schematic configuration diagram of an image display system showing a preferred embodiment of the present invention. It is the top view which looked at the straight type imaging terminal and the photographic subject from the direction perpendicular to the field containing an optical axis. It is the side view which looked at the rotation type imaging terminal from the optical axis direction. It is the top view which looked at the rotation-type imaging terminal and the imaging | photography object from the direction perpendicular | vertical with respect to the surface containing an optical axis. It is the top view which looked at the folding-type imaging terminal and imaging | photography subject from the direction perpendicular | vertical with respect to the surface containing an optical axis. It is the figure which looked at an observer's eyes and an image displayed on a display from the upper part of an observer. It is a 1st flowchart which shows operation | movement of an imaging terminal. It is a 2nd flowchart which shows operation | movement of an imaging terminal. It is a flowchart which shows operation | movement of an image display terminal.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image display system, 10 ... Imaging terminal, 20 ... Image display terminal, 30 ... Mobile communication network, 101a ... Left imaging lens, 101b ... Right imaging lens, 101 ... Imaging means, 102 ... Image processing part, 103 ... Data Compression unit 104 ... Focal distance adjustment unit 105 ... Focal distance detection unit 106 ... Lens position detection unit 108 ... Transmission unit 201 ... Reception unit 202 ... Data separation unit 203 ... Display position calculation unit 204 ... Image Display unit 205... Left video memory 206. Right video memory 207 Display 3D image display.

Claims (4)

An imaging means comprising a pair of imaging lenses;
Image processing means for generating left-side image data and right-side image data from respective images photographed by the pair of photographing lenses;
Lens arrangement information for specifying the relative position and direction of the pair of photographing lenses, the left image data and the right image data generated by the image processing means, and focal length information regarding the focal length of the photographing lens. A transmission means for transmitting;
An imaging terminal comprising: Lens arrangement information for specifying the relative position and direction of a pair of photographing lenses of the imaging terminal, left image data and right image data representing an image photographed by the pair of photographing lenses, and a focal point related to a focal length of the photographing lens Receiving means for receiving distance information;
Binocular parallax between the left image represented by the left image data and the right image represented by the right image data based on the lens arrangement information and the focal length information received by the receiving unit Display position calculating means for calculating the display position of the left image and the right image so as to produce
Image display means for displaying the left image and the right image at the display position calculated by the display position calculation means;
An image display terminal comprising: An image display system that displays an image captured by an imaging terminal as a stereoscopic image on an image display terminal,
The imaging terminal is
An imaging means comprising a pair of imaging lenses;
Image processing means for generating left-side image data and right-side image data from respective images photographed by the pair of photographing lenses;
Lens arrangement information for specifying the relative position and direction of the pair of photographing lenses, the left image data and the right image data generated by the image processing means, and focal length information regarding the focal length of the photographing lens. A transmission means for transmitting;
With
The image display terminal is
Lens arrangement information for specifying the relative position and direction of a pair of photographing lenses of the imaging terminal, left image data and right image data representing an image photographed by the pair of photographing lenses, and a focal point related to a focal length of the photographing lens Receiving means for receiving distance information;
Binocular parallax between the left image represented by the left image data and the right image represented by the right image data based on the lens arrangement information and the focal length information received by the receiving unit Display position calculating means for calculating the display position of the left image and the right image so as to produce
Image display means for displaying the left image and the right image at the display position calculated by the display position calculation means;
An image display system comprising: The imaging terminal further includes focal length detection means for detecting a focal length of the pair of photographing lenses,
The transmission unit includes focal length information related to a focal length detected by the focal length detection unit, lens arrangement information for specifying a relative position and direction of the pair of photographing lenses, and the image processing unit. Send left image data and right image data,
The image display system according to claim 3.

Images